Efficiency: creating vs. throwing an Exception

Question
1

Here’s a question that just came up at work: I’ve always taken for granted that you should avoid throwing Exceptions as part of a standard workflow, because it’s inefficient due to the stacktrace. Is that because of the throwing of the Exception, or the construction of it?

More precisely, the question at hand is whether using Future.failed() is any faster than throwing an Exception. I had sort of assumed it was, but folks have pointed out that that may well not be true (since Future.failed() still takes an Exception as a parameter), so I’m wondering if there’s an existing benchmark on the topic…

2

Construction is usually far more expensive than throwing. Both are expensive, but depending on details, construction is usually at least 5x and usually more like 30x more expensive in the microbenchmarks I’ve done.

So avoiding the throw isn’t going to help you much speedwise. If you can throw a stackless exception for control flow, that’s better. If the stackless exception doesn’t have far to go, sometimes the JVM can turn it into jumps and it can be way faster.

I think I gave a talk about related things at ScalaDays 2013 or something like that.

3

I have been under the assumption that creating an exception is slow, but throwing it is fast. Is that far from correct?

4

Here, I created a new benchmark (using Thyme):

import scala.util.control.ControlThrowable
val e = new Exception
trait T {
  def foo: Int
  def bar: Int
  def baz: Int
  def qux: Either[Throwable, Int]
  def yig: Either[Exception, Int]
}
class A extends T {
  def foo: Int = throw new Exception
  def bar: Int = throw e
  def baz: Int = throw new ControlThrowable {}
  def qux: Either[Throwable, Int] = Left(new ControlThrowable {})
  def yig: Either[Exception, Int] = Left(new Exception)
}
class B extends T {
  def foo: Int = throw new Exception
  def bar: Int = throw e
  def baz: Int = throw new ControlThrowable {}
  def qux: Either[Throwable, Int] = Left(new ControlThrowable {})
  def yig: Either[Exception, Int] = Left(new Exception)
}
class C extends T {
  def foo: Int = throw new Exception
  def bar: Int = throw e
  def baz: Int = throw new ControlThrowable {}
  def qux: Either[Throwable, Int] = Left(new ControlThrowable {})
  def yig: Either[Exception, Int] = Left(new Exception)
}
class D extends T {
  def foo: Int = throw new Exception
  def bar: Int = throw e
  def baz: Int = throw new ControlThrowable {}
  def qux: Either[Throwable, Int] = Left(new ControlThrowable {})
  def yig: Either[Exception, Int] = Left(new Exception)
}
val a = Array.tabulate(1000)(i => (i%4) match { case 0 => new A; case 1 => new B; case 2 => new C; case _ => new D })
val th = new ichi.bench.Thyme
def foo = {
  var i, s = 0
  while (i < a.length) {
    try { s += a(i).foo } catch { case _: Exception => s += 1 }
    i += 1
  }
  s
}
def bar = {
  var i, s = 0
  while (i < a.length) {
    try { s += a(i).bar } catch { case _: Exception => s += 1 }
    i += 1
  }
  s
}
def baz = {
  var i, s = 0
  while (i < a.length) {
    try { s += a(i).baz } catch { case _: Throwable => s += 1 }
    i += 1
  }
  s
}
def qux = {
  var i, s = 0
  while (i < a.length) {
    a(i).qux match {
      case Right(i) => s += i
      case _ => s += 1
    }
    i += 1
  }
  s
}
def yig = {
  var i, s = 0
  while (i < a.length) {
    a(i).yig match {
      case Right(i) => s += i
      case _ => s += 1
    }
    i += 1
  }
  s
}
th.pbenchOff(){ foo }{ qux }
th.pbenchOff(){ bar }{ qux }
th.pbenchOff(){ baz }{ qux }
th.pbenchOff(){ yig }{ qux }

Results (one run, but results are typical):

Benchmark comparison (in 617.3 ms)
Significantly different (p ~= 0)
  Time ratio:    0.00577   95% CI 0.00553 - 0.00601   (n=20)
    First     1.910 ms   95% CI 1.864 ms - 1.957 ms
    Second    11.03 us   95% CI 10.66 us - 11.40 us

Benchmark comparison (in 609.1 ms)
Significantly different (p ~= 0)
  Time ratio:    0.10376   95% CI 0.10127 - 0.10626   (n=20)
    First     107.6 us   95% CI 106.0 us - 109.3 us
    Second    11.17 us   95% CI 10.96 us - 11.37 us

Benchmark comparison (in 539.0 ms)
Significantly different (p ~= 0)
  Time ratio:    0.13197   95% CI 0.13019 - 0.13375   (n=20)
    First     81.38 us   95% CI 80.58 us - 82.18 us
    Second    10.74 us   95% CI 10.64 us - 10.84 us
  Individual benchmarks not fully consistent with head-to-head (p ~= 1.211e-12)
    First     91.14 us   95% CI 90.31 us - 91.98 us
    Second    10.52 us   95% CI 10.45 us - 10.59 us

Benchmark comparison (in 1.826 s)
Significantly different (p ~= 0)
  Time ratio:    0.00582   95% CI 0.00578 - 0.00587   (n=30)
    First     1.840 ms   95% CI 1.832 ms - 1.849 ms
    Second    10.71 us   95% CI 10.65 us - 10.78 us
  Individual benchmarks not fully consistent with head-to-head (p ~= 5.094e-07)
    First     2.106 ms   95% CI 2.082 ms - 2.131 ms
    Second    10.55 us   95% CI 10.49 us - 10.60 us

Conclusion: passing back a new stackless exception wrapped in a Left (I don’t know why you’d do this…) is about 8x faster than throwing that stackless exception, about 10x faster than throwing a pre-generated stack-containing exception, and about 170x faster than passing back a new exception with a stack or creating and throwing that exception.

Head-to-head of the pre-generated vs. freshly created exception, both with throwing, gives about a 20x advantage to pre-generation:

scala> th.pbenchOff(){ foo }{ bar }
Benchmark comparison (in 506.0 ms)
Significantly different (p ~= 0)
  Time ratio:    0.04942   95% CI 0.04867 - 0.05016   (n=20)
    First     1.594 ms   95% CI 1.579 ms - 1.609 ms
    Second    78.79 us   95% CI 77.86 us - 79.72 us
  Individual benchmarks not fully consistent with head-to-head (p ~= 8.028e-06)
    First     1.857 ms   95% CI 1.781 ms - 1.933 ms
    Second    78.38 us   95% CI 77.89 us - 78.88 us

Quick and dirty and various things could go wrong (and did during testing–sometimes I got weird unoptimized results), but this is roughly what I’ve seen before and is a good rule of thumb.

  • Return value without boxing–super fast!! (not shown here)
  • Return value boxed–fast!
  • Throw stackless or pre-generated exception–10x slower
  • Create stacked exception–20x slower again
1 Like
5

Very, very useful to know – thanks!