If you are used to benchmark your Ruby scripts or if you ever had to improve the performance of some strategic tasks, then this post won’t tell you nothing new because you should already know that Exceptions are slow. And this is not really a Ruby problem: .NET Exceptions are slow, JAVA Exceptions are slow just because the begin/raise/rescue (or try/throw/catch) architecture is slow by nature.
But how slow are Ruby Exceptions?
The answer to this question really depends on how complex is your code. Here I just want to show you a very simple example, extracted from a really strategic RoboDomain DNS sorting algorithm.
The code is fairly simple: the value for an A record is expected to be an IP Address while the value for a NS record is expected to be represented by a FQDN as string. The code should parse the data and return the normalized value depending on some conditions.
The first version of the algorithm is completely based on Exceptions. IPAddr raises an ArgumentError when the argument is not a valid IP Address. In this case, the script gracefully returns the value as string.
The second version checks against the record value (I agree with you that is quite empiric, but representative for the sake of this benchmark).
The third version is a less empiric alternative that uses some regular expressions to check whether the data looks like an IP before actually feeding the IPAddr class. Under the hood, the IPAddr class performs a really similar task, the only difference here is that in the latter case I’m not using Exceptions at all.
Results speak for themselves. *
(*) As pointed out by Curtis Summers in the comments,a huge amount of time is actually spent by IPAddr trying to resolve the hostname. For more benchmarks, also look at this test.
With Ruby 1.8.7, the algorithm (note, this is a super-simplified version of the original algorithm) is about 37 times slower when Exceptions are involved.
$ ruby if_vs_exception.rb
Rehearsal ------------------------------------------------
NS exception 4.330000 5.750000 10.080000 ( 37.599575)
NS if 0.140000 0.010000 0.150000 ( 0.136280)
NS regexp 0.450000 0.000000 0.450000 ( 0.454040)
A exception 2.010000 0.020000 2.030000 ( 2.047711)
A if 2.060000 0.020000 2.080000 ( 2.074642)
A regepx 2.030000 0.020000 2.050000 ( 2.054930)
-------------------------------------- total: 16.840000sec
user system total real
NS exception 4.420000 5.810000 10.230000 ( 38.350608)
NS if 0.130000 0.000000 0.130000 ( 0.130863)
NS regexp 0.450000 0.000000 0.450000 ( 0.447975)
A exception 2.000000 0.020000 2.020000 ( 2.016231)
A if 2.020000 0.020000 2.040000 ( 2.043085)
A regepx 2.030000 0.020000 2.050000 ( 2.048402)
The same story with Ruby 1.9.1.
Rehearsal ------------------------------------------------
NS exception 4.650000 5.800000 10.450000 ( 39.134858)
NS if 0.080000 0.000000 0.080000 ( 0.079427)
NS regexp 0.320000 0.000000 0.320000 ( 0.320389)
A exception 1.180000 0.010000 1.190000 ( 1.182892)
A if 1.200000 0.000000 1.200000 ( 1.209433)
A regepx 1.220000 0.000000 1.220000 ( 1.219345)
-------------------------------------- total: 14.460000sec
user system total real
NS exception 4.520000 5.720000 10.240000 ( 37.931455)
NS if 0.080000 0.000000 0.080000 ( 0.078286)
NS regexp 0.320000 0.000000 0.320000 ( 0.320988)
A exception 1.190000 0.000000 1.190000 ( 1.186198)
A if 1.210000 0.010000 1.220000 ( 1.220254)
A regepx 1.220000 0.000000 1.220000 ( 1.215634)
Does this mean I should forget about Exceptions?
Absolutely no! I love Exceptions and you should love them too. However, Exceptions should not be expected and, in some circumstances, an if can be much more convenient, or at least, efficient.
One important lesson to learn from these benchmarks is that exceptions should not be part of the regular application flow. This shouldn’t prevent you to use them to handle unexpected situations and exception performance shouldn’t be a big deal in this case.
A lesson learned from this specific case, is to avoid using exceptions in place of conditional statements to handle not exceptional situations.
UPDATE: for more interesting comments, check out the Reddit page.
What’s the problem with slow exceptions? Exceptions are not part of the normal control flow, and are by definition unexpected. Thus, nobody expects outstanding performance for exceptions…
Interestingly, on JRuby, the difference is much, much smaller: 2sec vs 1.5sec.
Thanks Vladimir!
I was planning to run the benchmarks on JRuby. This is a really interesting comparison, thank you for posting here the results.
Yes, exceptions are slow, but this is not a good example. There’s quite a bit more logic in IPAddr#initialize, and this example is simply circumventing all of that additional code:
http://ruby-doc.org/core/classes/IPAddr.src/M000734.html
In fact, it’s very likely that circumventing the call to IPSocket.getaddress is the majority of the time difference for this example.
Indeed, it depends on the specific code.
The example posted above was a simplification of the original code, where I actually saved up to 10 hours per 900K records just refactoring the logic in order to avoid exceptions.
You can write as many benchmarks as you wish, but you won’t be able to determine whether skipping Exception means to be x2, x4 or xN times faster.
Here’s an other example.
http://gist.github.com/275768#file_if_vs_exception_vs_regexp_zerodivision.rb
Here the difference is smaller compared to the original benchmark also because the execution stack is smaller resulting in a more lightweight script.
However, we all agree that exceptions cost. ;)
A key finding from this benchmark, IMHO, is that exceptions do not slow things down when they’re not raised. That is, having the rescue mechanism in place does not seem to slow down properly working code by any serious degree.. it’s only when exceptions are actually raised that they’re “slow.”
I agree with Thorsten Böttger. Since exceptions are not/should not be part of ordinary flow – hence their name – I don’t see their lack of performance as motivation for not using them in places where they really are “exceptional.” Just in case anyone gets the wrong idea from these findings.. ;-)
[...] How slow are (Ruby) Exceptions? [...]
[...] Ele indicou um post falando a respeito: http://www.simonecarletti.com/blog/2010/01/how-slow-are-ruby-exceptions [...]