The Cost of Exceptions

What do exceptions cost, and why you should use them with care.

A recent discussion with colleagues prompted me to write a simple program to get some measures on the performance cost of exceptions.

Update: a reader pointed out that my original program had a bug and the comparison wasn’t fair. Please see the update at the bottom for details.

I wrote two C++ functions that convert a string to an integer.

The first:

auto to_int_except(const std::string& s) noexcept(false) {
  return std::stoi(s);
}

The second:

auto to_int_noexcept(const std::string& s) noexcept {
  int result;
  auto [_, ec] = std::from_chars(s.data(), s.data() + s.size(), result);
  return make_pair(result, ec);
}

The second function looks complicated. It has a local variable, multiple function calls, and it returns a std::pair by value. While the first one is much more concise and just returns an integer. Which do you think is more performant? If you answered the first, you'd be wrong.

The two functions differ not only in implementation but also in interface. to_int_except returns the integer result and throws an exception when the argument cannot be converted, so the caller should enclose it in try {...} and catch the potential exception. to_int_noexcept returns the result with the error code, so the caller should check the error code before using the result.

I generated 10000 strings that can be converted to integers and 10000 strings that are not valid representation of integers, then test each function against the error-free case and the error case. The full program is on GitHub.

When I disabled compiler optimization, the results were:

good_nums(to_int_noexcept): 2739
bad_nums(to_int_noexcept): 1124
good_nums(to_int_except): 14529
bad_nums(to_int_except): 80702

When there was no error, to_int_noexcept was about 5 times faster than to_int_except. In the error case, the difference was almost 80 folds.

When I enabled optimization, the gap was even more pronounced:

good_nums(to_int_noexcept): 383
bad_nums(to_int_noexcept): 94
good_nums(to_int_except): 15411
bad_nums(to_int_except): 80406

The differences in the error-free case and the error case were 40 and 850 times respectively. It’s worth noting that to_int_noexcept was even faster when errors occur due to early returns. While to_int_except was much slower when there were errors than when there wasn't, because exceptions were thrown.

The performance cost of exceptions comes from a number of factors and depends on implementation. Besides the need to save and restore program context like a regular function call, exception handling also involves searching for the handler by exception type in multiple scopes. Besides taking more time, this also makes optimization hard. In the above example, compiler optimization made no difference to the code that uses exception. In some implementations, program context is saved whenever a try {...} block is encountered, so there is cost even when no exception is thrown at run time. Exception objects usually contain stack traces and error messages, which can also be expensive to construct.

Performance isn’t the most important reason to be prudent in the use of exceptions. Like goto, exceptions disrupts structrued control flow and can result in unexpected execution path. In fact, exceptions can be more implicit and harder to analyze than goto. It causes a lot of nuances in languages without garbage collection, such as C++. For those reasons, Google’s C++ style guide disallows exceptions. The same thinking was also embodied in the design of Go’s error handling.

By definition, exceptions are for exceptional cases. It’s best to restrict them to errors that are not related to program logic, such as I/O, network , and out-of-memory errors. Many people use exceptions to handle user input errors. However, user input validation is part of the application logic andnormal program flow and does not warrant the use of exceptions.

Both C++ and Java tried to make exception types part of a function’s interface/signature, but thoses efforts are generally considered failures due to how cumbersome they are in practice. Checked exceptions in Java are used less and less. C++17 deprecated throw() and considered it sufficient to distinguish throwing and non-throwing functions with noexcept. Because many programmers do not handle exceptions in a fine-grained manner, it's best for library designers not to expect the caller to handle each exception type. When it is not reasonable to terminate the program, it is better to return an error code and force the caller to handle it explicitly than to throw an exception.

---

A reader pointed out that my original program had a bug (see fix), and the two functions used different library functions to convert the strings, which made the comparison unfair. I did the test again with another function:

auto to_int_except2(const std::string& s) noexcept(false) {
  int result;
  auto [_, ec] = std::from_chars(s.data(), s.data() + s.size(), result);
  if (ec != std::errc()) {
    throw std::invalid_argument("invalid argument");
  } else {
    return result;
  }
}

The numbers without optimization are:

good_nums(to_int_noexcept): 693
bad_nums(to_int_noexcept): 390
good_nums(to_int_except1): 247
bad_nums(to_int_except1): 58822
good_nums(to_int_except2): 647
bad_nums(to_int_except2): 27478

And with optimization:

good_nums(to_int_noexcept): 113
bad_nums(to_int_noexcept): 26
good_nums(to_int_except1): 217
bad_nums(to_int_except1): 54962
good_nums(to_int_except2): 122
bad_nums(to_int_except2): 30247

At least for C++, the overhead of exceptions is only shown when they are actually thrown, so for errors that don’t frequently occur, performance shouldn’t be an important factor in deciding whether to use exceptions.