It is funny how they are still touting that C++ exceptions are/should not (be) slow and that we should "complain to your implementation purveyor" if they are. Yet, the two most used C++ compilers today with top-notch optimizations (GCC and Clang) are still slow with exceptions.
Took this code [2] and tested with O2 optimizations with gcc 11.x and clang 13.x versions:
$ ./exceptions-test-gcc
3906 return code errors in 0.000562 seconds
3906 exception errors in 0.005740 seconds
Exceptions are 10.2x slower than return codes
$ ./exceptions-test-clang
3906 return code errors in 0.000142 seconds
3906 exception errors in 0.004568 seconds
Exceptions are 32.2x slower than return codes
Performance benchmarks can be tricky, especially when used to evaluate a feature of a programming language. In this case, by adding "no_inline" to both the version and pre-initializing the exception object, the difference in performance on my machine decreased to 2.8 times from 22 times. The question is, which benchmark version provides a better assessment of the feature.
The discussion about exception performance mainly focuses on the runtime performance implications of the exception process, which should be negligible. However, due to the current semantics of exception handling, the throw/catch process significantly affects control and data flow, leading most compilers to be more pessimistic about optimization.
The debate between error codes and exceptions seems like a false choice to me. Both have their strengths and weaknesses and should be used interchangeably in the same codebase depending on the circumstances.
In my opinion, the main advantage of exceptions is when there is a large distance between the point of error generation and the point in the program where there is the best information to handle the error. In such cases, monadic error handling would require every intermediary computation to handle the error, whereas exception propagation allows unrelated computations to essentially ignore the error state.
I would even suggest that C++ needs something like "Conditions and Restarts" as a more general exception mechanism.
> In this case, by adding "no_inline" to both the version and pre-initializing the exception object, the difference in performance on my machine decreased to 2.8 times from 22 times.
That’s not a realistic benchmark though. I’ve never seen real-world non-benchmark code which pre-initializes exceptions. They’re always created completely from scratch. Beyond it being uncommon, pre-initializing would also unnecessarily waste memory when you don’t throw the exception.
When it comes to no_inline, I think one should go the opposite direction. Because no_inline here just showed that error codes can be made slower when compiler knows less about code, bringing it closer to exceptions. What’s disputed though is not that you can make ordinary code slower, but that compilers will realistically make your exception-throwing code faster. So I’d annotate with always_inline instead.
Your benchmark supports the position that exceptions should be used instead of return codes for better average case performance, at least for GCC.
If your function throws an exception 9% of the time, then it outperforms error codes. That's because you always pay for the cost of an error code, regardless of success, but you only pay for the cost of an exception on failure.
I suspect most functions don't throw an exception 9% of the time that they are called.
For clang, yes the situation is worse, indicating that if your function throws an exception ~3% of the time it's better to use error codes, but once again I don't think most functions throw an exception 3% of the time.
If you need to optimize for latency, or real time performance, then exceptions are not at all suitable and I think most performance sensitive developers who have those requirements already know that.
In said benchmark the function threw an exception 3906 / 1000000 = 0.4% of the time and it was from 10 to 32 times (edit: 73.2 times on my Mac Mini M1) slower on average. 9% would be much worse.
EDIT: updated the ratio to reflect the GP comment instead of the SO post.
The performance of the happy path matters a lot more, and I don't know of any compiler smart enough to return to different addresses for good and bad error codes. How much is the penalty for always checking a code after returning, compared to a try block that didn't catch anything (which should be free, apart from a compile time unwind table)?
Once I was in a training session on C++ STL, and the instructor told us not to use vector at() method but instead check the size and handle out-of-bounds issues using something else, because exceptions are slow.
Coming from another language, this sounded wild to me. There is a perfectly fine API in the standard library that we choose not to use but instead reinvent the wheel because of an "implementation detail" of a compiler, something a beginner or even an intermediate level C++ developer is likely unaware of.
Unfortunely C and C++ have too many folks that do performance analysis by gut feeling instead of using a profiler, many times asserting stuff that with modern hardware are completly irrelevant.
I don't want to give to much credibility to the benchmark OP posted because it is quite flawed, but MSVC performs absolutely terribly on that benchmark, far worse than GCC or clang with a 58x slowdown when compiled in 64-bit mode. I can only imagine how much worse it would be if compiled in 32-bit mode since MSVC in 32-bit mode does not use zero cost exception handling but uses a frame based, setup/teardown exception handling mechanism.
The exception handling you're referring to about pinging back to Win32 I presume is structured exception handling, which is not the default exception handling mechanism used by MSVC, is not recommended to be used, and is not at all performance sensitive.
What's the current state of automatic enforcement of these? I know that Microsoft has built tooling to enforce a subset of them, but I'm not aware of anything for Linux that specifically targets these.
Clang-tidy[0] supports a good set of them (cppcoreguidelines- prefix). It supports extending the checks too with your own checks that operate over the AST[1], so if there are extra guidelines internal to your company like mine has, there's room for extension there. For automation, I've seen a few examples of people integrating clang-tidy into IDEs, git hooks, and code review systems.
Readit News
Took this code [2] and tested with O2 optimizations with gcc 11.x and clang 13.x versions:
[1] https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines...[2] https://stackoverflow.com/a/78301673
The discussion about exception performance mainly focuses on the runtime performance implications of the exception process, which should be negligible. However, due to the current semantics of exception handling, the throw/catch process significantly affects control and data flow, leading most compilers to be more pessimistic about optimization.
The debate between error codes and exceptions seems like a false choice to me. Both have their strengths and weaknesses and should be used interchangeably in the same codebase depending on the circumstances.
In my opinion, the main advantage of exceptions is when there is a large distance between the point of error generation and the point in the program where there is the best information to handle the error. In such cases, monadic error handling would require every intermediary computation to handle the error, whereas exception propagation allows unrelated computations to essentially ignore the error state.
I would even suggest that C++ needs something like "Conditions and Restarts" as a more general exception mechanism.
That’s not a realistic benchmark though. I’ve never seen real-world non-benchmark code which pre-initializes exceptions. They’re always created completely from scratch. Beyond it being uncommon, pre-initializing would also unnecessarily waste memory when you don’t throw the exception.
When it comes to no_inline, I think one should go the opposite direction. Because no_inline here just showed that error codes can be made slower when compiler knows less about code, bringing it closer to exceptions. What’s disputed though is not that you can make ordinary code slower, but that compilers will realistically make your exception-throwing code faster. So I’d annotate with always_inline instead.
If your function throws an exception 9% of the time, then it outperforms error codes. That's because you always pay for the cost of an error code, regardless of success, but you only pay for the cost of an exception on failure.
I suspect most functions don't throw an exception 9% of the time that they are called.
For clang, yes the situation is worse, indicating that if your function throws an exception ~3% of the time it's better to use error codes, but once again I don't think most functions throw an exception 3% of the time.
If you need to optimize for latency, or real time performance, then exceptions are not at all suitable and I think most performance sensitive developers who have those requirements already know that.
EDIT: updated the ratio to reflect the GP comment instead of the SO post.
Coming from another language, this sounded wild to me. There is a perfectly fine API in the standard library that we choose not to use but instead reinvent the wheel because of an "implementation detail" of a compiler, something a beginner or even an intermediate level C++ developer is likely unaware of.
VC++ is much faster, as it pings back into Win32 exceptions, for example.
The exception handling you're referring to about pinging back to Win32 I presume is structured exception handling, which is not the default exception handling mechanism used by MSVC, is not recommended to be used, and is not at all performance sensitive.
This advice comes from Microsoft themselves:
https://learn.microsoft.com/en-us/cpp/cpp/exception-handling...
The C++ exception handling mechanism that is used by default does not "ping back" into Win32.
https://learn.microsoft.com/en-us/cpp/code-quality/using-the...
[0] https://clang.llvm.org/extra/clang-tidy/
[1] https://github.com/coveooss/clang-tidy-plugin-examples
Or you can get the DevOps team to install Sonar, PVS, and then it gets "automatically" enforced by not allowing merges with broken guidelines.