in Code, HowTo

Crash Reporting For MinGW 32 (Windows) and Clang (macOS) With Qt

I recently had a customer describe some very random crashes with my software. There didn’t seem to be a pattern or a way for me to reproduce the problems locally to debug them. So my first thought was to get him to install a version with some kind of crash reporting tooling so I could get a stack trace to help track down the issue.

I’d looked into implementing some form of crash reporting quite a while ago, but it was never a very high priority for me because I don’t get a lot of bug reports. In this case though it seemed like it would be easiest if I could produce a version of my software with some built-in stack tracing.

The first thing I did was to look at what libraries were available for this. My criteria were:

  • simple to use/integrate with a Qt application
  • works with the MinGW 32-bit compiler on Windows and the clang compiler on macOS
  • inexpensive (or free!)
  • usable in commercial software

The most promising were BreakPad (older) or CrashPad (newer) from Google. From what I understand, Breakpad no longer works on macOS which is why they switched to CrashPad. Unfortunately CrashPad doesn’t handle 32-bit MinGW builds. The reason I’m stuck with the 32-bit version is that Qt currently ships its MinGW builds of the libraries and toolchain using the 32-bit MinGW 4.9.2 compiler.

So after a lot of searching and piecing things together, I’ve created something that works and fits my criteria. It’s very simple – all it does is save the stack trace to a file that the user can send me – and requires some instructions to the user to work with it. If I wanted to get fancier I could have it automatically post the information to a web server, but for now this is simple and it works.

It might work on Linux too since the code path for macOS should be POSIX compliant, though I haven’t tried it. It could also be extended to handle MSVC compiles (or maybe it already does!), but I don’t use that compiler so I can’t test it.

I used many different sites in my search, but my primary sources were Catching Exceptions and Printing Stack Traces for C on Windows, Linux, & Mac by Job Vranish, Printing a Stack Trace with MinGW by Daniel Holden, and the C++ name mangling article on Wikipedia.

Code

The code, along with example usage, may be found on the asmCrashReport GitHub page.

What Makes This One Different?

None of the sources I found online handled the cases quite the same way and they didn’t give the results I was looking for (name demangling for example). What I’ve put together uses ideas from a bunch of different sources.

In addition, since I am targetting Qt applications using C++11, I took the liberty of using Qt classes and methods when putting this together.

Usage

In your .pro file, you need to include the asmCrashreport.pri file. e.g.:

This will define ASM_CRASH_REPORT for the preprocessor and modify the C/CXX and linker flags to include the debug symbols properly.

I usually wrap it in a config option so it can be included using “CONFIG += asmCrashReport” on the command line or in Qt Creator.

In your main.cpp, include the header:

This provides a simple API:

In your main() function, set your signal handler after you have declared your QApplication and set the application name and version number:

Example Results

(Aside: If you can figure out why the macOS stack trace gives junk data for the most recent frame I’ll owe you a beer!)

Some Code Details

I’m not going to go over everything in the code, just some highlights of things that are maybe different from the other posts on this subject.

I have shortened the code and split it up for this post. You can find the full source in the GitHub repo.

_writeLog()

The first part is fairly straightforward – the _writeLog() function writes the log and calls the callback if one was provided.

_addr2line()

This next function – _addr2line() – uses an external tool to translate memory addresses to file and line numbers. On macOS this is the atos tool, on Windows we use the addr2line tool from Cygwin.

Note that on Windows this means we need to ship the add2line tool and supporting DLLs to the end-user. See the GitHub repo README for more details.

If the external tool fails to run it will return an error. If it succeeded but the tool could not find the symbol, it will return an empty string. Otherwise it returns the file and line information as a string.

Windows-Specific

Next up is the Windows-specific code.

Note that I only include the stack frame layout for i386 since we are targetting the MinGW 32-bit compiler.

The other interesting thing here is what I’m doing with the result of addr2line() while walking the stack trace. The addr2line tool does not demangle the C++ symbols, so we end up with symbols like “_ZN9crashTest12divideByZeroEi“. To make them more readable, I identify them with a regular expression, use a function to demangle them properly (abi::__cxa_demangle()), and substitute the human-readable version of the symbol – e.g. “crashTest::divideByZero(int)“.

The last function – winExceptionHandler() – just converts the exception to a human-readable string, gathers the stack trace, and writes the log.

macOS-Specific

The macOS-specific code is similar, but we can use the backtrace() and backtrace_symbols() functions to simplify things.

These functions get us part-way there. The output of these gives us something like:

With a regular expression, I’m matching the _ZN* symbol and sending that to _addr2Line() to look up the file and line number with atos. If it finds it, I’m just substituting the result to get:

The other two functions here – _posixSignalHandler() and _posixSetupSignalHandler() set up the necessary plumbing to capture the signal, convert it a human-readable form, gather the stack trace, and write the log.

setSignalHandler()

Finally, we have the main setup function. All this does is set some of the static variables and call the appropriate signal handler function.

Final Notes

While what I have currently works for me, you might need to adapt it for your own use.

I’ve intentionally kept this code as simple as I can. Some of the possibilities for modification would be to include a working version for MSVC, include a Qt-based way to submit the crash log via a web server, expand the data included in the report to include machine information, etc..

If you make any changes, please feel free to submit pull requests on the github page.

I hope someone finds it useful!

Write a Comment

Comment