Avoiding dynamic memory allocation when throwing standard library exceptions

Many C++ use-cases don’t allow for heap allocation after some startup phase. This means that for those use-cases exceptions are off the table, since typical implementations will always lead to heap allocation when an exception is thrown. I tried to come up with a small library that solves this problem, but this is not more than a proof of concept.

Note: For the rest of this post, GCC and libstdc++ are assumed.

Allocating exceptions

Itanium ABI

The Itanium ABI, which is used by probably every C++ compiler except for MSVC, defines (among other things) what should happen when an exception is thrown.
It outlines the following broad steps:

1. Call __cxa_allocate_exception to allocate memory required to store the exception object
2. Evaluate the thrown expression and copy it into the buffer allocated in step 1
3. Call __cxa_throw, which does some bookkeeping before calling _Unwind_RaiseException, which in turn starts the process of stack unwinding

So what exactly happens when __cxa_allocate_exception is called? The Itanium ABI does not specify that the allocated memory HAS to be on the heap, only that

“This storage must persist while stack is being unwound, since it will be used by the handler, and must be thread-safe”
– Section 2.4.2 of the Itanium C++ ABI spec

Overwriting __cxa_allocate_exception

So, in order to stop our code from calling malloc, we have to overwrite __cxa_allocate_exception (and the correspoding __cxa_free_exception) so that exceptions will always be allocated in some kind of static buffer.

Fortunately this work has already been done in ApexAI’s static_exception library. We can download it, link to it, and now throw exceptions without calling malloc.
To test this, we can re-define malloc as

#include <dlfcn.h>
#include <exception>
#include <cstdio>

static bool g_forbid_malloc;

void *malloc(size_t size) {
  static void *(*real_malloc)(size_t) = nullptr;
  if (!real_malloc) {
    real_malloc = (void *(*)(size_t))dlsym(RTLD_NEXT, "malloc");
  }
  void *p = real_malloc(size);
  if (g_forbid_malloc) {
    fprintf(stderr, "malloc(%d) = %p\n", static_cast<int>(size), p);
    std::terminate();
  }
  return p;
}

Then run (be sure to compile with -O0)

int main() {
  g_forbid_malloc = true;
  try {
    throw 3;
  } catch(int i) {
    printf("Success!");
  }
}

And indeed, it prints Success!. While this is the hardest part, it’s unfortunately not the end of it.

Getting rid of allocations for standard library exceptions

The goal is to be able to throw standard library exceptions without dynamically allocating memory.
The following code snippet will still fail, however:

int main() {
  std::vector<int> v;

  g_forbid_malloc = true;
  try {
    v.at(1);
  } catch(const std::out_of_range& e) {
    printf("Success!");
  }
}

As it turns out, many of the exceptions thrown by the standard library will construct a string for their error message, and this string will allocate heap memory.

We can see what exactly is going on when looking at the libstdc++ source code. The vector.at() function is defined as:

void
_M_range_check(size_type __n) const
{
	if (__n >= this->size())
	  __throw_out_of_range_fmt(__N("vector::_M_range_check: __n "
				       "(which is %zu) >= this->size() "
				       "(which is %zu)"),
				   __n, this->size());
}
// ....
reference
at(size_type __n)
{
	_M_range_check(__n);
	return (*this)[__n];
}

So it will first perform a range check, and if this range check fails, it will call ____throw_out_of_range_fmt. As might be suspected from the name, this function will throw an instance of std::out_of_range.
It turns out that similar functions exist for all exceptions defined in the <stdexcept> header, and that they can be overridden.

StackString

First we will need a way to store the exception message without allocating on the heap. One way to do this is to create a StackString class, which stores the string in a stack buffer. The problem with this approach is that the exception message will have to be truncated if it is too long to store in the buffer, but this can be mostly prevented by choosing a relatively large buffer:

constexpr std::size_t stackBufferSize = 128;
const char truncatedMessage[] = "...<truncated>";
constexpr std::size_t truncatedMessageSize = sizeof(truncatedMessage);

static_assert(truncatedMessageSize < stackBufferSize);

class StackString {
public:
  StackString(const char *str) {
    std::size_t len = std::strlen(str);
    if (len >= stackBufferSize) {
      std::strncpy(buffer, str, stackBufferSize - truncatedMessageSize);
      std::strncpy(buffer + stackBufferSize - truncatedMessageSize, truncatedMessage,
                  truncatedMessageSize);
      buffer[stackBufferSize-1] = '\0';
    } else {
      std::strncpy(buffer, str, len);
      buffer[len] = '\0';
    }
  }
  StackString(const StackString &other) {
    std::strncpy(buffer, other.buffer, stackBufferSize);
  }
  StackString(StackString &&other) {
    std::strncpy(buffer, other.buffer, stackBufferSize);
  }

  const char* c_str() const { return buffer; }

private:
  char buffer[stackBufferSize];
};

Custom exception types

Using this, we can define our own exception types. These will be child classes of the standard library exceptions, so we don’t have to change the catch clauses.
One example would be:

class OutOfRange : public std::out_of_range {
public:
  // Call ctor of parent class with empty string, since it does not have a default ctor
  // No heap allocation because of small string optimization
  OutOfRange(const char *msg) : std::out_of_range(""), message(msg) {}

  const char *what() const noexcept override { return message.c_str(); }

private:
  StackString message;
};

Repeat for each standard exception type. Or to avoid code duplication:

template <typename Base>
class Exception : public Base {
public:
  Exception(const char *msg) : Base(""), message(msg) {}

  const char *what() const noexcept override { return message.c_str(); }
private:
  StackString message;
};

// Repeat using statement for all types
using OutOfRange = Exception<std::out_of_range>;

Tying it all together

The only step left now is to overwrite the throwing functions we found earlier, like so: Actually, this exception type has two throwing functions, the other being __throw_out_of_range_fmt. This second function is used to format a string before passing it to the constructor of OutOfRange

namespace std {
void __throw_out_of_range(const char *__s) {
  throw OutOfRange(__s); 
}
}

A list of all standard exceptions that allocate memory for a string can be found here. For each of these, the throwing function can be overwritten. Wrap it all in a library, compile to a shared object, and link against it - standard exceptions are now allocation free!

Pitfalls

Actually, it’s not quite as easy, and you probably should not use this in production. Some reasons are:

• The throwing functions which we overwrite here are implementation details of libstdc++. Clang's libc++ actually uses functions with the same name to throw exceptions from stadard library containers. However, they define these functions in the <stdexcept> header, so it's not possible to replace them in this way. They are not advertised as being supposed to be overwritten, so this is technically undefined behavior.
• This will probably not work if you statically link either this library or libstdc++
• This will only work if this library comes before libstdc++ in the list of dynamically linked libraries

Links

• I’ve thrown together a small library as a proof of concept. It builds on ApexAI’s static_exception library and extends it with the functionality shown in this post
• For a deep dive into the viability of using exceptions in embedded programming, I recommed Khalil Estell’s talk at last year’s CppCon