2026-05-11

Really Fast Microcontroller Black Box Sensor Logging

Or, DMA is all you need

When reading sensor values on a microcontroller (e.g. for a flight computer), it is important to record these readings for later analysis as a black box, or for mocking sensor data during testing.

It is possible to do this with almost no overhead by doing less work.

Background

This article assumes you are logging to flash storage (either an IC or an SD card).

It also assumes that you are writing your own drivers to talk to sensors. You should be! They aren’t that scary, and you’ll be able to get a lot higher performance out of your system.

Logging With a Buffer

You can’t write individual bytes to flash storage. Flash memory erases whole pages (typically 512 bytes) at a time to write new information. As a result, we must store data in an intermediate buffer that is written to storage in one go.

As we read sensor data, it is written into this buffer. Once the buffer is full, we write the whole thing to storage. After the data is written, we can reset the buffer and start putting more information in the start (in practice, you wouldn’t clear out the existing data, you would just write over it).

There are additional overheads in the protocols used when writing to flash storage, so it is more efficient to use a larger buffer size. This way, the costs are amortized across many sensor readings, lowering the average cost of logging a single data point.

If you’re using an SD card with a filesystem, you should match the file system sector size and your buffer size.

Double Buffering

While having a large buffer is great, when you reach the end of the buffer you still have to pause logging until it is done writing. With worst case SD card latencies in the 100s of milliseconds, this is unacceptable.

To solve this, we need two things. A second buffer, and the ability to write to storage in the background.

The hard part is writing to storage in the background. You (probably) only have one core, so how can you write bytes onto the hardware interface without occupying the CPU’s time? Fortunately, this is a common enough problem that there is already a solution in the hardware.

Most microcontrollers (hopefully yours!) have something called a Direct Memory Access (DMA) controller. At the most basic level, a DMA controller is instructed by the CPU to copy a sequence of bytes from one region in memory to another. It does this in the background, freeing up the CPU to do other work.

All peripherals on a microcontroller are memory mapped. That means that sending bytes over the hardware interface to storage (e.g. over SPI) is done by writing the data to a specific address in memory.

Instead of configuring our DMA controller to copy the buffer to another region of memory, we can configure it to copy each byte in the buffer to the specific transmit address used by our peripheral. By doing so, the microcontroller can transmit the log data without having to wait for the transfer to

The easier part is adding a second buffer. When one buffer fills up, you start writing it to storage and switch to the other one. That way, you can immediately begin logging more data, without having to wait for the write to finish. However, this introduces a potential race condition. The write must finish before the other buffer fills up. If it did not, you would try to start a second write to storage while the first one is still ongoing, which would break things. Make sure to measure the latency of your system to ensure that this will not happen.

Arena Allocator

Memory allocators are objects that provide programs with a specific amount of memory upon request. The program asks for a certain number of bytes, and the memory allocator returns a pointer to a region of that size. The program can then use that memory for any purpose, and then return the memory to the allocator when it is finished.

An arena allocator is a specific type of memory allocator where all allocations are freed at once. Internally, the arena has two things. A continuous span of bytes that it uses to serve memory requests, and a counter indicating where the end of used memory is. When it receives a memory allocation request (a malloc), it returns a pointer to the start of unused memory. It then bumps the counter to the new edge of used memory. When the arena is freed, the counter is reset to the start of the span of bytes.

This sounds a lot like our buffer! In fact, we can treat our log buffer as a memory allocator. Instead of storing our data on the stack, we can store it directly in the log buffer. You need some place in memory to store the sensor data while you operate on it, so it might as well be in the

This brings us to the final piece of the puzzle:

Log Raw Sensor Readings

In order to really maximize performance, you should log the raw bytes received from your sensor.

Many sensors support performing a burst read, where you receive all the data the sensor has at once. Using a burst read looks something like this:

char[BURST_READ_SIZE] packet;
sensor_spi_read_data(address, packet);
ParsedPacket parsed_packet = parse_packet(packet);

Before you can turn these bytes into values, you need to store them somewhere. Instead of storing them on the stack, you can store them directly in the log buffer. Like so:

char* packet = logger_malloc(BURST_READ_SIZE);
sensor_spi_burst_read_data(address, packet);
ParsedPacket parsed_packet = parse_packet(packet);

This way, you avoid having to copy any information into the log buffer, since it is there from the start.

Other Notes

This is a general overview of this technique, so there are a few things missing for an actual implementation. Firstly, you’ll need a byte before the sensor data that identifies the type of data it contains. You’ll probably also want a timestamp before it as well. You can save some more space by ignoring the higher bits of your clock, as long as there will be a packet recorded before the clock overflows.

You should also store metadata at the start of the file, before any data occurs. The most important thing to store here is a format version, so you can make breaking changes later. You may need to make sure that this data stays word aligned for your raw sensor packets to be usable.

For some further reading: