IA2: Hardware-Enforced Compartments Inside One C Process

A C program is rarely only its own code. It links a cryptography library, an image decoder, a compression library, a parser for some file format: tens or hundreds of thousands of lines of C, written by other people. C is not memory safe. An index past the end of an array, a length computed wrong, a pointer used after its memory was freed: each reads or writes whatever is at the address it produced, and nothing in the language stops it.

Such a bug threatens the entire process, because a process is a single trust domain. It falls out of how memory works: one address space, in which every instruction can reach every byte. Every access control built above it (users, file permissions, containers, seccomp) stops at the process boundary. An out-of-range write in a JPEG decoder can land on your session keys, and nothing in the machine will object, because there is no “yours” and “its”. Linking a library extends that trust to everyone who has ever committed to it.

There are two well-known fixes, and both are expensive. One: rewrite the library in a memory-safe language: correct, but often out of reach. Two, split the program into processes and communicate between them: also correct, but every call across the split becomes a message, shared data structures become copies, and each crossing is a trip through the kernel.

A third approach accepts that the library has bugs and bounds what they can reach. That is compartmentalisation: divide one program into compartments, give each memory only it can touch, and make every transfer of control between them go through a checked entry point. The idea is old. What it needs is a boundary cheap enough to cross as often as a program’s parts call each other.

IA2 compartmentalises C and C++ programs inside one process. It draws the boundaries where the build already drew them, at shared libraries, and has the CPU enforce them with features that ship in current hardware: memory protection keys on x86-64, and the Memory Tagging Extension on 64-bit Arm. Each compartment’s stack, heap, globals and thread-local storage are private to it. A call into another compartment passes through a generated call gate. An access to another compartment’s memory faults at the instruction that made it. One process, one address space, no messages.

Here’s how it works.

How IA2 Works

A shared library is already a boundary: it is compiled on its own, it has its own symbol table, and the functions it exports are its interface. IA2 turns that boundary into one the CPU enforces. (IA2’s internally calls shared libraries and executables DSOs, dynamic shared objects.)

Compartments

A compartment is the main executable or a shared library, with any further libraries assigned to it. Compartments are numbered from 1 to 15, and the main executable is compartment 1. Number 0 is not a compartment: it marks memory that every compartment may use, and it means “no compartment” for code that runs between two.

Each compartment has its own share of every kind of writable memory a C program has:

• Globals: the writable .data and .bss segments of its libraries.
• Thread-local storage (TLS): __thread variables, one copy per thread, each copy protected.
• Stacks: a stack of its own on every thread (4 MiB, made when the thread starts), so that a function never runs on another compartment’s stack, next to its frames.
• Heap: malloc and its relatives go to PartitionAlloc, an allocator borrowed from Chromium, set up with one partition per compartment. An allocation comes from the partition of the compartment that is running.

Shared by all of them:

• Read-only data: string literals, const tables, the code. IA2 assumes an attacker can read the program’s files from disk.
• Data marked shared: globals annotated IA2_SHARED_DATA, and memory from shared_malloc.
• Libraries no compartment claims, which by default include the C standard library.

All of this is set up before main runs. IA2 wraps main at link time, and the wrapper calls ia2_main, a function in which the program names its compartments. For each, IA2 finds the library’s writable segments and TLS block from its program headers, marks them as the compartment’s, and allocates its stacks. Then it switches to compartment 1 and calls the real main.

From then on, a call from one compartment into another goes through a call gate, a short routine generated for the function being called. The gate switches stacks, clears the caller’s registers, tells the CPU which compartment is now running, and calls the function; on return it does the same backwards.

What the CPU Checks

IA2 needs two things from the hardware: a way to mark memory as one compartment’s, and a way to say which compartment is running that is cheap enough to change on every call.

x86-64: memory protection keys. Memory protection keys (also pkeys) are part of the x86-64 page tables. Each page table entry names one of 16 keys, and a per-thread register, PKRU, holds two bits for each key: one denies all access to that key’s pages, the other denies writes. The processor checks every data access against PKRU, and Linux delivers a denied access as SIGSEGV, with si_code set to SEGV_PKUERR. Changing PKRU takes one unprivileged instruction, WRPKRU, and no system call. IA2 gives compartment N key N, applied with pkey_mprotect; while compartment N runs, PKRU allows keys N and 0 and denies the rest.

64-bit Arm: memory tagging. Arm’s Memory Tagging Extension (MTE, added in Armv8.5-A) works at a finer grain. Every 16-byte granule of memory carries a 4-bit allocation tag, stored apart from the data. Every pointer carries a 4-bit logical tag in bits 56 to 59. Those bits are part of the top byte, which address translation ignores. In memory mapped with PROT_MTE, each load and store compares the two tags. In the synchronous mode IA2 uses, a mismatch faults at the instruction that made the access, and Linux delivers SIGSEGV, with si_code set to SEGV_MTESERR. Memory mapped without PROT_MTE is never checked.

MTE was built to find memory-safety bugs: an allocator gives each allocation a tag of its own, so a pointer that strays into a neighbour, or is used after its memory has been reused, usually meets a different tag. IA2 uses the same four bits to name compartments.

Each compartment’s writable memory is mapped PROT_MTE and tagged with the compartment’s number; shared memory is left untagged, and so unchecked. A modified Clang reserves register x18 for the running compartment’s tag, held in its top byte, and rewrites each load and store so that the pointer’s top byte is replaced by x18’s. Therefore, the loads and stores the compiler emits carry the running compartment’s tag (the few that do not are listed under What It Protects Against).

The last row is the difference that matters most. On x86-64 the permission is ambient: PKRU belongs to the thread, so even code IA2 never compiled is held to the running compartment’s key. On Arm the permission travels in the pointer, and code IA2 did not compile uses whatever tag it was handed. Tags in Pointers is about what follows from that.

A Call Across the Boundary

IA2’s test suite has a two-compartment program short enough to show. The executable is compartment 1 and its library compartment 2. Each calls a function in the other, and each tries to read the other’s secret, which must fault:

INIT_RUNTIME(2);
#define IA2_COMPARTMENT 1
#include <ia2_compartment_init.inc>

void ia2_main(void) {
  ia2_register_compartment("main", 1, NULL);
  ia2_register_compartment("libtwo_keys_minimal_lib.so", 2, NULL);
}

uint32_t secret = 0x09431233;
static bool steal_plugin_secret = false;
bool clean_exit IA2_SHARED_DATA = false;

void print_message(void) {
  cr_assert(secret == 0x09431233);
  if (steal_plugin_secret) {
    cr_assert(CHECK_VIOLATION(plugin_secret) == 0x78341244);
  }
}

#define IA2_COMPARTMENT 2
#include <ia2_compartment_init.inc>

uint32_t plugin_secret = 0x78341244;

void start_plugin(void) {
  cr_assert(plugin_secret == 0x78341244);
  print_message();
  if (!clean_exit) {
    cr_assert(CHECK_VIOLATION(secret) == 0x09431233);
  }
}

INIT_RUNTIME(2) declares two compartments. IA2_COMPARTMENT, with the ia2_compartment_init.inc it includes, says which compartment a library is, and ia2_main names the file for each. clean_exit is read by both sides, so it is shared; the secrets are not. CHECK_VIOLATION fails the test unless its argument faults, and the test cases set steal_plugin_secret and clean_exit to choose which violation a run makes. Compiled for Arm, compartment 2’s read of secret is:

ldr  x10, [x10, #0xf80]               // the address of secret, from the GOT
and  x8, x18, #0xff00000000000000     // the top byte of x18: tag 2
and  x9, x10, #0xfffffffffffffc       // the address, its top byte cleared
orr  x8, x8, x9                       // the address, carrying tag 2
ldr  w8, [x8]                         // secret's granules carry tag 1: SEGV_MTESERR

The address of secret comes from the global offset table, which the dynamic linker fills in, without a tag; the clamp gives it x18’s. Where it can, the compiler does the same in two instructions: mov x9, x18 copies the tag, and bfxil x9, x8, #0, #56 inserts the low 56 bits of the address beneath it.

The calls between the two files are plain C. IA2’s rewriter, a tool built on Clang, finds the ones that cross a boundary, and writes a gate for each and a file of linker arguments for each compartment:

--wrap=start_plugin
--wrap=ia2_compartment_destructor_1

--wrap=start_plugin makes the linker send compartment 1’s calls to start_plugin to __wrap_start_plugin, which is the gate. (The second line gates the compartment’s destructor, which runs at exit.) This is the gate generated for Arm, shortened where it repeats:

stp  x29, x30, [sp, #-16]!          // 1. save, on the caller's stack
mov  x29, sp
sub  sp, sp, #160
str  x19, [sp, #0]                  //    x19 to x28, then d8 to d15
...
str  x8, [sp, #144]                 //    and the struct-return pointer

mrs  x9, tpidr_el0                  // 2. switch stacks: find this thread's
adrp x10, :gottprel:ia2_stackptr_1  //    slot for compartment 1's stack,
ldr  x10, [x10, #:gottprel_lo12:ia2_stackptr_1]
add  x10, x10, x9
orr  x10, x10, #0x100000000000000   //    a pointer with tag 1,
mov  x12, sp
str  x12, [x10]                     //    and park the caller's sp there
adrp x10, :gottprel:ia2_stackptr_2  //    then compartment 2's slot,
ldr  x10, [x10, #:gottprel_lo12:ia2_stackptr_2]
add  x10, x10, x9
orr  x10, x10, #0x200000000000000   //    with tag 2,
ldr  x11, [x10]
mov  sp, x11                        //    and run on compartment 2's stack

bl   __libia2_scrub_registers       // 3. scrub
movz x18, #0x0200, LSL #48          // 4. from here, every access carries tag 2
bl   start_plugin                   // 5. call

...                                 // 6. return: the same steps backwards,
movz x18, #0x0100, LSL #48          //    ending with tag 1 back in x18
...
ret

1. Save the caller’s frame pointer, return address, callee-saved registers and struct-return pointer on the caller’s own stack. The callee is not trusted to preserve them.
2. Switch stacks. Each compartment’s stack pointer lives in a thread-local slot of its own, ia2_stackptr_N, which is that compartment’s memory and carries its tag. The gate builds a pointer with tag 1 to park the caller’s stack pointer, and one with tag 2 to fetch compartment 2’s.
3. Scrub all 32 SIMD registers, the condition flags, and every general-purpose register except x18, the frame pointer and the link register. Argument registers are saved around the scrub; start_plugin has none.
4. Switch compartments: tag 2 into the top byte of x18.
5. Call, then return the same way backwards, keeping the return value.

The x86-64 gate has the same structure, with WRPKRU in place of the movz. It executes four of them: to a value that allows both compartments’ keys while it switches stacks, then to the callee’s, and the same pair on the way back. Both copy any arguments passed on the stack. Pointers are passed as they are, so a pointer argument is useful to the callee only if it points at shared memory or the callee’s own; a buffer passed between compartments has to be allocated shared.

Indirect calls. A function pointer could lead into any compartment, so its gate is split in two. At the call site, a first half leaves the caller’s compartment for number 0, where x18 is 0 (on x86-64, PKRU allows key 0 only). The pointer itself points at a second half, generated for the target function, which enters the target’s compartment. Between the halves the running code has no compartment’s access, so neither side has to trust the other, and an overwritten pointer has no compartment’s access unless it points at a real entry. To keep every indirect call on this path, the rewriter turns each function pointer type into a pointer-sized struct (struct IA2_fnptr_...), called through IA2_CALL(...) and made with IA2_FN(...). A function pointer that passes through void * without an annotation no longer compiles, instead of silently losing its gate.

Using IA2

Using IA2 adds a step to a C build:

1. Mark the compartments in the source, as in the example.
2. Record the compile commands in compile_commands.json (CMake can write one; bear records one for other build systems), with each file’s compartment passed as -DPKEY=N.
3. Run the rewriter. It writes the rewritten sources, the gates and the linker arguments into a directory of their own, and leaves the originals alone.
4. Build the rewritten sources with the project’s own build system, a few extra flags, and IA2’s runtime and allocator. On x86-64 the compiler and linker are ordinary ones: GCC or Clang, and GNU ld or LLVM’s lld. On Arm the compiler is IA2’s modified Clang, and every library in the process, the C library included, must be built with it.

IA2’s CMake functions do all four. Its video player example is two calls to one of them:

add_ia2_compartment(video_decoder LIBRARY
  SOURCES video_decoder.c
  PKEY 2
  INCLUDE_DIRECTORIES include
)
add_ia2_compartment(video_player EXECUTABLE
  SOURCES video_player.c
  LIBRARIES video_decoder ${SDL2_LIBRARIES} ${FFMPEG_LIBRARIES}
  PKEY 1
)

The annotations that stay in the checked-in source are the ones no tool can infer:

• Data that is shared. nginx’s configuration parser, for example, runs in nginx’s compartment and reads tables that belong to whichever module defined a directive:

  nginx-rtmp-module/ngx_rtmp_codec_module.c

  static ngx_conf_enum_t ngx_rtmp_codec_meta_slots[] IA2_SHARED_DATA = {
      { ngx_string("off"),            NGX_RTMP_CODEC_META_OFF  },
      { ngx_string("on"),             NGX_RTMP_CODEC_META_ON   },
      { ngx_string("copy"),           NGX_RTMP_CODEC_META_COPY },
      { ngx_null_string,              0 }
  };

• Function pointers handed to code IA2 does not rewrite. A qsort comparator keeps its raw type (IA2_IGNORE), and a signal handler is registered through a wrapper that enters its compartment (IA2_SIGHANDLER).

• Functions that run in their caller’s compartment, between IA2_BEGIN_NO_WRAP and IA2_END_NO_WRAP. nginx does this for its string, hash and memory-pool utilities.

Each missing annotation is a fault, found one at a time. For porting, IA2 has a permissive mode: a fault handler that logs each violation and lets the access go ahead (on x86-64 by running the instruction with full access, on Arm by retagging the granule), so that one run lists them all.

Goals & Prior Art

Broadly speaking, IA2 has these goals:

1. Compartments at the granularity of shared libraries, so that a library becomes a compartment without being rewritten.
2. One address space. Data shared between compartments is shared through a pointer, not a copy.
3. Checks in hardware, so that isolation does not cost a software check on each access.
4. Annotations only where they cannot be inferred.
5. Inspectable. Everything the framework inserts, the gates included, can be read as source.
6. No changes to existing compilers and linkers.
7. No changes to the operating system or the dynamic linker.

Separate Processes

The strongest boundary available, and the one browsers are built on.

What it asks for is a design rather than a mechanism: someone has to decide which structures become messages and which pointers stop crossing, and a call that cost a few instructions becomes a round trip through the kernel. For an existing code base, that means rewriting every interface the split crosses. It fails goals 1 and 2.

Software Sandboxes

Compile the library to WebAssembly, translate it back to native code, and run it against a region of memory it cannot address outside of. RLBox, with the wasm2c translator, is the mature example: Firefox ships six libraries this way (Graphite, Hunspell, Ogg, Expat, WOFF2 and SoundTouch).

The isolation is strong (a pointer inside the sandbox is an offset into its memory, with nothing else to name) and needs no special hardware. But the library must compile to WebAssembly, data crossing the boundary is copied in and out of the sandbox’s memory, and every value that comes out must be checked before use, which RLBox enforces by wrapping it in a type of its own. It fails goals 2 and 4.

Capability Hardware

CHERI makes each pointer a capability that carries its own bounds and permissions, checked by the hardware on every use. That gives spatial memory safety object by object, and compartments far finer than a library. It needs a CHERI processor. SCI Semiconductor announced the first commercial CHERI silicon, a microcontroller, in March 2026; for an application processor that runs Linux, there is Arm’s Morello, a research prototype board. IA2 works with hardware that is already deployed.

Tags in Pointers

On Arm the compartment boundary is in the tag each pointer carries, so the port has to get the tag right on every pointer, in code it compiles and in code it does not.

Every Library Is Built by IA2’s Compiler

Code that IA2’s compiler did not build uses whatever tag a pointer carries, and it may use x18 as an ordinary register: the Arm calling convention leaves x18 to the platform, and Linux does not reserve it. Therefore, the whole application, from the standard library upwards, must be built with an IA2-aware compiler.

Pointers That Leave Compiled Code

The clamp covers the loads and stores the compiler emits. A pointer passed to code that does not clamp, such as the C library’s hand-written assembly (sigsetjmp, for one) or the kernel, is checked against the tag it carries. Pointers to stack and heap memory already carry the right tag, because the stack pointer and the allocator’s results are tagged. The address of a global, computed with adrp and add, is not. So the compiler stamps x18’s tag onto every address of a global or thread-local variable that it computes, and at startup the runtime patches the pointers held in static initializers, by walking each library’s relocations.

The kernel is one of these consumers. In synchronous mode it makes what its documentation calls a “best effort” to check its own accesses to user memory against the pointer’s tag, and a mismatch makes the system call fail rather than raise a signal. A buffer with the wrong tag produces an error return, not a fault at the line that passed it.

What a Pointer Is, Now

Stamping tags changes what a pointer is. The hardware does not mind; the program does. == compares all 64 bits, - subtracts all 64, and so does any code that turns a pointer into an integer. If some computations of &x carry the tag and others do not, they stop being equal while naming the same bytes. Porting nginx met this three times.

Equality. nginx’s red-black tree tests for emptiness by comparing a stored &sentinel, which carried the tag, with a freshly computed one, which did not. They never matched, and the server walked a tree that was not there. The fix was to stamp every computation of a global’s address, not only the ones that escape, so that each address has one form.

The constant folder. The HLS and DASH playlist writers end with write(fd, buffer, p - buffer). Clang compiles that subtraction to sub i64 %p.int, ptrtoint (ptr @buffer to i64), and the second operand is a constant expression of integer type, which the stamping did not yet look inside. One side carried the tag and the other did not, so the difference kept it:

write() was asked for 2⁵⁷ + 73 bytes and returned EFAULT, so the playlists were never written. The 1,085 other writes from the same compartment, the media data, went through, because their lengths came from struct fields. The stream produced good video segments that no player could find. A tag that leaks into an integer need not fault where it leaks: it can become a length or an index, and surface far from where it came from.

Code that computes on raw addresses. IA2’s per-compartment initialisation code is compiled as compartment code. Once every global address was stamped, its pointer to its own table of finalisers carried the tag, while the addresses it derived from the ELF header (__ehdr_start) stayed raw. Comparisons between the two failed, and it wrote a tagged address into the library’s DT_FINI_ARRAY entry where an offset belonged. Tagging memory breaks whatever was reading it raw.

Tags Belong to the Memory

Linux allows PROT_MTE only on anonymous memory and on files held in RAM (tmpfs, memfd); on any other file mapping, it fails with EINVAL. A library’s writable data is mapped from its file, so IA2 copies the data out, maps anonymous PROT_MTE memory in its place, tags that, and copies the data back.

The kernel also documents that memory released with madvise(MADV_DONTNEED) may have its tags cleared to 0. PartitionAlloc releases free heap spans exactly that way, and later reuses them without a system call. A reused span came back with tag 0, and the compartment faulted on its own memory. What triggered it in nginx was not how many requests arrived but how many were in flight at once: bursts of more than a hundred simultaneous connections, alternating with idle periods. Releasing a span now also removes all access to it (PROT_NONE), and taking it back re-tags it.

What It Costs

At each crossing. The Arm gate above is 74 instructions in full, and it calls the 62-instruction __libia2_scrub_registers twice: about 200 instructions for a round trip, against two for a plain call and return. The x86-64 gate has the same shape, with four WRPKRU. We’re working on measuring the practical performance cost of this in real applications.

Inside a compartment, on x86-64, ordinary code runs unchanged: the rewriter changes only calls and function pointers, and the key check is the hardware’s.

Inside a compartment, on Arm, each load or store carries two or three extra instructions to clamp its pointer (fewer where the compiler can reuse a clamped address), and each computed address of a global carries more. We are working on measuring the performance impact on this instrumentation as well.

Memory. A 4 MiB stack for each compartment on each thread, and a page of padding on each library’s thread-local block, so that no two compartments' thread-local variables share a page.

The build. The rewriter runs before every build. On Arm, the toolchain, the C library and every other library in the process are rebuilt.

What It Protects Against, and What It Does Not

What IA2 provides coarse spatial memory safety. A bug in one compartment that reads or writes where it should not (an out-of-bounds index, a wild pointer) cannot reach another compartment’s private stack, heap, globals or thread-local storage: the access faults, at the instruction that made it. It holds in both directions, so the program is protected from the library and the library from the program. That is narrower than memory safety, and the gaps are worth stating plainly.

Bugs inside a compartment. IA2 bounds what a bug reaches; it does not prevent the bug.

Read-only data is shared by design, so a secret in const storage is readable from every compartment.

Shared data is shared. In the nginx demo, roughly two thousand function pointers that nginx core calls sit in shared memory: the event-loop table, the connection and event structures, all 70 module descriptors. A compromised module that overwrites one gains none of core’s private data, since the indirect gate drops to tag 0 first, but it chooses what runs, and it can call back into core through legitimate gates with arguments of its choosing.

Control flow. IA2 confines what a compartment’s code does as compiled. An attacker who takes over control flow can jump straight to a gate’s movz x18 or WRPKRU.

IA2’s design relies on control-flow integrity to close that gap. Branch targets must be used to ensure that an attacker can’t pivot the instruction pointer directly into the juicy part of a call gate to skip the boundary checks.

Variadic functions are not gated. A gate is generated once per function, and copies stack arguments by the size the signature gives, but a variadic function’s arguments differ at every call site. So a variadic call across a boundary runs in the caller’s compartment: on its stack, with its tag, with no scrub. In the nginx demo the RTMP module makes about 160 such calls to nginx’s two logging functions, and about 90 to three formatting functions that the demo runs in the caller’s compartment by design anyway. The rewriter reports each of the five, and walks the call graph to report any compartment-private data they could reach. Closing the gap would mean a gate per call site rather than per function, which turns the rewriter from a tool that rewrites declarations into one that rewrites call expressions. This work has not yet been done.

System calls that change memory. On x86-64, an optional tracer (a separate process using ptrace, a seccomp filter and Landlock) checks every call that maps, unmaps or changes the protection of memory against which compartment owns it. Arm does not currently have this protection.

Pointers with a forged tag, on Arm. The compiler does not re-stamp a pointer passed as an ordinary argument, so code that does not clamp (the kernel, hand-written assembly such as strcpy, inline assembly, a few explicit SVE vector intrinsics) uses whatever tag the pointer carries. A pointer whose top byte has been overwritten with another compartment’s tag, and then passed to write(), would have the kernel read that compartment’s memory, with no fault.

Fifteen compartments, on both architectures. The C runtime and the dynamic loader are outside IA2’s scope.

Where It Stands

The nginx demo runs with enforcement on: HTTP served from compartment 1, a stream published over RTMP, HLS and DASH written from compartment 2, and the playlist served back and decoded.

Given a C program, and libraries you did not write and cannot afford to rewrite, IA2 gives each of them a stack, a heap and globals that an out-of-bounds read or write in another cannot reach, checked by the processor on every access, without splitting the program up. What it costs is a gate at every crossing, annotations for what is shared, and on Arm, a toolchain of its own.

There is much more to be done here to close gaps and measure performance. We continue to research ways to close gaps in the C memory safety model, and improve the boundary between compartments without unduly sacrificing performance. We believe that IA2 has the potential to serve as a lightweight solution to compartmentalising legacy code, where a reimplmentation in a safer language is not available.