Gramine Library OS

The main project page is here.

Gramine was previously known as Graphene, but recently renamed to avoid conflicts with other project names.

What is the Gramine library OS?

A Library OS is a light-weight guest OS, designed to run one application in a VM-like environment. Existing library OSes lack full support of Multi-process abstractions, such as fork, execve, signaling, etc, which are required by many legacy Unix or Linux applications. such as shell scripts, Makefiles, or server applications.

Gramine extends the library OS model to support multi-process applications, and aims to ultimately provide Linux compatibility.

Gramine library OS slightly modifies the GNU C library to redirect system calls to Gramine as library calls, or uses seccomp to redirect system calls to Gramine. Gramine can support most of the native executables and libraries from Linux platform without modifying the binaries or recompiling the source code. The only prerequisite to run a binary on Gramine is to provide a manifest file which specifies the static environment where the binary can start. A reasonable manifest file usually includes URIs of the binaries, parameters of the file systems, and other important parameters that the system requires.

Gramine library OS is designed to run on top of Linux as its host, although we are working on ports to other host kernels. Gramine is programmed against a narrowed, but generic host ABI---a variant of the Drawbridge library OS, with minimal extensions to support Linux personality and improve performance and security isolation for multi-process applications.

How to obtain Gramine library OS?

Gramine source code and documentation are available at the Gramine source code repository.

How does Gramine support multi-process applications?

Similar to other library OSes, Gramine implements the operating system APIs, such as system calls of Linux platform, inside the library OS. For a multi-process application, each guest, or picoprocess, has its own library OS instance. Multiple library OSes in an application cooperate to support shared multi-process abstractions such as fork, execve, signaling or even System V message queues and semaphores. Gramine includes a framework for coordinating all of these guest-specific multi-process abstractions using RPC streams across picoprocesses---providing a narrow, easy-to-secure security isolation boundary.

figure 1

Figure 1 shows how signaling and exit notification are serviced in Gramine as an example of multi-process coordination. To exchange the state of signal or exit status, one library OS will send a RPC message over a RPC stream to other library OS. In each library OS, an internal thread called IPC helper thread will listen on all the connection RPC streams, triggering asynchronous events or responding to remote requests. For example, when the IPC helper thread receives a signal or an exit notification, it updates the bookkeeping inside the library OSes, and wakes up appropriate application threads to handle the signal or exit status.

We observe that the primary challenge of coordinating multi-process abstractions across library OSes is to maintain a globally consistent namespace state. Maintaining namespace state requires keeping an consistent mapping from the name of the shared resources to the picoprocesses that manage the resource state. We avoid broadcasting or completely replicating the states to reduce costs. Instead, we choose one of the library OSes as a centralized server of namespace management (these library OSes are called leaders, but still distribute and coordinate the state across multiple library OSes. Although the multi-process abstractions Gramine supports are quite diverse, we are able to identify namespace management as a common theme, and develop a set of tools to simplify implementation of each additional abstraction.

The following is a list of multi-process abstractions that are currently supported in Gramine:

  • fork (with PID allocation) / execve
  • Exit notification / waitpid
  • Signaling
  • /proc/[pid] to get process metadata
  • System V message queues and semaphores

What is the benefit of running applications in Gramine library OS?

Compared to running an application in a VM with a complete legacy guest OS, which requires hundreds of MB to GB of memory, Gramine's memory overheads are closer to a Linux process (tens of MB). Gramine still has higher performance overheads than Linux (which we are working to improve), but maintains the qualitative benefits of virtualization, such as security isolation. Gramine also introduces some useful mechanisms for application sandboxing.