Using Peios

A process is a program that is running. When a program starts, the system gives it a block of private memory that only it can see, a table of the files and other resources it holds open, and a name the system tracks it by. That running program, plus everything the system keeps for it, is a process. When the program finishes, its process goes away.

A process can also do several things at once by running more than one thread. A thread is a single line of execution — one sequence of steps the system is working through. A process always has at least one thread (its initial execution is its first thread), and it can start more. Every thread in a process shares the same private memory and the same open resources. What each thread keeps to itself is its own place in its sequence: each runs its own steps, at its own pace, possibly in parallel with the others.

The actors of the system #

Processes and threads are what carry out work on the system. Every action — opening a file, sending data over a network, starting another program — is performed by some thread. When the system decides whether an action is allowed, the question it answers is "is this thread allowed?" When it records that something happened, it records which thread did it.

A thread always acts as someone — a person, a service, or the system itself. Peios carries that identity along with the thread on an object called a token (see Tokens). Two facts matter here:

• Every thread is always acting as someone. There is no "nobody" state.
• When one process starts another, the new process begins acting as the same identity as the process that started it.

What a process has #

The things the system keeps for every process:

This is not the full list — a process also carries other per-process state, such as its Process Security Block (PSB), which holds its security-related settings.

Where to start #

Continue with The process and thread model for how a thread and a process relate, and why the thread — not the process — is the more basic unit, with a "process" being one particular way of using them.

This topic also covers:

• Creating processes — how a process starts another, and what the new one begins with.
• Process lifecycle — how a process ends, and how the system cleans up after it.
• Process relationships and job control — the process tree, process groups, and sessions (distinct from the logon sessions in Logon sessions).

A running program has two separable parts: the threads, which run, and the process, which contains them.

The thread is the part that runs — one line of execution, a single sequence of steps the system works through. A machine with several cores runs that many threads genuinely in parallel, one per core; a single core runs many threads by switching between them. Which thread runs, on which core, and for how long is decided by the system's scheduler — a subject of its own. Some tools and logs call a thread a task; it is the same thing.

The process is the boundary around a group of threads. It does not run — it contains. A process holds the things its threads share: their private memory, their open files and connections, and their identity. Every thread inside one process sees the same memory and the same open resources; threads in two different processes do not. That separation is what stops one program from reaching into another's memory or files.

The thread does the work; the process is the shared space the work happens in. A program that does one thing at a time is a process with a single thread. A program that does several things at once is a process with several threads, all sharing one space.

Why the thread is the more basic idea #

Threads, not processes, are the fundamental unit: what the system runs is threads, and a "process" is the answer to the question "what does this thread share, and with whom?" Bundle some threads together so they share one memory, one set of open resources, and one identity, and that bundle is a process. Give a thread its own fresh, separate memory instead of sharing an existing one, and you have made a new process.

This matters because it explains creation. Making a new thread and making a new process are the same mechanism with a different answer to "how much does the new line of execution share with the one that made it?" Share everything, and the result is another thread in the same process. Share nothing, and the result is a new process. Everything in between is possible too. How the operation is actually performed is covered in Creating processes.

What threads share, and what they don't #

Within one process, every thread shares:

Each thread keeps its own place in its sequence — which step it is on right now. That is the point of having more than one: each thread makes progress through its own work independently of the others.

There is one exception to the shared identity. A single thread can temporarily act as a different principal — for example, a service handling a request can act as the requester for that one piece of work, then revert. Only that one thread is affected, and only until it reverts. This is covered in Impersonation; here it is enough to know that "every thread in a process shares the process's identity" is the default, not an unbreakable rule.

Naming a process #

The system gives each process a number called its process ID, or PID, so that people and other programs can refer to it — to inspect it, signal it, or wait for it to finish. Threads are individually identifiable too, by a thread ID — see thread operations.

A PID is short and convenient, but it only names a process while that process exists. Once a process ends, its PID can later be reused for a completely different process. A PID tells you which process is which right now, but it is useless as a lasting record — two unrelated processes can hold the same PID at different times.

For that, every process also has a Process GUID (globally unique identifier): a name that is never reused, assigned to each process when it starts and fixed for the rest of its life. Where the PID is the convenient short handle for the moment, the Process GUID is the permanent one — it identifies that one specific process and no other, even long after it has ended. This is what lets a record of something that happened on the system name exactly the right process, with no chance of confusing two that happened to share a PID.

Both the PID and the GUID refer to a process. Neither says who the process is acting as — that is a separate question, answered by its identity rather than by any number or name.

Where to go next #

To see how a process or a thread is created — and what a new one starts out with — read Creating processes.

Every process on the system was started by another process — a process that is already running has to create it. Creation is built from two operations, which combine to cover every case.

Splitting: one process becomes two #

A running process can split into a copy of itself. The original keeps going, and a second process appears beside it that begins as a near-identical copy. The original is called the parent and the copy the child.

The child starts with:

• Its own private memory, copied from the parent. From the moment of the split the two memories are separate — a change one process makes is invisible to the other.
• Copies of the parent's open resources — the files and connections the parent had open, the child has open too.
• The same identity — the child begins acting as the same principal as its parent. (It inherits the parent's token; how the token carries across a split is in Token lifecycle.)

It also gets things of its own that are not copied: a new PID, and a new Process GUID — the child is its own distinct process, not a continuation of the parent. A few pieces of per-process state start fresh rather than being inherited, too.

This operation is called fork.

Replacing: same process, different program #

A process can also replace the program it is running with a different one. It remains the same process — same PID, same Process GUID, same identity — but from that point on it runs entirely different code. Whatever the previous program was doing is discarded, replaced wholesale by the new one.

This operation is called exec.

Putting them together #

The two operations combine to launch a different program as a new process: a process splits, and the child immediately replaces itself with the program to run. This split-then-replace pair is how most programs are started — when you run a command, something forks and the child execs your command. There is also a single combined step, spawn, that does both at once for the common case.

Underneath, splitting is one setting of a more general operation, clone, that lets the creator choose how much the new line of execution shares with it. Share everything, and the result is another thread in the same process; share almost nothing, and the result is a new process — the two ends of the range from the process and thread model. Everyday code rarely uses the general form directly; it makes threads or spawns programs, which are particular settings of it.

The exact contract — every clone sharing flag, precisely what fork passes to a child, and the variants of exec — is collected in the process creation reference.

Getting hold of the new process #

When a process creates a child, it usually needs to keep track of it — to wait for it to finish, or to send it a signal later. For this it can hold a handle to the child: a reference to that one specific process, called a pidfd. It is the same kind of handle the system hands out for an open file.

A process can be given a pidfd for its child at the moment it creates it, or ask for one for a process that already exists. Either way the pidfd is tied to that one specific process and can never be mistaken for another, so waiting on it or signalling through it always reaches the intended target.

This is what makes a pidfd safer than a bare PID. A PID can be reused once the process it named has ended; code that holds a bare PID risks later acting on a completely different process that inherited the number. A pidfd never has that problem — it refers to the one process it was created for and nothing else, for as long as the program holds it.

Where to go next #

A process that has been created eventually ends. How it finishes, and how the system cleans up after it, is covered in Process lifecycle.

Every process eventually ends. A process's life has three parts — it is created, it runs, and it finishes — and the system ensures that when it finishes, its result is collected and its resources are reclaimed.

The states it passes through #

While it exists, a process is in one of a handful of states, and the system moves it between them. Most of the time it is either running — executing, or ready to run the moment a core is free — or sleeping, paused while it waits for something to happen, such as data to arrive or a timer to fire. A sleeping process uses no processor time until what it is waiting for is ready; most processes spend most of their lives asleep.

A few states are worth knowing by name:

• Stopped — suspended (for example by job control when a job is paused at the terminal); it does nothing until told to resume.
• Traced — stopped under the control of a debugger that has attached to it, so the debugger can inspect and step it.
• Uninterruptible sleep — waiting on something the system will not interrupt, almost always a brief piece of disk or device I/O. A process here cannot be woken or even killed until the operation finishes; it normally lasts an instant, and a process stuck in it is a sign that hardware or a filesystem is stuck too.
• Zombie — finished, but still held as a result waiting to be collected (see below).

Once its result has been collected the process is gone, removed from the system entirely.

Two ways a process ends #

A process ends in one of two ways.

Most often it ends on its own: it finishes its work and stops. A process that ends this way leaves behind an exit status — a small record of how it went, usually just "succeeded" or "failed" (and, on failure, a number giving a rough reason). The act of a process ending itself is called exit.

The other way is that a process is stopped from outside — something tells it to stop, and it ends whether or not its work was done. (The messages that do this are signals, a subject of their own.) A process ended this way also leaves an exit status, marked to show it was stopped rather than finishing on its own.

Either way, a record of the process remains until its result is collected.

Collecting the result #

Whenever a child changes state — ends, is stopped, or resumes — the system sends its parent a signal, SIGCHLD, prompting it to check. The parent then reads the child's exit status. This is called waiting for the process.

Between the moment a process ends and the moment its parent collects the result, the finished process is a zombie: no longer running, but with the system still holding its exit status so the parent can read it. Despite the name, a zombie is harmless — it does no work and uses almost nothing; it is a result waiting to be collected. The instant the parent collects it, the zombie is gone for good.

If a parent never collects, zombies accumulate — finished processes whose results no one read. That is a fault in the parent, not normal behaviour.

A parent holding a pidfd for its child can wait on it directly — the dependable way to be told the exact moment the child ends.

The exact calls — every wait variant, and how an exit status is read — are in the process lifecycle reference.

When the parent ends first #

A parent does not always outlive its children. If a parent ends while one of its children is still running, that child becomes an orphan.

Orphans are not lost. The system's first process — PID 1, which on Peios is peinit — adopts every orphan and stands in as its parent from then on: when the orphan eventually ends, peinit collects its result, so no finished process is ever left uncollected. Adoption is automatic; the orphan keeps running unaffected, only with a new parent.

PID 1 is the catch-all, but a process can arrange to catch orphans within its own subtree. By marking itself a subreaper (with prctl(PR_SET_CHILD_SUBREAPER)), a process becomes the one that adopts any orphan descended from it — its grandchildren and below — instead of letting them reparent all the way up to PID 1. Service managers and container managers use this to keep responsibility for everything they launched: when something deep in their subtree loses its immediate parent, it reparents to the manager, which still learns when it finally ends.

What the system cleans up #

When a process ends, the system reclaims what it was using. Its private memory is freed and its open files and connections are closed.

It also releases its identity. A process acts as a principal, carried on its token; when the process ends, its hold on that token is released. If other processes were sharing the same identity — for example, siblings from the same login — the identity lives on for them, and the system clears it away only once nothing is using it any more. The details are in Token lifecycle.

The process's PID becomes available for the system to assign to a future process. Its Process GUID is not reused — it remains a permanent marker of that one process, so records of what it did still point unambiguously back to it long after it is gone.

Where to go next #

Processes are arranged into a family tree and grouped together in ways that matter for running them. That is Process relationships and job control.

Every process is related to others in two different ways: by descent — who started whom — and by grouping — which processes are handled together when they are controlled at a terminal.

The family tree #

Every process was started by another, so every process has a parent: the process that created it. A process can refer to its parent by the parent's PID, sometimes called the PPID (parent process ID).

Follow the parents upward and they all lead back to the same place — PID 1, the first process, which the system starts at boot. Every other process descends from it, directly or through a chain of parents, so the processes on a running system form a single family tree rooted at PID 1. This is the same tree that makes orphan adoption work: when a process loses its parent, PID 1 takes over the role, as covered in the lifecycle.

A few processes at the very root of the tree are special. PID 1 is the first process started at boot — on Peios, peinit — and the ancestor of all the rest; the system protects it, refusing to kill it with any signal it has not chosen to handle, because losing it would bring down the whole tree. Beneath the userspace processes the kernel runs threads of its own: PID 0 is the idle process (what a core runs when it has nothing else to do), and PID 2, kthreadd, is the parent of every kernel thread. The kernel's threads — among them the worker threads (kworker) that run deferred work and the threads that service hardware interrupts — appear in a process listing as the system's own work rather than as programs loaded from disk.

Process groups #

Several processes often do one job together. In a pipeline — the output of one command feeding into the next — each command is its own process, but they form one job: you stop, pause, or resume the whole job at once, not a piece at a time.

To make that possible, related processes are collected into a process group. A process group is a set of processes treated as a unit, so that an action — most often a signal, like the one that stops a job — can be delivered to all of them together instead of one by one.

Sessions and the terminal #

Process groups are themselves collected into a larger unit: a session, tied to the terminal the work is running at.

A terminal can only take input for one job at a time, so within the session one process group is the foreground group: the one connected to the keyboard, the one typing reaches, the one a stop-or-interrupt keystroke acts on. Any other groups are in the background, running without holding the keyboard. Moving a job between foreground and background — and starting, stopping, and resuming jobs — is job control.

A different kind of "session" #

The word "session" has two meanings here, and they are unrelated:

• A job-control session, described above, organises processes at a terminal — which groups share a terminal, which one is in the foreground. It says nothing about who anyone is.
• A logon session records one sign-in — a single authentication event — and ties together every identity token that came from it. It is part of the identity model, covered in Logon sessions.

They often line up in practice — signing in at a terminal creates a logon session, and the processes run there share a job-control session — but they are separate things answering separate questions: "which processes share this terminal?" versus "where did this identity come from?"

Where to go next #

Beyond its place in the tree and its groups, every process carries a bundle of its security-related settings — its Process Security Block. That is The Process Security Block.

A process's identity — who it is acting as — is carried on its token, and can change moment to moment, since a thread can impersonate another principal. A process has a second aspect that is independent of identity: what it is. What program is it running? How trusted is that program? How hardened is it against attack? Who is allowed to operate on it?

Those facts are gathered in one place — the Process Security Block, or PSB. Every process has one. Where the token answers who this process is acting as, the PSB answers what this process is. Unlike the token, the PSB never changes when a thread impersonates: impersonation changes who, never what.

What the PSB holds #

• Its permanent name. The process's Process GUID — the never-reused identifier — lives on the PSB.
• How trusted its program is. When a process starts running its program, the system checks the program's cryptographic signature and records from it how trusted the program is. This is the process's PIP (Process Integrity Protection) label, and it decides which other processes are allowed to inspect, signal, or interfere with this one. The barrier is based on what program is running, not on who is running it: even a fully privileged process cannot disturb a more-trusted one. The full treatment is Process integrity protection.
• How it is hardened. A set of mitigations — restrictions the process carries on what it may do with its own memory and code, so that a bug or injected code has far less room to do harm. They can only ever be tightened, never loosened. The catalog and rules are Process mitigations.
• Who may operate on it. Every process has its own security descriptor — the rules for who is allowed to act on the process itself: inspect it, signal it, and so on. It lives on the PSB alongside the rest.

A few more specialised settings live here too — a process can be marked so that it may no longer create children, for instance — but those four are the core.

Inspecting and managing the PSB #

Because the PSB is where a process's trust level and hardening live, there is a dedicated tool for working with it: the psb command, which inspects a process's PSB and manages its mitigations and PIP mode.

Where to go next #

The two largest parts of the PSB each have a topic of their own: Process integrity protection, for the trust label that governs which processes may interfere with which, and Process mitigations, for the self-hardening flags and how they are applied.

This is the detailed counterpart to Creating processes. That page explains the ideas; this one gives the exact contract — what each call does, what a new process inherits, and what each clone sharing flag controls.

fork #

fork() creates a new process by duplicating the caller. The child is a near-identical copy that runs independently from the point of the call.

The child inherits:

• a private copy of the parent's memory, made lazily (copy-on-write), so the duplication is cheap and changes on either side stay invisible to the other
• copies of the parent's open file descriptors — each refers to the same open file description, so file offset, status flags, and signal-driven-I/O settings are shared between parent and child
• open directory streams and open message-queue descriptors
• open-file-description locks (flock, OFD locks) through those shared descriptors
• the parent's signal dispositions (the handler set for each signal)
• the file-creation mask (umask), the current directory, and the root directory
• resource limits and the timer-slack value
• the parent's identity — it begins acting on the same token (see Token lifecycle)
• the parent's mitigations and protection, carried on its PSB

The child does not inherit:

• the parent's PID — it receives a new PID and a new Process GUID
• memory locks (mlock, mlockall)
• memory regions marked MADV_DONTFORK (absent in the child) or MADV_WIPEONFORK (present but zeroed)
• process resource usage and CPU-time counters — reset to zero
• pending signals — the child's set starts empty
• semaphore adjustments (semadj)
• process-associated record locks (POSIX fcntl locks — distinct from the OFD and flock locks above, which are shared through the inherited descriptors)
• timers (setitimer, alarm, timer_create)
• outstanding asynchronous I/O
• directory-change notifications (dnotify)
• the PR_SET_PDEATHSIG parent-death-signal setting (reset)
• I/O-port access permissions (ioperm)

The child's termination signal — what its parent is notified with when it ends — is SIGCHLD.

The child is created with a single thread — the one that called fork(). If the parent had other threads, they do not exist in the child.

vfork #

vfork() is a lightweight variant of fork() for the one case where the child will immediately replace itself with another program. It differs in two ways:

• the caller is suspended from the call until the child either execs or exits;
• the child runs in the parent's memory — no copy is made — until that point.

Because the two share memory and the parent is frozen, the child must do almost nothing first. The contract: the child must not return from the function that called vfork(), must not modify any variable other than the one holding the return value, and must not call any other function before a successful exec or _exit. Doing otherwise is undefined behaviour.

vfork is an optimisation for the create-then-replace pattern. Ordinary code should use posix_spawn (below) rather than reach for it directly.

clone and clone3 #

clone() is the general creation call. fork and thread creation are both clone with particular flag sets; clone exposes the choice directly. The caller gives a set of flags controlling what the new task shares with the creator, a stack for the new task, and locations for thread-ID and thread-local-storage bookkeeping.

clone3() is the modern form, taking a structure instead of positional arguments so it can grow new fields over time without changing the call:

The structure is size-versioned: the caller passes the size it knows, the kernel reads up to the size it knows, and any unknown trailing fields must be zero. This is how new fields are added without breaking existing programs.

Requesting a specific thread ID (set_tid) is a privileged operation — it exists for checkpoint-and-restore tools that must recreate a process with its original ID. The privilege required follows the capability model.

clone sharing flags #

Each flag makes the new task share something with its creator instead of getting its own copy. The flags that build threads and control creation:

A thread is CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGHAND | CLONE_THREAD (plus TLS and thread-ID bookkeeping) — everything shared, same process. A fork is the opposite end, with almost nothing shared.

Being in one thread group has consequences beyond shared memory. Every thread returns the same PID from getpid (the thread-group ID), and signal actions are process-wide — an unhandled fatal signal delivered to any thread ends them all — though each thread keeps its own signal mask. A thread that ends does not notify the thread that created it and cannot be collected with a wait call; only once every thread in the group has ended is the process's parent sent SIGCHLD. The new thread shares the creator's parent, and CLONE_THREAD requires CLONE_SIGHAND (and hence CLONE_VM).

Two further flag groups select separation rather than sharing, each documented where it belongs:

• The CLONE_NEW* flags create namespaces — separate views of system resources (mounts, process IDs, networking, and more), covered under namespaces.
• CLONE_INTO_CGROUP places the new process into a resource group at creation, covered under resource management.

CLONE_DETACHED is a historical flag with no effect and is ignored.

The exec family #

An exec replaces the program running in the current process with a different one. The process keeps its PID, its Process GUID, and its identity — only the program changes (see Token lifecycle for the identity detail).

What survives an exec #

A successful exec keeps the process itself but replaces the program it runs. Preserved across the call:

• the PID, the parent PID, and the Process GUID
• the process's identity (its token), its process group and session, and its controlling terminal
• open file descriptors — except those marked close-on-exec, which are closed
• the current directory, root directory, file-creation mask, and resource limits
• pending signals, and the dispositions of signals that were ignored or left at their default

Reset or discarded:

• all memory — the program's mappings, stack, heap, and data are replaced; memory locks are dropped and MADV_* region markings are gone
• every thread except the one calling exec — the new program starts single-threaded
• handlers for caught signals — reset to the default; the alternate signal stack is dropped
• attached System V shared memory, POSIX shared-memory mappings, open POSIX message-queue descriptors and named semaphores, and in-process synchronisation objects (mutexes, condition variables)
• POSIX timers, outstanding asynchronous I/O, open directory streams, and registered exit handlers
• the floating-point environment, and the process name (set to the new program)

What an exec does to a process's identity when the program file is marked to run as another principal is not the Linux set-user-ID model — Peios handles that through the token, covered in setuid and uid0.

Whether a program is allowed to run at all is a separate, execution-policy question — covered under binary signing — not part of these calls.

posix_spawn #

posix_spawn() is the standard library function for the create-then-replace pattern: it makes a new process and runs a named program in it with a single call, with controls for arranging the child's file descriptors and a few attributes first. It is built on the calls above and is the recommended way to launch a program, in preference to assembling fork/vfork and exec by hand.

Process handles (the pidfd family) #

A pidfd is a file descriptor referring to one specific process — the reliable handle introduced in Creating processes. CLONE_PIDFD hands one back at creation; a few calls work with them afterwards:

pidfd_getfd reaches into another process, so it is not something any process may do to any other: it is gated by the right to act on the target, governed by that process's security descriptor.

See also #

• Creating processes — the conceptual counterpart to this page.
• Process lifecycle reference — ending a process and collecting its result.
• Thread operations reference — per-thread calls: TIDs, exit notification, TLS.

This is the detailed counterpart to Process lifecycle. That page explains how a process ends and is cleaned up; this one gives the exact calls — how a process terminates, what its exit status encodes, and how a parent collects it.

Ending a process #

A process can end itself in a few ways, differing in how much tidying happens first:

Returning from the program's entry point is equivalent to calling exit() with the returned value.

At the lowest level the raw _exit system call ends only the calling thread; the library _exit and ordinary termination route through exit_group so the whole process ends. Ending one thread while the rest of the process keeps running is the unusual case.

A process can also be ended from outside, by a signal rather than by a call of its own — covered under signals. Either way, it leaves an exit status behind.

The exit status #

When a process ends it leaves a small exit status for its parent to read. It records one of two outcomes:

• Ended on its own — carries an exit code: the low 8 bits of the value passed to exit/_exit (so 0–255, with 0 conventionally meaning success).
• Ended by a signal — carries the number of the signal that ended it, plus a flag for whether the process produced a core dump.

The status is a packed value, read not directly but through a set of macros:

Collecting the result #

A parent collects a finished child's status — and in doing so clears the zombie — with one of the wait calls. They differ in which child they target and how much they report:

waitpid's target is chosen by a single number: a positive value waits for that exact PID; -1 for any child; 0 for any child in the caller's own process group; and a value below -1 for any child in the process group whose ID is its absolute value.

Two further options matter for threads and other clone-created children, which may not notify their parent with SIGCHLD the way an ordinary child does: __WCLONE waits only for such clone children, and __WALL waits for every child regardless of type.

See also #

• Process lifecycle — the conceptual counterpart to this page.
• Process creation reference — fork, clone, exec, and the pidfd calls.

This is the thread-level companion to the process and thread model — the operations that act on an individual thread rather than on the process as a whole.

Thread IDs #

Every thread has its own thread ID (TID), returned by gettid. Within a multi-threaded process all threads share one PID — the value getpid returns, which is really the thread-group ID (TGID) — but each thread's TID is unique. In a single-threaded process the TID, PID, and TGID are all the same number.

The TID is how the system targets one specific thread rather than the whole process: sending a signal to a single thread, setting one thread's CPU affinity, or addressing it under /proc. It names a thread the way a PID names a process — and, like a PID, it is a reference, not an identity. What a thread is acting as is its token, never its TID.

The TID is also distinct from the opaque handle a threading library returns when it creates a thread; that is a library-level identifier, not the kernel's TID.

Thread-exit notification #

A thread can have the kernel notify waiters when it exits — the mechanism threading libraries build thread join on. set_tid_address sets a clear-tid address for the calling thread (and returns its TID). When that thread later terminates, if it shares memory with other threads, the kernel writes 0 to the integer at that address and wakes one thread waiting on it (a single futex wake).

A thread waiting for another to finish blocks on that address; the kernel clears it and wakes the waiter at the exact moment the other thread ends. This is the same machinery as the CLONE_CHILD_CLEARTID flag in process creation — that flag arranges the clear-tid address at thread creation, where set_tid_address sets it afterwards.

Thread-local storage #

Thread-local storage (TLS) gives each thread its own private copy of a variable — a value that is per-thread rather than shared across the process, even though the threads share the same memory. The per-thread data lives in an ordinary block of memory; what makes it per-thread is a register pointing at this thread's block, so the same code reaches a different copy depending on which thread is running.

The kernel's part is maintaining that per-thread pointer. It is set when the thread is created (the CLONE_SETTLS flag carries the address) and, on x86-64, a running thread can change it through arch_prctl with ARCH_SET_FS (the older set_thread_area serves the same purpose on 32-bit). Everything above that — how thread-local variables are laid out and reached in code — is handled by the compiler and the threading library.

See also #

• The process and thread model — the conceptual counterpart to this page.
• Process creation reference — the clone flags that create threads and arrange TLS and clear-tid addresses.

peinit is the init system of Peios — the first userspace process, running at PID 1 for the lifetime of the machine. It is a single-threaded program with one job: manage services, from the first one started at boot to the last one stopped at shutdown. Every supervised process on a Peios system — a long-running daemon, a one-shot setup task, a health check, a scheduled job — is started, watched, and reaped by peinit.

What makes peinit its own thing is that identity is built in. peinit does not just launch processes; it launches them as someone. Each service runs under a KACS token that decides what it can reach, and each service is itself a securable object with a security descriptor that decides who may start, stop, or query it. Service management and access control are the same system, not two systems bolted together.

This page sketches what peinit is responsible for, names the handful of ideas that make it unlike other init systems, and points at the pages that cover each one.

peinit in one sentence #

peinit is the sole service manager and PID 1 — it reads service definitions from the registry, starts them in dependency order under per-service identities, supervises them through a defined state machine, and tears them down in reverse at shutdown.

Everything in this topic is an elaboration of that sentence: what a service definition is, what dependency order means, what the state machine looks like, how identity is materialised, and how you drive the whole thing from a shell.

Three kinds of object #

peinit is easiest to understand through the three first-class objects it works with. Keeping them straight is most of the battle when you read a status output or an event log.

A service is the what; a job is what actually ran; an operation is what someone asked for. A single restart command, for example, is one operation, which stops the current job and starts a new one, all against one service. Jobs and operations both carry GUIDs and are emitted to eventd as they complete, which is how the history of a service is reconstructed after the fact. They get their own page: Jobs and operations.

Underneath all three sits the fourth thing peinit is always handling — the token, the identity a service runs under. Tokens are not peinit's invention; they are the system-wide identity object. peinit's role is to obtain the right token for each service and install it before the process runs. That is the subject of Service identity and privileges.

What peinit is not #

peinit borrows vocabulary from init systems you may already know, and the familiarity is a trap. Clearing three wrong mental models is most of understanding what peinit actually is.

It is not systemd. There are no unit files. peinit does not read, parse, or translate .service files, and there is no migration shim. A service definition is a key in the registry under Machine\System\Services\, made of typed registry values — not a text file under /etc. Some interfaces are deliberately compatible where that helps: peinit speaks the sd_notify readiness protocol, and timer schedules use systemd's OnCalendar calendar-expression format. The compatibility stops at those wire formats; the model underneath is different.

It is not Windows SCM. The influence is real — services as securable objects with per-service descriptors, token-based service identity, an access-controlled control interface, a structured state machine — but it is architectural, not interface-level. peinit does not implement the SCM RPC protocol, Windows service types, or Windows control codes.

It is not a logging system. peinit holds a service's stdout/stderr pipes at birth, so it controls where output goes — but storage, indexing, and queries are eventd's job. peinit forwards; eventd is the historian. The same split applies to job and operation history: peinit emits structured events and forgets them; eventd keeps them. See Service output and logging.

A fourth, smaller correction: peinit does not supervise forking daemons. A service that double-forks to "daemonise" is solving a problem that does not exist when the manager tracks the process it spawned. peinit tracks every process by pidfd; a legacy binary that insists on double-forking must be wrapped at the package layer.

The one operational invariant #

peinit is single-threaded PID 1, and that one fact explains a surprising amount of its design. In a single-threaded PID 1, a blocking system call blocks everything — child reaping, watchdog expiry, shutdown signals, every other event stops while the call is stuck. So peinit's hard rule is that its event loop never blocks on a userspace service.

Two consequences you will meet repeatedly:

• peinit operates on an in-memory snapshot of the registry, not a live view. It reads all service definitions synchronously once, during boot, and thereafter works from an in-memory model that it updates from change notifications. Supervision never waits on the registry being available. This is why a configuration change does not always take effect immediately — see Defining a service.
• Anything that could block is pushed off the main loop. Filesystem condition checks run in a short-lived forked helper rather than a stat() on the event loop; token requests to authd are driven by a timed state machine, not a blocking call; log delivery is best-effort and never exerts back-pressure on peinit.

You do not need to think about the event loop to operate peinit, but it is the reason behind several behaviours that would otherwise look arbitrary.

Where peinit sits in the system #

peinit is part of the Trusted Computing Base: a compromise of peinit is a compromise of the whole machine. It runs as SYSTEM with all privileges, and it never drops that identity. It depends on, and is depended on by, the other TCB components:

The bootstrapping puzzle — authd needs the registry, the registry needs to be started by something, and that something is peinit running before any of them exist — is resolved by the boot model in Boot and boot modes.

Where to start #

If you are configuring a system, start with Defining a service — what a service definition is, where it lives, and the rule that explains why some changes take effect immediately and others wait for a restart.

If you want to operate a running system — start, stop, query, and troubleshoot services — go to Controlling services for the command set and The service lifecycle to read what a status actually tells you.

If you care about what runs as whom, read Service identity and privileges and Who can manage a service — the two independent halves of the security model.

If you are investigating boot behaviour, Boot and boot modes covers Full, Safe, and Recovery modes and the escalation between them.

And when something is wrong, Troubleshooting peinit works backward from the symptom.

A service is defined by a single key in the registry, under Machine\System\Services\<name>. The key's name is the service name, and the typed values inside it are the definition — the binary, who it runs as, what it depends on, how it is supervised. There is no file under /etc; there is a registry key.

peinit reads these definitions in two situations: once at boot, to build the service graph, and on demand, when an administrator starts a service or runs reload-config. Between those reads it works from an in-memory copy. That last fact is the source of the most common "why didn't my change take effect?" question, and the second half of this page is devoted to it.

Service names #

A service name must be made of characters from [A-Za-z0-9._-] and be 1–128 bytes long. Anything else is a validation error. Two characters are pointedly excluded:

• / — names map directly onto cgroup ids and registry key names, and a slash would be ambiguous in both.
• : — reserved for peinit-internal synthetic names (for example, the way a hook job is labelled).

The name is how you refer to the service everywhere: in peiosctl commands, in another service's dependency list, in a ServiceSecurity descriptor, and in the per-service SID derived from it.

The definition schema #

A definition is a set of typed registry values. Rather than list all of them in one wall of rows, the tables below group the fields by what they are for, with a pointer to the page that explains each group in depth. Every field is optional unless noted; the only hard requirement is ImagePath. The full type-and-default catalog lives in the Registry key reference.

What to run — the binary and its execution context.

What kind of service — type and readiness. See Simple and Oneshot services.

When to start — triggers and conditions. See Triggers and timers.

Who it runs as — identity and privileges. See Service identity and privileges.

How it relates to other services — dependencies. See Dependencies and ordering.

How it is kept alive — supervision and health. See Keeping services running.

The transition phases — hooks and timeouts. See The execution environment and The service lifecycle.

The remaining knobs — scheduling, notify, fds, metadata.

Forward compatibility #

The schema version lives at Machine\System\Services\SchemaVersion (currently 1). peinit is deliberately forward-compatible:

• Unknown values are ignored. A definition written for a newer peinit does not break an older one.
• A newer schema version does not block boot. peinit logs a warning and continues.
• The schema only grows. New capability arrives as new optional fields, never as a breaking change to an existing one.

One defensive rule cuts the other way: a known field must not appear more than once in a collected definition. A duplicated known field is a validation error, even though the registry would ordinarily give you at most one value per name.

peinit works from a snapshot, not the live registry #

Here is the idea that explains most surprises. peinit does not re-read the registry every time it touches a service. It reads definitions at well-defined moments and operates on an in-memory model in between.

Two layers of snapshotting stack on top of each other:

• Boot generation. At the start of boot, peinit reads all definitions, builds the dependency graph, and validates it. The whole boot runs against that one snapshot. Changes made during boot — by an install script, a post-hook — do not perturb the boot in progress.
• Activation generation. When peinit starts a specific service, it snapshots that service's definition for the entire start. Pre-exec hooks, the token request, the readiness timeout, the first health checks — all use the values captured at activation. Edit a field while the service is Starting, and the edit waits for the next start.

peinit learns about registry edits through change notifications: it subscribes to Machine\System\Services\ and Machine\System\Init\ at boot, and processes events in its event loop at a time of its choosing. If the notification queue overflows — a bulk admin operation, a flurry of scripted writes — peinit detects the overflow marker and does a full reload-config to resynchronise. You never have to think about the queue; you do have to know that a change is picked up, not pushed, and that when it takes effect depends on the field.

Field mutability: when a change takes effect #

Every field falls into one of four mutability classes. This table is the one to keep handy.

The practical reading: changing what a process is or runs as (ImagePath, Identity, Type, privileges, ErrorControl) is fundamental enough that it only applies when a fresh process starts — you must restart. Changing policy that peinit consults each time it acts (timeouts, restart behaviour, health checks) is picked up the next time peinit acts. And ServiceSecurity is special: it is re-read on every control request, so an access-control change takes effect on the very next command without touching the running service.

For inactive services the rules are simpler, because there is no running process to protect: a new service entry becomes available once the change notification is processed; a changed timer arms on the next evaluation; a changed dependency takes effect on the next start.

Removing a service definition #

Deleting a definition from the registry does not kill a running instance. peinit learns of the removal through the same notification path and behaves according to whether the service is running:

• Not running (Inactive, Failed, Completed, Skipped, Abandoned): peinit discards the in-memory entry immediately. There is nothing to supervise.
• Running (Active, Starting, Reloading, Backoff, Stopping): the running process is a job, and a job outlives its definition. peinit marks the entry definition-removed, keeps the cached definition only to finish supervising the existing instance, and does not restart it when it exits. Once it exits, the entry — and any stored fds — are discarded.

While an entry is definition-removed it keeps satisfying its dependents (it is still running), stop still works so you can drain it cleanly, but start, restart, and reload are rejected with UNKNOWN_SERVICE — there is no definition to start from. A status query reports it with a definition_removed: true flag so the draining instance is never invisible.

Where to start #

To understand the Type and Readiness fields and the Simple/Oneshot split, read Simple and Oneshot services.

To understand how a definition becomes a running, supervised process — and what each state in a status output means — read The service lifecycle.

For the complete type-and-default catalog of every registry key peinit reads, see the Registry key reference.

A service's type answers one question: does the process keep running, or does it run once and exit? peinit supports exactly two answers — Simple and Oneshot — set by the Type field (default Simple). The type is independent of when the service starts; a Oneshot can be boot-triggered, timer-triggered, or demand-only, exactly like a Simple service.

Simple services #

A Simple service is a long-running daemon. peinit forks, installs the token, and execs the binary. The process is the service: while it runs, the service is running; when it exits, the service has stopped. This is the default and covers the large majority of services — registryd, authd, sshd, application daemons.

The interesting question for a Simple service is when does it count as started — the moment its dependents are allowed to begin. That is the readiness model, set by the Readiness field.

Readiness: Notify vs Alive #

The distinction matters because it is a promise to dependents. With Notify, a service that declares itself a Requires target is telling peinit "do not start anything that needs me until I say READY=1." With Alive, peinit can only promise that the process was forked — not that it is functional.

Once ready, a Simple service transitions to Active and stays there until its process exits or it is stopped. What happens on exit — clean exit, crash, restart — is the subject of The service lifecycle and Keeping services running.

Oneshot services #

A Oneshot service is a run-to-completion task. peinit forks, installs the token, execs the binary, and waits for it to exit. It is the right type for setup work: creating directories, initialising a database, running a schema migration, applying a one-time fixup at boot.

For a Oneshot, readiness is exit, not a signal. The Readiness field is ignored entirely — READY=1 is meaningless for a process whose whole job is to finish. Success and failure are decided by the exit code:

• Exit 0 (or any code listed in SuccessExitCodes) → the service succeeded.
• Any other exit, or death by signal → the service failed, and the failure policy applies.

SuccessExitCodes is for binaries that signal a meaningful outcome with a non-zero code — for example, a tool that exits 2 to mean "nothing to do." Each entry is a decimal 0–255; code 0 is always success and need not be listed.

Completed, and RemainAfterExit #

A successful Oneshot transitions to Completed, and what happens next depends on RemainAfterExit:

Both forms satisfy dependents; the only difference is what a later status query shows. Use RemainAfterExit=1 when "this task is done" is a state worth seeing — a boot-time migration you want to confirm ran, say — rather than a service that quietly returns to Inactive.

Oneshot timing and hooks #

Two details distinguish a Oneshot's start sequence from a Simple one:

• StartTimeout covers the entire run. For a Simple service the timeout covers startup until readiness; for a Oneshot it covers everything from the first pre-hook to process exit. A long-running Oneshot must raise StartTimeout accordingly (or send EXTEND_TIMEOUT_USEC as it works — see Keeping services running).
• ExecStartPost runs only on success. Post-hooks fire after the successful exit. If the Oneshot fails, ExecStartPost does not run.

Restarting a Oneshot #

A Oneshot is not restarted on success, regardless of RestartPolicy — even Always. A successful exit is the goal, not a failure to retry. RestartPolicy governs only the response to failure (a non-zero exit). To re-run a Oneshot on a schedule, give it a timer trigger; that is the intended mechanism.

Simple vs Oneshot at a glance #

The sd_notify contract #

A Notify-readiness service signals readiness by sending READY=1 over the sd_notify protocol — a datagram to the socket named in the NOTIFY_SOCKET environment variable, which peinit sets for every service. READY=1 is only the most visible message; the same channel carries watchdog keepalives (WATCHDOG=1), graceful-stop notice (STOPPING=1), timeout extensions (EXTEND_TIMEOUT_USEC), status strings (STATUS=), and the fd store. These are covered where they belong — readiness here, the rest in Keeping services running and Controlling services.

Readiness is always per start generation. A READY=1 left over from a previous incarnation of the process is never valid for the current start — peinit ties every notify message to the specific process it is currently supervising, verified by pidfd.

Who is allowed to send these messages is governed by NotifyAccess, which currently has a single value: Main (0, the default). Under Main, only the tracked main PID — the process peinit forked and supervises — is authorised to send sd_notify readiness and status messages; a message arriving from any other PID (a child, a helper, an unrelated process) is ignored. So if a worker your service spawns sends READY=1, peinit will not act on it — the readiness signal must come from the main process itself.

No forking daemons #

peinit does not support services that double-fork to background themselves. The whole reason a traditional daemon double-forks — to detach from the launcher — is moot when the launcher tracks the process it spawned. peinit obtains a pidfd for every process at fork time, which is a kernel handle to that exact process, immune to PID-reuse races. There is no MAINPID= mechanism for a service to redirect supervision onto a different process; peinit supervises what it forked.

If a legacy binary insists on daemonising, the fix lives at the packaging layer — wrap it so it stays in the foreground (commonly a --no-daemon/--foreground flag) — not at the init layer.

Where to start #

To control when either type of service starts, read Triggers and timers.

To follow a service from definition to running process and read its states, read The service lifecycle.

To configure restart, health checks, and watchdogs for a Simple service, read Keeping services running.

A trigger decides when a service starts on its own. A service's triggers are independent of its type: a boot-triggered Oneshot and a timer-triggered Simple service are both perfectly ordinary. Triggers live in the Triggers field as a list, each entry written type or type:argument.

A service with no triggers is demand-only: it never starts itself, and is brought up only by an explicit control command or because another service depends on it. Many services are demand-only by design.

Waiting for the boot to settle #

boot:settled is boot with one difference: the service starts once the Phase 2 boot set has stopped moving — every service in it reached a state it will not leave on its own — rather than during it.

It exists for services that write to a terminal. peinit reports its progress to /dev/console, and a service holding that same terminal writes there too, so a prompt started mid-boot gets written over: login emits Username: with no newline and waits, peinit appends a service line, and the result is unreadable until you press Enter and login redraws.

The wait is capped (Machine\System\Boot\SettleTimeout, 5 seconds by default). A service stuck in Starting therefore costs a deferred service a short delay rather than preventing it — which matters most exactly when something is broken and a console prompt is what you need.

A boot:settled service is deliberately not part of the boot plan. It does not consume the parallel-start budget, is not counted towards boot success, and cannot block another service. Waiting for quiet is a preference; it must not change what the boot means.

You can list more than one trigger, including more than one of the same type. ["timer:*-*-* 02:00:00", "timer:*-*-* 14:00:00"] runs at 2 am and 2 pm. The model is built to grow — future trigger types (path, device, event) will slot into the same list without a schema change.

The Disabled flag #

Disabled=1 suppresses automatic activation. A disabled service will not be started by any trigger — boot, timer, or anything added later — but it can still be started by hand through the control interface. It is the switch for "configured, but not running on its own right now."

Two related controls are easy to confuse with it:

Enabling and disabling are registry writes performed by admin tooling, not peinit commands. peinit picks up a Disabled change through a registry notification or on reload-config. A disabled service's definition is still loaded into peinit's model, so it is ready for an on-demand start the instant you ask.

Calendar expressions #

Timer schedules are written in systemd's OnCalendar format. peinit's grammar is normative and self-contained — what follows is the whole language. The general shape is:

DayOfWeek Year-Month-Day Hour:Minute:Second Timezone

Every field is optional and has a default, so most real schedules are short.

So 02:00:00 is a complete expression — date defaults to every day, and it means "every day at 02:00." Hour:Minute with no seconds defaults the seconds to 00.

Component syntax #

Each numeric component — year, month, day, hour, minute, second — and the weekday accept the same operators:

Weekdays are English names, case-insensitive, abbreviated (Mon) or full (Monday), and take lists and ranges.

Last day of the month #

A ~ in place of the - between Month and Day counts the day from the end of the month: ~01 is the last day, ~02 the second-to-last, and so on. So *-*~01 is the last day of every month. Repetition combines with it — Mon *-05~07/1 is "the last Monday in May."

Named shortcuts #

These stand in for the expression shown:

Timezones and precision #

A timezone specifier follows the IANA database — Europe/London, US/Eastern, UTC. With none, the expression is interpreted in system-local time.

Precision is second-level. Unlike systemd, peinit does not accept sub-second fractions; a fractional second is a parse error. Service scheduling has no use for finer granularity.

Examples #

What happens when a timer fires #

A timer firing does not blindly start the service — peinit considers the service's current state first, so a firing never collides with a run already in progress.

The Oneshot "pending" flag is deliberately not a queue: many firings that land while a Oneshot is busy collapse into one catch-up run, taken when the current run ends. A service cannot stampede itself.

That single flag is shared across all of a Oneshot's timers, not one per timer. If a Oneshot has several timer triggers and two of them fire while it is busy, at most one run is still pending when the current one finishes. The timers are otherwise fully independent — each keeps its own next-firing time and its own last-run timestamp — but they can only ever queue up a single shared catch-up between them.

Each timer firing creates a normal start operation, so a timer-started service is observable exactly like an admin-started one — same GUIDs, same events.

Persistent timers and catch-up #

TimerPersistent (default 1) controls whether runs missed while the machine was off are caught up after a reboot.

peinit records the last-run timestamp of each timer in the registry. On boot, for each persistent timer it reads that timestamp, computes the next firing after it, and — if that time is already in the past — fires once, immediately, then resumes the normal schedule.

The catch-up is always a single run, never one-per-missed-occurrence. A daily timer that was off for five days fires once on the next boot, not five times. A non-persistent timer (TimerPersistent=0) ignores history entirely and simply computes its next future firing from now.

One subtlety worth knowing: the last-run timestamp is keyed by the schedule string. If you change a timer's schedule, the old timestamp is orphaned and the new schedule has no history, which causes exactly one spurious catch-up run. Schedule changes are rare and the cost is one extra run, so peinit accepts it rather than tracking trigger identity.

Jitter #

TimerJitter (default 0) adds a random delay of 0 to TimerJitter seconds to each firing, recomputed every time. A daily timer with TimerJitter=900 fires at a slightly different moment between 00:00 and 00:15 each day. Jitter only ever delays — a timer never fires before its scheduled time. It is the standard remedy for a fleet of machines all hammering the same backend at exactly midnight.

When the clock jumps #

Calendar timers are wall-clock schedules: *-*-* 02:00:00 means "02:00 on the wall," not "every 86,400 seconds." peinit arms them as absolute real-time timers that are cancelled and recomputed whenever the system clock is stepped, so the schedule stays anchored to the wall clock across corrections.

Daylight-saving is the case most likely to surprise you, and it only affects timers that name an IANA zone that observes DST (a bare 02:30:00 in system-local time follows whatever the system does). Say you schedule *-*-* 02:30:00 Europe/London:

• Spring forward — the clock jumps from 01:59 straight to 03:00, so 02:30 does not exist on that day. The timer does not fire; it simply resumes at the next valid 02:30.
• Fall back — the clock reaches 02:30, then rewinds and reaches 02:30 a second time. The timer fires exactly once, on the first occurrence, not twice.

This is distinct from peinit's interval timers — the watchdog, health-check intervals, restart backoff, and the start/stop/reload phase timeouts. Those are genuine durations measured on the monotonic clock, so "wait 30 seconds" means 30 elapsed seconds and never lurches when the wall clock is set.

There is one boot-time case worth calling out on its own. When peinit reads a timer's stored last-run timestamp at boot and finds it is in the future relative to the current wall clock — meaning the clock has gone backwards since that run — it can't trust the timestamp, so it treats the timer as having an unknown last run and fires the persistent catch-up immediately. This only applies to the boot-time check; the runtime handling above is unaffected.

Where to start #

To see how a timer-started run appears as a job and an operation, read Jobs and operations.

To understand the states a timer-started service moves through, read The service lifecycle.

For the exact registry keys that store last-run timestamps, see the Registry key reference.

A service managed by peinit is, at every instant, in exactly one state. The state is what a status query reports, what gates dependents, and what decides which commands are valid. Alongside the state, peinit records the cause of the most recent transition — the why behind the where. Reading a status is reading these two things together: "Failed, because RestartBudgetExhausted" tells a very different story from "Failed, because ValidationError."

This page is the reference for both. It is worth internalising before Controlling services and Troubleshooting, because both lean on it.

The states #

Three of these — Active, Completed, Skipped — satisfy dependents. Everything else blocks them. That single column is the rule the whole dependency system turns on: a service waiting on a Requires target does not move until that target reaches one of those three.

The common path #

Most of a service's life is a small loop. The diagram below shows the states a healthy Simple service moves through, plus the two ways it leaves Active.

stateDiagram-v2
    [*] --> Inactive
    Inactive --> Starting: start / trigger / dependency
    Starting --> Active: ready (READY=1 or alive)
    Active --> Stopping: stop / shutdown / conflict
    Stopping --> Inactive: exited (clean stop / shutdown)
    Stopping --> Failed: exited (conflict / BindsTo eviction)
    Active --> Backoff: crash + restart allowed
    Backoff --> Starting: backoff delay elapsed
    Active --> Failed: crash + no restart
    Starting --> Failed: timeout / hook / setup failure
    Failed --> Starting: explicit start

A few things this picture makes concrete:

• An automatic restart always routes through Backoff, never through Failed. Backoff → Starting is the retry; Failed is reached only when restarts are exhausted or disabled. This is why OnFailure fires once at the end, not on every retry.
• Starting can fail before any process exists — a parent-side setup error, a failed pre-hook, a condition or assert. The cause records which.
• Failed → Starting is how a manual start (or a recovery path) revives a dead service; automatic restarts never originate from Failed.
• Stopping has two exits. A clean stop or a shutdown lands in Inactive. But a service stopped because it lost a Conflicts race (ConflictEviction) or because its BindsTo target went away (BindsToPropagation) comes to rest in Failed, carrying that cause — so an evicted or bound-out service shows as Failed in status and needs a reset (or, for a bound service, its target returning) before it starts again. It was not shut down on purpose, so peinit does not treat it as cleanly Inactive.

Oneshot services follow a parallel path through Completed instead of Active: Starting → Completed, and then either staying there (RemainAfterExit=1) or passing through to Inactive. See Simple and Oneshot services.

Transition causes #

Every transition carries a cause. peinit keeps the cause of the most recent transition on the service, and emits all of them to eventd. The full taxonomy, grouped by what kind of thing happened:

Why a service started

Why a service stopped

Why a service failed or restarted — the diagnostic causes

Why a service was rejected — definition and graph problems

The grouping matters for what peinit does next: the diagnostic causes are mostly restart-eligible (peinit consults restart policy), whereas the definition/graph causes are never restarted — retrying a broken definition cannot help. The distinction is spelled out in Keeping services running.

Readiness gating during boot #

The dependent-satisfaction rule is what makes boot orderly. A service's dependents do not start until it satisfies them:

• A Simple service satisfies dependents when it signals READY=1 (Notify) or when its process exists (Alive).
• A Oneshot service satisfies dependents when it exits successfully and reaches Completed.
• A Skipped service satisfies dependents immediately — it succeeded by not needing to run.

If a service does not reach readiness within its StartTimeout, its dependents diverge by relationship: Requires dependents transition to Failed (DependencyFailure); Wants dependents start anyway. That difference is the whole point of the two relationship types — see Dependencies and ordering.

The Abandoned state #

Abandoned is the one state that reflects a kernel-level problem rather than a service-level one. peinit reaches it when it has sent SIGKILL to a service's process group but the processes are still there after a grace period — the post-kill timeout, 5 seconds by default. If the service's cgroup has not emptied within it, the processes are wedged in uninterruptible kernel sleep (D-state), typically behind a hung mount or a broken storage controller, and the service is marked Abandoned (its cgroup leaked).

peinit cannot kill a D-state process; nothing in userspace can. So it stops trying: it marks the service Abandoned, leaks the cgroup (it cannot be removed while populated), and moves on rather than hanging. The leak is never silent — it shows up in the service's warnings and, on a later start, peinit creates a fresh "generational" cgroup so the new instance is unaffected by the stuck old one.

An Abandoned service is cleared with reset, which re-checks the cgroup: if it finally emptied, peinit cleans up; if it is still populated, peinit leaves it leaked and warns you. An Abandoned service is a sign of an underlying I/O fault that needs investigating, not something to paper over with a restart.

Invariants #

A handful of rules hold without exception:

1. A service is in exactly one state at any moment.
2. Only the transitions peinit defines are valid — there are no others.
3. Only peinit changes a service's runtime state. No external process can reach in and set it.
4. A service's securable identity (its ServiceSecurity descriptor — who can manage it) is independent of its process identity (its token — what it can access).
5. Readiness is per start generation — a READY=1 from a previous incarnation is never honoured for the current start.

Where to start #

To understand what happens after a service crashes — restart policy, backoff, health checks, watchdogs — read Keeping services running.

To understand how dependents block on and are released by these states, read Dependencies and ordering.

To see which commands are valid in which state, read the command × state matrix in Controlling services.

Once a service is Active, peinit's job becomes keeping it that way — or deciding, deliberately, when to stop trying. That decision is governed by a small set of policies: a restart policy with a throttling budget, optional active health checks, an optional watchdog, and an error-control level that decides how serious a final failure is. This page covers all of them.

Restart policy #

When a service fails in a way that might be transient, peinit consults RestartPolicy:

Not every failure is restart-eligible. peinit splits causes into three groups:

• Restart-eligible — peinit consults the policy and budget: ProcessCrash, WatchdogTimeout, HealthCheckFailure, ReadinessTimeout, PreHookFailure, PreExecFailure, ParentSetupFailure. (The startup failures are eligible on purpose — a pre-hook that failed on a momentarily-missing mount deserves another try.)
• Never restarted — retrying cannot help: an explicit stop or reset, shutdown, conflict eviction, a broken definition (ValidationError, CycleDetected, DependencyFailure, AssertionError), an already-exhausted budget, or an unkillable process.
• Budget-exempt — BindsToRecovery, where a bound service came back and its dependents are revived. The dependent did not fail on its own, so this restart does not count against its budget.

The budget and backoff #

Restarting an instantly-crashing service in a tight loop is worse than useless, so every restart is throttled by two mechanisms working together.

Exponential backoff. Before each restart the service sits in the Backoff state for a delay that doubles on each consecutive failure, starting from RestartDelay (default 1 s) and capped at 60 s. So a crash-looping service backs off 1 s, 2 s, 4 s, 8 s, … up to a minute between attempts.

The budget. A consecutive-failure counter tracks how many times the service has failed in a row. Once it reaches RestartMaxRetries (default 5), the next failure is not restarted: the service transitions to Failed with cause RestartBudgetExhausted.

The reset. The counter resets to zero once the service stays Active for RestartWindow seconds (default 120) without failing. This is the crucial detail: the budget is not "N restarts ever" — it is "N restarts faster than the service can sustain RestartWindow of health." A service that crashes once a day and recovers cleanly each time never exhausts its budget, because every crash starts from a counter of zero. Only failures recurring faster than the service can stay healthy accumulate.

flowchart LR
    A["Active"] -->|crash| B{"budget left?"}
    B -->|yes| C["Backoff<br/>delay doubles"]
    C -->|delay elapsed| D["Starting"]
    D -->|ready, stays Active<br/>RestartWindow| A
    B -->|no| E["Failed<br/>RestartBudgetExhausted"]

Clean exits #

For a Simple service, exiting with code 0 (or a SuccessExitCodes match) is success, not a crash — a daemon chose to quit. What peinit does next depends on the policy:

• Under Never or OnFailure, the cause is CleanExit and the service goes straight to Inactive, with no restart-policy consultation at all.
• Under Always, the cause is CleanExitRestart and the service is restarted — but through the same backoff and budget as a failure, so a daemon that exits cleanly in a tight loop cannot bypass throttling. Logs and status make clear it exited successfully and was restarted only because the policy is Always.

(A Oneshot clean exit is never restart-eligible regardless of policy — see service types.)

Health checks #

A live process is not always a working one — it can lose its database connection, wedge in a bad state, or return errors while still "up." An active health check closes that gap by running a command periodically and treating its exit code as a verdict: 0 is healthy, non-zero is a failure.

Health checks apply to Simple services only. peinit starts a service's health-check timer once the service becomes Active; a Oneshot has no long-running process to probe, so it never has one. HealthCheck and its companion fields are inert on a Oneshot — no timer is started — so don't set them there expecting supervision.

HealthCheckRetries consecutive failures mark the service unhealthy and restart it through the normal restart policy — backoff and budget included. A single success resets the failure count. If a check is still running when the next interval fires, the new one is skipped rather than stacked.

The health field in a status response reflects this: healthy, unhealthy (failing but not yet failed out), unknown (configured, no result yet), or null (no health check).

The flap guardrail. Health-check throttling only works if a failure cycle is shorter than the restart window — otherwise the counter resets between failures and the service restarts forever. peinit enforces this at validation time as a hard rule:

HealthCheckRetries × HealthCheckInterval < RestartWindow

A definition that violates it is a validation error, not a warning — the service is marked Failed with ValidationError rather than allowed to restart indefinitely.

A health check stuck in D-state is handled like any other stuck helper: its sub-cgroup is leaked and surfaced as a warning, but — unlike a stuck main process — it does not push the service to Abandoned. A health check holds no service resources; it is only a probe.

The watchdog #

The watchdog inverts the health check: instead of peinit asking the service if it is alive, the service must periodically tell peinit. Set WatchdogTimeout (seconds; 0 disables) and the service must send WATCHDOG=1 via sd_notify at least that often. Miss the deadline and peinit treats it as a failure (cause WatchdogTimeout) and applies the restart policy.

Like health checks, the watchdog is a Simple-only supervision mechanism. peinit arms the watchdog timer only once a Simple service becomes Active; on a Oneshot WatchdogTimeout is inert — no timer is started, and setting it changes nothing.

A service can adjust its own watchdog at runtime by sending WATCHDOG_USEC=<microseconds>: a positive value re-arms the timer with the new interval immediately; 0 disables the watchdog. This is for services whose phases have different latencies — a database might run a tight 5 s watchdog normally but widen it to 60 s during a compaction pass, because it knows its own phases better than the schema author did. The runtime value does not persist: a restart reverts to the schema's WatchdogTimeout.

Timeout extension #

A service that does genuinely slow work during a transition can ask for more time by sending EXTEND_TIMEOUT_USEC=<microseconds> while it is Starting, Stopping, or Reloading. Each message replaces the current phase deadline (it is not additive); a service that keeps sending them keeps proving it is making progress, and if it stops, the last deadline fires and peinit escalates normally.

The extension is capped to prevent a buggy service from extending forever:

During shutdown a second, global cap also applies — the deadline can never exceed the remaining time in the global shutdown timeout, and the stricter of the two caps wins. A message received outside a transition (the service is Active, Failed, …) is ignored — there is no timeout to extend.

OnFailure #

When a service enters Failed, an OnFailure field names another service to start in response — a fallback for graceful degradation (the main web UI failed; bring up a minimal emergency endpoint). It is not a monitoring or alerting hook; that is eventd's job.

OnFailure fires for runtime failures — ProcessCrash, WatchdogTimeout, HealthCheckFailure, the startup-failure causes, and non-Critical budget exhaustion. It deliberately does not fire for:

• ShutdownWave — nothing new starts during shutdown.
• ValidationError, CycleDetected, DependencyFailure, AssertionError — these are definition or graph breakage, not a running service degrading. The fallback would likely sit in the same broken graph anyway.

Because a fallback can itself fail and carry its own OnFailure, peinit bounds the chain: it tracks which services it has already started for one originating failure, refuses to start one twice, and never follows the chain beyond a depth of 16. When the guard trips, it logs the loop and stops.

ErrorControl: when failure is not an option #

ErrorControl decides how serious an irrecoverable failure is:

A Critical service is one the system genuinely cannot run without — the platform daemons registryd, authd, lpsd, eventd are all Critical. "Irrecoverable" means the restart budget was exhausted: peinit tried, backed off, retried up to RestartMaxRetries, and the service still would not stay healthy. At that point the fastest path to a known-good system is a reboot.

A few consequences worth holding onto:

• Critical failure outranks OnFailure. When a Critical service exhausts its budget, peinit reboots; it does not start the OnFailure handler.
• ErrorControl=Critical implies SafeMode. A Critical service is always eligible to start in Safe mode.
• Critical services are OOM-protected. peinit sets their oom_score_adj to -1000, so the kernel will not pick them as out-of-memory victims.
• A runtime Critical crash does not enter Safe mode — it follows the reboot path. Safe mode is only for boot-time configuration errors (a cycle or conflict involving a Critical service). The escalation between reboots, the boot-attempt counter, and Recovery mode are covered in Boot and boot modes.

Where to start #

To see exactly which states these policies move a service through, read The service lifecycle.

To understand how a failed service propagates to the ones that depend on it, read Dependencies and ordering.

When a service will not stay up, work backward from the symptom in Troubleshooting peinit.

Services rarely stand alone. A web app needs its database; a network daemon needs the loopback interface up; two implementations of the same role must never run at once. peinit expresses all of this with four relationship fields — Requires, Wants, BindsTo, and Conflicts — that together decide the order services start in, what happens when one stops, and how a failure spreads to its neighbours.

The relationships are declared on the dependent (the service that needs something), naming the target (the service it needs) by name. Conflicts is the exception — it is symmetric, and one side declaring it is enough.

The four relationships #

Requires — a hard dependency #

If A Requires B:

• Start. B must reach a satisfying state before A starts. If B is not running, peinit starts it first (cause DependencyStart). If B fails, A is marked Failed with DependencyFailure and never even attempts to start.
• Stop. Stopping B does not stop A. Requires is a start-ordering constraint, not a runtime leash — A keeps running.
• Runtime failure. If B crashes while A is already Active, A is unaffected. B's own restart policy handles B's recovery.

Requires is the workhorse: "I need this to have started, and if it can't, neither can I."

Wants — a soft dependency #

If A Wants B:

• Start. peinit starts B before A if B exists and is not Disabled — but if B fails to start, or does not exist, A starts anyway.
• Stop / failure. No effect either way. A and B are runtime-independent.

Wants is best-effort ordering: "start this first if you can, but I work without it."

BindsTo — a runtime coupling #

If A BindsTo B:

• Start. Identical to Requires — B must satisfy before A starts.
• Stop. If B stops for any reason — explicit stop, conflict, crash, shutdown — A is stopped too, transitioning to Stopping with cause BindsToPropagation. When A finishes stopping it comes to rest in Failed (carrying BindsToPropagation), not Inactive — so a status query shows the bound service as Failed, making it clear it was taken down by its target rather than shut down cleanly.
• Recovery. When B returns to Active, peinit automatically restarts A from that Failed state (cause BindsToRecovery). This is reactive — peinit watches B's transitions — and it is budget-exempt: A did not fail on its own, so the restart does not count against its budget. If B never comes back, A stays Failed until you reset (or start) it.

BindsTo is Requires plus a runtime leash: "I need this to start, and I should not outlive it." It is the right choice for a sidecar that is meaningless without its principal. BindsTo implies Requires; listing both for the same target is harmless, and BindsTo semantics win.

Conflicts — mutual exclusion #

If A Conflicts with B:

• Start. Starting A while B is Active creates a stop operation for B (source ConflictResolution), evicting it (cause ConflictEviction) before A starts — and vice versa. If the loser will not stop within its StopTimeout, SIGKILL escalation applies. The evicted loser does not land in Inactive: it comes to rest in Failed (carrying ConflictEviction), so it shows as Failed in status and needs a reset before it will start again. That is deliberate — an evicted service is not a clean stop, and leaving it Failed stops it from quietly restarting straight back into the conflict.
• Symmetry. Conflicts is two-way. If A declares Conflicts=["B"], starting either stops the other; B need not declare it back.

Conflicts is for true mutual exclusion — two services binding the same port, or two implementations of one role where exactly one must run — not for ordinary resource contention.

At a glance #

Graph validation #

Before peinit starts anything, it builds the dependency graph and validates it. Validation runs once per graph build — at boot for the whole boot graph, and per request for an on-demand start's transitive closure. It is not incremental.

Cycles. peinit topologically sorts the graph; if the sort fails, there is a cycle. Every service in the cycle is marked Failed with CycleDetected, and peinit logs the full path (dependency cycle: A → B → C → A) so you can find and break it. A service may not depend on itself — a self-reference is rejected as a cycle. If any service in the cycle is ErrorControl=Critical, peinit downgrades to Safe mode rather than rebooting, because a reboot would just hit the same cycle.

Missing targets. What a missing target means depends on the relationship:

Unresolvable conflicts. If two boot-triggered services conflict with each other, there is no way to honour both, so both are marked Failed with ValidationError. As with cycles, if either is Critical, Safe mode applies.

Warnings. Some conditions are logged but do not block boot — most notably a Readiness=Alive service that others Requires (see service types). Warnings appear in the logs and do not change state.

Validation errors. Some conditions do fail the service — for example a health-check configuration that violates the flap constraint. These mark the service Failed with ValidationError.

When more than one finding applies to the same service, peinit records a single primary cause by precedence — CycleDetected > ValidationError > DependencyFailure — but it logs all findings, so the primary cause never hides the others.

Parallel start #

After validation, peinit starts services whose dependencies are all satisfied, and it does so in parallel up to a configurable limit:

The scheduler is simple: every service with no unsatisfied dependencies is eligible; peinit starts up to MaxParallelStarts of them; as each one reaches a satisfying state its dependents become eligible and join the queue. The result is that independent subtrees of the graph come up at the same time, while ordering constraints are still honoured exactly.

Failure propagation #

When a service enters Failed during graph execution, the failure spreads along Requires and BindsTo edges — and only those:

1. Every service that Requires the failed one transitions to Failed with DependencyFailure.
2. Every service that merely Wants it is unaffected and starts normally.
3. Propagation is transitive: if A requires B and B requires C, and C fails, then B fails, then A fails — each with DependencyFailure.

This is the payoff of the Requires/Wants distinction. A hard dependency failing takes its dependents down with it; a soft one failing is shrugged off. Choosing the right relationship is choosing how far a failure is allowed to travel.

On-demand starts #

When you start a service explicitly rather than at boot, peinit does the same graph work on a smaller scope — the requested service's transitive closure:

1. Collect all transitive Requires and BindsTo dependencies (and best-effort Wants).
2. Validate that sub-graph (cycles, missing targets).
3. Resolve Conflicts — stop anything that conflicts.
4. Start the sub-graph with the same parallel scheduler.

Anything already in a satisfying state (Active, Completed, Skipped) is left alone — its dependency is already met, so there is no needless restart. Dependencies pulled in this way start with cause DependencyStart. If two on-demand starts need the same dependency at once, their start operations merge rather than racing.

Shutdown reverses the graph #

peinit does not need a separate stop-ordering configuration. Shutdown simply reverses the dependency graph: services with no dependents stop first, and services that others depend on stop last. A service is never stopped until everything that Requires or BindsTo it has already stopped.

The very last services to stop are the TCB daemons everything rests on — eventd, then authd, then lpsd, then registryd — with registryd stopped dead last, mirroring its position as the first service ever started. The full shutdown sequence is in Shutdown.

Where to start #

To see the states services move through as they start and stop in order, read The service lifecycle.

To understand how a target's failure interacts with the dependent's own restart policy, read Keeping services running.

To follow how dependency-driven starts appear as operations, read Jobs and operations.

Every service process runs under a KACS token — the kernel object that carries identity into every system call and is the input to every access decision. peinit's responsibility is to get the right token for a service and install it on the child process before the binary execs, so the service runs as the intended principal from its very first instruction.

One rule frames everything else: peinit never shares its own SYSTEM token with a service. Even a service configured Identity=SYSTEM gets a separately materialised token of its own — never peinit's. This keeps every service's identity independent and auditable, and it is one of peinit's security invariants.

How a token is materialised #

The Identity field decides where the token comes from:

The SYSTEM path #

For a SYSTEM service, peinit mints an independent SYSTEM token using its own token as the template: it reads its own identity (user SID S-1-5-18, the group list, the privilege set) and mints a fresh primary token with the same identity, adding the service's per-service SID. The minted token is fully independent — the privilege trimming that follows affects only this token, never peinit's.

This path exists because of a bootstrapping problem: the platform services that make the normal token flow possible cannot use it, because they have to start before it exists. registryd, authd, lpsd, and eventd all run as SYSTEM and are all minted this way during boot, before authd is available to mint anything.

The authd path #

For any other identity, peinit asks authd for a token: it sends the Identity value verbatim, and authd routes it to the right source —

• Well-known principals (LocalService, NetworkService, …) → a built-in identity with a predefined minimal privilege set;
• Local service accounts → lpsd, the local principal store;
• Domain accounts → the directory connector;

— resolves the principal's SIDs, mints a token, creates a logon session, and hands the token back to peinit to install. peinit neither knows nor cares whether the identity is local or domain; routing is authd's job.

The per-service SID #

Every service token — minted or authd-issued — carries a per-service SID in its group list. It uses authority S-1-5-80 and is derived deterministically from the service name: the SHA-1 of the uppercased name (as UTF-16LE), with the digest split into five sub-authorities. authd adds it automatically for the tokens it mints; peinit computes and adds it itself for SYSTEM tokens, no authd involvement needed.

The point of the per-service SID is fine-grained access control even when services share an identity. A dozen services might all run as LocalService, but each has a unique service SID, so an ACL can grant access to exactly one of them without inventing a dedicated account. It is also what keeps the platform services — all running as SYSTEM — distinguishable to AccessCheck. See Well-known principals for the SID landscape this fits into.

Privilege restriction #

A token comes from its source (authd or the SYSTEM mint) with a default set of privileges. RequiredPrivileges lets a definition trim that set down to only what the service needs:

• peinit reads the RequiredPrivileges allow-list and removes every privilege not on it from the token before exec.
• Restriction is purely subtractive. peinit removes privileges; it never adds them. A service cannot acquire a privilege its token's source did not grant.
• If RequiredPrivileges is absent, the token's default privilege set is used unchanged.

This is least-privilege made concrete: eudev, for instance, runs as SYSTEM (it needs device access during early boot) but with its privilege set stripped to the minimum it actually uses, so a compromise of eudev does not hand an attacker the full SYSTEM privilege set. Removal is permanent for the life of that token — it is not a disable that the service can re-enable. See Privileges for what the individual privileges grant.

Which identity runs what #

A service is not just its main process — it has hooks, health checks, and a reload command, and each runs under a defined identity:

Token materialisation for hooks follows the same rules: a HookIdentity of SYSTEM is minted; any other principal is requested from authd, at the point the hook runs.

HookIdentity exists so hooks can run with different — usually higher — privileges than the service itself: creating directories in privileged locations, or running a database migration as an admin identity, before the long-running daemon drops to a lesser identity.

Security invariants #

The identity model rests on a few rules peinit never breaks:

1. peinit never shares its SYSTEM token. Even Identity=SYSTEM services get a separately minted token.
2. peinit never drops its own SYSTEM identity. PID 1 runs as SYSTEM for the life of the system — its identity is axiomatic, granted by the kernel at boot.
3. Privileges are subtractive only. RequiredPrivileges removes; it can never add.
4. Identity is deterministic. Every service runs as a known principal. SYSTEM must be declared explicitly; an empty Identity is the minimal LocalService, never SYSTEM.

Where to start #

Identity decides what a service can do; the ServiceSecurity descriptor decides who can manage it — two independent concerns. Read Who can manage a service for the other half.

To see how the token is installed alongside the rest of the process setup, read The execution environment.

For the identity primitives themselves — tokens, SIDs, privileges — start at Tokens.

When peinit starts a service, the process it hands you is built deliberately — a clean context with a known identity, a known environment, and only the resources peinit chose to give it. This page covers everything about that context except the token (its own page): the cgroup the process lives in, its standard streams, the environment it sees, the limits on it, and the hooks and checks that run around the start.

A clean context #

A service inherits only what peinit explicitly hands it — its standard streams and any stored file descriptors. Every other descriptor peinit holds — the control socket, the notify socket, the event-loop fd, the registry and authd and eventd connections — is opened close-on-exec, so it closes automatically at exec and can never leak into a service. peinit also resets the child's signal state to defaults (it runs with all signals blocked for its own signalfd; a service must not inherit that). The guarantee is that a service starts from a clean slate, not from peinit's privileged one.

The cgroup tree #

Every service runs in its own cgroup tree under /sys/fs/cgroup/peinit/, with separate sub-cgroups for the main process, hooks, and health checks:

/sys/fs/cgroup/peinit/<service>/
                       ├── main/     the main process
                       ├── hooks/    pre/post hooks
                       └── health/   health checks

peinit uses cgroups for two things only: tracking every process a service spawns (so a child that forks its own children is still accounted to the service), and clean kill (terminating the entire tree at once, so nothing is orphaned). It does not use cgroups for resource accounting or limits.

The split into sub-cgroups serves a real purpose: it satisfies cgroup v2's "no internal processes" rule, and it lets peinit kill a service's hooks or health checks without touching the main process.

When an old tree cannot be removed — because a process survived SIGKILL in D-state and leaked it — peinit creates a fresh generational tree (<service>.gen<N>) for the next start, so a stuck old instance never blocks a new one. Leaks are surfaced in the service's warnings; they are a sign of an underlying I/O fault, covered in Keeping services running.

Standard streams #

peinit wires a service's standard streams before exec:

• stdin is redirected to /dev/null. peinit never gives a service an interactive input channel; a service that needs input must obtain it explicitly (a socket, a stored fd).
• stdout and stderr are redirected to pipes peinit holds, so it can capture and forward every line. This is the subject of Service output and logging.

Attaching a terminal #

A service that sets TTYPath to an absolute terminal path — /dev/console, /dev/tty1, a serial line — gets that terminal on all three standard streams instead of the wiring above, and becomes a session leader owning it as its controlling terminal. That last part is what makes job control work: Ctrl-C interrupts, fg/bg behave, and a hangup reaches the process group. It is what a boot shell or a login prompt needs.

It carries a path rather than a flag because which terminal is a per-service question: a maintenance shell on tty1 while the kernel console is a serial line is an ordinary thing to want.

The environment #

peinit constructs each service's environment in layers, lowest precedence first — it does not pass through its own near-empty startup environment.

1. Compiled-in base. A fixed floor peinit always provides — currently just PATH: /bin.
2. Global EnvVars. Each value under Machine\System\Init\EnvVars\ becomes a variable (value name → variable name). EnvVars\PATH overrides the base PATH; other names add. Every service gets this layer except registryd, which serves the key itself — see the warning below for why that exemption exists.
3. Per-service Environment. The definition's own KEY=VALUE entries, which override layers 1–2.
4. Protocol variables. NOTIFY_SOCKET (always) and LISTEN_FDS / LISTEN_FDNAMES (only when stored fds are injected). These have the highest precedence and cannot be overridden — a service that clobbered NOTIFY_SOCKET would break sd_notify.

A change to EnvVars\ takes effect on a service's next start, like the per-service Environment field — it is not pushed into running services.

peinit deliberately does not set HOME, USER, LOGNAME, SHELL, or TERM. Peios identity is a token (a SID), not a passwd entry, so there is no canonical home directory or login shell to fill in. A service that needs any of these supplies it through EnvVars\ or its own Environment.

Working directory and limits #

peinit also sets oom_score_adj: -1000 (OOM-immune) for ErrorControl=Critical services, and the default 0 for everyone else — so the kernel never picks a Critical platform daemon as an out-of-memory victim.

Runtime directories #

RuntimeDirectories gives a service private scratch space under /run without an ExecStartPre that has to mkdir it by hand. Each name you list becomes a directory created directly under /run — RuntimeDirectories=myapp yields /run/myapp — set up immediately before the main process starts. Each one gets a security descriptor granting full access to SYSTEM, Administrators, and the service's own SID, so the service can write to its directory while other principals cannot. If a directory can't be created, or its descriptor can't be applied, the start fails with ParentSetupFailure.

peinit does not remove these directories when the service stops. /run is a boot-scoped tmpfs, so they simply disappear at the next boot rather than being torn down on each stop — a restarting service finds its runtime directory (though not necessarily its contents) already in place.

For the exact naming rules on each entry and the field's type and default, see the Registry key reference.

Pre-exec and post-exec hooks #

Hooks let a service do work around its main process without baking it into the binary.

• ExecStartPre runs before the main binary. Each command runs in sequence, in the hooks/ sub-cgroup; any hook exiting non-zero aborts the start (the whole cgroup is killed and the service Fails with PreHookFailure). Use it for prerequisites that must hold before the service starts — creating a runtime directory, waiting on a precondition.
• ExecStartPost runs after readiness (Simple) or after a successful exit (Oneshot). A post-hook failure is logged but does not fail the service — by the time it runs, the service is already up. (A Oneshot that fails never runs its post-hooks.)

Hooks run under HookIdentity if set, otherwise the service's own identity. Their output is captured and tagged like the main process's, labelled with the hook (e.g. jellyfin/ExecStartPre[0]).

Command string parsing #

The command fields — ExecStartPre, ExecStartPost, ExecReload, HealthCheck — are parsed into an argument list with simple, predictable rules. peinit never invokes a shell.

• Whitespace splits the command into arguments; double quotes group text containing spaces into one argument and are not retained (--name="hello world" → one argument --name=hello world).
• There is no shell expansion, variable substitution, or globbing. Backslash is literal; a single quote is an ordinary character.
• The first argument is the executable and must be an absolute path beginning with /. Relative names and PATH search are validation errors.
• An empty or whitespace-only command, or an unclosed quote, is a validation error.

If a command genuinely needs shell features, wrap it in a script and point the field at the script.

ExecReload can also be a signal, written signal:<NAME> — for example signal:SIGHUP. The name must be an exact, canonical Linux signal name (uppercase, no aliases, no numbers); SIGKILL and SIGSTOP are rejected because they cannot be handled as a reload. With no ExecReload at all, reload sends SIGHUP. The reload lifecycle is covered in Controlling services.

Conditions and asserts #

Before peinit runs any hook or the main process, it evaluates conditions and asserts — start-time checks in the same type:argument form. The difference is what failure means:

• A failed Condition skips the service — it transitions to Skipped, which satisfies its dependents (it succeeded by not needing to run). Conditions express "only run here if it makes sense."
• A failed Assert fails the service — it transitions to Failed with AssertionError. Asserts express "this was expected to run, and a precondition is missing."

Conditions are checked first; only if all pass are asserts checked. All entries are AND-ed. The check types:

Two constraints follow from peinit being single-threaded PID 1, which must never block:

• registry: checks are cache-only. A registry: check may only name a key peinit already caches (under Machine\System\Services\ or Machine\System\Init\); it is evaluated against the in-memory model, never a live read. Naming any other key is a validation error.
• Filesystem checks run off the main loop. path:/file:/directory: checks are performed by a short-lived forked helper, not a stat() on the event loop, because stat() can wedge on a hung mount.

That timeout is the PreStartCheckTimeout field, which defaults to 5 seconds. A filesystem check that does not finish within it — say a stat() wedged on a hung mount — is treated as not satisfied, fail-safe: peinit kills the helper and treats the check as unmet, so a Condition skips the service and an Assert fails it, rather than letting the event loop hang.

The fd store #

The fd store lets a service preserve file descriptors across restarts — most usefully a listening socket, so a stateful daemon can restart without dropping connections. It is opt-in: FdStoreMax (default 0) sets the maximum number of fds peinit will hold; 0 disables it.

The mechanics ride on sd_notify:

• A service stores an fd by sending FDSTORE=1 with the fd attached (and optionally FDNAME=<name>; the default name is stored). peinit holds it.
• It removes one with FDSTOREREMOVE=1 and FDNAME=<name>.
• On the next start, peinit injects the stored fds starting at fd 3, sets LISTEN_FDS to the count and LISTEN_FDNAMES to the colon-separated names, then clears the store. (This is the same LISTEN_FDS convention systemd uses, so software with existing fd-passing support works unmodified.)

The store survives an automatic restart (crash → restart policy → new start) — that is the whole point. It is cleared on an explicit stop or shutdown (the service is not coming back), and when a removed definition is finally discarded.

Where to start #

For the identity half of the launch — how the token is chosen and installed — read Service identity and privileges.

For what happens to the stdout/stderr pipes peinit holds, read Service output and logging.

For how conditions, asserts, and hooks fit into the start sequence and its states, read The service lifecycle.

peiosctl is the command-line tool for driving peinit at runtime — starting and stopping services, querying their state, reloading configuration, and shutting the system down.

peiosctl <command> [service] [flags]

$ peiosctl status jellyfin        # current state of one service
$ peiosctl start jellyfin         # start it, wait until Active or Failed
$ peiosctl list                   # every service you can query
$ peiosctl shutdown reboot        # graceful reboot

Underneath, peiosctl is a thin client over peinit's control socket at /run/services/peinit/control.sock. The wire protocol — not the CLI — is the normative interface, so everything here (commands, rights, semantics) holds regardless of which front-end you use.

How the control interface works #

The socket speaks newline-delimited JSON: one request object per line, one response object per line. A request and its success response look like this:

{"command": "start", "service": "jellyfin", "wait": true}
{"status": "ok", "operation_id": "a1b2c3d4-...", "service": "jellyfin", "state": "active", "cause": "explicit_start", "warnings": []}

Two properties are worth knowing even if you only ever use peiosctl:

• Every command is access-controlled. When you connect, peinit captures your token from the kernel and runs AccessCheck against the target service's descriptor for every command. There is no "trust localhost," no override. Who may do what is the subject of Who can manage a service.
• Lifecycle commands create operations. A start/stop/restart/reload/reset returns an operation_id — a GUID you can poll. Conflict resolution between concurrent commands happens at the operation layer, which is why two simultaneous starts merge instead of colliding.

Service commands #

$ peiosctl restart jellyfin
$ peiosctl reload nginx
$ peiosctl reset failed-migration     # clear a Failed state without starting

System commands #

$ peiosctl shutdown poweroff
$ peiosctl reload-config
$ peiosctl operation-status a1b2c3d4-...

Wait semantics #

By default, a lifecycle command blocks until its operation reaches a terminal state — start waits for Active (or Failed), stop waits for Inactive, and so on. Pass --no-wait to get the operation_id back immediately and poll with operation-status instead.

reload is the exception — it returns immediately by default, because a reload may have no observable completion. With --wait, the response carries a mode:

The detection window is a fixed 2 seconds — it is built in and is not configurable via the registry. If the service says nothing within that window, the reload resolves as advisory and the service stays Active.

The command × state matrix #

A command sent to a service in an unexpected state returns an error, not a silent no-op. This matrix is the authority on what each command does in each state:

Legend:

• MERGE — an operation of this type is already running; your command merges into it and you get its GUID.
• DEFER — your start is accepted and creates a Pending start operation, but it does not run yet: it waits out the service's existing backoff deadline and then starts. If a deferred start is already waiting, you merge into it and get its GUID.
• ALREADY — already in the target state; returns the current status, not an error.
• QUEUE — queued as Pending; runs after the current operation finishes.
• NOOP — no effect; returns the current status.
• Clear / Cancel — clear the state to Inactive / abort the current operation, then proceed.
• ERROR — invalid for this state; returns an error with an explanation.

The Backoff column is the subtle one: the service is down with an automatic restart pending, so start is deferred — it creates a Pending start that honours the remaining backoff delay and only runs once that deadline expires (a second start merges into the one already waiting), stop cancels the pending restart and any deferred start (the service goes Inactive), restart cancels the automatic one and does an admin restart, and reload/reset are invalid because no process exists.

Reading status #

status returns the full picture for one service:

{
    "status": "ok",
    "service": "jellyfin",
    "state": "active",
    "cause": "explicit_start",
    "status_text": "Listening on port 8096",
    "current_job": {"id": "a1b2...", "type": "service_main", "pid": 1234, "started_at": "...", "identity": "jellyfin-svc"},
    "current_operation": {"id": "e5f6...", "type": "start", "source": "admin"},
    "health": "healthy",
    "uptime_seconds": 86400,
    "definition_removed": false,
    "warnings": []
}

• state and cause are the lifecycle pair — read them together.
• status_text is the latest STATUS= string the service sent via sd_notify (null if never sent; cleared on each restart).
• current_job and current_operation are the job and operation GUIDs, or null.
• health is healthy, unhealthy, unknown, or null (no health check).
• definition_removed is true when the definition was deleted but an instance is still draining.
• warnings lists leaked sub-cgroups and other operator-relevant notices.

list returns a compact summary of every service you can query — services you lack SERVICE_QUERY_STATUS on are simply omitted, not denied:

{"status": "ok", "services": [
    {"service": "jellyfin", "state": "active", "cause": "explicit_start", "health": "healthy"},
    {"service": "registryd", "state": "active", "cause": "dependency_start", "health": null}
]}

operation-status returns one operation by GUID; an unknown or expired GUID is the UNKNOWN_OPERATION error. (Operations are dropped after a short retention grace once terminal — long enough for a polling client to read the result, not forever.)

Error codes #

An error response is {"status": "error", "code": "...", "message": "..."}. The code is one of:

The message is human-readable and non-normative — read it for context, key off the code.

Connection limits #

peinit enforces hard limits on the control socket, all tunable under Machine\System\Init\:

Exit status #

Where to start #

To understand the states the matrix refers to, read The service lifecycle.

To understand the operations every lifecycle command creates and how to poll them, read Jobs and operations.

To configure who may run each command, read Who can manage a service.

A service is a securable object. Just as a file or a registry key carries a security descriptor that decides who may touch it, a service carries one that decides who may start, stop, query, or reload it. peinit enforces it on every control command — there is no command that skips the check.

This is the second, independent half of the security model. The first half, Service identity and privileges, is about what a service can do (its token). This page is about who can manage the service — a completely separate question, answered by a completely separate descriptor.

Two descriptors, two questions #

A service is associated with two descriptors that are easy to conflate but answer different questions and are enforced by different components:

The registry key SD is ordinary registry access control and not peinit's concern — peinit reads definitions as SYSTEM, which has full access. The ServiceSecurity SD is peinit's domain, and the rest of this page is about it.

They are genuinely independent. An administrator might be able to query a service's status (ServiceSecurity grants SERVICE_QUERY_STATUS) but not read its configuration (the registry key SD denies read) — or the reverse. Runtime control and configuration access are separate concerns, and both combinations are valid.

Service access rights #

The ServiceSecurity descriptor grants these rights:

restart requires both SERVICE_STOP and SERVICE_START, since it is a stop followed by a start. reset requires SERVICE_STOP.

When peinit evaluates the descriptor it maps the generic rights as follows, so a descriptor written with generic rights behaves sensibly:

How a command is authorised #

Every control command runs the same gate:

1. peinit captures the caller's token from the kernel (kacs_open_peer_token) — the caller's effective identity at connection time, so if the caller is impersonating, the impersonated identity is what is checked.
2. peinit resolves the target service and its ServiceSecurity descriptor.
3. peinit runs AccessCheck: the caller's token against the descriptor, for the right the command needs.
4. Denied → return ACCESS_DENIED and log the attempt (caller SID, target service, requested right).
5. Granted → execute the command.

The default descriptor #

If a service has no ServiceSecurity value, it inherits its parent key's. If no ancestor sets one either, peinit applies a built-in default:

• SYSTEM (S-1-5-18) — full access.
• Administrators (S-1-5-32-544) — query and stop only.

So out of the box, administrators can see and stop a service but not start or reload it unless a descriptor grants more — a conservative default that you widen deliberately.

ServiceSecurity is hot-reloaded: a change to the value in the registry takes effect on the next control request, with no service restart. peinit picks the change up through a registry notification. This is why ServiceSecurity is in its own mutability class — access policy should be able to change without disturbing a running service.

The system control descriptor #

Some operations are not about any one service — shutdown and reload-config act on the whole system. These are checked against peinit's own descriptor, stored at Machine\System\Init\ControlSecurity:

Its generic mapping deliberately gives GENERIC_READ nothing — there is no "read" of the system control object, only the two actions:

The default grants SYSTEM full access and Administrators both rights. peinit loads this descriptor at boot and hot-reloads it on registry change, exactly like ServiceSecurity.

The list command filters, it does not deny #

list is access-control-aware in a quieter way: it returns only the services the caller has SERVICE_QUERY_STATUS on, and simply omits the rest. A caller with no query rights gets an empty list, not a denial. This means a low-privilege principal cannot even enumerate the services it cannot see — the existence of a service is itself information the descriptor controls.

The boundaries that hold #

A few invariants are worth stating outright, because they are what make this trustworthy:

• peinit never bypasses AccessCheck for a control operation. No backdoor, no override flag, no "trust localhost."
• The descriptors are the only policy inputs. peinit consults the ServiceSecurity and ControlSecurity descriptors and nothing else — not config files, not environment variables, not hardcoded lists.
• One service's state is never exposed to another without a check. status is per-service access-controlled; list filters.

Where to start #

For the other half of the security model — what a service can reach once it is running — read Service identity and privileges.

For the commands these rights gate, read Controlling services.

For the descriptor and AccessCheck machinery itself, read Security descriptors and Access decisions.

A service is a long-lived definition. But when you read a status output or query the event log, two shorter-lived objects show up alongside it: the job (one process execution) and the operation (one requested action). They are what make peinit observable — every run and every command carries a GUID you can follow. This page covers both, and the special case of ad-hoc jobs.

Jobs: what actually ran #

Every time peinit forks a process — a service's main binary, a pre- or post-hook, a health check, or an ad-hoc run — that single execution is a job, with its own GUID and exit result. If a service is the what, a job is what actually ran. A restart does not reuse a job; it creates a new one. The service always tracks its current job GUID, and a status query reports it.

Jobs come in five types:

Their lifecycle is simpler than a service's, because a job only tracks a process — not policy:

A job record carries the things you would want for forensics: the resolved identity and a token summary, the image path and arguments, created/started/ended timestamps, the exit code or signal, the cgroup, the failure cause, and the operation_id that created it (null for ad-hoc jobs). The clean division of ownership is: the service owns policy (restart, dependencies, health schedule, current state); the job owns the execution facts (PID, exit, timestamps, identity, cgroup, log correlation).

Retention and log correlation #

peinit keeps only active jobs in memory. When a job reaches a terminal state it emits a structured event (through KMES) carrying the full record, then drops the job. peinit keeps no job history — eventd is the historian, consuming those events from the kernel ring buffer. The lifecycle events are job.created, job.started, and job.ended.

Separately, all of a job's stdout/stderr is forwarded to eventd tagged with the job's GUID. That is what lets a query like "show me the logs for job X" return exactly that execution's output — not interleaved with the run before or after it.

Ad-hoc jobs #

An ad-hoc job is an arbitrary process a service asks peinit to run on behalf of one of its clients. It is not tied to a persistent definition — it runs once, reports its result, and is cleaned up. The motivating case: a service has impersonated a user and wants a process run as that user, but supervised centrally by peinit rather than by the service itself.

The obstacle is identity delegation: a service cannot simply forward an impersonation token over a socket. JFS (the Job Forwarding Subsystem) solves this in the kernel — it captures the caller's effective token and delivers it to peinit along with the job request. peinit forks, installs that captured token, and supervises the result like any other job.

An ad-hoc request carries a deliberately small subset of the service fields — ImagePath, Arguments, Environment, WorkingDirectory, Timeout, Description. A couple are worth pinning down: Timeout is in seconds, and 0 means no limit (the job runs for as long as it needs). WorkingDirectory defaults to / when you leave it out. The policy fields (restart, dependencies, health checks, ErrorControl, triggers) are not available — those belong to persistent definitions. An ad-hoc job runs once in its own cgroup, routes its output to eventd, and on exit emits job.ended and is dropped. If it overruns its Timeout, peinit escalates SIGTERM → SIGKILL exactly as a service stop does.

Ad-hoc jobs bypass the operation model entirely — there is no service to "start," so the request creates a job directly. The job is the whole lifecycle.

Operations: what was requested #

An operation is a first-class object representing a requested state-machine action on a service. Rather than letting control commands mutate state directly, peinit turns each one into an operation that is validated, queued, and executed by its event loop. This is what gives concurrent callers — admin tools, automated triggers, dependency propagation — explicit, observable conflict resolution instead of races.

There are five operation types — Start, Stop, Restart, Reload, Reset — and each carries a source recording why peinit created it:

The source is why a status showing current_operation.source: "restart_policy" tells you the service is being auto-restarted, while "admin" tells you a person asked.

Operation lifecycle #

flowchart LR
    P["Pending"] --> R["Running"]
    R --> C["Completed"]
    R --> F["Failed"]
    R --> A["Aborted"]
    P --> M["Merged"]
    P --> X["Cancelled"]
    P --> F

Cancelled and Aborted are the same idea at two points: cancelled never ran, aborted was running. The reason (admin action, supersession) is a property of the event, not the state.

Conflict resolution and merging #

When a command arrives for a service that already has an operation in flight, peinit resolves the collision deterministically — the same logic the command × state matrix summarises:

• Same type (start over start, stop over stop, reload over reload) → merge. The new caller transparently receives the existing operation's GUID; from their side, their request is in progress.
• Stop wins over start. An explicit stop cancels a pending start or aborts a running one. The admin said stop.
• Later supersedes earlier, and a start while a stop is in flight is queued to run after the stop.

Timeout and retention #

An operation inherits its target's timeout as its maximum lifetime — StartTimeout for start/reload/reset, StopTimeout for stop, and the sum of both legs for restart. Crucially, the clock starts at operation creation, including queue time — from the caller's perspective they have been waiting since they sent the command, so a long queue can time an operation out before it even runs.

Terminal operations are emitted as events (operation.requested, .started, .completed, .failed, .cancelled, .merged, .aborted) and dropped from memory after a short grace (default 60 s — long enough for a polling client to read the result). As with jobs, peinit keeps no operation history; eventd does.

How they surface #

You meet jobs and operations in three places:

• status — current_job and current_operation give the GUIDs of the service's active execution and active action (or null).
• operation-status <id> — polls one operation to a terminal state; a --wait lifecycle command does this for you under the hood.
• eventd — the durable history. Every job and operation lifecycle transition lands there as a structured event, which is how you reconstruct what happened after the objects themselves are gone from peinit's memory.

Where to start #

To create and poll operations from the command line, read Controlling services.

To understand the service states an operation drives a service through, read The service lifecycle.

To query the durable job and operation history, read Auditing.

peinit is not a logging system — storage, indexing, and queries belong to eventd. But peinit holds the stdout/stderr pipes of every service at the moment it forks, so it is unavoidably in charge of where output goes, and it has to cope with the window early in boot before eventd even exists. This page is about that responsibility.

Capturing output #

When peinit forks a service it creates pipes for stdout and stderr, redirects the child's streams to the write ends, and keeps the read ends — monitoring them in its event loop. (The child's stdin is /dev/null; see The execution environment.) It reads output line by line and tags each line with:

• the service name,
• the stream (stdout or stderr),
• a timestamp,
• the job GUID.

The job GUID is the important one: it is what lets a later query return exactly one execution's output, never interleaved with the run before or after it. Hook output is captured too, labelled with the hook (e.g. jellyfin/ExecStartPre[0]), and so is health-check output — invaluable for diagnosing why a check failed.

Logs are best-effort; audit events are not #

This is the distinction to internalise, because it explains why some things survive a crash and others do not. peinit handles two different streams of information with two different guarantees:

Service logs are a convenience: useful, but lossy by design. Audit events are part of the security posture and go through KMES, which persists them in the kernel from the instant the module loads. So peinit keeps a pre-eventd buffer for logs but needs none for events — eventd picks events up from the ring buffer whenever it attaches, no matter how late.

The pre-eventd buffer #

Before eventd is running there is nowhere to send logs — its socket does not exist yet. peinit therefore buffers captured output in a fixed-size in-memory pre-eventd buffer (default 1 MB); when it fills, the oldest entries are dropped.

In practice the only services that run before eventd are registryd, lpsd, authd, and eudev. The first three are Peios-owned with controlled output; eudev is the real overrun risk (verbose device enumeration), and the restart budget naturally bounds output from a crash loop.

Flood protection #

A noisy service must never starve peinit's event loop. Several limits enforce that:

Two mechanisms back these up:

• A per-iteration read budget. Each pass of the event loop reads at most a bounded number of bytes from service pipes before moving on to other events — so no single chatty service can monopolise a loop iteration.
• Back-pressure, not dropping. If a service writes faster than peinit reads, the kernel pipe buffer fills and the service's own write() calls block. This is deliberate: the service slows down. While reading the pipe, peinit does not drop output — the no-silent-drop guarantee covers reading the pipe. It is only downstream (the pre-eventd buffer, and eventd's datagram socket) that delivery becomes lossy.

The eventd handoff #

When eventd reaches Active, peinit switches from buffering to forwarding:

1. It begins sending to eventd's log datagram socket (path from Machine\System\eventd\LogSocketPath).
2. It replays the pre-eventd buffer, oldest first, preserving each line's timestamp and metadata. This replay is best-effort — the records are datagrams on a loss-tolerant socket and some may be dropped under load; peinit never blocks waiting to deliver them.
3. It switches to real-time forwarding — new output goes out as it arrives.
4. It clears the buffer.

From then on peinit is a pipe relay: read a line, tag it, forward it as a datagram. The log socket is non-blocking and loss-tolerant — if eventd cannot drain it fast enough the kernel drops further datagrams silently, because log ingestion must never exert back-pressure on the whole system through PID 1. peinit keeps no unbounded outbound buffer and never blocks on a send to eventd.

If eventd itself crashes after starting, peinit (which supervises it like any service) notices, re-enables the pre-eventd buffer, and repeats the handoff when eventd comes back. There is a log gap across the restart, bounded by the buffer size — but audit events are unaffected, because they were going to KMES the whole time and eventd resumes consuming them from the last persisted sequence.

Console output #

peinit writes its own operational messages to /dev/console:

• Phase 1 progress (mount results, registryd start),
• Phase 2 progress (service starts and failures, dependency errors),
• shutdown progress,
• recovery-mode entry,
• critical service failures.

Service stdout/stderr is not echoed to the console by default — the console is for peinit's status messages only. This keeps the console legible during boot and, crucially, usable in Recovery mode, where it may be the only interface an administrator has.

Where to start #

To see how the pipes are wired and what else the process inherits, read The execution environment.

To follow the job GUID that tags every log line, read Jobs and operations.

For where the logs and events actually go — storage, indexing, and queries — read Auditing.

peinit's boot has two phases and a chicken-and-egg problem to solve. The problem: peinit reads everything it does from the registry, but the registry is served by a daemon that something has to start first — and the identity authority that would mint that daemon's token does not exist yet either. The solution is to split boot into a hardcoded bootstrap that needs no registry, and a registry-driven phase that starts once the registry is up. The boundary between them is registryd.

Where peinit takes over #

peinit is PID 1, but it is not the first thing that runs. The initramfs assembles and mounts the real root — decryption, RAID/LVM, the root filesystem itself — and then hands control to peinit. The contract peinit relies on is narrow:

• The real root is already mounted read-write, and /proc, /sys, /dev are mounted and moved into it.
• peinit is exec'd as PID 1 from a fixed path on the real root.

peinit does not assemble, decrypt, repair, or even re-mount the root — those need tools and configuration that belong to the initramfs. It also does not fsck the root or mount non-root storage (a data partition is mounted by an ordinary Oneshot service, not by peinit). For the trust and identity side of this handoff — signatures, the SYSTEM token peinit inherits — see peinit at PID 1.

Phase 1: the hardcoded bootstrap #

Phase 1 is compiled into peinit. It cannot change at runtime and touches no registry. It does the minimum to make Phase 2 possible:

1. Confirm the root is writable with a single probe write. registryd's storage needs a writable root even for reads, so a read-only root cannot support Phase 2 → Recovery.
2. Mount the remaining virtual filesystems — /dev/pts, /dev/shm, /run, /sys/fs/cgroup — mounting each only if absent. A failure here → Recovery.
3. Restore the persisted random seed from /var/state/peinit/random-seed, mixing it into the kernel's entropy pool early so anything that needs randomness during boot gets it. A missing seed is normal — first boots and stateless live boots have none — so peinit just carries on; a seed problem is never fatal and never sends boot to Recovery.
4. Establish the machine-id from /lcl/etc/machine-id — a stable, opaque identifier for this install (used for log correlation, instance identity, and software compatibility). It is not a security principal: it is not a SID, an account, or a credential, and no authorisation decision depends on it. If the file is missing, empty, or malformed, peinit generates a fresh 128-bit ID and writes it before continuing.
5. Set the clock from the hardware RTC, so early timestamps and the boot counter are meaningful. A failure here → Recovery.
6. Start registryd and wait for it to signal readiness, then probe-read the schema-version key to confirm it is actually serving reads. Any failure → Recovery — there is no Phase 2 without a registry.
7. Provision boot-time paths. With registryd up and before Phase 2 starts, peinit applies the entries under Machine\System\Init\ProvisionedPaths\ — the registry-driven equivalent of tmpfiles.d, creating directories and files (with Peios security descriptors) that no single service owns. Best-effort entries that fail are logged and skipped, but an entry marked Required=1 that cannot be provisioned sends boot → Recovery. The individual keys are cataloged in the registry key reference.
8. Infrastructure setup — create the control socket, open the JFS device for ad-hoc jobs, bring up the loopback interface. Control-socket failure → Recovery; the other two are logged as warnings and boot continues.

Most Phase 1 failures are fatal to a normal boot, because none of the later machinery can run without this foundation — the only outcome is Recovery mode. The exceptions are the fail-soft steps called out above: a missing or unusable random seed, a regenerated machine-id, and best-effort provisioned paths all let boot continue.

registryd and loregd #

registryd is an interface, not a specific program. It is the path peinit execs to get a registry source daemon — the component that implements the registry's persistent storage and answers peinit's reads. The implementation behind that interface can vary; by default it is loregd.

This split matters in exactly one place: Recovery mode. In normal operation you only ever deal with the registryd abstraction — peinit starts it, treats it as opaque, and reads the registry through it. But when the registry itself is what broke, you need tools that work without a running registry, and those tools talk to the implementation directly. That is why the recovery tooling is named loregd (loregd --inspector, --recover-from-backup, …): in recovery you are working with the storage implementation, not the registry abstraction. It is the one context where the distinction is visible to an administrator.

Phase 2: the registry-driven boot #

With registryd serving reads, peinit boots the rest of the system from the registry:

1. Read all definitions under Machine\System\Services\. (A registry read timing out here → Recovery.)
2. Build and validate the dependency graph from the boot-triggered services and their transitive dependency closure. Validation runs before anything starts.
3. Start services in dependency order, in parallel up to MaxParallelStarts, with readiness gating releasing each service's dependents as it becomes satisfied.

Only services with a boot trigger are start candidates; demand-only services are pulled in only if something boot-triggered depends on them, and Disabled services are excluded from the graph (but kept in the model for on-demand start). The whole boot runs against one snapshot — mid-boot registry edits do not perturb it.

The platform daemons come up first because everything rests on them. They are all SYSTEM, all minted by peinit (no authd yet), and all ErrorControl=Critical:

Once authd is up, every subsequent service gets its token through the normal authd flow. The order above is emergent from the standard role definitions' dependencies, not hardcoded — change the dependencies and the order changes. In practice login services such as sshd come up last, so the system is fully operational before it starts accepting user sessions.

The three boot modes #

The three modes form an escalation path from normal operation to last-resort maintenance.

flowchart TD
    F["Full boot"] -->|success| OK["operational<br/>counter reset"]
    F -->|cycle/conflict w/ Critical| S["Safe mode<br/>(no reboot)"]
    F -->|Critical failure| RB["sync + reboot"]
    S -->|success| OKS["operational (reduced)<br/>counter reset"]
    S -->|Critical failure| RB
    RB --> CNT["counter increments"]
    CNT -->|counter ≥ N| REC["Recovery mode"]
    F -.->|Phase 1 failure| REC

Full mode #

The default. All boot-triggered services start in dependency order. A boot is successful — and the boot-attempt counter resets to 0 — when every Critical service has reached and held a dependent-satisfying state (Active, Completed, or Skipped) for a grace period (BootSuccessGrace, default 30 s). The criterion is "satisfying," not "Active," so that a Critical Oneshot that reaches Completed can still mark the boot good.

Safe mode #

Safe mode starts a reduced set of boot-triggered services and rebuilds the graph from scratch using only those, dropping dependencies on excluded services. Two categories are eligible:

• Critical services (ErrorControl=Critical) — must start; a Critical failure here still follows the normal reboot path.
• SafeMode services (SafeMode=1) — best-effort; if they fail, Safe mode carries on without them.

Everything else is skipped. Safe mode is entered when:

• graph validation finds a cycle involving a Critical service, or
• an unresolvable conflict involving a Critical service, or
• the kernel command line says peios.safemode=1.

The first two are configuration errors — rebooting would just hit them again — so peinit downgrades without rebooting. Safe mode is not entered by a Critical service crashing at runtime; that follows the reboot path. And it is purely a boot-sequencing concern: once booted, you can start any service by hand exactly as in Full mode. It is the mode for "the configuration is broken but the TCB is healthy."

Recovery mode #

Recovery mode is the last resort, and it offers no TCB guarantee — the administrator gets an unrestricted SYSTEM shell on the console and must treat it with corresponding care. It is a maintenance environment, not a degraded boot.

In Recovery, peinit completes the Phase 1 basics, tries to start registryd (ignoring failure — the shell must appear regardless), skips all Phase 2 services, and execs a SYSTEM shell on /dev/console (/bin/recsh if present, else /bin/sh), respawning it if it exits. It is entered when:

• the boot-attempt counter reaches N (default 3),
• the kernel command line says peios.recovery=1, or
• registryd fails during Phase 1 — entered immediately, with no reboot and no counter increment, because there is no Phase 2 to attempt.

Because the registry itself may be what broke, recovery provides tools that work without it — talking to the loregd implementation directly:

The boot-attempt counter #

The counter is what turns a crash-looping Critical service into an eventual Recovery shell instead of an infinite reboot loop. It is a plain integer in a file at /.peinit/boot-attempts — not in the registry, because the registry may be the very thing that is broken.

• peinit reads it at startup, before choosing a mode. The Recovery threshold (counter ≥ N) is checked against this pre-increment value, so the default N of 3 admits exactly three attempts before Recovery. A missing file counts as 0; a corrupt or unreadable one → Recovery. Override N with peios.bootattempts=N on the kernel command line, or set it to 0 to disable the check when the counter is itself the fault.
• peinit increments it once per boot, right after confirming the root is writable and before Phase 2 — never before the root is known writable, or a read-only root would silently lose the increment and defeat escalation.
• The counter is reset to 0 on a successful Full or Safe boot (after the grace period).
• If the counter file cannot be written (disk full), peinit treats it as 0 and continues — a write failure must not by itself trigger Recovery.

A Critical service exhausting its restart budget — at boot or at runtime — triggers a sync and reboot, which increments the counter on the next boot. Repeat that enough and the counter crosses N, and peinit stops trying and hands you a Recovery shell. The counter deliberately does not try to catch a peinit too broken to reach its own increment; that is a binary-integrity problem, not a boot-loop problem.

Boot configuration #

The kernel command line #

peinit reads four peios.* tokens. They are deliberately few: everything peinit can read after registryd is serving belongs in the registry instead, where it can be inspected, secured and changed without editing a boot entry. What is left is either a per-boot mode decision or a Phase 1 value — one peinit needs before there is a registry to ask.

Console noise: peios.quiet #

peinit reports its progress to /dev/console, and so does any service that claims that terminal with TTYPath. Two writers, one device: a login prompt started while peinit is still narrating gets written over, and the reader cannot tell input from log. peios.quiet decides who yields.

It sets two independent rules, which is why it is a level and not a flag:

The rules stack rather than scale: errors are never less visible at 2 than at 1. An error overrides a requested blackout — silence was a preference, and an error is news — but it does not override terminal ownership, which is not peinit's to override. Only losing the machine (dropping to Recovery, halting with no shell, a Critical service about to force a reboot) is worth one corrupted line of somebody else's session.

prelude honours peios.quiet=2 as well, since it writes to the same console. The other two levels mean nothing there — prelude exits before the first service starts, so no terminal has an owner yet.

A few lines escape all of this: whatever peinit and prelude print before they have read the command line. Neither can honour a preference it has not seen yet, and those lines are also the only evidence either started at all.

Unknown peios.* tokens are ignored, as is a malformed value on either of the two valued tokens — this parser runs before anything exists to report a diagnostic to, and refusing to boot over a typo in a tuning knob is the worse outcome.

Where to start #

To understand the graph validation and parallel start that drive Phase 2, read Dependencies and ordering.

To understand the Critical-failure reboot path that feeds the boot counter, read Keeping services running.

For the trust and token side of early boot, read peinit at PID 1.

Shutdown is boot run in reverse. peinit stops services in reverse dependency order — dependents before the things they depend on — escalating from a polite SIGTERM to a forced SIGKILL, all bounded by a global timeout, and then unmounts, syncs, and performs the final power action. This page covers how it is triggered and how it runs.

How shutdown is triggered #

There are four paths into shutdown, and they are not equal — three are graceful, one is not.

A control command. An administrator with SYSTEM_SHUTDOWN runs peiosctl shutdown <type>:

Signals. As PID 1, peinit treats a handful of signals as shutdown requests:

The power button. On Linux, a physical power-button press is a graceful poweroff — peinit watches the machine's input devices under /dev/input and treats a KEY_POWER press (the key going down, not its release or auto-repeat) as a shutdown request. This path is deliberately minimal and fail-soft: if /dev/input is missing, an input device cannot be opened, or a device stops responding, peinit just keeps running and drops that device — the button quietly stops working, but the machine stays up and you can still shut down over the control socket or with a signal. It is not a full power-management policy engine; a future acpid/logind-style daemon can layer richer policy on top by sending control-socket commands.

A Critical service failure — the ungraceful path. When an ErrorControl=Critical service exhausts its restart budget, peinit syncs and reboots immediately. There is no service-stop ordering: the system is in an undefined state and the fastest route to a known one is a reboot. This is what feeds the boot-attempt counter, and a Critical service that fails identically every boot becomes a reboot loop that the counter eventually breaks by escalating to Recovery.

The graceful sequence #

For a poweroff, reboot, or halt, peinit runs an ordered sequence:

1. Enter the shutdown state. A flag is set, and while it holds: no new services may start, timer triggers are disarmed, and new control commands are rejected with INVALID_STATE — except status queries, which keep working so you can watch progress.
2. Suspend Critical-failure semantics. If a Critical service fails during shutdown, it is logged but does not trigger a reboot — the system is already going down, and rebooting would loop.
3. Clear Completed services. Oneshot services sitting in Completed (with RemainAfterExit) have no process; peinit moves them to Inactive so their dependents can be stopped cleanly.
4. Stop services in reverse dependency order, in waves. Each graceful-stop-eligible service (those in Active or Reloading) gets SIGTERM and StopTimeout seconds to exit; if it does not, peinit SIGKILLs its whole cgroup. A service is never stopped until everything that Requires or BindsTo it has already stopped. Services that are Starting are not stopped gracefully — their startup is cancelled and the cgroup SIGKILLed.
5. Enforce the global timeout. The whole sequence is bounded by ShutdownTimeout (default 90 s). When it expires, all remaining services are SIGKILLed; any whose cgroups do not empty within the post-kill timeout (default 5 s) become Abandoned (leaked), and shutdown continues regardless.
6. Save the random seed. With every service stopped and before any filesystem is touched, peinit writes a fresh seed drawn from the kernel's CSPRNG to /var/state/peinit/random-seed — this is the entropy the next boot starts from. The write is crash-conscious (written to a temporary file, flushed, then atomically swapped in), and it is best-effort: if it fails, peinit logs it and carries on. Shutdown never stalls on it, and the forced and Critical-reboot paths skip it entirely in favour of getting the machine down fast.
7. Unmount filesystems. peinit snapshots the mount table and tries to unmount every remaining non-root mount in its namespace — not just the ones it mounted itself — working from the deepest mount points outward. That includes the Phase 1 set (/proc, /sys, /dev, /dev/pts, /dev/shm, /run, /sys/fs/cgroup) wherever those are still mounted. A mount that is already gone counts as done. If one won't unmount because it is busy, peinit falls back to remounting it read-only; anything it still can't clean up is logged and left for the final sync() and the kernel, never blocking shutdown. The root filesystem (/) is never unmounted — once the rest are handled, peinit remounts root read-only.
8. Sync and finish. sync() to flush pending writes, then the final action — power off, reboot, or halt.

The reverse ordering falls straight out of the dependency graph; there is no separate stop-order configuration. The last services to stop are the TCB daemons everything rests on — eventd, then authd, then lpsd, then registryd dead last, mirroring its position as the first ever started.

STOPPING=1 and timeout extension #

A service that begins shutting down on its own — say, in response to an internal error — can tell peinit by sending STOPPING=1 via sd_notify. peinit acknowledges it and, importantly, stops sending SIGTERM to that service: it is already on its way down, and a redundant signal could interfere.

STOPPING=1 is a courtesy, not a timeout extension. The StopTimeout keeps running; if the process has not exited when it expires, peinit escalates to SIGKILL regardless. A service that genuinely needs longer must say so with EXTEND_TIMEOUT_USEC (see Keeping services running) — and during shutdown those extensions are capped not only by the per-service StopTimeout × 4 but also by the remaining global ShutdownTimeout, whichever is smaller.

Forced shutdown #

When something is wedged and you need the machine down now, repeated SIGINT forces it: three Ctrl-Alt-Del presses within five seconds skip the graceful sequence entirely — SIGKILL everything, sync, reboot. It is the escape hatch for a graceful shutdown that is itself stuck behind an unresponsive service.

Shutdown during boot #

A shutdown requested while peinit is still in Phase 2 boot takes effect immediately: services still Starting are SIGKILLed (they never reached Active, so there is nothing to stop gracefully), services that did reach Active are stopped per the normal sequence, and the boot is abandoned.

How peinit handles signals #

peinit handles all signals through a signalfd read from its event loop — every signal is blocked and read as data, so there are no async signal handlers and no async-safety hazards. PID 1 cannot be killed by any signal; the kernel protects it.

All other signals are ignored.

Where to start #

To understand the reverse ordering and which relationships gate it, read Dependencies and ordering.

To understand the Critical-failure path that bypasses graceful shutdown, read Keeping services running.

For the boot side of the lifecycle and the reboot/recovery escalation, read Boot and boot modes.

Almost every peinit problem is diagnosed the same way: read the state and the cause. A status query gives you both, and the cause taxonomy tells you what the cause means. This page works backward from common symptoms to the cause and the fix.

$ peiosctl status <service>      # state + cause + health + warnings
$ peiosctl list                  # everything you can see, at a glance

For history beyond what peinit holds in memory — past jobs, prior failures, the exact log line a service died on — query eventd; peinit emits every job and operation transition there.

A service won't start #

status shows it Failed or Skipped. The cause says why:

A service keeps restarting #

The service flaps — up, down, up, down. This is restart policy doing its job, but it points at an unstable service.

• It eventually settles in Failed with RestartBudgetExhausted. It crashed RestartMaxRetries times faster than it could stay healthy for RestartWindow. The fix is the service, not the policy — read its logs for the crash. Raising the budget only delays the inevitable.
• It flaps forever and never exhausts the budget. It is briefly reaching Active each cycle and resetting the counter. Either genuinely stabilise it, or — if a health check is failing it after it starts — check the health command; a flaky check restarts a perfectly good service.
• It restarts on a clean exit you did not expect. RestartPolicy=Always restarts even successful exits (cause CleanExitRestart). If the service is meant to exit, use OnFailure instead.

The machine keeps rebooting #

A reboot loop almost always means a Critical service is failing. A ErrorControl=Critical service that exhausts its budget makes peinit sync and reboot; if it fails the same way every boot, you loop.

The boot-attempt counter is designed to break this: after N attempts (default 3) peinit stops and drops you into Recovery mode. From the Recovery shell, find the failing Critical service (its logs are in eventd, which persists across reboots), fix or disable it, and reboot. If you need to intervene before the counter trips, boot with peios.recovery=1 on the kernel command line.

A service is Abandoned #

Abandoned means peinit SIGKILLed the process but it survived — stuck in uninterruptible kernel sleep (D-state), the signature of hung I/O (a dead NFS mount, a failing disk). Nothing in userspace can kill a D-state process.

• The underlying I/O fault is the real problem — investigate the storage or mount, not peinit.
• The service's cgroup is leaked and shows in warnings. A later start uses a fresh generational cgroup, so the new instance is unaffected.
• peiosctl reset <service> clears the Abandoned state and re-checks the cgroup: if the stuck process finally died, peinit cleans up; if not, it stays leaked and warns you. Leaked cgroups clear fully only on reboot.

I changed the config and nothing happened #

peinit works from an in-memory snapshot, and each field has a mutability class that decides when a change applies:

• Immutable at runtime (ImagePath, Type, Identity, RequiredPrivileges, ErrorControl) → takes effect only on restart.
• Apply on next start (dependencies, conditions, OnFailure) → next start or graph reload.
• Reloadable at runtime (timeouts, restart policy, health checks, environment, hooks…) → next time peinit acts on the service.
• Hot-reloaded (ServiceSecurity) → next control request.

If a change is not landing, check its class first. For a wholesale re-read of every definition, run peiosctl reload-config — it rebuilds and validates the whole graph atomically and swaps it in only if validation passes (running services are untouched). And remember peinit pulls changes from registry notifications rather than having them pushed, so there can be a brief lag.

ACCESS_DENIED #

A control command returns ACCESS_DENIED. peinit ran AccessCheck on your token against the target's descriptor and the right was not granted:

• A service command needs the matching right in the service's ServiceSecurity descriptor (start needs SERVICE_START, restart needs both stop and start, …).
• shutdown and reload-config need SYSTEM_SHUTDOWN / SYSTEM_RELOAD_CONFIG in peinit's control descriptor.
• The check uses your effective identity at connect time — if you are impersonating, that is what is checked.

Every denial is logged with the caller SID, target, and requested right. Walk it through Debugging a denial.

A dependent never started #

You started (or booted) a service, but something that depends on it never came up:

• A Requires/BindsTo dependent stays blocked until its target reaches a satisfying state (Active, Completed, or Skipped). If the target is stuck in Starting or Failed, so is the dependent — fix the target.
• A dependent that connected to its target on boot and got connection refused usually means the target uses Readiness=Alive — peinit only waited for the process to exist, not to be serving. Switch the target to Readiness=Notify so it signals READY=1 when actually ready. (peinit logs this as a validation warning at boot.)
• A Wants dependent starts regardless of its target — if you needed it to wait, Wants was the wrong relationship; use Requires.

Booted into Safe mode #

Safe mode starts only Critical and SafeMode=1 services. You land here when graph validation finds a cycle or unresolvable conflict involving a Critical service, or when peios.safemode=1 is on the kernel command line. The TCB is healthy; the configuration is broken.

Fix the graph: the logs name the cycle path (A → B → C → A) or the conflicting pair. Once the cycle is broken or the conflict resolved, a normal boot returns. You can start any service by hand from Safe mode in the meantime — it is only auto-start that is restricted.

Booted into Recovery mode #

Recovery mode gives you a SYSTEM shell on the console and starts nothing else. You reach it when the boot counter hits N, when peios.recovery=1 is set, or when registryd failed in Phase 1 (entered immediately, no counter).

If the registry itself is the problem, use the offline tools that bypass it — they talk to the loregd implementation directly:

$ loregd --inspector               # read the storage directly to diagnose
$ loregd --recover-from-backup     # restore the backup taken each registryd start
$ loregd --dangerously-clear-database   # last resort — wipe; roles re-supply config

Recovery has no TCB protections — it is a SYSTEM shell, full stop. Treat it accordingly, and remember it needs console access (physical, IPMI, or serial); there is no remote recovery yet.

Where to start #

To read states and causes fluently, keep The service lifecycle handy.

For restart, backoff, and Critical-reboot behaviour, see Keeping services running.

For the commands and their outputs, see Controlling services.

This page is the reference catalog for everything peinit reads from or writes to the registry: the full service-definition schema with types and defaults, and every key peinit touches outside the service definitions. For the meaning of each item, follow the link in its row; this page is for looking up a type, a default, or a path.

Registry value types #

Service definition fields map onto registry value types as follows:

The service definition schema #

Every service is a key under Machine\System\Services\<name>; these are the values inside it. Only ImagePath is required. Fields are grouped by purpose; for full semantics see Defining a service and the linked pages.

Execution — see The execution environment

Type and readiness — see Simple and Oneshot services

Activation — see Triggers and timers

Identity — see Service identity and privileges

Dependencies — see Dependencies and ordering

Supervision and health — see Keeping services running

Transition phases — see The service lifecycle and Controlling services

Notify, fds, security, metadata

Service definition keys #

Boot configuration #

Under Machine\System\Boot\:

Operational parameters #

Under Machine\System\Init\:

Keys peinit reads from other subsystems #

Where to start #

For how these fields combine into a working definition and when changes take effect, read Defining a service.

To look up the meaning, valid range, and effect-timing of any key on a live system, use regman.

A mount policy is the per-superblock setting that controls how FACS treats a filesystem. The policy tells the kernel: does FACS apply here at all? When a file has no SD, what should happen? When the kernel synthesises a default SD because the file doesn't have one, where does the template come from? These questions are uniform within a filesystem — every file on a single mount gets the same treatment — and the answer is the mount policy.

There are four policy classes in v0.20. Three are FACS-managed (the kernel applies KACS access control); one is unmanaged (FACS doesn't apply at all). The class is decided when the filesystem is mounted, and changes apply lazily — existing handles are unaffected, but the next access against any file on the mount sees the new policy.

This page introduces the four classes at a glance. Policy classes covers each in detail with the missing-SD and corrupt-SD rules; SD storage by filesystem covers how each filesystem type physically stores the SD; Managing mounts covers the syscalls and the generation counter for cache invalidation.

Why mount policies exist #

A FACS-managed file needs a security descriptor. The DACL on it is the access decision input. The kernel needs to be able to look at any FACS-managed file and find an SD.

But not every filesystem can store an SD on every file. ext4 stores SDs in xattrs, but only up to 4 KB without special features; XFS supports xattrs natively up to 64 KB; tmpfs has no persistent storage at all; FAT and exFAT have no xattr support; remote filesystems like NFS may not surface SDs to the client. The kernel needs different behaviours for these cases.

That's what mount policies provide. The policy is the kernel's per-filesystem answer to "what to do when a file does not have an SD I can read":

• For some filesystems, refuse access (the file should have an SD and doesn't, so something is wrong).
• For some, synthesise an SD on the fly in memory (and don't try to write it back).
• For some, synthesise an SD and try to write it back so future accesses don't need to re-synthesise.
• For some (the kernel's own pseudo-filesystems), don't apply FACS at all.

Each of these is one of the four classes. The class is a property of the mount, not the file.

The four classes at a glance #

unmanaged is not settable via the public ABI — only the kernel itself sets this class, at boot time, for the pseudo-filesystems it manages. The other three are administratively settable.

A mounted filesystem has exactly one of these classes at any time. Changing the class is a single operation that affects every file on the mount going forward (subject to the lazy-invalidation behaviour).

Per-superblock, not per-path #

A mount policy is per-superblock, not per-path or per-bind-mount. All paths visible through the same underlying filesystem instance share the same policy.

The implications:

• A bind mount of /etc/peios to /old/etc/peios does not let you set different policies for the two paths. They are the same superblock; one policy.
• A read-only bind mount and the underlying read-write mount of the same filesystem share the policy.
• Two separate mounts of the same filesystem (uncommon but possible in some namespacing setups) each have their own superblock instance and can have different policies.

The per-superblock rule keeps mount policy simple. The alternative — per-mount or per-path — would mean a single inode could be subject to different policies depending on how it was reached, which would be operationally confusing.

What the policy decides #

For a FACS-managed mount (any of the three managed classes), the policy decides three things:

1. What happens when an inode has no SD attached. Different classes handle this differently — deny, synthesise-in-memory, or synthesise-and-persist.
2. The mount-level SD template. Each FACS-managed mount can carry a default SD template, used when synthesising an SD for a file that has no parent-derivable SD (root of the filesystem, typically).
3. Whether mutations to synthesised SDs are persisted. Ephemeral synthesises in memory only; persistent writes back to the filesystem.

For the unmanaged class, the policy decides one thing: FACS doesn't apply. Files under this mount are governed by whatever per-operation rules the kernel has for that pseudo-filesystem.

What the policy does not decide #

A few clarifications:

• The policy does not change what filesystem operations are available. Read, write, mmap, all the usual operations work normally on every class. The policy affects how the access check runs, not what operations exist.
• The policy does not control persistence of file data. A file's contents are persisted (or not) by the underlying filesystem; the policy is only about SD persistence.
• The policy does not affect already-open handles. The handle model caches access masks on fds; changing the policy does not retroactively change cached masks. Future opens see the new policy.
• The policy does not propagate inheritably. Changing a parent's mount policy doesn't affect mounted-elsewhere child directories — each mount carries its own policy on its own superblock.

Where to start #

If you want each class in detail — what facs_deny_missing actually does at runtime, how synthesize_ephemeral works for FAT, what happens when a synthesised SD turns out to be problematic — read Policy classes.

If you want to know how each filesystem stores SDs — ext4's ea_inode, XFS's native xattr support, NTFS via system.ntfs_security, the no-xattr-support filesystems — read SD storage by filesystem.

If you want the operational story — kacs_set_mount_policy, kacs_get_mount_policy, the generation counter that makes invalidation lazy, why mount policy changes don't walk the filesystem — read Managing mounts.

The commands #

The concept pages above describe the model; three command-line tools put it to work. mount attaches a filesystem to the mount tree and is where a KACS mount policy (and its optional SD template) is chosen at attach time — the one place the policy classes above meet a command line. umount detaches a filesystem again, by mount point or by source, with lazy, forced, recursive and all-targets variants. lsblk lists the block devices available to mount, with their filesystems and, like ls -l, their SD-derived owner and mode. Use these when you want the operational reference for a specific flag rather than the model behind it.

The four policy classes are the answer to one question: when FACS needs an SD for a file and the filesystem doesn't have one, what should happen?

The three managed classes — facs_deny_missing, facs_synthesize_ephemeral, facs_synthesize_persistent — answer the question in three different ways. The fourth, unmanaged, answers it by stepping out of the question entirely: FACS doesn't apply, so the question doesn't arise.

This page covers each class in detail, the synthesis chain that produces an SD when one is needed, and the universal corrupt-SD rule.

facs_deny_missing #

A FACS-managed mount where missing SDs are treated as an error. Every file on a facs_deny_missing mount must have a valid SD; one that does not is unreachable.

The runtime behaviour:

1. The kernel reads the file's SD from the filesystem (typically an xattr).
2. If no SD is present, the access fails. The kernel returns -EACCES for any operation that would require an SD-based access check.
3. The file is effectively unreadable, unwritable, undeletable, unmodifiable. Every access fails because there is no policy to evaluate against.

This is the strict mode. It is appropriate for filesystems where every file should have been provisioned with an SD — system mounts, application storage, anywhere the operator has set up the filesystem deliberately.

The defaults for Peios system mounts (the root filesystem, /home, /var) are facs_deny_missing. The image-build process ensures every file in the base image has an SD; subsequent file creation always inherits an SD from the parent. There is no path by which a file without an SD legitimately ends up on these filesystems.

If somehow a file does end up without an SD — a backup-restore tool that skipped xattrs, a misbehaving filesystem driver — that file is unreachable. The fix is to give it an SD (kacs_set_sd, requires WRITE_DAC, which the caller may not have if they cannot read the file).

Practical upshot: a facs_deny_missing mount is strict. Files without SDs stay unreachable until given one. This is the right setting for filesystems where you trust the provisioning.

facs_synthesize_ephemeral #

A FACS-managed mount where missing SDs are synthesised in memory, but not written back to the filesystem. The synthesised SD is used for the access check; it does not become part of the file's persistent state.

The runtime behaviour:

1. The kernel reads the file's SD from the filesystem.
2. If no SD is present, the kernel synthesises one using the synthesis chain (covered below).
3. The synthesised SD is used for the access check.
4. The synthesised SD is not written back to the filesystem. It exists only in the kernel's cache for the duration this inode is in memory.
5. On a subsequent open of the same file (after the inode has been evicted from cache, say), the SD is synthesised again. The same inputs produce the same output, so the same SD comes out — but the synthesis is repeated.

This class is appropriate for filesystems where you can't or don't want to write SDs:

• Removable media (USB drives, optical media). You don't want to modify the media just because you read a file on it.
• FAT and exFAT. These filesystems have no xattr support; the SD has nowhere to go even if you wanted to write it.
• NFS client mounts. The actual file lives on a remote server; modifying its SD via xattr would have unpredictable effects.
• tmpfs and devtmpfs. Per-instance pseudo-filesystems with no real persistence.

The synthesis-only-in-memory pattern lets FACS apply access control to these filesystems without changing their on-disk content. The trade-off is that the cost of synthesising is paid every time the inode is cold.

facs_synthesize_persistent #

A FACS-managed mount where missing SDs are synthesised and written back. Each file is synthesised once; from then on it has a real SD.

The runtime behaviour:

1. The kernel reads the file's SD from the filesystem.
2. If no SD is present, the kernel synthesises one using the synthesis chain.
3. The synthesised SD is used for the access check — immediately, from the kernel's cache.
4. The synthesised SD is also written back to the filesystem (typically as the SD xattr). The file now has a persistent SD.
5. On a subsequent open, the SD is read from storage — no re-synthesis needed.

The write-back in step 4 does not happen during the access — it is deferred until just after the triggering operation finishes (when the kernel can safely write without holding the locks the access is using). In practice the SD is on disk by the time the syscall that first touched the file returns to userspace, so for an observer it is effectively immediate.

Because the synthesised SD is a deterministic function of its inputs (the parent's SD, or the mount template), the on-disk copy is a cache of a value the kernel can always recompute. If the write-back is interrupted — the inode is evicted first, or the process exits in the gap — nothing is lost: the next access re-synthesises the identical SD and tries the write-back again. A file is never left with the wrong SD, only occasionally with the SD still in memory rather than on disk. (A mount-template change in that gap simply discards the pending SD and re-synthesises against the new template, so a superseded SD is never frozen onto the disk.)

This class is for filesystems being adopted into Peios. A previously-unmanaged filesystem (say, an ext4 volume from a Linux system without KACS) can be mounted with facs_synthesize_persistent; every file accessed gets an SD on first access; over time, every file ends up with an SD.

The use case: migrating an existing filesystem to KACS-managed without a single-pass conversion tool. The synthesis-on-access pattern lets the conversion happen incrementally as files are touched. After enough time, every regularly-accessed file has been adopted; rarely-touched files get adopted the next time they are read.

After all files have SDs, the mount can be switched to facs_deny_missing for strict mode. The transition is a single kacs_set_mount_policy call; existing SDs are unaffected, and any straggler files without SDs become unreachable (which is the desired effect of switching to strict).

unmanaged #

The unmanaged class is the special case for filesystems FACS shouldn't apply to. The kernel uses its own per-operation access rules for files under this mount; no SD-based access check runs at FACS's level.

Specifically:

• /proc is unmanaged. Access to /proc/<pid>/* files is governed by the process-level checks (process SD + PIP, per the two-check rule), not by FACS.
• /sys is unmanaged. Writes are restricted to BUILTIN\Administrators and SYSTEM by hardcoded rule.
• /sys/kernel/security/kacs/* has explicit SDs set by the kernel; reading these uses the kernel's own logic.

The unmanaged class cannot be set via the public ABI. The kacs_set_mount_policy syscall rejects attempts to set this class with -EINVAL. Only the kernel itself sets this class — at boot, for the pseudo-filesystems it manages.

The reason for restricting this: making a regular filesystem unmanaged would mean FACS has no say over it at all, which would be a meaningful operational decision but also a security-relevant one. The kernel reserves the class for its own use.

If you mount a regular filesystem and don't want FACS to apply, the closest you can get is facs_synthesize_ephemeral with a permissive mount template — files get a permissive synthesised SD that effectively grants access. This is not the same as no FACS, but it is the closest path available through the public ABI.

The synthesis chain #

For the two synthesising classes, when the kernel needs to produce an SD for a file with none, it uses a chain of sources. The first source that yields a usable SD wins.

flowchart LR
    A["File has no SD"] --> B["Inherit from parent directory"]
    B -->|parent has inheritable ACEs| F["Use computed SD"]
    B -->|parent has none / no parent| C["Mount-level SD template"]
    C -->|template is set| F
    C -->|no template| D["Hardcoded fallback SD"]
    D --> F

In order:

1. Parent directory inheritance. The kernel reads the parent directory's SD and computes what a newly-created child would inherit (per the Inheritance rules). If this produces a usable SD (with inheritable ACEs from the parent), that SD is used.
2. Mount-level SD template. Each FACS-managed mount can carry a default SD template — set via kacs_set_mount_policy along with the policy class. The template is a complete self-relative SD (max 64 KB). If the parent did not yield an SD, the template is used.
3. Hardcoded fallback. If neither the parent nor the template produces an SD, the kernel uses a hardcoded fallback: GENERIC_ALL to SYSTEM and BUILTIN\Administrators; GENERIC_READ | GENERIC_EXECUTE to Everyone. The owner is set to SYSTEM, group to SYSTEM.

The fallback is conservative: administrators get full control, others get read-and-execute. It is what every filesystem mounted with facs_synthesize_* and no specific configuration falls back to, and what every root-of-mount file ends up with on a freshly-installed Peios system.

For most mounts, the fallback is the safety net — typical files have parents with inheritable ACEs, and synthesis lands on step 1. The template is used for files at the root of the mount (where there's no parent on this filesystem) or in unusual cases where parent-inheritance doesn't apply.

Corrupt SD handling #

A universal rule across all FACS-managed classes: a corrupt SD is treated as a denial. If the SD on a file exists but fails structural validation — bad header, malformed ACL, invalid SID, exceeds the size limit, anything that prevents the parser from making sense of it — the access check fails with -EACCES.

The kernel does not fall back to synthesis when an existing SD is corrupt. The synthesis path is for files with no SD; a file with a broken SD is different. The kernel:

1. Detects the corruption when reading the SD.
2. Returns -EACCES for the access.
3. Emits an audit event (one per inode per cache population — the same corrupt SD encountered repeatedly during one mount's lifetime produces one audit event for that inode, not one per access).

The corruption audit event is part of the audit stream and useful for diagnostics. A misbehaving backup-restore tool that produced corrupt SDs on a restored set of files generates a flurry of these events when the files are first accessed.

The corrupt-SD rule is the same across all three FACS-managed classes. facs_deny_missing denies missing; facs_synthesize_* synthesises missing; all three deny corrupt. The "missing" and "corrupt" cases are different — missing is "no SD was attached" and is recoverable through synthesis or operator intervention; corrupt is "the SD that was attached is broken" and requires either fixing the SD or accepting denial.

Class transitions #

Changing a mount's policy class is allowed (subject to access rules covered in Managing mounts). The interesting transitions:

• facs_synthesize_persistent → facs_deny_missing. Common during migration. As files are adopted by synthesize_persistent, they accumulate SDs. Once enough have been adopted, switch to deny_missing to enforce strictness. Any file still missing an SD becomes unreachable; an administrator can either accept that or set SDs on the holdouts.
• facs_synthesize_ephemeral → facs_synthesize_persistent. Less common. Would convert an ephemeral mount to a persistent one — the next access of any file without a stored SD would write the synthesised SD back. This effectively "snapshots" the synthesis on first access.
• facs_deny_missing → facs_synthesize_*. Unusual; would relax strictness. The kernel allows it but the operational reasoning is rarely good — synthesis is a recovery mechanism, not an everyday setting.

A class transition is just a policy update; no files are touched at the transition. The new policy applies to future accesses. Existing cached state (synthesised SDs in memory) may need to be re-evaluated — see the generation counter in Managing mounts.

Where to go next #

For how each filesystem physically stores the SD, read SD storage by filesystem.

For the syscalls that set and read a mount's policy, read Managing mounts.

For choosing a policy from the command line at attach time, read mount.

A security descriptor is structurally a single piece of metadata that has to live somewhere. Different filesystems have different ways to hold metadata; some have built-in security-attribute support (NTFS), some have generic extended attribute support that fits (ext4, XFS, Btrfs), some have neither (FAT, exFAT). The mount-policy class you choose for a filesystem depends partly on what the filesystem can store and partly on what behaviour you want.

This page covers the storage mechanism each filesystem type uses and the implications.

The canonical xattr #

For filesystems that support extended attributes, Peios stores the SD in the security.peios.sd xattr. The name is in the security.* namespace, which requires privileged access to read or write at the filesystem layer — which doesn't matter for FACS-managed access (FACS denies direct xattr operations on the SD xattr unconditionally; access goes through kacs_get_sd / kacs_set_sd), but does mean the xattr survives operations that respect security xattrs.

For NTFS, the xattr name is system.ntfs_security — the native NTFS security attribute, accessed through the same xattr interface. Using the NTFS-native name means SDs round-trip cleanly between Peios and other operating systems that read NTFS.

The choice of xattr name is filesystem-specific:

For other filesystems (with no xattr support or no convention), the SD is held in memory and re-synthesised on each cold open. See the per-filesystem sections below.

ext4 #

ext4 supports xattrs natively. SDs up to about 4 KB fit in the inline xattr space; larger SDs require the ea_inode feature.

ea_inode is an ext4 feature that lets a single xattr value spill into a dedicated inode rather than being limited to the size of the inode's inline xattr area. With ea_inode enabled at filesystem creation time (or enabled later via tune2fs), an SD can be up to the standard 65,535-byte SD limit.

mke2fs on Peios enables ea_inode by default for ext4, and raises the inode size to 512 bytes so that a typical SD fits inline in the first place — see mke2fs. Real-world SDs rarely exceed the inline xattr space, but pathological cases (deeply-nested ACEs, lots of inherited ACEs from a complex parent) can push past 4 KB; ea_inode accommodates them.

If an ext4 filesystem without ea_inode is mounted on Peios and a file has an SD larger than its inline xattr space could hold, the filesystem's xattr write would fail. The kernel handles this by either:

• Refusing the kacs_set_sd call with an error indicating the filesystem cannot hold the SD.
• Or, in the facs_synthesize_persistent case, refusing the write-back and treating the file as if it has no SD (re-synthesising next time).

The cleanest fix is to enable ea_inode. For everyday use, the inline xattr space is sufficient.

XFS #

XFS supports xattrs natively, up to 64 KB per attribute by default. This is comfortably more than the SD size limit (65,535 bytes), so SDs fit without any special configuration.

XFS is a fine choice for FACS-managed filesystems. No ea_inode-style consideration is needed; SDs just work.

Btrfs #

Btrfs supports xattrs natively. Small xattrs are stored inline; larger ones get their own extent. SDs of any practical size fit without special handling.

Btrfs's snapshot and clone behaviour is interesting for SDs: a Btrfs snapshot of a directory includes the SDs of every file within. A snapshot is a point-in-time view; the SDs in the snapshot reflect what they were when the snapshot was taken. If the original files' SDs are subsequently modified, the snapshot still has the old SDs. This is the expected behaviour but worth noting for backup/snapshot workflows.

tmpfs and devtmpfs #

tmpfs is an in-memory filesystem. It supports xattrs, but everything it stores lives in RAM and is lost on unmount or reboot. SDs on tmpfs are present while the filesystem is mounted; they vanish when it is unmounted.

tmpfs is typically mounted with facs_synthesize_ephemeral. The synthesis happens in memory anyway (the tmpfs storage is memory), so there's no operational difference between "store an SD in the tmpfs xattr" and "synthesise an SD into the kernel's per-inode cache".

devtmpfs is the kernel-managed filesystem for device nodes. SDs on devtmpfs are applied by udev rules, not by FACS-driven synthesis. The udev daemon (or its Peios equivalent) sets the SD on each device node as the node is created. This is a different model from the synthesis-based pattern other filesystems use; it works because the device-node population is controlled centrally and the SDs can be set deterministically.

devtmpfs uses the facs_synthesize_ephemeral class, but in practice the synthesis path is rarely hit because udev provides the SDs.

NTFS — round-trip via ntfs3 #

NTFS is the Windows-native filesystem. Peios mounts it through the ntfs3 kernel driver, which exposes the NTFS security attribute via the standard xattr interface under the name system.ntfs_security.

The implications:

• SDs written by Peios to an NTFS volume use the same on-disk format as Windows. A Windows system reading the same volume sees the SD natively.
• SDs written by Windows to an NTFS volume are readable by Peios. Round-tripping a volume between the two operating systems preserves SDs.

This is the "binary-compatible" property the SD format gives. The wire format is the same; the filesystem driver translates between the xattr interface and the on-disk security attribute.

NTFS volumes are typically mounted facs_synthesize_ephemeral rather than facs_deny_missing. The reasoning: an NTFS volume from Windows may have files without Peios-recognised SDs (the SDs are Windows-native and may use principals that don't exist on the Peios system). Ephemeral synthesis lets such files be accessible without modifying the volume's stored SDs.

FAT and exFAT #

FAT and exFAT have no xattr support at all. There is no place to store an SD; the on-disk format simply doesn't have the metadata channel.

FACS handles this by using facs_synthesize_ephemeral for FAT/exFAT mounts. Every file gets a synthesised SD in memory; no SD is ever written back. The synthesised SD applies for the file's time in the inode cache; when the file is evicted, the SD is gone and will be re-synthesised on next access.

This means FAT/exFAT files cannot be given persistent KACS-style permissions. The synthesised SD is the same every time (assuming the same inputs — parent SD, mount template), so the access decision is deterministic, but there is no way to customise per-file.

For most FAT use cases this is fine — FAT is typically used for removable media or boot partitions where uniform-permissions semantics is acceptable. The mount-level SD template can be configured to grant whatever access pattern is appropriate for the mount as a whole.

NFS — synthesise locally, enforce remotely #

NFS client mounts are a unique case. The actual files live on a remote server; the local filesystem driver is a network protocol implementation, not a real filesystem.

The mount class is facs_synthesize_ephemeral. FACS synthesises a local SD per the synthesis chain (typically yielding a sensible default from the mount template) and uses it for local access control. The local FACS check decides whether to forward the operation to the server.

If the local check passes, the operation goes to the server. The server has its own access control (potentially also Peios with FACS, potentially another OS with different rules); the server's access control decides whether the operation actually proceeds. If the server denies, the operation fails with whatever error the protocol returns (typically -EACCES or -EIO).

This is "dual authority" — both client and server have a say. The client's denial is local-final; the server's denial is remote-final. There is no single source of truth for the access decision.

The implications were covered in Special cases under "NFS — dual authority": don't trust local FACS results for security, expect I/O errors from server-side denial, the local synthesised SD is not the server's actual SD.

/proc and /sys #

/proc and /sys are kernel pseudo-filesystems. Their mount-policy class is unmanaged — set by the kernel at boot, not changeable via the public ABI.

/proc doesn't have an SD per-file in the FACS sense. Access to /proc/<pid>/* is gated by the per-process rules (process SD + PIP, from the two-check rule). The kernel implements these checks directly when serving /proc file operations.

/sys similarly has its own per-file rules. The /sys/kernel/security/kacs/* entries have explicit SDs the kernel maintains; other /sys files have hardcoded rules ("writes restricted to BUILTIN\Administrators and SYSTEM").

The unmanaged class is what tells FACS to not interfere with these. The kernel knows what it is doing with its own pseudo-filesystems; FACS stays out of the way.

Stacked filesystems — overlayfs and StrataFS #

A stacking filesystem presents files that physically live somewhere else. overlayfs merges a read-only lower layer with a writable upper one; StrataFS composes several directories into one view. Neither stores a security descriptor of its own, and neither needs to: the descriptor belongs to the file, and the file is on the layer underneath.

So a stack forwards. When the kernel needs the SD of a file in the merged view, it asks the layer that actually holds it, and what comes back is that layer's effective descriptor — which is to say, whatever an access check on the underlying file would have used. If the layer underneath stores an SD, the stack sees the stored one. If the layer underneath synthesises, the stack sees the synthesised one. Synthesis composes upward through the stack; it is not a private arrangement between the kernel and the bottom layer.

That is what makes the common live-boot arrangement work:

overlay          <- the merged root, facs_deny_missing
  upper: tmpfs   <- writable scratch, stores real SDs
  lower: squashfs <- read-only image with no SDs, facs_synthesize_ephemeral

The overlay stores nothing and can still be facs_deny_missing, because neither layer beneath it ever answers "missing": the tmpfs has stored descriptors, and the squashfs synthesises one for every inode.

Two consequences worth holding onto:

The stack's own policy class governs only what happens when every layer beneath it has nothing to offer. Pick it for that case. Over a synthesising lower, facs_deny_missing is the strict-and-correct choice; over an unmanaged lower it would lock the whole view.

A copy-up does not carry the lower's descriptor up with it. When overlayfs copies a file from the lower layer to the upper to make it writable, it creates a genuinely new inode in the upper, and that inode gets its descriptor by ordinary inheritance from its parent in the upper — not by duplicating the lower's. This is deliberate: security.peios.sd is not userspace-writable (SD mutation is kacs_set_sd's job), so a verbatim copy could not be performed even in principle, and the inherited answer is the right one anyway. Every other extended attribute is copied up normally.

Summary #

The pattern: real on-disk filesystems with xattr support get one of the facs_* classes depending on policy needs. Pseudo-filesystems and the kernel's own filesystems are unmanaged. Filesystems without xattr support default to ephemeral synthesis as the only viable mode. Stacking filesystems store nothing and defer to the layer beneath.

Where to go next #

For what each policy class does with a missing SD, read Policy classes.

For setting and reading a mount's policy at runtime, read Managing mounts.

For the structure of the SD being stored, read Security descriptors.

A mount's policy is set via a syscall, read via a syscall, and otherwise sits as kernel-internal state on the filesystem's superblock. Mount-policy changes are administrative — they require SeTcbPrivilege — and they take effect lazily, without the kernel walking the filesystem to update anything.

This page covers the two syscalls (kacs_set_mount_policy, kacs_get_mount_policy), the generation counter that makes lazy invalidation work, and what does and does not happen at a policy change.

kacs_set_mount_policy #

The write syscall:

result = kacs_set_mount_policy(fd, args)

Where fd is a file descriptor referring to any object on the target superblock (the kernel uses the fd to identify which superblock — the file itself does not need to be the root of the mount). args is a kacs_mount_policy_args struct:

The kernel:

1. Validates the fd and identifies the superblock.
2. Validates the policy value. Attempts to set KACS_MOUNT_POLICY_UNMANAGED (which is not settable via this ABI) are rejected with -EINVAL.
3. Validates the template SD if provided — must parse, must be within size limits.
4. Checks the caller's privileges. SeTcbPrivilege is required.
5. Atomically updates the superblock's policy class, the template SD, and increments the generation counter.
6. Writes the new generation to args.generation.
7. Returns 0.

The change is atomic at the superblock level — every future access against any inode on this superblock sees the new policy. There is no transitional state where some inodes are under the old policy and others under the new.

Privilege requirement #

The reasoning: changing a mount's policy is an administrative decision that affects the entire filesystem's access semantics. It belongs in the TCB tier, not in the regular-administrator tier. An ordinary administrator who wants to change a mount's policy goes through a tool that itself has the privilege.

The template SD #

The optional template SD is the default SD used during synthesis when the parent's inheritance does not yield one. The kernel stores the template on the superblock; subsequent synthesis-on-missing calls use it.

The template is a complete self-relative SD — owner, primary group, DACL, optional SACL. The kernel validates the template at policy-set time:

• Must be in self-relative format.
• Must include an owner (an SD without one is malformed and rejected).
• Must fit within 65,535 bytes total; the template specifically is capped at 64 KiB.
• ACLs and ACEs must parse.

A bad template at policy-set time is rejected with -EINVAL; the current policy and template are unchanged.

The template can be omitted entirely (zero pointers) for mounts that don't need one. The synthesis chain then falls through to the hardcoded fallback for any file whose parent doesn't yield an SD.

kacs_get_mount_policy #

The read syscall:

result = kacs_get_mount_policy(fd, args)

Same fd and args semantics, returning the current policy class, current template (if buffer provided), and current generation counter.

The kernel:

1. Validates the fd, identifies the superblock.
2. Checks the caller's privileges. SeTcbPrivilege is required to read.
3. Writes the policy class, generation, and template to args (template only if a buffer was provided).
4. Returns 0.

Like the write syscall, this is SeTcbPrivilege-gated. The mount policy is system-level state, exposed only to TCB-tier callers.

This is the kernel's read-back-the-current-policy interface. A management tool reads the policy, perhaps applies a transformation, and writes the new value back. The read-modify-write pattern is what most policy management code uses.

The generation counter #

A subtle but important feature: the kernel maintains a monotonic generation counter per superblock. Each successful kacs_set_mount_policy (or template change) increments it.

The purpose is lazy invalidation.

Without the counter, a policy change would either need to immediately propagate to every cached SD on the filesystem (expensive — could be millions of inodes) or accept that cached SDs from before the change remain valid (incorrect — the new policy might say different things).

With the counter, the kernel can do something smarter:

1. Each in-kernel cached SD or synthesised entry records the generation it was derived from.
2. At access time, the kernel compares the cached entry's generation against the superblock's current generation.
3. If they match, the cached entry is current; use it.
4. If they differ, the cached entry is stale; discard it and re-derive.

The kernel does not walk the filesystem at the policy change. The walk happens implicitly, one inode at a time, as accesses arrive. An access that arrives 30 seconds after the policy change is the first to invalidate that specific inode's cached state; the access pays a small cost to re-derive, but every subsequent access uses the new cached state.

This is why mount policy changes are cheap to apply. The expensive work (walking inodes) never happens; what happens instead is per-inode re-derivation on first access after the change.

What the generation counter affects #

The counter affects in-kernel cached state derived from the mount policy:

• Missing-SD synthesised entries. If a file had a missing SD and was synthesised on-the-fly, the synthesised SD is cached. A policy change invalidates this cache; the next access re-synthesises.
• Ephemeral-synthesis caches for facs_synthesize_ephemeral mounts. Same.
• Template-derived information — the kernel's representation of the mount template is regenerated on next access.

The counter does not affect:

• On-disk SDs. A file that has a stored SD continues to have that SD. The policy change doesn't rewrite stored SDs.
• Already-open file descriptors. The cached granted mask on an fd is independent of the mount-policy generation; it's the result of the access check at open time. Policy changes after open don't affect open fds. (This is the standard check-at-open model.)
• Inodes not currently in cache. They have nothing to invalidate; the next access just goes through the normal path with the current policy.

Reading the counter #

The generation counter is exposed in the output of kacs_get_mount_policy. A management tool can read the counter to know when policy was last changed, or compare against a previously-read value to detect external changes.

The counter is per-superblock; different filesystems have unrelated counter values. The counter starts at some implementation-defined value at mount time and increases monotonically with each policy change.

What happens at a policy change #

To summarise, when kacs_set_mount_policy succeeds:

1. The superblock's policy class field is updated.
2. The superblock's template SD is updated.
3. The superblock's generation counter is incremented.
4. Open file descriptors against files on this mount are unaffected (cached granted masks intact).
5. Cached in-memory state derived from the old generation will be invalidated on first access after the change.
6. Files with stored SDs continue to have those SDs.
7. The filesystem itself is not walked, not modified, not reorganised.

The change is essentially a small superblock update plus a lazy invalidation marker. The work happens at future accesses, distributed across actual usage.

What policy changes do not do #

A few clarifications:

• They don't rewrite SDs on disk. A change from facs_synthesize_ephemeral to facs_synthesize_persistent doesn't go back and write synthesised SDs to disk. Files that were synthesised under the old policy continue to have no on-disk SD; the next access re-synthesises (now writes back, per the new persistent policy).
• They don't close open files. Processes with handles to files on the mount continue to have those handles. The cached granted masks are unchanged.
• They don't propagate across mounts. A policy change on one mount doesn't affect any other mount, even if they reference the same underlying device (per the per-superblock rule).
• They don't change the FACS handle model. The new policy applies to future access checks; existing handles use their cached masks per usual.

Use patterns #

A handful of common patterns for managing mount policy:

Boot-time policy. peinit applies the policy class and template for each mounted filesystem at boot. The configuration source is typically registry-based; peinit reads the configuration and calls kacs_set_mount_policy for each mount.

Migration from a non-Peios filesystem. Mount with facs_synthesize_persistent initially. Let usage gradually adopt files into having stored SDs. Once enough have been adopted, switch to facs_deny_missing (or leave it as synthesize_persistent if you want the synthesis-on-missing safety net).

Removable media insertion. When a removable filesystem is mounted (a USB drive, an optical disc), the mount setup chooses facs_synthesize_ephemeral. The volume's contents become reachable without modifying the on-disk metadata; ejection cleanly leaves the media untouched.

Policy uplift on a hardened deployment. A deployment starting with facs_synthesize_ephemeral everywhere (permissive) can migrate to facs_synthesize_persistent (filling in stored SDs) and then to facs_deny_missing (strict mode). Each transition is one kacs_set_mount_policy call per mount.

Errors #

Possible failures from kacs_set_mount_policy:

Possible failures from kacs_get_mount_policy:

In normal operation both calls succeed. Failures are typically privilege issues or malformed inputs.

See also #

• Policy classes — what each settable class does.
• mount — setting the policy from the command line at attach time.
• Privileges — the privilege model behind the SeTcbPrivilege gate.

mount attaches a filesystem to the Peios mount tree. With no operands it instead lists what is currently mounted. It is a faithful reworking of the util-linux mount(8) surface for Peios, with two structural differences: Peios has no /etc/fstab and no /etc/mtab, so every mount is described entirely on the command line and live mount state is read from the kernel; and a mount can apply a KACS mount policy to the new filesystem at attach time (see Mount policies).

mount [-t TYPE] [-o OPTIONS] SOURCE TARGET
mount [-o remount,OPTIONS] TARGET
mount --bind|--rbind|--move SOURCE TARGET
mount --make-shared|--make-private|... TARGET
mount [-l] [-t TYPE]

Internally mount uses the fd-based mount API (fsopen / fsconfig / fsmount / move_mount, plus open_tree and mount_setattr), never the classic single-shot mount(2). This matters in one place you can see: when a mount fails, the kernel's fs_context message log is drained and printed as the reason, even without -v.

Operands and argument shapes #

Because there is no fstab to consult, operand resolution is strict:

• No operands, no verb — list mode (see below).
• Two operands — SOURCE TARGET.
• One operand — an error. In util-linux a lone operand is resolved through fstab; Peios has none, so a single operand cannot be turned into a SOURCE TARGET pair. Supply both, or use --source / --target.
• A lone target is valid only for -o remount and for a standalone propagation change (--make-*), which act on an existing mount point.

--source SRC and --target DIR name the operands explicitly and may be combined with a single positional. --target-prefix DIR prepends DIR/ to the target after it is chosen. Options may be interspersed with operands (mount SRC -o ro TGT), matching util-linux.

Paths, -o key=value values, labels and UUIDs are handled as opaque byte strings, never assumed to be UTF-8; an embedded NUL is a usage error.

Canonicalisation #

By default source and target paths are canonicalised (made absolute, symlinks resolved). -c / --no-canonicalize disables that; X-mount.nocanonicalize[=source|target] is the -o form and can disable it for just one side. The final path handed to the kernel is protected against a TOCTOU symlink swap on its last component.

Operation modes (verbs) #

mount performs one of several operations. The structural verbs — bind, rbind, move, beneath and remount — are mutually exclusive with one another. A propagation change is not a structural verb: it may stand alone on a target, or trail another verb or a new mount in the same command (applied afterwards as one or more mount_setattr steps), and several may be combined.

Propagation flags #

Each of these sets the propagation type of the mount at the target. The --make-r* (and r-prefixed -o) forms apply recursively to the whole subtree; the recursive form is all-or-nothing (it either applies to the entire subtree or to none of it).

A freshly created mount, bind or move is private by default unless its destination's parent is shared, in which case it joins that peer group — the standard kernel rule.

Source specification #

Filesystem type #

-t TYPE names the type explicitly. -t auto, or omitting -t entirely, asks libblkid to safely probe the source; an ambiguous or multiply-signed device is refused (exit 32) rather than guessed at. X-mount.auto-fstypes=LIST constrains the probe candidates. Type lists and no<type> negation are not accepted in the mounting path (they remain meaningful only as a listing filter).

The -o option language #

-o takes a comma-separated list of key or key=value tokens and is repeatable (all occurrences are joined). A key="..." double-quoted value protects embedded commas; the key=value split is on the first =. Every token is sorted into one of the following categories.

Per-mount attributes #

Applied to the mount itself. Each accepts its negation; ro/rw additionally accept a =vfs / =fs / =recursive scope qualifier (bare is =vfs, non-recursive; =fs targets the superblock read-only flag; =recursive applies to the whole subtree).

The atime tokens share a single mode field; the last one specified wins.

Superblock flags #

Filesystem-specific parameters #

Any token not recognised above is forwarded verbatim to the filesystem: key=value as a string parameter, a bare key as a flag. There is no built-in allow-list and no length limit; a rejection by the filesystem is reported with the offending key and the kernel's message.

For example, StrataFS takes its precedence-ordered directory stack through strata=:

mount -t stratafs none /bin -o 'strata=/lcl/bin+create:/usr/bin+ro'

See StrataFS for the stack flags, merge and write-routing model, and the stratafs inspection command.

Userspace-only tokens #

These never reach the filesystem. They include the meta-verbs remount, bind, rbind, move; the loop controls loop / loop=/dev/loopN, offset=, sizelimit= (numeric values accept K/M/G/T and KiB/MiB/… suffixes); the propagation tokens listed above; and defaults, which expands to rw,suid,dev,exec,async (later tokens override it).

Functional X-mount.* options #

X-mount.idmap and X-mount.owner / group / mode are not supported and are rejected with a usage error: Peios ownership is SID/security-descriptor based, so the fix is to add a SID/ACE to the descriptor rather than remap or chown.

KACS mount policy #

This is the genuinely Peios-specific part of mount. A mount policy is a per-superblock setting that governs how FACS treats the filesystem; the full model is described in Mount policies and its detail pages. mount can set that policy at attach time:

policy=unmanaged is not user-settable; only the kernel sets the unmanaged class, for its own pseudo-filesystems. policy= is valid only on a new mount of a real filesystem — combining it with bind/move/remount/propagation, or with list mode, is a usage error. See Policy classes for what each class does and SD storage by filesystem for how the SD is physically stored.

The policy is applied to the detached filesystem before it is attached: if setting it fails, nothing is ever published with an unintended policy (no rollback needed). Because setting a mount policy is a SeTcbPrivilege-gated operation (see Managing mounts), a caller without that privilege gets a clean EPERM (exit 1) and no mount. --synth-sddl is validated client-side first — it must be well-formed SDDL and must include an owner.

Peios applies no coarse uid==0 check anywhere: the mount applet is not installed set-user-ID, and every privileged action is authorised per-operation by KACS.

Remounting #

-o remount changes the options of an existing mount without detaching it. Peios does not perform the util-linux "read the old flags and re-supply them" dance — the fd-based API applies deltas rather than resetting, so each remount touches only what you name. Per-mount attributes (ro/rw, nosuid, atime, …) are changed via mount_setattr; superblock flags, filesystem parameters and ro=fs/rw=fs go through the superblock reconfigure path. =recursive remounts the whole subtree, all-or-nothing.

A bind mount shares its source's superblock, so any superblock-level option on a bind remount (a Category-B flag, a filesystem parameter, or ro=fs/rw=fs) is refused as a usage error rather than silently mutating the shared superblock for every mount of that filesystem. Only per-mount VFS attributes are valid on a bind remount.

Loop devices #

A regular-file source is backed by a loop device automatically when the type is unspecified or the filesystem is recognised by libblkid; X-mount.noloop suppresses this. -o loop forces auto-allocation of a free device, loop=/dev/loopN names one, and offset= / sizelimit= select a region of the file (they imply loop and are an error against a real block device). A backing file already attached at the same offset and size is reused rather than doubly attached, to avoid corruption. Loops created by mount are auto-cleared by the kernel on unmount, so they do not leak; umount -d force-clears the rest (see umount).

Other flags #