Peios Learn
Products
PePeios pkpekit PvProvium UDUniversal Directory TrTrail Ckclackatoo PrProject WiWispist
Using Peios Security Basics Technical Documentation Source
Using Peios Security Basics Technical Documentation Source
Peios

peinit

Technical Reference Manual — the init system and service manager: boot, the service model, identity, supervision, jobs and operations, and shutdown.

Single-page view · as markdown

1.1 Overview

Peios / Peios technical reference / peinit / Introduction

peinit is PID 1. It is the only service manager on a Peios system: every supervised process on the machine — a platform daemon, an application service, a startup hook, a health probe, a job some other service asked for on a user's behalf — is forked by peinit and watched by peinit until it exits.

It is a single-threaded Rust process. That is the constraint the rest of its design answers to. A blocking syscall in PID 1 stops everything: child reaping, watchdog expiry, shutdown signals, the control socket. So peinit keeps a complete in-memory model of every service it knows about, reads the registry synchronously only at boot and when it reloads — whether an administrator asked for the reload or a registry change notification set it off, which takes the same path (§10.4) — and pushes anything that could block off the main loop into a forked helper or a pollable descriptor.

1.1.1 What makes it not systemd #

Two things, and both run deep enough to change the shape of the daemon.

Services are securable objects. A service carries a Security Descriptor that says who may start it, stop it, query it, or reload it, and peinit evaluates that descriptor against the caller's KACS token on every control request. There is no "root can do anything" path, because there is no root — there is a token, and AccessCheck is the only thing that decides.

Service identity is a token, not a user. peinit never sets a UID, a GID, or a Linux capability on a service process. It obtains a KACS token — minted from its own for the platform daemons that start before an authority exists, requested from authd for everything else — restricts its privileges to what the definition asked for, and installs it on the child before exec. Every service also carries a per-service SID derived from its name, so two services sharing an identity are still distinguishable to an access check.

Configuration follows from the second one: service definitions live in the registry, under Machine\System\Services\, where they are protected by the registry's own descriptors rather than by file permissions. There are no unit files, no generators, and no translation layer.

1.1.2 The shape of the daemon #

Boot is two phases with registryd as the boundary. Phase 1 is compiled in and has no registry dependency at all: it confirms the root is writable, mounts what is missing, restores entropy, settles the clock, and starts registryd. Phase 2 reads the service graph out of the registry and boots the system from it. When Phase 1 cannot complete, there is no Phase 2 to fall back to, and peinit drops into a recovery shell.

Once booted, peinit is an event loop over a handful of descriptors: a signalfd, the control socket, the jobs socket, the notify socket, one timerfd per armed timer, a pidfd per supervised process, a pipe pair per supervised process's output, and the registry's change-notification descriptor. Work arriving on most of them turns into an operation — a queued, observable request to move a service through its state machine — and executing an operation eventually forks a job. Work arriving on the jobs socket skips the operation: a submitted job has no service and no state machine to contend for, so a submission forks a job directly.

Those two objects are how peinit stays comprehensible under concurrency. Operations exist so that two administrators issuing conflicting commands get a defined answer instead of a race. Jobs exist so that "what actually ran" is a thing with an identifier, a token summary, an exit status and a log correlation key, rather than a PID that may already have been reused.

peinit keeps no history of either. A job or an operation that reaches a terminal state is emitted as a structured event into the KMES kernel ring buffer and dropped. eventd is the historian.

1.1.3 What peinit is not #

It does not assemble storage. The initramfs delivers a mounted, writable root, and peinit neither decrypts, nor assembles, nor checks it. It has no mount feature beyond the fixed Phase 1 set — mounting a data partition is a Oneshot service's job.

It does not store logs. It holds the pipes at birth, tags each line, and forwards it to eventd; before eventd exists it buffers, and when the buffer fills it drops the oldest.

It does not authenticate anyone. authd mints tokens; peinit installs them. It does not resolve identities, and it does not know or care whether a principal is local or from a domain.

And it does not support forking daemons. It tracks the process it spawned, through a pidfd obtained at fork, and there is no mechanism for a service to point supervision somewhere else.

1.2 What This Manual Covers

Peios / Peios technical reference / peinit / Introduction

This manual describes peinit as it is built: the boot sequence, the service model and its registry schema, how a service acquires its identity, the exact sequence between "start this" and "the binary is running", the state machine and its causes, dependencies, jobs and operations, timers, output handling, shutdown, and the security model.

1.2.1 What is a contract and what is not #

Three of peinit's interfaces are specified separately. PSPU §4 gives the control socket, spoken by administrative tools and by any program that manages services or observes jobs, and the notification socket, spoken by every service and every submitted job that reports readiness, status, keepalives, or a stored descriptor. PSPU §7 gives the jobs socket, spoken by any process that asks peinit to run and supervise a program on its behalf. Those are contracts. A third party implements one side of each, and what is written there binds both.

This manual covers the other side of that boundary — how peinit fulfils them, and everything that is not a contract at all:

  • how a command becomes an operation, and what happens when two of them collide (§8.2, §8.3)
  • what a command does to a service in each state (§10.3)
  • how a notification is authenticated and applied (§10.5, §10.6)
  • how a submission becomes a running job, and what it is checked against afterwards (§8.5, §10.7)

Where a chapter touches the contract it references PSPU §4 rather than restating it.

The service definition schema is here rather than in PSPU. It is registry configuration, protected by the registry's descriptors and administered by the same tools as the rest of the registry, and it is documented for the people who write service definitions rather than specified as a wire format. §3.2 is the reference.

1.2.2 What this manual does not cover #

  • KACS — tokens, Security Descriptors, AccessCheck, and the per-service SID algorithm peinit reproduces. Peios Kernel TRM §3.
  • LCS — the registry syscalls, watches, and layer resolution peinit reads through. Peios Kernel TRM §5.
  • KMES — the ring buffer peinit emits its events into. Peios Kernel TRM §2.
  • loregd — the storage behind registryd. Its own TRM.
  • eventd — where the logs and events go. Its own manual.
  • authd — token minting and identity routing. peinit's requirements of it are described in §4.3; its interface is authd's own.
  • How a token is conveyed on a socket, and what the kernel checks when it is — peer-token capture, KACS_SCM_TOKEN, the impersonation gates and the level ratchet that job submission relies on. Peios Kernel TRM §3.5.
  • Using peinit — writing service definitions, the svctl command surface including svctl job, and everyday administration are covered in Using Peios.

1.2.3 Constants and keys #

Registry keys and compiled-in constants are collected in the appendices, so a chapter's reference material does not interrupt the prose that explains it. Where an appendix defines a value the body references it rather than repeating it.

1.3 Terminology

Peios / Peios technical reference / peinit / Introduction

Terms defined where they are introduced — activation generation, boot generation, transition cause, cgroup generation, start generation — are not repeated here.

Service. A named, supervised unit of execution: a definition in the registry, a runtime state, a Security Descriptor, and at most one running main process. Services are the primary unit of management and the thing dependencies are expressed between.

Job. One supervised process execution. Every fork peinit performs is a job — a service's main binary, a pre-exec hook, a health check invocation, a submitted job — with a GUID, a lifecycle, a token summary and a log correlation key. Jobs are the observable unit of what actually ran.

Submitted job. A job whose definition arrived on the jobs socket rather than from a service definition. It has no service, no operation and no policy; it belongs to its submitter and is managed under its own Security Descriptor. §8.5.

Submitter. The identity a jobs socket connection carried when a submit was read: recorded on the job, counted for the quota, and the owner of the job's descriptor.

Job identity. The identity a submitted job's process runs as — the submitter's own primary token, or a token the kernel attached to the submit message. Never named by a field. §8.5.

Operation. A first-class object representing a requested state machine action on a service. Control commands do not mutate state directly; they create operations that are validated, queued, resolved against whatever else is in flight, and executed by the event loop.

Trigger. A rule in a service definition saying when the service starts automatically: at boot, once the boot set has settled, or on a schedule. A service with no triggers starts only when something asks for it.

Phase 1. The compiled-in bootstrap: root writability, the remaining virtual filesystems, the persisted random seed, the local machine ID, the clock, registryd, boot-time path provisioning, and infrastructure setup. No registry access happens before registryd is serving.

Phase 2. The registry-driven boot: read the service definitions, build and validate the dependency graph, and start the boot-triggered services in dependency order.

ErrorControl. The per-service policy for an irrecoverable failure. Normal leaves the service Failed; Critical syncs the filesystems and reboots.

Token. The per-thread KACS identity object — a user SID, group SIDs, a privilege bitmask, an integrity level, and metadata. Tokens are the sole identity mechanism: peinit sets no UIDs, GIDs, or Linux capabilities on service processes. Peios Kernel TRM §3.2.

Security Descriptor (SD). The KACS structure controlling access to a securable object. peinit evaluates three kinds: a ServiceSecurity descriptor per service, controlling who may manage it; its own control descriptor, controlling system-level operations; and a descriptor per submitted job, controlling who may observe, stop or signal it. Peios Kernel TRM §3 and PCDS §5.

AccessCheck. The KACS function that evaluates a token against a descriptor to produce a decision. peinit calls it for every control request. Peios Kernel TRM §3.8.

Per-service SID. A SID under authority S-1-5-80 derived deterministically from the service name, carried in the group list of every service token. Two services running as the same principal are still distinguishable to an access check. §4.4.

registryd. The userspace registry source daemon serving the persistent hives. peinit starts it in Phase 1 from a compiled-in definition and treats it as opaque thereafter. Its implementation is loregd, a distinction visible only in recovery mode.

Registry. The configuration system LCS and its sources provide together. peinit reads service definitions from Machine\System\Services\, boot configuration from Machine\System\Boot\, and its own parameters from Machine\System\Init\.

KMES. The kernel-mediated event subsystem. peinit emits its structured events — job and operation lifecycle, audit records — into its ring buffer, where they survive eventd restarts and reboots. Peios Kernel TRM §2.

pidfd. A descriptor referring to one specific process, obtained atomically at fork through clone3(CLONE_PIDFD). Every process peinit supervises is tracked by one, which is what makes supervision immune to PID reuse.

cgroup. peinit uses cgroups v2 for process tracking and clean kill only — not for resource accounting or limits. Every service gets its own tree under /sys/fs/cgroup/peinit/, and every submitted job one under /sys/fs/cgroup/peinit/jobs/. §5.1.

sd_notify. The datagram protocol services and submitted jobs use to report readiness, status, progress, keepalives and stored descriptors, over the socket named by NOTIFY_SOCKET. Specified in PSPU §4.

TCB. The Trusted Computing Base: the kernel, KACS, LCS, KMES, peinit, registryd, authd, lpsd and eventd. A compromise of any of them compromises the system.

1.4 Compatibility and Prior Art

Peios / Peios technical reference / peinit / Introduction

peinit is not a port or a reimplementation of anything. The choices that give it its shape — identity as a token, configuration in the registry, operations as objects, jobs as observable executions — were made for Peios. But several interfaces deliberately match existing conventions, because the convention is good and breaking it buys nothing.

1.4.1 sd_notify #

peinit speaks systemd's sd_notify datagram protocol: a service reports readiness, keepalives, status and stored descriptors by sending KEY=VALUE lines to the socket named in NOTIFY_SOCKET. Existing software that supports sd_notify works unmodified.

The compatibility is not total. MAINPID= is not supported, because peinit does not supervise forking daemons — it tracks the process it forked through a pidfd, and there is no way to redirect supervision somewhere else. BUSERROR= is not supported because Peios has no D-Bus.

The protocol as peinit speaks it, including which fields it accepts and how a sender is authenticated, is specified in PSPU §4.

1.4.2 The fd store #

FDSTORE=1, FDNAME=, FDSTOREREMOVE=1 and FDPOLL=0 work as they do under systemd, and descriptors come back to a restarted service at fd 3 onwards with LISTEN_FDS and LISTEN_FDNAMES set. A daemon written to survive a restart without dropping its listening sockets keeps working.

1.4.3 Calendar expressions #

Timer schedules use systemd's OnCalendar format, including weekday names, lists, ranges, repetition, the ~ last-day-of-month form, IANA timezone suffixes and the named shortcuts. §9.1 gives the grammar peinit actually parses; the one deliberate subtraction is sub-second precision, which service scheduling has no use for.

1.4.4 Windows Service Control Manager #

The service model owes its architecture to the Windows SCM: services as securable objects carrying their own descriptors, identity as a token rather than a user account, an access-controlled control interface, and a structured state machine. The debt is architectural. peinit implements none of the SCM's RPC protocol, none of its service types, and none of its control codes.

1.4.5 What is deliberately absent #

There is no systemd unit file support: no reader, no parser, no generator, no migration path. Service definitions are registry keys. Roles are how a package declares them.

There is no socket activation in the systemd sense — peinit does not listen on a service's behalf and hand it a connection. The fd store covers the case that matters, which is a service keeping its own listener across its own restart.

There is no resource control. peinit uses cgroups for tracking and for clean kill, and sets RLIMIT_NOFILE and RLIMIT_CORE if a definition asks, but it does not do accounting, slices, or limits.

1.4.6 Features that belong to other components #

ConcernComponent
Authentication and token mintingauthd
The local identity databaselpsd
Log storage, indexing and querieseventd
Registry storageregistryd, over LCS
Packaging and installationpeipkg and the role system
Device managementeudev
Network configurationa dedicated service
File access control enforcementFACS

1.5 Service Manager Boundaries

Peios / Peios technical reference / peinit / Introduction

These design notes explain the boundaries behind the operator workflows. The implementation chapters specify each mechanism in detail.

1.5.1 peinit in one sentence #

peinit is the sole service manager and PID 1 — it reads service definitions from the registry, starts them in dependency order under per-service identities, supervises them through a defined state machine, and tears them down in reverse at shutdown.

Everything in this topic is an elaboration of that sentence: what a service definition is, what dependency order means, what the state machine looks like, how identity is materialised, and how you drive the whole thing from a shell.

1.5.2 Three kinds of object #

peinit is easiest to understand through the three first-class objects it works with. Keeping them straight is most of the battle when you read a status output or an event log.

ObjectWhat it isLifetime
ServiceA definition — a named unit of execution with identity, policy, and configuration, stored in the registry. It has a runtime state (the state machine) and, when running, a process.Long-lived. Persists across restarts and reboots.
JobA single process execution. Every time peinit forks — a service's main process, a hook, a health check, a submitted job — that is one job, with its own GUID and exit result.Short-lived. A restart creates a new job.
OperationA requested action on a service — start, stop, restart, reload, reset. Control commands create operations that are validated, queued, and executed.Short-lived. Resolves to a terminal state, then is dropped.

A service is the what; a job is what actually ran; an operation is what someone asked for. A single restart command, for example, is one operation, which stops the current job and starts a new one, all against one service. Jobs and operations both carry GUIDs and are emitted to eventd as they complete, which is how the history of a service is reconstructed after the fact. They get their own page: Jobs and operations.

Underneath all three sits the fourth thing peinit is always handling — the token, the identity a service runs under. Tokens are not peinit's invention; they are the system-wide identity object. peinit's role is to obtain the right token for each service and install it before the process runs. That is the subject of Service identity and privileges.

1.5.3 What peinit is not #

peinit borrows vocabulary from init systems you may already know, and the familiarity is a trap. Clearing three wrong mental models is most of understanding what peinit actually is.

It is not systemd. There are no unit files. peinit does not read, parse, or translate .service files, and there is no migration shim. A service definition is a key in the registry under Machine\System\Services\, made of typed registry values — not a text file under /etc. Some interfaces are deliberately compatible where that helps: peinit speaks the sd_notify readiness protocol, and timer schedules use systemd's OnCalendar calendar-expression format. The compatibility stops at those wire formats; the model underneath is different.

It is not Windows SCM. The influence is real — services as securable objects with per-service descriptors, token-based service identity, an access-controlled control interface, a structured state machine — but it is architectural, not interface-level. peinit does not implement the SCM RPC protocol, Windows service types, or Windows control codes.

It is not a logging system. peinit holds a service's stdout/stderr pipes at birth, so it controls where output goes — but storage, indexing, and queries are eventd's job. peinit forwards; eventd is the historian. The same split applies to job and operation history: peinit emits structured events and forgets them; eventd keeps them. See Service output and logging.

A fourth, smaller correction: peinit does not supervise forking daemons. A service that double-forks to "daemonise" is solving a problem that does not exist when the manager tracks the process it spawned. peinit tracks every process by pidfd; a legacy binary that insists on double-forking must be wrapped at the package layer.

1.5.4 The one operational invariant #

peinit is single-threaded PID 1, and that one fact explains a surprising amount of its design. In a single-threaded PID 1, a blocking system call blocks everything — child reaping, watchdog expiry, shutdown signals, every other event stops while the call is stuck. So peinit's hard rule is that its event loop never blocks on a userspace service.

Two consequences you will meet repeatedly:

  • peinit operates on an in-memory snapshot of the registry, not a live view. It reads service definitions at boot and on configuration reload, and thereafter works from an in-memory model that it updates from change notifications. Supervision never waits on the registry being available. This is why a configuration change does not always take effect immediately — see Defining a service.
  • Anything that could block is pushed off the main loop. Filesystem condition checks run in a short-lived forked helper rather than a stat() on the event loop; token requests to authd are driven by a timed state machine, not a blocking call; log delivery is best-effort and never exerts back-pressure on peinit.

You do not need to think about the event loop to operate peinit, but it is the reason behind several behaviours that would otherwise look arbitrary.

1.5.5 Where peinit sits in the system #

peinit is part of the Trusted Computing Base: a compromise of peinit is a compromise of the whole machine. It runs as SYSTEM with all privileges, and it never drops that identity. It depends on, and is depended on by, the other TCB components:

peinit relies onFor
KACSService identity (installing tokens on children), control-interface authentication (peer token + AccessCheck), and per-service access control. See Access decisions.
LCS / the registryService definitions, boot configuration, and its own parameters. See The registry.
registrydThe registry source daemon — the first service peinit starts. peinit treats it as an opaque dependency. (registryd is an interface; the default implementation is loregd — see Boot and boot modes.)
authdMinting tokens for non-platform services. peinit requests; authd mints.
eventdService logs (forwarded over a socket) and job/operation/audit events (emitted through KMES).
KACS, againConveying a job identity: when a process submits a job over peinit's jobs socket and attaches a token, the kernel gates the attach — so peinit can run a program as a service's client without ever judging the identity itself. See Jobs and operations.

The bootstrapping puzzle — authd needs the registry, the registry needs to be started by something, and that something is peinit running before any of them exist — is resolved by the boot model in Boot and boot modes.

1.5.6 How a token is materialised #

The Identity field decides where the token comes from:

Identity valueToken source
SYSTEMMinted by peinit from its own SYSTEM identity.
Any other principal name or SIDRequested from authd.
Absent or emptyauthd, defaulting to LocalService.

1.5.6.1 The SYSTEM path #

For a SYSTEM service, peinit mints an independent SYSTEM token using its own token as the template: it reads its own identity (user SID S-1-5-18, the group list, the privilege set) and mints a fresh primary token with the same identity, adding the service's per-service SID. The minted token is fully independent — the privilege trimming that follows affects only this token, never peinit's.

This path exists because of a bootstrapping problem: the platform services that make the normal token flow possible cannot use it, because they have to start before it exists. registryd and authd use this bootstrap path during boot. Standard eventd and lpsd deployments now use Identity=Service, take the authd path, and start after authd (§4.2).

Note

There is no allow-list restricting which services may be SYSTEM. The security boundary is the registry key descriptor on Machine\System\Services\ — an administrator who can create service definitions is already trusted to assign any identity, so a second gate would add nothing. The dangerous case is never implicit, though: SYSTEM must be written out explicitly; an empty Identity defaults to the minimal LocalService, never to SYSTEM.

1.5.6.2 The authd path #

For any other identity, peinit asks authd for a token: it sends the Identity value verbatim, and authd routes it to the right source —

  • Well-known principals (LocalService, NetworkService, …) → a built-in identity with a predefined minimal privilege set;
  • Local service accounts → lpsd, the local principal store;
  • Domain accounts → the directory connector;

— resolves the principal's SIDs, mints a token, creates a logon session, and hands the token back to peinit to install. peinit neither knows nor cares whether the identity is local or domain; routing is authd's job.

authd records each request as an authd.service.attested event: the service (object.service.name), the identity it got (object.token.sid), its session and the privileges policy granted it. A refused request is recorded too, with the reason, such as account-restricted for an identity that may not be used for a service logon. The event is standard, so the emission policy can switch it off on a machine that restarts services often.

Important

Every non-SYSTEM service start depends on authd. peinit interacts with it over a non-blocking, timed channel — if authd is unreachable or unresponsive, the service start fails rather than hanging PID 1. A broken authd therefore prevents non-SYSTEM service starts, including any platform service configured that way, while SYSTEM-minted services do not need this request. The authd interface is ServiceAttest on /run/logon.sock, specified in PGSS §2.19 and used through libauthd (§4.3).

1.5.7 The per-service SID #

Every service token — minted or authd-issued — carries a per-service SID in its group list. It uses authority S-1-5-80 and is derived deterministically from the service name: the SHA-1 of the uppercased name (as UTF-16LE), with the digest split into five sub-authorities. authd adds it automatically for the tokens it mints; peinit computes and adds it itself for SYSTEM tokens, no authd involvement needed.

The point of the per-service SID is fine-grained access control even when services share an identity. A dozen services might all run as LocalService, but each has a unique service SID, so an ACL can grant access to exactly one of them without inventing a dedicated account. It also keeps services configured as SYSTEM distinguishable to AccessCheck. See Well-known principals for the SID landscape this fits into.

1.5.8 A clean context #

A service inherits only what peinit explicitly hands it — its standard streams and any stored file descriptors. Every other descriptor peinit holds — the control socket, the notify socket, the event-loop fd, the registry and authd and eventd connections — is opened close-on-exec, so it closes automatically at exec and can never leak into a service. peinit also resets the child's signal state to defaults (it runs with all signals blocked for its own signalfd; a service must not inherit that). The guarantee is that a service starts from a clean slate, not from peinit's privileged one.

1.5.9 The cgroup tree #

Every service runs in its own cgroup tree under /sys/fs/cgroup/peinit/, with separate sub-cgroups for the main process, hooks, and health checks:

/sys/fs/cgroup/peinit/<service>/
                       ├── main/     the main process
                       ├── hooks/    pre/post hooks
                       └── health/   health checks

peinit uses cgroups for two things only: tracking every process a service spawns (so a child that forks its own children is still accounted to the service), and clean kill (terminating the entire tree at once, so nothing is orphaned). It does not use cgroups for resource accounting or limits.

The split into sub-cgroups serves a real purpose: it satisfies cgroup v2's "no internal processes" rule, and it lets peinit kill a service's hooks or health checks without touching the main process.

When an old tree cannot be removed — because a process survived SIGKILL in D-state and leaked it — peinit creates a fresh generational tree (<service>.gen<N>) for the next start, so a stuck old instance never blocks a new one. Leaks are surfaced in the service's warnings; they are a sign of an underlying I/O fault, covered in Keeping services running.

2.1 The Initramfs Contract

Peios / Peios technical reference / peinit / Boot

peinit starts with the root filesystem already assembled. Everything that makes a root mountable — LUKS decryption, LVM activation, RAID assembly, any filesystem check — belongs to the initramfs and has happened before peinit exists. peinit performs no root assembly, no decryption, no repair, and no fsck, and it does not assume one has been done.

2.1.1 The handoff #

The initramfs transfers control by chroot-ing into the assembled root and exec'ing peinit there. Not switch_root, and not pivot_root: the kernel refuses to relocate onto the initramfs rootfs, so that route is closed. The consequence is that the initramfs rootfs does not go away. It remains the mount-namespace root — emptied, and unreachable from peinit's view, but present. peinit therefore never assumes a clean single-root mount topology, and never attempts pivot_root.

peinit is installed in package storage at /usr/bin/peinit2 and reached through the fixed runtime path /bin/peinit2. The boot-image tooling sets the kernel init= to the runtime path. Before the transfer, the initramfs has assembled the base StrataFS topology, including the /bin and /sbin views, because peinit reaches every binary it execs through them.

At handoff:

  • the real root is mounted read-write at /;
  • /proc, /sys and /dev are mounted and have been moved into the real root;
  • the environment holds TERM and nothing else, and argv is just peinit's own path.

The read-write requirement is registryd's, not peinit's. loregd's storage backend needs to write its write-ahead log and shared-memory files even to answer a read, so a read-only root cannot support Phase 2 at all. Delivering the root writable is the initramfs's job; peinit only confirms it.

peinit inherits nothing from that environment. It does not rely on having been given anything, and it does not pass its own near-empty startup environment through to services — the environment a service receives is constructed from scratch (§5.5).

2.1.2 What stays outside #

Non-root storage is not peinit's concern. Data partitions and additional filesystems are mounted at the services layer, typically by a Oneshot service that runs mount. peinit has no mount feature beyond the fixed Phase 1 set.

Note

The initramfs itself is assembled by mkirf from hook scripts that packages contribute; mkirf resolves the ordering and bakes the sequence its PID 1 runs. What peinit depends on is the handoff contract above, not how the initramfs arranged to satisfy it.

2.2 Bootstrap Identity

Peios / Peios technical reference / peinit / Boot

The steady-state identity flow is: peinit asks authd for a token, authd mints it, peinit installs it on the child. That flow cannot start the system, because authd needs a token of its own, and registryd has to be running before it. The bootstrap model breaks the circle.

2.2.1 Platform services run as SYSTEM #

A service whose definition says Identity=SYSTEM gets a token peinit mints from its own, with kacs_create_token (§4.2). No authd interaction is involved — which is the point, since authd does not exist when the first of these services starts.

Two services use it:

ServiceWhy
registrydStarts before authd exists at all.
authdNeeds SeTcbPrivilege and SeCreateTokenPrivilege; it is the minter for everything else.

lpsd and eventd used it too, until each moved to an account of its own (Identity=Service). Both now require authd, start after it, and take the authd path like any other service.

Nothing restricts which services may declare Identity=SYSTEM. There is no allowlist, because an allowlist would be enforcing a boundary that is already enforced somewhere better: the Security Descriptor on Machine\System\Services\. Anyone who can create a service definition is by definition trusted to choose its identity, and adding a second list to maintain would only create a way for the two to disagree.

Every SYSTEM token peinit mints carries the service's per-service SID in its group list, computed by peinit itself from the service name (§4.4). That is what keeps platform services distinguishable to an access check despite all of them running as S-1-5-18.

Note

This is the same arrangement Windows uses: the SCM, LSASS and the core platform services all run as LocalSystem. The trust anchor is that peinit is SYSTEM, and the kernel guarantees that by handing PID 1 the boot token.

2.2.2 After authd #

Once authd is running, every subsequent service gets its token through the ordinary authd flow (§4.3). A definition with no Identity field defaults to LocalService — a well-known principal with a minimal privilege set — and authd adds the per-service SID to the token it mints.

2.3 Phase 1

Peios / Peios technical reference / peinit / Boot

Phase 1 is compiled into peinit. It does not change at runtime and has no registry dependency, because its whole purpose is to reach the point where a registry exists. It does the minimum needed to make Phase 2 possible, and most of its failures are fatal to the boot.

2.3.1 Step 1: Confirm the root is writable #

The initramfs delivers the root mounted read-write. peinit does not remount it — mount flags belong to the initramfs, and a redundant remount of an already-writable or overlay root can fail for reasons that have nothing to do with the root being usable.

Instead peinit probes. It creates /.peinit/ if it is absent, writes a uniquely named file there — the name is derived from peinit's own PID and a namespace identifier, so two probes cannot collide — writes to it, and removes it. If any part of that fails, the root is not usable for Phase 2 and peinit enters recovery mode.

2.3.2 Step 2: Mount what is missing #

/proc, /sys and /dev are already mounted. peinit does not blindly mount them again: a redundant mount stacks a second filesystem over the populated one, and on some kernel and flag combinations returns EBUSY instead.

peinit reads /proc/self/mountinfo to find out what is already there, and mounts only what is not:

Mount pointFilesystemFlagsProvided by
/procprocnosuid, nodev, noexecinitramfs
/syssysfsnosuid, nodev, noexecinitramfs
/devdevtmpfsnosuidinitramfs
/dev/ptsdevptsnosuid, noexecpeinit
/dev/shmtmpfsnosuid, nodevpeinit
/runtmpfsnosuid, nodevpeinit
/sys/fs/cgroupcgroup2nosuid, nodev, noexecpeinit
/sys/kernel/securitysecurityfsnosuid, nodev, noexecpeinit

Each mount(2) passes the filesystem name as both the source and the filesystem type, passes only the listed flags, and passes null mount data. Mount points that do not exist are created first.

/dev/pts additionally gets a KACS mount policy immediately after it is mounted : SYNTHESIZE_EPHEMERAL, with a template descriptor granting SYSTEM and Administrators full control and Authenticated Users read, write and traverse. devpts cannot store security descriptors, and its slave nodes are materialised by the kernel when a terminal opens /dev/ptmx — never through a create path that could stamp or inherit one — so under the default DENY_MISSING every pseudo-terminal on the machine is unopenable. The policy is set through SYS_KACS_SET_MOUNT_POLICY on an O_PATH descriptor of the mount point. The template applies to every SD-less inode of the mount alike, so any authenticated principal may open any slave; per-owner slave descriptors need kernel support for stamping the opener's identity at materialisation, and are recorded as future work, not provided here.

/sys/kernel/security gets the same kind of policy, for the same reason: securityfs cannot store descriptors either. Its template grants SYSTEM full control and Authenticated Users read and traverse :

O:SYG:SYD:(A;;GA;;;SY)(A;;GRGX;;;AU)

KACS publishes kacs/self and kacs/sessions there (see Logon sessions), and each makes its own check when read, so opening sessions is allowed to anyone signed in while reading it stays with Administrators and SYSTEM. Only SYSTEM may write: lockdown is the one writable file on the mount, and raising the kernel's lockdown level is not routine administration.

There is a bootstrap wrinkle in reading mountinfo at all: the file lives in /proc, which is one of the things being checked for. If the read fails with ENOENT or ENOTDIR, peinit mounts /proc from the table and retries. Any other failure to read or parse mountinfo sends peinit to recovery.

For the three initramfs-provided rows, an already-mounted filesystem is success, and so is an EBUSY from an attempted mount. For the five peinit owns, a mount failure sends peinit to recovery.

After successful mount setup, peinit ensures /run/lock exists as a directory. Mounting /run hides the directory shipped by fsbase, so peinit recreates it in the live filesystem before starting services. A newly created directory inherits /run's descriptor; an existing directory is preserved. This step also runs when /run was already mounted. A creation failure enters recovery.

2.3.2.1 Seeding descriptors on the new filesystems #

Three of the five filesystems peinit mounts are fresh and empty: /dev/shm, /run and /sys/fs/cgroup. Under KACS an inode with no Security Descriptor is denied to every caller, and there is nothing on a newly mounted tmpfs for a new inode to inherit from — so peinit stamps the mount root with a descriptor that grants SYSTEM and Administrators full control, inheritable by both containers and objects, and grants Everyone read and execute, inheritable by containers only:

O:SY G:SY D:(A;OICI;GA;;;SY)(A;OICI;GA;;;BA)(A;CI;GRGX;;;WD)(A;OICIIO;GA;;;S-1-3-0)

Everything created underneath — the notify socket, per-service runtime directories, the cgroup hierarchy — inherits from it; the two sockets that need a different population are stamped explicitly in step 9. This is why peinit never sets mode bits on the sockets it creates; under KACS they would mean nothing, and the descriptor is the thing that does the work. It is the same descriptor the initramfs seeds onto the root filesystem but for the inheritance flags on that last ACE, and the two are kept in step on purpose: this one inheritable ACL is, in practice, the access policy of everything under these mounts, so an ACE missing here is missing from every per-service directory under /run/services.

The Everyone ACE grants execute as well as read because on Peios execute is traverse — FILE_TRAVERSE and FILE_EXECUTE are the same bit — and an explicit chdir is the one traversal that does not get the SeChangeNotifyPrivilege bypass. A service running as anything other than SYSTEM cannot otherwise reach its own runtime directory. SYSTEM and Administrators keep GenericAll rather than read/write/execute because GenericAll carries WRITE_DAC, WRITE_OWNER and DELETE, and peinit re-stamps descriptors throughout /run.

The last ACE is CREATOR OWNER, and it is what makes the container-only Everyone ACE survivable. A file created under these mounts inherits the SYSTEM and Administrators ACEs and nothing else — and that is a non-empty DACL, so the creating token's default DACL is never consulted. Without it a service would create runtime state it could not read back. It is inherit-only, so it grants nothing on the directory it sits on; it is resolved against whoever creates each object, and carried onward down each container.

The Everyone ACE is inheritable by containers only, which is the one place this descriptor differs from the root filesystem's. What a service needs here is to walk to its own directory, and container inheritance gives it that by itself. Object inheritance would additionally put Everyone-read on every file anything creates beneath these mounts — a service's runtime state, and shared memory segments, readable by any principal through inheritance alone. The root filesystem needs object inheritance because a service has to read and execute the binaries beneath it; nothing under these three mounts is a binary.

This descriptor is not the one prelude's seed-sd writes by default. That one is narrower, and stays narrower: it also stamps /dev, where every ACE is inherited by the next hot-plugged block device, so a read ACE for Everyone there would be every filesystem descriptor on the machine bypassed by opening the raw disk.

Failure to apply the descriptor sends peinit to recovery. Without it every file peinit later creates on that filesystem would be unreachable to everything, including peinit.

/proc, /sys and /dev are not stamped: they arrive from the initramfs already populated.

2.3.2.2 Device node policy #

/dev arrives with that same inheritable descriptor on its root and on every node, which is the right default — whatever the root grants, a disk hot-plugged later inherits, so the root must be acceptable on a raw block device — but it leaves /dev/null unusable by anyone who is not an administrator. A single inherited descriptor cannot say "/dev/null for everyone, the disks for administrators", so peinit enumerates the exceptions. Once the mounts are up it stamps each of these nodes with a descriptor of its own:

NodeDACL
/dev/null, /dev/zero, /dev/full, /dev/random, /dev/urandom, /dev/tty, /dev/ptmxD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FRFW;;;WD)

Everyone may open the node for reading and writing and may stat it; nobody but SYSTEM and Administrators may change its descriptor. Only the DACL is replaced — owner and group stay as the seed left them — and the ACEs carry no inheritance flags, because a device node has no children. /dev/console is deliberately not on the list: it is the SYSTEM console.

The step is advisory. A node that cannot be stamped is reported as a warning and stays on the inherited default, usable by administrators and denied to everyone else; a node that does not exist is noted and skipped. Neither sends peinit to recovery.

2.3.3 Step 3: Restore the persisted random seed #

peinit restores the seed at /var/state/peinit/random-seed once /dev is available and before registryd starts. The seed is a machine-local entropy cache for the kernel CSPRNG. It is not configuration, and shipping one in a packaged image, live ISO or VM template would hand every instance of that image the same starting entropy.

If the file is absent, that is an ordinary first boot or a stateless live boot, and peinit continues silently. When a seed is present peinit mixes it into the kernel pool, preferring the interface that credits entropy for a locally persisted seed; if that fails it mixes the bytes without crediting and records the failure. A seed file that is empty, or larger than 4096 bytes, is treated as an error.

Nothing in this step can send peinit to recovery. A system with no entropy cache still boots; it just starts with less entropy, which is a problem for the image builder to solve with a hardware or virtio RNG rather than with a seed baked into the image.

The initramfs may perform the same restore earlier, once the persistent root is mounted. peinit's restore stays as the fallback for initramfs images that do not participate and for boots that have no initramfs.

2.3.4 Step 4: Ensure the local machine ID #

/lcl/etc/machine-id holds a stable local install identifier used for software compatibility, log correlation and instance identity. It is not a security principal: not a credential, not a SID, not an account, and not an input to any authorisation decision.

The format is 128 bits as exactly 32 lowercase hexadecimal characters followed by one newline. A valid existing file is left alone. A file that is absent, empty, all zeroes, the wrong length, not hexadecimal, or missing its trailing newline is replaced : peinit draws 128 bits from the kernel CSPRNG and writes a valid file atomically, through a temporary file and a rename, with the result flushed.

The write creates /lcl/etc/ if it is not there. This step runs before the path-provisioning machinery of step 7, and /lcl is a StrataFS view the initramfs assembled, so whether the directory exists is a property of the image rather than something peinit can assume.

Only a CSPRNG failure sends peinit to recovery. If the kernel cannot produce random bytes, nothing else on the machine can be trusted either.

An unreadable file or an unwritable path does not. The boot continues with an identifier valid for this boot only, and says so :

peinit warning: machine-id not persisted (<reason>); using an identifier
for this boot only

A read failure takes the same path as finding no file, because it leaves peinit in the same position — unable to say whether a valid identifier is on disk. It is reported rather than treated as a clean generation, because it may have replaced something valid.

This is deliberately fail-soft. The identifier is a local opaque install ID and not an authorisation input, so failing a boot over it would be disproportionate — and an ephemeral ID is already the defined behaviour for a stateless live boot.

Images and templates are expected to ship with no machine ID, or with an empty file as a reset marker. Clone tooling that wants a new identity removes or truncates the file and lets the next boot generate one. Stateless live boots without a persistent overlay get an ephemeral ID for that boot.

2.3.5 Step 5: Set the clock from the hardware RTC #

peinit reads the hardware clock and calls clock_settime() before registryd starts, so that timestamps on registry operations, log entries and the boot attempt counter mean something.

It opens /dev/rtc, falling back to /dev/rtc0 if that device is absent , and reads it with RTC_RD_TIME. The returned struct rtc_time is interpreted as UTC and converted to CLOCK_REALTIME seconds with zero nanoseconds.

Every failure in this step sends peinit to recovery: no openable RTC device, a failed read, a value that is invalid or before the Unix epoch, a failed clock_settime, and — deliberately — a failure to close the descriptor after a successful read. Leaking a descriptor in PID 1 during bootstrap is a symptom of something being badly wrong, not a detail to swallow.

Note

NTP corrects the clock properly later in the boot. This step only gets it into the right decade, so that early timestamps are not absurd and the timer subsystem has a wall clock to anchor to.

2.3.6 Step 6: Start registryd #

peinit holds a compiled-in definition for registryd — the only compiled-in service definition there is :

FieldValue
ImagePath/sbin/registryd
ArgumentsMachine=/var/state/loregd/Machine.hive, Users=/var/state/loregd/Users.hive
IdentitySYSTEM
ReadinessNotify
ErrorControlCritical

The hive paths are peinit's choice, not registryd's: where the machine registry lives is a boot-policy decision, and it is made here because this is the one service start that cannot consult configuration.

peinit mints a SYSTEM token including registryd's per-service SID, creates the cgroup tree, forks with the token installed, and execs /sbin/registryd through the runtime StrataFS view. Two separate timeouts bound the start, both 30 seconds: one on process setup, driven synchronously because there is no event loop yet, and one on readiness.

registryd's READY=1 means "accepting and serving registry requests", not "the process is alive" — it does not signal until its storage backend is open, its schema is validated and it can answer a read.

2.3.6.1 The schema-version guard #

After readiness, peinit ensures the base registry structure exists and then probes it. Ensuring comes first: peinit creates Machine\System, Machine\System\Services and Machine\System\Init if they are absent and stamps Machine\System\Services\SchemaVersion with the current schema version, 1. Only then does it read the value back.

The read is what verifies registryd is genuinely serving. A value that is present but not a REG_DWORD, or that is a REG_DWORD of the wrong length, fails the probe. A key or value that is absent reads as zero and passes — the structure was just created, so absence at this point means the write did not take effect, and the failure that matters is the provisioning failure, which is reported directly.

The consequence is that the guard is self-healing. An unprovisioned first boot, or a registry cleared by the recovery tools, comes up with an empty Machine\System\Services\ and boots into a Phase 2 with no services rather than into recovery.

2.3.6.2 Keeping registryd #

When registryd passes readiness and the probe, peinit retains the activation as an ordinary runtime service instance: its state, its pidfd-tracked main process, its cgroup generation and paths, its output pipe ownership, its notify generation, its job identity, and any cleanup evidence. Ownership is not dropped at the Phase 2 boundary.

During Phase 2 the registry's own definition of registryd, if there is one, is merged onto the retained activation. peinit does not create a second inactive record and does not restart registryd because a definition has appeared. If the registry definition is absent or invalid, the ordinary graph validation rules apply.

If registryd fails to start, its readiness times out, or the probe fails, peinit enters recovery. There is no Phase 2 without a registry.

2.3.7 Step 7: Autorun scripts #

Between registryd starting and path provisioning, peinit runs every non-directory entry in /lcl/policy/autorun.d, in sorted order, by absolute path , with the working directory / and PATH=/sbin:/bin . Each runs under peinit's own SYSTEM token.

The step is fail-open at every point: a missing directory, an unreadable directory, a spawn failure and a non-zero exit are all console warnings and none of them stops the boot. Its console output bypasses the quiet policy (§2.6), because a script that ran this early and went wrong needs to be visible.

2.3.8 Step 8: Provision boot-time paths #

Covered in §2.4.

2.3.9 Step 9: Infrastructure setup #

Three things, before Phase 2 begins:

  1. The control socket at /run/services/peinit/control.sock, which serves every runtime command for the lifetime of the system, stamped so that any authenticated principal may connect and every command is then checked on its own (§10.1).
  2. The jobs socket at /run/services/peinit/jobs.sock, on which any authenticated principal may submit a job once Phase 2 runs, stamped so that connecting is the submission permission (§10.7).
  3. Loopback: peinit brings up lo over netlink, because services that bind 127.0.0.1 need it.

Either socket failing to bind or to take its descriptor sends peinit to recovery — without the first there is no way to administer the system, and a jobs socket with the wrong descriptor is either unreachable or open to everything. A loopback bring-up failure is a warning that lets Phase 2 proceed.

2.3.10 Failure summary #

FailureResponse
Root writability probe failsRecovery
A mount point cannot be createdRecovery
A peinit-owned filesystem fails to mountRecovery
A mounted filesystem cannot be stamped with its descriptorRecovery
/proc/self/mountinfo unreadable or unparseableRecovery
/proc, /sys or /dev already mounted, or EBUSYTolerated as success
A device node in the policy list cannot be stampedWarning; node keeps the inherited default
A device node in the policy list does not existNoted; boot continues
Random seed absent, oversized, empty, or unrestorableWarning; boot continues
Machine ID generation fails (CSPRNG)Recovery
Machine ID unreadable, or unwritable at its pathWarning; boot continues with an identifier for this boot only
Machine ID absent, empty, or malformedRegenerated; boot continues
Any RTC or clock failureRecovery
registryd fails to start, or setup times outRecovery
registryd readiness times outRecovery
Base registry provisioning failsRecovery
Schema-version probe returns a wrong type or lengthRecovery
An autorun script is missing, unspawnable, or exits non-zeroWarning; boot continues
A provisioning entry is malformedWarning; entry skipped
An optional provisioned path failsWarning; boot continues
A required provisioned path failsRecovery
Control socket creation or stamping failsRecovery
Jobs socket creation or stamping failsRecovery
Loopback bring-up failsWarning; boot continues

2.4 Path Provisioning

Peios / Peios technical reference / peinit / Boot

Some filesystem objects belong to no single service. A directory two packages both write into, a state file created before anything runs, a path that has to exist with a particular Security Descriptor before the first service that uses it starts — none of these has an owner in the service model, and creating them from a service's pre-exec hook makes their existence depend on start ordering.

Boot-time path provisioning is the registry-backed answer, and the equivalent of the tmpfiles.d role elsewhere. peinit applies it after registryd is serving and before any Phase 2 service is planned or started.

2.4.1 Entries #

Each child key under Machine\System\Init\ProvisionedPaths\ is one entry. Unknown values on an entry are ignored.

ValueTypeRequiredDefaultMeaning
Kindstringyes—directory or file.
Pathstringyes—Absolute path to create or verify.
Securitybinarynobuilt-inThe Peios file Security Descriptor to apply.
Requireddwordno0If 1, failing this entry prevents Phase 2.

For Kind=directory peinit ensures the path exists as a directory; for Kind=file it ensures the path exists as a regular file. In either case a path that exists with a different file type fails the entry. An existing file is opened rather than created, so provisioning never truncates one.

peinit does not create parent directories. The parent is checked, and has to already be a directory. A package that needs a hierarchy declares each directory explicitly, or depends on the package that owns the parent — which keeps the ownership of every directory traceable to a package rather than to whichever entry happened to run first.

2.4.2 Descriptors #

When Security is present, peinit applies the supplied binary descriptor. A malformed or rejected descriptor fails the entry.

When it is absent, peinit applies a built-in default :

O:SY G:SY D:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FR;;;BU)

SYSTEM and Administrators get full control; ordinary users get FILE_GENERIC_READ.

2.4.3 Failure #

Entries with Required=0 are fail-soft: peinit logs and continues. Entries with Required=1 are fail-closed: peinit logs and enters recovery before Phase 2 starts.

An entry that is malformed — a missing or unrecognised Kind, a missing or relative Path, a value of the wrong type — is logged as a warning and skipped, regardless of Required. Required marks a path as essential to boot; it does not make a broken entry more dangerous than a missing one.

2.5 Phase 2

Peios / Peios technical reference / peinit / Boot

With registryd serving, peinit reads the service graph and boots the system from it. Phase 2 is entirely registry-driven.

2.5.1 Reading the definitions #

peinit reads every key under Machine\System\Services\. The reads are bounded by LCS's request timeout: if registryd hangs mid-read, peinit receives ETIMEDOUT and enters recovery.

Decoding is per key. A definition that fails to decode — an invalid service name, a malformed trigger, an unclosed quote in a command, a registry: check naming an uncacheable key, a duplicate known field, an unrecognised value for an enumerated dword — fails that service, which is marked Failed with cause ValidationError. The boot proceeds with every other definition, and anything that depended on the failed service fails in turn through the ordinary dependency propagation.

peinit holds no definition for such a service, so the entry behind it is a placeholder: status reports it Failed with cause ValidationError, marked definition-removed (§3.8), with the decode error as its description. start, restart, reload and reset are refused with UNKNOWN_SERVICE, as for any definition-removed service. The placeholder leaves with its key: a reload-config that re-reads a repaired key restores the entry, and one that finds the key deleted discards it, since Failed does not retain a definition-removed entry.

Only services carrying a boot trigger are root candidates. A service with no triggers is demand-only and is not a root, though it can still be pulled into the boot transaction as somebody's dependency. A service with Disabled=1 is excluded from the boot graph entirely, but its definition is still loaded into the in-memory model so it can be started by hand later.

2.5.2 Building and validating the graph #

The boot graph is every boot-triggered root candidate plus the transitive closure of their Requires, BindsTo, and existing non-disabled Wants dependencies. Requires and BindsTo pull their target in even if the target has no boot trigger. Wants targets come in as best-effort members; missing or disabled ones are ignored. A missing or disabled Requires or BindsTo target blocks the dependent with cause DependencyFailure, and that blocking propagates.

peinit topologically sorts the graph and validates it before starting anything. The rules are in §7.2; the outcomes that matter here are:

  • A cycle fails every service in it. A cycle involving a Critical service downgrades the boot to Safe mode without rebooting.
  • An unresolvable conflict fails both services. Same downgrade if either is Critical.
  • A missing Requires target fails the dependent.
  • Warnings — a Readiness=Alive service with dependents that require it — are logged and do not prevent boot: a [ WARN ] peinit warning: line on the console and a peinit.graph.validation.warned event under phase boot, whether or not the same graph also has findings.

2.5.3 Starting #

peinit walks the graph and starts services, parallelising wherever the graph allows, up to a configurable limit:

KeyDefaultMeaning
Machine\System\Boot\MaxParallelStarts10Services starting concurrently.

An absent key uses the default. A value of zero, a type mismatch, or a malformed payload is invalid boot configuration and sends peinit to recovery — running the scheduler with an effective limit of zero would hang the boot rather than fail it.

As each service reaches a dependent-satisfying state — Active for Simple, Completed for Oneshot with or without RemainAfterExit, Skipped for a service whose conditions did not hold — its dependents become eligible and join the start queue. A Oneshot without RemainAfterExit passes through Completed, releasing its dependents, and then goes Inactive.

Dependents blocked on a Requires or BindsTo target wait for that target to reach a satisfying state. Dependents blocked on a Wants target wait only for it to reach any terminal state, satisfying or not — which is what makes Wants ordering rather than dependency.

2.5.3.1 The typical order #

Nothing below is hardcoded. It falls out of the dependency graph that the standard role definitions produce, and an administrator who changes a dependency gets a different order.

  1. eudev — device management. SYSTEM, with RequiredPrivileges stripped to the minimum it needs. Starts before authd exists.
  2. lpsd — the local identity database. Depends only on registryd.
  3. authd — the identity authority. Depends on registryd and lpsd. Once it is ready, token minting is available.
  4. eventd — logging and audit. Services started before it log into peinit's pre-eventd buffer.
  5. Networking — the first service to receive a token from authd.
  6. Application services, in dependency order.
  7. Login services, last, so the system is operational before it accepts a session.

2.5.3.2 The bootstrap matrix #

ServicePhaseIdentityToken fromReadinessErrorControl
registryd1SYSTEMminted by peinitNotifyCritical
eudev2SYSTEMminted by peinit, privileges strippedAliveNormal
lpsd2SYSTEMminted by peinitNotifyCritical
authd2SYSTEMminted by peinitNotifyCritical
eventd2SYSTEMminted by peinitNotifyCritical
networking2service tokenauthdNotifyNormal
sshd2service tokenauthdAliveNormal
application services2service tokenauthdper-serviceNormal

2.5.4 Deferred starts #

A service whose trigger is boot:settled is not part of the boot plan at all. It is not a root, it does not consume the parallel-start budget, it is not counted towards boot success, and it cannot block or delay anything. peinit starts it after the plan, once the boot has stopped moving.

"Stopped moving" is precise: every service in the plan — those that were started and those that were blocked — is in a state it will not leave without help. Active, Completed, Failed, Skipped, Abandoned and Inactive all count as settled. Starting, Reloading, Stopping and Backoff do not, because a service between restart attempts is going to produce more output.

A deadline bounds the wait:

KeyDefaultMeaning
Machine\System\Boot\SettleTimeout5Seconds before deferred services start regardless.

The deadline is measured from the moment the plan was observed. An absent key uses the default; a type mismatch or a bad length sends peinit to recovery. Zero is legal and means "start on the next turn, settled or not".

Whichever comes first — the set settling or the deadline expiring — the deferred services start, once, each independently. A start that fails is recorded and dropped: one refusing service does not stop the others and does not affect the boot. The dispatch carries a flag saying whether the deadline expired rather than the set settling, so a service that cares whether the boot was still moving can be told.

Note

The motivating case is a console login prompt being scribbled over by peinit's own progress messages. That is a scheduling preference, not a dependency — the prompt does not require anything to have started, it just wants the console to itself. Expressing it as a dependency would mean naming every service that might print, which changes every time the system does.

2.5.5 Boot success #

A boot is successful once every Critical service has held a dependent-satisfying state continuously for a grace period:

KeyDefaultMeaning
Machine\System\Boot\BootSuccessGrace30Seconds of held health before the boot counts.

The criterion is satisfying, not Active. A Critical Oneshot reaches Completed and never reaches Active, so a test for Active would make such a service unable to ever mark a boot successful. Skipped counts too.

Success resets the boot attempt counter to zero (§2.7). The control socket's boot command says where a boot stands: which Critical services it is waiting for, or when it will count, or that it has (§10.8).

2.5.6 Failure summary #

FailureResponse
A service definition fails to decodeThat service Failed with cause ValidationError; the rest of the boot proceeds
A registry read times outRecovery
Invalid MaxParallelStarts or SettleTimeoutRecovery
A dependency cycleAll services in it Failed; Safe mode if any is Critical (see below)
An unresolvable conflictBoth Failed; Safe mode if either is Critical (see below)
A missing Requires targetThe dependent Failed
A Critical service fails during bootRestart budget, then reboot
A non-Critical service failsFailed; its Requires dependents fail; the rest continue
authd unavailable when a service needs a tokenThat service Failed

The two Safe-mode rows carry a caveat. When the downgrade fires, peinit rebuilds the graph in Safe mode and discards the Full-mode one, so the services that caused it are not marked Failed — they are never entered into the blocked set at all. What records them is the boot-level downgrade finding described in Boot modes.

2.6 Boot Modes

Peios / Peios technical reference / peinit / Boot

peinit boots in one of three modes, forming an escalation path from normal operation to last-resort maintenance.

Full boot ---+-- success ----------------> counter reset, operational
             |
             +-- cycle w/ Critical ------> Safe mode (no reboot)
             |
             +-- conflict w/ Critical ---> Safe mode (no reboot)
             |
             +-- Critical failure -------> sync + reboot --+
                                                           |
Safe boot ---+-- success ----------------> counter reset,  |
             |   operational (reduced)                     |
             +-- Critical failure -------> sync + reboot --+
                                                           |
                             counter increments <----------+
                                        |
                             counter >= N ---------> Recovery mode
                             Phase 1 failure ------> Recovery mode

2.6.1 Full mode #

The default. Every boot-triggered service starts in dependency order, as §2.5 describes.

2.6.2 Safe mode #

Safe mode starts a reduced set. Eligibility is a filter within the boot-triggered set, not a replacement for it: a service with no boot trigger does not auto-start in Safe mode whatever its SafeMode or ErrorControl says, and remains demand-only. Within the boot-triggered set, two categories are eligible:

  • Critical services (ErrorControl=Critical) start. If one fails, the ordinary Critical failure path applies — restart budget, reboot, counter increment, eventually recovery.
  • SafeMode=1 services are attempted best-effort. If one fails, Safe mode continues without it.

ErrorControl=Critical implies SafeMode, so a Critical service does not have to declare both.

2.6.2.1 What caused the downgrade #

The rebuild discards the Full-mode graph, so a service that forced Safe mode and is then excluded from the rebuilt graph is never entered into the blocked set and never marked Failed. That is deliberate: Safe mode was never going to start it, and a Failed state would say something about its own health that is not true. status should keep meaning "this service is broken".

A service that survives into the rebuilt graph gets no such protection. Two Critical services that require each other are both eligible in Safe mode, so the rebuild contains the same cycle and blocks them both, exactly as it would anywhere else — and there is no order that could have started either.

The reason is therefore recorded at boot level rather than per service. Every finding that forced the downgrade — each critical cycle, each critical boot conflict — is written to the console and emitted as a peinit.boot.downgraded KMES event naming the services involved, an essential type the emission policy cannot switch off.

All of them are reported, not just the first. A machine can be downgraded by a cycle and a conflict at once, and an operator who fixed only the one they were shown would reboot straight back into Safe mode.

They stay available for the rest of the boot: the control socket's boot command reports the mode, why, and every finding (§10.8).

peinit rebuilds the dependency graph from scratch using only the eligible services. Dependencies on excluded services are dropped: if A depends on non-Critical B and B is excluded, A's dependency on B does not exist in the Safe mode graph. This is what makes Safe mode useful — it is a graph in which the broken parts of the configuration are simply not present, rather than a graph in which they are present and failing.

A successful Safe boot resets the boot attempt counter.

Note

Safe mode is purely a boot-sequencing concern. Once booted, an administrator starts anything by hand exactly as in Full mode. It is for a system whose configuration is broken but whose TCB is healthy.

2.6.2.2 Entry #

  • A cycle involving a Critical service at boot. Graph validation detects it and peinit downgrades in place, without rebooting — the cycle is a configuration error, and rebooting would find it again.
  • An unresolvable conflict involving a Critical service at boot. Same reasoning.
  • peios.safemode=1 on the kernel command line.

Safe mode is not entered because a Critical service crashed at runtime. That follows the ordinary path: restart budget, reboot, counter increment, recovery.

2.6.3 Console output #

peios.quiet=N bounds what peinit writes to the console:

ValueBehaviour
0Write unconditionally.
1Do not write to a terminal held as the controlling terminal of a running service, except to announce loss of the system. This is the default.
2Additionally drop ordinary progress everywhere, while still emitting errors.

The two rules are independent, and an error is never less visible at 2 than at 1. A terminal is matched by device rather than by path, since /dev/console and /dev/ttyS<n> can name the same device ; where the device cannot be determined, peinit treats the terminal as free and writes to it. Guessing wrong in that direction costs a scrambled line, which is recoverable; guessing wrong the other way costs the operator their console output at the moment the machine is least able to explain itself. Suppressed messages are discarded rather than buffered.

The autorun step (§2.3) bypasses the policy: a script that ran that early and went wrong is worth interrupting a login prompt for.

2.6.4 Kernel command line #

ParameterEffect
peios.safemode=1Force Safe mode.
peios.recovery=1Force recovery mode regardless of the counter.
peios.bootattempts=NSet the recovery threshold; 0 disables the check.
peios.quiet=NConsole verbosity, as above.
peios.notifysocket=PATHOverride the notification socket path.

A malformed value is ignored in favour of the default rather than failing the boot. Nothing exists this early to report a diagnostic to.

2.7 The Boot Attempt Counter

Peios / Peios technical reference / peinit / Boot

The counter is what turns a repeated failure into an escalation. peinit keeps it at /.peinit/boot-attempts, as a plain decimal integer in a file on the root filesystem — deliberately not in the registry, because the registry may be the reason the boot is failing.

2.7.1 The cycle #

peinit reads the counter at startup, before selecting a boot mode. The recovery threshold is evaluated against this pre-increment value, so a default threshold of 3 admits exactly three boot attempts before recovery.

  • Absent file: treated as 0.
  • Unreadable, empty, non-decimal, carrying trailing non-whitespace data, or overflowing the counter representation: recovery. A counter that cannot be read cannot be trusted to escalate.

peinit increments once per boot, after the root is known writable and before Phase 2 begins. Incrementing before the root is known writable would silently lose the increment on a read-only root and defeat escalation entirely.

The increment happens after the mount, seed, machine ID and clock steps rather than immediately after the writability probe, because reading the kernel command line requires /proc. One consequence is that a recovery entered from a mount failure, a machine ID failure, or an unreadable command line does not advance the counter.

A write failure — a full disk, say — is treated as a counter of 0 and the boot continues. A failure to record an attempt is not itself a reason to escalate.

The counter resets to 0 on a successful Full or Safe boot, after the grace period.

The control socket's boot command reports the count this boot began with, the threshold, and whether this boot has reset the counter yet (§10.8).

2.7.2 Recovery threshold #

The threshold is peios.bootattempts=N on the kernel command line, defaulting to 3. It is a command-line value rather than a registry one because the check runs in Phase 1, before registryd is serving.

peios.bootattempts=0 disables the check entirely — the escape hatch for a system whose counter is itself the fault.

peios.recovery=1 forces recovery without consulting the counter, but does not suppress the increment.

Note

The counter catches boots where peinit runs but the system never reaches health — a crash-looping Critical service being the usual case. It deliberately does not try to catch a peinit too broken to reach its own increment. Recovery mode is itself a peinit mode, so a peinit that cannot start cannot deliver recovery either, and a higher count could not help. That failure belongs to binary integrity, not to the counter.

2.8 Recovery Mode

Peios / Peios technical reference / peinit / Boot

Recovery mode is the last resort. There is no TCB guarantee and no degraded boot to speak of — it is a maintenance environment that hands the administrator an unrestricted SYSTEM shell on the console.

2.8.1 Entry #

  • The boot attempt counter reaching the threshold (§2.7).
  • peios.recovery=1 on the kernel command line.
  • Any of the Phase 1 failures listed in §2.3, most importantly a registryd that will not start or will not serve.
  • A Phase 2 registry read that fails or times out, or invalid boot configuration.
  • A required provisioned path that cannot be created or secured.
  • The runtime loop failing on an error about supervision itself: the event wait, event source registration, a listener, the event ring refusing even the small peinit.event.dropped, shutdown finalisation. An error peinit can attribute to one service is contained to that service instead and does not enter recovery (§8.2).

2.8.2 What peinit does #

peinit records the reason as a peinit.recovery.entered KMES event — essential, so no emission policy switches it off — with the reason in outcome.reason and what went wrong, in the error's own words, in outcome.detail; a Phase 2 plan that would not build adds its finding as a peinit.graph.validation.failed under phase phase2-boot , then:

  1. Completes Phase 1 steps 1–5 if they have not been reached yet.
  2. Ensures the base registry structure exists, so the shell sees a normal layout even on a system that has never been provisioned.
  3. Attempts to start registryd. A failure here is ignored — recovery delivers a shell whatever registryd's state.
  4. Skips all Phase 2 services.
  5. Starts a shell on /dev/console from a compiled-in definition, with no registry dependency: /bin/recsh if it is present and executable, otherwise /bin/sh , running as SYSTEM with a fixed environment of PATH=/sbin:/bin, TERM=linux and HOME=/. peinit does not care where either binary comes from.
  6. Logs the failure reason to the console.

If the shell exits, peinit respawns it. Recovery never exits to an unmanaged PID 1.

If neither /bin/recsh nor /bin/sh can be exec'd, peinit cannot deliver a shell at all. It logs the reason to the console, syncs, and halts — PID 1 exiting would panic the kernel. A missing shell is a binary-integrity failure and sits outside the boot-attempt machinery's remit.

The shell receives /dev/console duplicated onto its standard streams, but peinit does not call setsid() or acquire a controlling terminal for it. The shell is not a session leader, so job control is not available in the recovery shell.

Step 1 runs on every entry. The steps are individually idempotent, so completing them and ignoring the failures is what "if they have not been reached yet" amounts to in practice — including the step that failed, which by the time the operator has a shell may well succeed.

Step 3 runs only if Phase 1 has not already reached its own registryd start, and it runs at most once. A recovery entered from a Phase 2, provisioning or runtime failure starts no registryd: Phase 1 already did, and its activation is retained for the runtime. A recovery entered from the registryd start failing also starts none — the attempt has been made, and a process may have forked before the failure was reported. In both cases a second daemon would bind over the first's notify socket, which succeeds rather than reporting EADDRINUSE because the bind unlinks the path first, and would then open the same hive files behind the first daemon's back.

Where recovery does start one, it uses the settings this boot parsed, so peios.notifysocket= and peios.quiet apply to it.

Note

Recovery mode is not a degraded boot; it is a maintenance environment. There are no security protections beyond what the kernel provides. The administrator has a SYSTEM shell and the corresponding responsibility.

2.8.3 Registry diagnosis and recovery limits #

Recovery provides a console maintenance shell when one is available as described above; it does not define an offline storage-repair interface for the registryd provider. Preserve the console failure reason and source startup output, then identify the deployed provider and version, its hive declarations, and the actual database paths. See LCS and sources.

Entering Recovery does not establish that the source is stopped. As explained above, a registryd started or attempted earlier in the boot can remain active. The loregd exit contract also permits a source to keep running after storage-operation failures. Do not launch a second source against the same hive files.

The source-checked loregd implementation takes one or more HiveName=DatabasePath declarations and rejects arguments without =. It has no offline inspection, backup-recovery or database-clearing switches and does not create an automatic startup backup. Startup WAL recovery and orphan cleanup are not backup restoration or a database-corruption repair facility. The linked references pin these findings to the implementation revision checked; identify the installed provider and revision before choosing a recovery procedure.

Offline inspection, repair or database replacement therefore needs a procedure supported by the installed provider and version. Establish the source's state and verify the existence and coverage of a recovery copy before planning a replacement. This manual supplies no offline stop, repair or replacement sequence. Role definitions can re-supply service configuration; they do not constitute a backup of every registry value.

The ordinary Backup and restore workflow is separate: LCS coordinates operations through a working source. Backup requires a read-only snapshot transaction; restore replaces a target subtree, including descriptors, in a read-write transaction. Neither is a way to repair a source that cannot serve those operations.

Note

/bin/recsh exists so a system can ship a purpose-built recovery shell — one that bundles the offline registry tools, presents guidance, or curates a command set — without making it mandatory. /bin/sh is the floor where it is absent. peinit treats both as opaque executables.

2.8.4 Remote recovery #

Recovery requires console access: physical, IPMI or serial. Two post-v1 features address headless servers — rolling the registry back to a last-known-good state from the recovery shell, and an emergency sshd started without registry involvement.

3.1 Services

Peios / Peios technical reference / peinit / The Service Model

A service is the primary unit of management: a definition in the registry, a runtime state, a Security Descriptor, and at most one running main process. Definitions live under Machine\System\Services\<name>, where the key name is the service name. peinit reads them at Phase 2 boot and on an explicit reload-config.

3.1.1 Names #

A service name is 1 to 128 bytes drawn from [A-Za-z0-9._-]. Any other byte makes the name invalid.

Two exclusions are deliberate. / is out because names map directly onto cgroup identifiers (§5.1) and onto registry key names, and a name containing a separator would mean something different in each. : is reserved for peinit's own synthetic naming.

3.1.2 The two types #

3.1.2.1 Simple #

A long-running daemon. peinit forks, installs a token, and execs the binary; the process is the service. When it exits, the service has stopped. This is the default and covers nearly everything — registryd, authd, sshd, application services.

Readiness comes from the Readiness field. Notify, the default, waits for READY=1. Alive treats the process as ready the moment it exists.

The service goes Active on readiness and stays Active until the process exits or something stops it.

3.1.2.2 Oneshot #

A run-to-completion task: database initialisation, a schema migration, a directory that has to exist. peinit forks, installs a token, execs, and waits for the exit.

Readiness is ignored. A Oneshot's readiness is always "it exited successfully", because READY=1 is meaningless from a process whose job is to finish. Success means exit code 0, or any code listed in SuccessExitCodes.

The differences from Simple are:

  • A successful exit goes to Completed. With RemainAfterExit=1 it stays there; without, it passes through Completed to release dependents and then goes Inactive.
  • A non-zero exit goes to Failed.
  • ExecStartPost runs after the successful exit rather than after a readiness signal, and does not run at all if the Oneshot failed.
  • StartTimeout covers the entire execution, from the first pre-hook to the process exiting.

RemainAfterExit matters when the Completed state itself is the useful information — so a status query shows a migration as finished rather than as inactive.

3.1.3 Forking daemons #

peinit does not support them. A service that double-forks to daemonise itself is working around a problem that does not exist when the service manager tracks the process it spawned, and peinit tracks its child through a pidfd obtained at fork. There is no MAINPID=, and no way to point supervision at a different process.

A legacy binary that insists on double-forking is wrapped by whoever packages it — a script with a --no-daemon flag, typically. That is a packaging concern.

3.2 The Definition Schema

Peios / Peios technical reference / peinit / The Service Model

Every field a service definition can carry, with its registry type and its default. The semantics of each are in the section named alongside.

Value names are matched case-insensitively, so ImagePath and imagepath are the same field — and therefore a definition carrying both is a duplicate, not two fields.

FieldTypeDefaultMeaning
ImagePathstringrequiredAbsolute path to the service binary.
Argumentsmulti_string—Arguments passed to the binary.
Typedword0 (Simple)0 Simple, 1 Oneshot. §3.1
Triggersmulti_string—When the service starts automatically. §3.4
Disableddword0If 1, no trigger activates the service.
SafeModedword0If 1, attempt this service in Safe mode. Implied by ErrorControl=Critical. §2.6
IdentitystringLocalServiceSYSTEM, shared LocalService/NetworkService, Service for the service’s own virtual account, or a designated principal. §4.1
RequiredPrivilegesmulti_string—Privileges to keep; all others are removed. §4.5
Requiresmulti_string—Hard dependencies. §7.1
Wantsmulti_string—Soft dependencies. §7.1
BindsTomulti_string—Runtime coupling. §7.1
Conflictsmulti_string—Mutual exclusion. §7.1
Providesmulti_string—Roles this service fills: virtual names a dependency may target (network), and that peinit derives edges to. A role is a bare name — the level in network:routed comes from the depending service's readiness requirement, not from this value. §7.6
OnFailurestring—Service to start when this one fails. §6.3
ErrorControldword0 (Normal)0 Normal, 1 Critical.
RemainAfterExitdword0Oneshot only: stay Completed after a successful exit.
SuccessExitCodesmulti_string—Non-zero exit codes treated as success.
ExecStartPremulti_string—Commands run before the main binary, sequentially. §5.3
ExecStartPostmulti_string—Commands run after readiness or successful exit. §5.3
HookIdentitystring—Principal for the hook processes. Falls back to Identity. §4.1
ExecReloadstring—Reload command, or signal:<NAME>. Absent means SIGHUP. §6.5
PreStartCheckTimeoutdword5Seconds before a filesystem check helper is killed. §3.5
StartTimeoutdword30Seconds for the entire start sequence. §5.3
StopTimeoutdword10Seconds after SIGTERM before SIGKILL.
WatchdogTimeoutdword0Seconds between expected WATCHDOG=1 pings; 0 disables. §6.6
HealthCheckstring—Command run periodically. Exit 0 is healthy. §5.6
HealthCheckIntervaldword30Seconds between health checks.
HealthCheckTimeoutdword5Seconds before a health check is killed and counted failed.
HealthCheckRetriesdword3Consecutive failures before the service is unhealthy.
RestartPolicydword1 (OnFailure)0 Never, 1 OnFailure, 2 Always. §6.4
RestartMaxRetriesdword5Consecutive restarts before Failed. §6.4
RestartWindowdword120Seconds of sustained health that reset the restart counter.
RestartDelaydword1Seconds before a restart; doubles each consecutive failure, capped at 60.
Readinessdword0 (Notify)0 Notify, 1 Alive. Ignored for Oneshot.
NotifyAccessdword0 (Main)Who may send notifications. Main is the only mode. §10.5
FdStoreMaxdword0Maximum descriptors held for the service; 0 disables the store. §10.6
TimerPersistentdword1Catch up a missed timer run after a reboot. §9.3
TimerJitterdword0Maximum random delay added to each firing. §9.4
Environmentmulti_string—KEY=VALUE pairs added to the environment. §5.5
WorkingDirectorystring/Working directory for the process.
TTYPathstring—Terminal to attach as the standard streams and controlling terminal. §5.4
TTYPrecedencedword0Who wins that terminal when several services want it at once. Higher wins. Refused without a TTYPath. §11.6
RuntimeDirectoriesmulti_string—Private directories under /run, created before the main process.
LimitNOFILEdword—RLIMIT_NOFILE.
LimitCOREdword—RLIMIT_CORE, in bytes.
Conditionsmulti_string—Start-time conditions; failure skips the service. §3.5
Assertsmulti_string—Start-time assertions; failure fails the service. §3.5
DisplayNamestring—Human-readable name for status display.
Descriptionstring—What the service does.
ServiceSecuritybinaryinheritDescriptor controlling runtime operations on the service. §4.6

3.2.1 Registry types #

Schema typeRegistry type
stringREG_SZ, UTF-8
multi_stringREG_MULTI_SZ, an ordered list
dwordREG_DWORD, 32-bit unsigned
binaryREG_BINARY

A value whose registry type does not match the field's is a decode error. So is a dword carrying a value outside an enumerated field's range.

3.2.2 Schema version and forward compatibility #

Machine\System\Services\SchemaVersion is a dword, currently 1. peinit creates it if it is absent (§2.3).

Unknown values on a service key are ignored, which is what lets the schema grow additively: a definition written for a newer peinit still loads on an older one, minus the fields it does not understand. A newer schema version does not prevent boot.

Known fields are the opposite. A known field appearing more than once in a collected definition is a decode error rather than a last-one-wins, because a definition that says two different things about the same field has no defensible reading. Since names match case-insensitively, this catches ImagePath and imagepath in the same key.

3.2.3 What a decode failure costs #

A definition that fails to decode fails that one service, and the answer differs by caller.

At boot the key is marked Failed with cause ValidationError and the boot proceeds with every other definition. Anything that depended on the failed service fails in turn through the ordinary dependency propagation (§7.4), so the cost is bounded by what actually needed it.

On reload-config the same: that service is Failed with cause ValidationError — or, if it is running, left running with its definition marked removed — and every other definition loads (§10.4).

3.3 Field Formats

Peios / Peios technical reference / peinit / The Service Model

Rules that apply to how a field's value is written, as distinct from what it means.

3.3.1 Strings #

A string field that is present is non-empty, unless this section says otherwise for that field. Three fields treat the empty string as absence:

  • Identity — empty is the same as absent, and defaults to LocalService.
  • HookIdentity — empty is the same as absent, and falls back to the service's Identity.
  • DisplayName and Description — empty is the same as absent.

WorkingDirectory, when present, is a non-empty absolute path. Whether it exists, is a directory, and is reachable is checked when the service starts, not when the definition is read.

TTYPath is checked for emptiness before absoluteness: an empty value means no terminal, and a non-empty relative path is rejected.

3.3.2 Identity and HookIdentity #

Either a well-known principal name — SYSTEM, LocalService, NetworkService — matched case-insensitively and canonicalised, or a literal SID string such as S-1-5-18. Anything else is passed to authd verbatim to resolve (§4.3).

3.3.3 RequiredPrivileges #

Privilege names, matched case-sensitively against the published privilege table. A name that does not match exactly fails token materialisation and therefore the service start, so SeTCBPrivilege does not start a service that SeTcbPrivilege would.

3.3.4 RuntimeDirectories #

Each entry names one directory directly under /run. Entries are non-empty relative names; an entry equal to . or .. is rejected, as is any entry containing /, \, a NUL, or a control character. A dot inside a name is fine — app.sock.d is a valid entry.

For an entry foo, peinit creates /run/foo immediately before launching the service's main process, with a descriptor granting full access to SYSTEM, Administrators, and the service's own SID. Hook processes inherit the service's environment and identity rules but do not cause provisioning — the directories belong to the main start.

If creation or descriptor assignment fails, the start fails with ParentSetupFailure.

peinit does not remove runtime directories when a service stops. /run is a boot-scoped tmpfs and the next boot clears it.

3.3.5 Environment #

Each entry is KEY=VALUE, with a non-empty key containing no NUL. An entry that does not split into that shape is a decode error.

3.3.6 SuccessExitCodes #

Each entry is a decimal integer from 0 to 255 — a process exit code. Signal names and ranges are not accepted. Code 0 is always success and does not need listing. Duplicates are collapsed.

3.3.7 Dependency and handler names #

Entries in Requires, Wants, BindsTo and Conflicts, and the value of OnFailure, are validated when the definition is read. A dependency naming something outside [A-Za-z0-9._-] is a decode error rather than an unresolved dependency discovered later — the difference being that a typo containing an illegal character is caught immediately, while a typo that is still a legal name is caught at graph validation as a missing target.

An entry in the four list fields may carry a readiness level, written after a colon (netd:routed). The name and the level are validated separately, so the colon is not one of the illegal characters there. OnFailure takes a bare service name and no level: it names a handler to run, not a state to wait for.

3.3.8 Timeouts and intervals #

Every timeout and interval in the schema is in whole seconds unless the field says otherwise. The two sd_notify fields that carry durations, WATCHDOG_USEC and EXTEND_TIMEOUT_USEC, are in microseconds because that is what the protocol specifies.

3.3.9 Identifiers peinit generates #

Job identifiers, operation identifiers, and every other GUID peinit mints are UUIDv7. UUIDv7 is time-ordered, so identifiers sort by creation time — which is what keeps eventd's time-range and recency queries over jobs and operations cheap.

3.4 Triggers

Peios / Peios technical reference / peinit / The Service Model

A trigger says when a service starts by itself. Triggers are independent of service type: a Simple service can have a timer, and a Oneshot can start at boot.

Triggers is a multi_string, each entry either type or type:argument.

TriggerFormMeaning
BootbootStart during the Phase 2 boot sequence.
Deferred bootboot:settledStart once the boot set has settled, or the deadline expires. §2.5
Terminal handovertty:releasedStart when this service's TTYPath is let go of. §11.6
Timertimer:<schedule>Start on a schedule. §9.1

A service with no triggers is demand-only: it starts only when something asks for it, whether an administrator, a dependency, or an OnFailure handler.

Multiple triggers of the same type are allowed. A service with ["timer:*-*-* 02:00:00", "timer:*-*-* 14:00:00"] runs at 2am and 2pm, and each trigger is independent — its own timerfd, its own next-firing computation, its own history.

3.4.1 Arity is enforced #

boot takes no argument. boot:<something> accepts only the sub-triggers in the table above, and any other value is malformed rather than an unknown trigger type to be ignored. tty carries the same rule: the bare word names no event, and tty: accepts only released. timer with no schedule is malformed, as is timer: with an empty one.

That strictness is the point. Unknown values on a service key are ignored for forward compatibility, and if unknown trigger types were ignored too, boot:setled would be silently accepted and the service would simply never start. Instead it fails to decode, loudly.

3.4.2 Triggers that need a field #

tty:released is about the terminal this service names, so a definition carrying it without a TTYPath is refused rather than ignored — it describes a service that would silently never start, and starting is the entire point of the trigger. TTYPrecedence is refused on the same grounds. Emptying TTYPath (§3.3) takes both down with it.

3.4.3 Disabled #

Disabled=1 suppresses automatic activation and nothing else. No trigger fires: not boot, not boot:settled, not tty:released, not a timer, and not any trigger type added later. A disabled service's timers are neither armed nor serviced.

The definition is still loaded into the in-memory model, and an explicit start still works. To stop a service being started at all, deny SERVICE_START in its ServiceSecurity descriptor — that is an access control question, not a trigger question, and answering it with the Disabled flag would mean a flag that anyone who can write the key can clear.

Enabling and disabling are registry writes performed by administrative tools, not peinit commands. peinit picks the change up through a registry change notification, or on the next reload-config.

3.4.4 Extensibility #

The type:argument shape is designed to grow. Path, device and event triggers slot into the same array with no schema change, because a trigger is a string in a list rather than a field of its own.

3.5 Conditions and Asserts

Peios / Peios technical reference / peinit / The Service Model

Both are start-time checks in the same type:argument form, over the same four check types. What differs is the consequence of a failure.

CheckFormPasses when
pathpath:<path>The path exists, of any type.
filefile:<path>A regular file exists there.
directorydirectory:<path>A directory exists there.
registryregistry:<key>The registry key exists.

Conditions describe when a service applies. A failed condition skips the service: it transitions to Skipped, which satisfies its dependents, because a service that does not apply has succeeded by not needing to run.

Asserts describe what a service needs. A failed assert fails the service, with cause AssertionError — the service was expected to run and a precondition it depends on is missing.

All entries of a kind are AND'd. Conditions are evaluated first; asserts only if every condition passed. Both are evaluated before dependency resolution and before any pre-exec hook, and an entry with an empty argument or an unrecognised type is a decode error.

3.5.1 Evaluation without blocking #

peinit is single-threaded PID 1 and its event loop cannot block while a check runs. Two constraints follow, and they are why the check types behave differently from one another.

3.5.1.1 Registry checks are cache-only #

A registry: check is evaluated against the in-memory model, never by a live registry read. Accepted paths are exactly Machine\System\Services, Machine\System\Init, and Machine\System\Services\<name> with one nonempty service-name component. Init descendants, deeper Services paths, and trailing separators are decode errors, caught at load rather than silently failing at start. The same rule applies to Conditions and Asserts.

The two roots resolve as present. A registry:Machine\System\Services\<name> check is true when that service exists in the model; it does not read an arbitrary registry key. A missing service is a valid check whose result is false.

3.5.1.2 Filesystem checks run in a helper #

stat() can block uninterruptibly on hung I/O — a dead NFS mount, a failing disk controller — so peinit does not call it from the event loop. It forks a short-lived helper into a dedicated checks/ cgroup under the service's tree, using the same clone3 path as any other child. The helper stats the paths and reports over a non-blocking pipe; peinit waits on the pipe and the helper's pidfd through epoll, and never blocks.

PreStartCheckTimeout, default 5 seconds, bounds the helper. A check that does not report in time is treated as not satisfied — the fail-safe direction, so a condition skips the service and an assert fails it. peinit then SIGKILLs the helper's cgroup and unregisters the result descriptor, and the event loop is never held up by a hung check.

3.5.2 When results are computed #

Checks are evaluated once, before the service's dependencies start, and the result is cached for the rest of that activation. A service that waits a long time for a dependency starts on the answer that was true when the wait began, not on a fresh one.

3.6 Command Strings

Peios / Peios technical reference / peinit / The Service Model

Four fields hold executable commands: ExecStartPre, ExecStartPost, the command form of ExecReload, and HealthCheck. All four are parsed the same way.

3.6.1 Parsing #

The string is split on whitespace into an argv, with double quotes grouping. There is no shell — no expansion, no substitution, no globbing, and peinit never invokes one. A command that needs shell features is wrapped in a script.

Whitespace means exactly six characters: space, horizontal tab, line feed, carriage return, form feed and vertical tab. Every other Unicode whitespace code point is an ordinary argument character, which keeps the split independent of the Unicode version peinit was built against.

Double quotes group text into one argv element and are not retained in the result. Grouping may happen inside an argument, so --name="hello world" becomes the single entry --name=hello world. An empty quoted string is preserved as an empty argv entry.

Backslash has no escape semantics and is copied literally. A single quote is an ordinary character.

An empty or whitespace-only command is invalid, and so is an unclosed double quote.

3.6.2 The executable #

For all four fields, argv[0] is the executable path and begins with /. Relative names, empty names and PATH-searched execution are decode errors. Everything after argv[0] is an opaque string and is not path validated.

3.6.3 ExecReload signals #

ExecReload may instead name a signal, as signal:<NAME>. The name is an exact canonical Linux signal name. Numeric values, realtime signal expressions, aliases, lowercase spellings and names with surrounding whitespace are all invalid.

SIGKILL and SIGSTOP are invalid for reload, because a service cannot handle either as a request to re-read its configuration.

The accepted set is the standard non-realtime Linux signals other than those two:

SIGHUP, SIGINT, SIGQUIT, SIGILL, SIGTRAP, SIGABRT, SIGBUS, SIGFPE, SIGUSR1, SIGSEGV, SIGUSR2, SIGPIPE, SIGALRM, SIGTERM, SIGSTKFLT, SIGCHLD, SIGCONT, SIGTSTP, SIGTTIN, SIGTTOU, SIGURG, SIGXCPU, SIGXFSZ, SIGVTALRM, SIGPROF, SIGWINCH, SIGIO, SIGPWR, SIGSYS.

Names are validated when the definition is read and again before delivery, and mapped to host signal numbers at that point. ExecReload absent means SIGHUP.

3.7 Configuration Generations

Peios / Peios technical reference / peinit / The Service Model

peinit operates on snapshots, not on a live view of the registry. Two generation concepts govern when a registry change takes effect.

3.7.1 The in-memory model #

peinit maintains a full in-memory model of every service definition. The registry is read synchronously exactly twice: during Phase 2 boot, before meaningful supervision has started, and during a reload-config, which an administrator initiated and which is bounded. At all other times peinit works from the model.

Note

The model exists because the event loop cannot block on a userspace service during normal supervision. In single-threaded PID 1 a blocking syscall blocks everything — child reaping, watchdog expiry, shutdown signals, every other event stops while the syscall is stuck. Reading the registry means waiting on registryd, and registryd is a service peinit supervises.

Change notifications arrive as events on a pollable descriptor. peinit subscribes to Machine\System\Services\ and Machine\System\Init\ at boot, using the LCS watch mechanism — a persistent subscription that delivers change events on a key descriptor, with an OVERFLOW event when the kernel-side queue is exceeded (Peios Kernel TRM §5).

Any drained watch event triggers a full reload of the configuration, not a targeted re-read of the changed key. That covers the OVERFLOW case by construction, and it is why an administrator writing one value causes every definition to be re-read.

3.7.2 The boot generation #

At the start of Phase 2, peinit reads all definitions, builds the dependency graph and validates it. The plan and the graph are fixed at that point, and the boot executes against them.

The watches are armed as the event loop starts, which is after the plan is fixed but while boot-plan services are still starting. A registry write during that window — from an install script, a post-hook, a package transaction — triggers a reload like any other, and the reload applies like any other, with one exception: a boot-plan service whose launch has not yet been attempted keeps the definition the plan was built from, and the new one is recorded as pending, exactly as for a running service — the boot executes against its snapshot (§2.5). Such services are listed under deferred in the reload's summary, said on the console, and recorded as peinit.config.reload.deferred. Once every planned launch has been attempted or decided against — each service terminal, or in Backoff with its dependents held for it (§6.1) — peinit runs one more reload that applies what was deferred, recorded as peinit.config.reload.applied naming the deferred services in graph.services; nothing is emitted when nothing was deferred. A service already running keeps its pinned definition as always.

3.7.3 The activation generation #

When peinit starts a service, it snapshots that service's definition. The snapshot governs the whole start lifecycle: pre-exec hooks, the token request, the readiness timeout, the initial health checks. A field changed while the service is Starting does not take effect until the next start.

A service in Inactive or Failed has no activation snapshot, so starting it uses the current model. That gives the expected behaviour for the common edits:

  • A new service entry is available once the change notification is processed.
  • A timer change on an inactive service takes effect at the next trigger evaluation.
  • A dependency change takes effect on the next start for an inactive service, and on the next restart for an active one.

3.7.4 Field mutability #

Which class a field falls into depends on when its value is consumed.

3.7.4.1 Pinned to the running definition #

A change takes effect only when the service is restarted:

ImagePath, Type, Identity, RequiredPrivileges, ErrorControl, RemainAfterExit, Triggers, Disabled.

Triggers and Disabled are pinned only while the service is running. On a service that is not running, both take effect as soon as the notification is processed — which is what arms a timer added to an inactive service.

3.7.4.2 Applied on the next start #

A change takes effect at the next start or explicit graph reload, not while services are running:

Requires, Wants, BindsTo, Conflicts, OnFailure, Conditions, Asserts.

3.7.4.3 Reloaded at runtime #

A change takes effect at the next relevant event, with no restart:

Arguments, SuccessExitCodes, every timeout and retry value (StartTimeout, StopTimeout, WatchdogTimeout, RestartDelay, RestartMaxRetries, RestartWindow, PreStartCheckTimeout), HealthCheck and its three parameters, RestartPolicy, Environment, WorkingDirectory, ExecStartPre, ExecStartPost, ExecReload, HookIdentity, Readiness, NotifyAccess, LimitNOFILE, LimitCORE, FdStoreMax, TTYPath, TTYPrecedence, RuntimeDirectories, Provides, TimerPersistent, TimerJitter, SafeMode, DisplayName, Description, and ServiceSecurity.

ServiceSecurity is the one whose reload is immediately observable: a change takes effect on the very next control request against that service (§4.6).

3.8 Service Removal

Peios / Peios technical reference / peinit / The Service Model

When a definition disappears from Machine\System\Services\, peinit learns of it through the ordinary change-notification path. What happens next depends on whether anything is running.

3.8.1 Not running #

An entry in Inactive, Failed, Completed, Skipped, Abandoned or Backoff is discarded immediately. There is no process to consider.

Backoff belongs in this list rather than the next one, which is not obvious: a service between restart attempts looks like it has something pending. It does not. It has no process, and once the definition is gone there is nothing left to restart it from, so there is neither an instance to supervise nor a restart to wait for.

3.8.2 Running #

An entry in Active, Starting, Reloading or Stopping is not killed. The running process is a job, and a job's lifecycle is independent of the definition that produced it — removing a definition stops future management, it does not terminate work in progress.

peinit marks the entry definition-removed and keeps the cached definition, solely to go on supervising the instance it already has. When that instance exits it is not restarted — RestartPolicy is moot, because there is nothing left to restart from — and peinit then discards the entry.

A definition-removed entry never has dependents. A read in which one service Requires another the read does not define is refused whole, with MissingHardDependency — so a removal that would leave a dangling hard dependency does not land at all, and the administrator has to remove the dependents first.

While an entry is definition-removed:

  • stop is accepted, so an administrator can drain it cleanly, using the cached StopTimeout;
  • start, restart and reload are rejected with UNKNOWN_SERVICE — there is no definition to work from;
  • status reports the current runtime state with definition_removed set, so the draining instance is visible rather than silent.

definition-removed is a flag on the existing runtime state, not a state of its own. The state machine (§6.1) and the command × state matrix (§10.3) are unchanged by it.

A restart already in its stop phase is a case of its own. The stop phase still drains the existing instance; the start phase does not begin, because there is nothing to start from. When the instance exits, the operation is aborted with the reason definition_removed_during_restart_stop_leg and the entry takes the ordinary removal discard. This is the window a package upgrade opens — remove the definition, then restart the service.

Once the instance exits or is stopped, the entry — including anything in its fd store (§10.6) — is discarded. A crash counts: the exit is routed to Failed rather than into a restart, because there is no policy left to apply. The fd store goes with it, which a crash would otherwise preserve for the restart that is not going to happen.

4.1 Token Materialisation

Peios / Peios technical reference / peinit / Service Identity

Every service process runs with a KACS token that determines its identity and its access rights. peinit obtains or creates that token and installs it on the child before exec. It never shares its own token — even an Identity=SYSTEM service receives a separately materialised token of its own.

Which route the token comes from depends on the Identity field:

IdentitySourceMechanism
SYSTEMMinted by peinit from its own identitykacs_create_token. §4.2
ServiceauthdA virtual account unique to this service. §4.3
LocalService / NetworkServiceauthdA shared service account, with this service’s SID as a group. §4.3
Another principalauthdA principal designated for service logon. §4.3
Absent or emptyauthdDefaults to LocalService.

peinit reads its own token with kacs_open_self_token requesting the real token rather than any impersonation, and opens it query-only: it is a template to copy from, never a thing to hand out.

4.1.1 Where a token is materialised #

Materialisation happens at the point of use, per launched process, not once per service. A service that runs a pre-exec hook, a main process, and then a health check materialises three tokens.

ContextIdentity used
Main processIdentity
ExecStartPre / ExecStartPostHookIdentity if set, otherwise Identity
Health checksIdentity, always
ExecReload external commandIdentity, always
Submitted jobsThe prepared primary token: the submitter's own, or the token the kernel attached to the submission (§8.5)

Health checks and reload commands deliberately do not honour HookIdentity. A health check reports on the service's own health and should see what the service sees; a reload command acts on the running service. HookIdentity exists for setup work — creating directories in privileged locations, running a migration — which is a different job from either.

Note

HookIdentity typically grants more than the service itself. peinit does not validate filesystem permissions on hook binaries, so where it grants elevated privileges the administrator is responsible for the hook binary and its parent directories not being writable by anything lower-privileged.

If materialisation fails at any point — authd unreachable, an identity that cannot be resolved, a KACS error — no child exists yet, and the start fails with ParentSetupFailure for the main process or PreHookFailure for a hook.

4.2 The SYSTEM Path

Peios / Peios technical reference / peinit / Service Identity

For Identity=SYSTEM, peinit mints a token itself. This is what breaks the bootstrap circle: registryd and authd need tokens, and authd — the thing that mints tokens — is one of them. eventd and lpsd took this path too until they moved to accounts of their own (Identity=Service); they now take the authd path (§4.3) and start after authd.

4.2.1 Minting #

peinit reads its own token as a template and builds a new primary token carrying the same identity: user SID S-1-5-18, the same group list, the same privilege set, the same integrity level. The mint requires SeCreateTokenPrivilege, which the boot SYSTEM token carries; the kernel refuses the call with EPERM otherwise.

Two details of the copy matter.

The logon session comes from the token's statistics. peinit takes the auth_id from the source token's TokenStatistics — not from the independent interactivity_scope field, and not from a hard-coded well-known SYSTEM LUID. The minted token therefore stays associated with the real SYSTEM logon session peinit was given at boot, while carrying its own interactivity scope, which is zero for a platform service. Substituting either of the other two values would associate platform services with a session that does not exist.

The logon SID group is dropped from the copy. The kernel re-appends the session's logon SID when it creates the token, and rejects a create whose group list already contains it. So peinit filters that group out of the template before building.

Two groups are added that the template does not carry. The per-service SID (§4.4), which is what keeps platform daemons distinguishable while they all run as SYSTEM; and the Service group S-1-5-6, which every token authd mints for a service logon carries by derivation from the logon type.

The second is load-bearing rather than cosmetic. S-1-5-6 is the grantee on anything reachable by services as a class — peinit's own notification socket first among them (§10.5) — so a SYSTEM service minted here without it would be unable to report itself ready, while an otherwise identical service on an authd-minted token could. A service is no less a service for having been started before the authority was.

The minted token carries a default DACL the template cannot supply. A token's default DACL is the DACL an object gets when the token creates it with no parent to inherit from — an abstract socket, a key under a container written without inheritable ACEs. The kernel has no fallback of its own: a token whose default DACL is empty leaves such an object with a null DACL, which grants everything to everybody. The bootstrap SYSTEM token carries D:(A;;GA;;;SY)(A;;GA;;;BA) (see Bootstrap tokens), but a token's default DACL is not a field peinit can read back from its own handle, so peinit states the same value itself and sets it on every SYSTEM token it mints. Non-SYSTEM services do not pass through here; their default DACL is the authority's, from the DefaultDacl value of the principal's policy record (§4.3).

peinit also asserts that its own token is a primary token and that its user SID really is S-1-5-18 before minting, and fails the start with a message naming what it found otherwise. PID 1 minting from something that is not the boot SYSTEM token is not a situation to proceed from.

The minted token is fully independent. The privilege restriction that follows (§4.5) operates on it alone and cannot affect peinit's own.

Note

Minting is an interim mechanism. The intended model is for peinit to derive the token from its own handle, through a KACS duplicate-with-additions operation — kernel-attested as a descendant of peinit's real token, and requiring no token-minting privilege at all. That operation does not exist in KACS yet, so peinit mints.

4.3 The authd Path

Peios / Peios technical reference / peinit / Service Identity

For any identity other than SYSTEM, the token comes from authd. peinit does not resolve identities, does not know whether a principal is local or from a domain, and does not want to: routing is authd's whole purpose.

The division is not that peinit lacks the privilege to mint — it plainly has it, and uses it for the SYSTEM path (§4.2). It is that the privilege set and integrity level an identity carries are policy, and policy is authd's. A second component deciding it in parallel is a disagreement waiting to happen, and peinit has already shipped one: its hand-written privilege bit table had four wrong entries, silently stripping privileges a service had asked to keep.

4.3.1 The request #

peinit sends a ServiceAttest on /run/logon.sock, specified as PGSS Logon §2.19. It carries two strings:

FieldValue
identityThe definition's Identity value, verbatim
serviceThe name of the service being started

authd answers with AccessGranted carrying the token as an SCM_RIGHTS descriptor, or AccessDenied. There is no credential exchange, because a service identity has no credential and must never acquire one — a machine that could authenticate its own services unattended would have to hold their secrets.

authd derives the per-service SID from service and adds it to the token, so the SID that distinguishes one service from another comes from the authority on this path rather than from peinit (§4.4). Note that authd cannot verify service: the process it names does not exist yet, so there is no token to interrogate. peinit is trusted on it, which is the reason for the restriction below.

4.3.2 This request must come from PID 1 #

authd authorises a ServiceAttest on two facts: that the peer's token names SYSTEM, and that the peer is PID 1. The second is not redundant. SYSTEM alone is every platform daemon on the machine, so without it, compromising any one of them would yield a token for any service identity authd would mint.

That makes it a standing constraint on this crate: the connection is opened by peinit's own process and must never be opened by a forked helper. Nothing in the code enforces it. If it is ever broken the symptom is every non-SYSTEM service failing to start at boot with an authorisation error, which reads like a fault in authd rather than like a refactor.

4.3.3 The token is weaker than a user's, deliberately #

An attested token is minted one rung lower on the impersonation ratchet than a credentialled logon: Impersonation rather than Delegation.

The evidence behind it does not travel. A credential a principal source verified is something another machine could in principle have checked too; peinit's word is evidence on this machine only. A service token that could be forwarded off the box on the strength of it would let one system's PID 1 act as an arbitrary account on another.

Because the level is a ratchet, nothing derived from a service's token can exceed it either.

4.3.4 Failure is a failure #

Every non-SYSTEM service start depends on authd being reachable, and every one of them fails if it is not. Only services using the SYSTEM path are unaffected; platform services such as eventd and lpsd use the authority too.

peinit orders around that dependency rather than merely suffering it. The authority declares Provides = ["authn"], and peinit derives a hard dependency on whoever fills that role for every service whose Identity — or whose HookIdentity, where it has hooks — is not SYSTEM (§7.6). Nothing is written on the consumer side: the edge comes from the same field that creates the requirement, so a new service cannot forget it.

Until that existed the ordering was left to each definition, and all three LocalService definitions in the tree omitted it — so a service needing a token routinely reached this code before the socket was bound and failed its launch with ENOENT, on most boots, recovering only through its restart policy.

This is the intended behaviour rather than a limitation. The client that preceded it returned a freshly minted SYSTEM token whenever it could not do better, so a service declaring LocalService ran with peinit's entire privilege set — and status output reported it as LocalService, which is what kept the problem invisible from outside. A service that does not start is visible; a service running as the wrong principal is not.

summarize_token now refuses to describe a token as an identity it does not carry: where the declared identity predicts a user SID, a token carrying a different one fails the launch rather than being reported under the name that was asked for. It is only checked where the identity predicts a SID — a principal name is authd's to resolve, and peinit has nothing to compare it against.

4.3.5 What may be attested #

authd accepts the platform's own service identities — SYSTEM, LocalService, NetworkService, and the per-service virtual account selected by Service — which no principal source holds and for which no credential could exist.

Anything else must be designated in the principal's own record as usable for a service logon, and until that designation exists authd refuses it. So a definition naming an ordinary principal does not start. That is the property the design exists to guarantee: without it, the service manager would be an oracle that mints a credential-free token for anybody on the machine.

4.3.6 The service’s own virtual account #

Identity=Service asks authd to use the per-service SID (§4.4) as the user SID, as well as including it in the token’s groups. The spelling Service is case-insensitive. Each service name therefore selects a separate virtual account without creating an account in a principal source or enabling a credentialled logon.

The derived groups include Everyone (S-1-1-0), Authenticated Users (S-1-5-11), Local (S-1-2-0), and Service (S-1-5-6), together with the kernel’s per-session logon SID. It does not acquire SYSTEM or LocalService membership. Privileges, integrity, owner and default DACL still come from authd’s policy evaluation; choosing a distinct user SID grants no extra privileges by itself.

With Identity=LocalService, the user SID is the shared S-1-5-19 instead. The service’s own SID remains a group, so ACLs can still distinguish two services that share that account. Use Service when their user identities should also be distinct.

Service takes the same derived hard dependency on the providers of the authn role as other non-SYSTEM identities (§7.6). On the standard platform that provider is authd; an explicit Requires=authd is unnecessary.

4.3.7 The interface is authd's #

peinit speaks the protocol through libauthd, which is its definition, rather than encoding the message itself. A second implementation of a wire format that mints identity is the kind of duplication that fails silently and in the worst place.

4.4 Per-Service SIDs

Peios / Peios technical reference / peinit / Service Identity

Every service token carries a SID derived from the service's name, in its group list, alongside whatever the principal's own identity brings.

The derivation is the one KACS defines (Peios Kernel TRM §3.2). The authority is S-1-5-80. The service name is uppercased and encoded as UTF-16LE, SHA-1 is taken over those bytes, and the 20-byte digest is split into five little-endian 32-bit sub-authorities:

S-1-5-80-<sub1>-<sub2>-<sub3>-<sub4>-<sub5>

peinit computes this itself, from the name alone, with no involvement from anything else. authd computes the same value independently when it mints a token, and the two implementations are pinned against each other by a shared test vector.

4.4.1 Why they exist #

Per-service SIDs are what make access control useful when services share a principal. Every platform daemon runs as SYSTEM, and every service with no Identity runs as LocalService; without something to tell them apart, an ACL could grant a right to "LocalService" and thereby grant it to a dozen unrelated services.

With a per-service SID, an ACE can name one specific service. It costs nothing — no account, no registry entry, no allocation — because it is a hash of a name that already has to be unique.

They are load-bearing in more than access checks. peinit uses the service SID directly when it provisions a service's runtime directories, stamping /run/<name> with a descriptor that grants full access to SYSTEM, Administrators, and that service's SID and nothing else (§3.3).

4.4.2 The uppercasing rule #

The name is uppercased before encoding, using full Unicode case mapping — the one that expands ß to SS and fi to FI rather than mapping each code unit in place. peinit uppercases in the code-point domain, before the UTF-16 encoding, so any expansion happens first.

For an ASCII service name, which is every real service, the choice is invisible. It becomes visible only for a name containing a character whose uppercase form is longer than itself.

4.5 Privilege Restriction

Peios / Peios technical reference / peinit / Service Identity

RequiredPrivileges is a list of the privileges a service needs. Everything else is removed from its token before exec.

4.5.1 Subtractive only #

peinit removes privileges. It never adds one, on either path — there is no code that constructs anything but a removal. A service cannot acquire a privilege by naming it in RequiredPrivileges; if the token it was given does not have it, listing it changes nothing.

The removal targets the token's present bitmask, through KACS_IOC_ADJUST_PRIVS on the token descriptor — one kacs_priv_entry per removed privilege, carrying the privilege-removed attribute. The right required on the descriptor is KACS_TOKEN_ADJUST_PRIVS, which a freshly minted token always has.

peinit does not use KACS_IOC_RESTRICT. That builds a restricted-SID token — a different mechanism with different semantics — and does not touch the privilege bitmask at all. It is available in the bindings and would be a plausible-looking mistake.

Removing a privilege clears its present, enabled and enabled-by-default bits together, and irreversibly. The privileges that survive keep the enable state their source gave them: peinit does not enable, disable or re-order anything. Enable policy belongs to whoever minted the token.

peinit iterates all sixty-four privilege bits rather than only the ones it has names for, so a privilege this build does not know about is stripped along with the rest. The safe direction is to remove what was not asked for, including what cannot be named.

If RequiredPrivileges is absent, peinit does not query or adjust the token at all, and the source's default privilege set stands unchanged.

4.5.2 Names #

Privilege names are matched case-sensitively against the published privilege table. A name that does not match exactly is not silently ignored: it fails token materialisation, and therefore fails the service start.

Two privileges KACS enforces are absent from the published table — SeTakeOwnershipPrivilege and SeRelabelPrivilege — and so cannot be named in RequiredPrivileges at all. A service that needs either declares nothing and takes the source token's defaults, or fails to start if it tries to name one.

4.6 Service Security Descriptors

Peios / Peios technical reference / peinit / Service Identity

A service carries two independent descriptors, and they answer different questions.

The registry key descriptor on Machine\System\Services\<name> controls who may read and write the service's definition. It is enforced by LCS at key-open time and is not peinit's concern — peinit reads definitions as SYSTEM.

The ServiceSecurity descriptor controls who may perform runtime operations on the service through the control interface. It is stored as a binary ServiceSecurity value on the same registry key, but enforced by peinit rather than by LCS.

The two are genuinely independent. An administrator might be able to query a service's status without being able to read its configuration, or the reverse. Runtime control and configuration access are separate concerns and there is no reason for one to imply the other.

4.6.1 Access rights #

RightBitGrants
SERVICE_QUERY_STATUS0x0001Query state, PID, cause, health, warnings.
SERVICE_START0x0002Start the service.
SERVICE_STOP0x0004Stop the service.
SERVICE_INTERROGATE0x0008Reload the service.
SERVICE_ALL_ACCESS0x000FThe union of the four.

Restart requires SERVICE_START and SERVICE_STOP together. Reset requires SERVICE_STOP, because clearing a Failed or Abandoned state is the tail of stopping something rather than the head of starting it.

The generic mapping peinit passes to AccessCheck:

Generic rightMaps to
GENERIC_READSERVICE_QUERY_STATUS
GENERIC_WRITESERVICE_START | SERVICE_STOP | SERVICE_INTERROGATE
GENERIC_EXECUTESERVICE_START | SERVICE_STOP | SERVICE_INTERROGATE
GENERIC_ALLSERVICE_ALL_ACCESS

4.6.2 Inheritance and the default #

A service whose definition carries no ServiceSecurity value takes the one on Machine\System\Services itself. The lookup is a single step to that key, not a walk up the hierarchy, which is exact for the flat layout definitions actually use. That includes the compiled-in registryd, which takes it once Phase 2 has read the key and on every reload after; a registry definition of registryd (§2.3) may give it a descriptor of its own like any other service.

If that key has no ServiceSecurity either, peinit applies a built-in default:

O:SY G:BA D:(A;;0x000F;;;SY)(A;;0x000F;;;BA)(A;;0x0001;;;AU) S:(AU;FA;0x000F;;;WD)

SYSTEM and Administrators both get SERVICE_ALL_ACCESS. The SACL audits every refusal, for everyone, which is what records a denial (below). An administrator who may stop any service and shut the machine down gains nothing by being unable to start one, so the default does not attempt a narrower grant; a service that wants one carries its own ServiceSecurity value.

Every authenticated principal gets SERVICE_QUERY_STATUS: what a service is doing is not, by default, a secret from the people using the machine, and a service whose state is a secret carries a ServiceSecurity that says so.

4.6.3 The check #

When a control command arrives, peinit:

  1. Takes the caller's token, captured when the connection was accepted.
  2. Resolves the target service and its ServiceSecurity descriptor. A command naming no definition and no addressable definition-removed entry returns UNKNOWN_SERVICE; peinit does not invent a descriptor to check against.
  3. Calls AccessCheck with the caller's token, that descriptor, the generic mapping above, the right the command needs, and an audit context naming the service, {kind: "service", service: {name}}.
  4. On denial, returns ACCESS_DENIED. The attempt is recorded by KACS, not by peinit: when the descriptor's SACL asks for it — and the built-in default's does — the check writes a kacs.audit.access.checked record with object.kind service, object.service.name, the caller's token, and the access requested and granted (§8.4).
  5. On grant, proceeds.

4.6.4 The SACL is the audit policy, and who may drop it #

A service's SACL decides which of its decisions are recorded. A descriptor written to the registry is used as it is: one with no SACL records nothing, and one that narrows the default's audits less.

ServiceSecurity is a whole registry value. Whoever may write it — on the service's key, or on Machine\System\Services, whose value every service without its own takes — may replace the descriptor, SACL included, without holding SeSecurityPrivilege, which is what changing an object's SACL otherwise needs. So those keys must be writable only by principals trusted to change audit policy. By default they are: both inherit the Machine hive root, which grants write to SYSTEM and Administrators alone, and Administrators hold SeSecurityPrivilege. Delegate a service through its ServiceSecurity DACL, never by loosening the key's own descriptor; and a tool that changes only the DACL must write the SACL back unchanged. The same holds for the control descriptor in Machine\System\Init (§10.2).

4.6.5 Hot reload #

ServiceSecurity changes take effect on the next control request, with no restart. A registry change notification triggers a configuration reload, and the reload re-reads every descriptor. There is no cached decision to invalidate — the check runs against the current descriptor every time.

4.6.6 Filtering, not denying #

list returns only the services the caller has SERVICE_QUERY_STATUS on. Services the caller cannot query are omitted, not denied: a caller with no query rights anywhere receives an empty list and a successful response. The denials are not surfaced to the caller; each service's SACL decides whether KACS records them.

What the filtering protects is a service's state, which is what SERVICE_QUERY_STATUS grants. That a service exists is not protected: a command naming one is answered UNKNOWN_SERVICE or ACCESS_DENIED according to whether it exists, and the definitions under Machine\System\Services are readable. A client that can read the definitions may therefore show a service list left out as one whose state the caller may not see.

4.7 The Control Descriptor

Peios / Peios technical reference / peinit / Service Identity

Three commands are not about any one service: shutting the system down, re-reading the configuration, and asking how the machine booted. They are checked against peinit's own descriptor, stored at Machine\System\Init\ControlSecurity as a binary value.

4.7.1 Access rights #

RightBitGrants
SYSTEM_SHUTDOWN0x0001Initiate poweroff, reboot or halt.
SYSTEM_RELOAD_CONFIG0x0002Re-read all definitions from the registry.
SYSTEM_QUERY_STATUS0x0004Read peinit's own status: how this boot went, with boot (§10.2).

The generic mapping:

Generic rightMaps to
GENERIC_READSYSTEM_QUERY_STATUS
GENERIC_WRITESYSTEM_RELOAD_CONFIG
GENERIC_EXECUTESYSTEM_SHUTDOWN
GENERIC_ALLall three

GENERIC_READ maps to the one right that changes nothing, as it maps to SERVICE_QUERY_STATUS on a service. Before boot existed the control descriptor governed two actions and no queries, and GENERIC_READ on it conveyed nothing; a descriptor written then that grants GENERIC_READ now lets its holder ask how the machine booted, and nothing more.

4.7.2 The default #

Absent a value in the registry, peinit applies:

O:SY G:BA D:(A;;0x0007;;;SY)(A;;0x0007;;;BA)(A;;0x0004;;;AU)

SYSTEM and Administrators both get every right, as they both get every service right in the ServiceSecurity default (§4.6). An administrator who can stop services one at a time can already stop the system, so withholding shutdown would be theatre.

Authenticated Users get SYSTEM_QUERY_STATUS and nothing else. How a machine booted is machine status, like what /proc shows, not a secret; and it is what a person signed in to the machine needs to tell that it came up in Safe mode, or is a failed boot or two from recovery. The ServiceSecurity default lets the same group query every service for the same reason.

The default applies only while ControlSecurity is absent or malformed. A descriptor already written to the registry is used as it stands: one written before SYSTEM_QUERY_STATUS existed grants it only where it grants GENERIC_READ, GENERIC_ALL or the bit itself, and a machine that wants every signed-in user to read its boot status adds an ACE for them.

4.7.3 Loading #

peinit loads the descriptor during Phase 2 boot and hot-reloads it on registry change notification, on the same path as the service descriptors. Until it is loaded — during Phase 1 and the early part of Phase 2 — the built-in default applies, which matters because the control socket exists from Phase 1 infrastructure setup onwards.

5.1 The Cgroup Tree

Peios / Peios technical reference / peinit / Starting a Service

peinit uses cgroups v2 for exactly two things: knowing which processes belong to a service, and killing all of them at once. It does no resource accounting and sets no limits.

Every service gets a tree:

/sys/fs/cgroup/peinit/<cgroup-id>/          service root
/sys/fs/cgroup/peinit/<cgroup-id>/main/     the main process
/sys/fs/cgroup/peinit/<cgroup-id>/hooks/    hooks and reload commands
/sys/fs/cgroup/peinit/<cgroup-id>/health/   health check invocations
/sys/fs/cgroup/peinit/<cgroup-id>/checks/   pre-start filesystem check helpers

The sub-cgroups satisfy cgroups v2's "no internal processes" rule, which applies whenever controllers are enabled, and give hooks and probes containment of their own so that killing one does not touch the service.

They are not all created at once. The root and hooks/ are created when the first thing needs them — the first pre-exec hook, if there is one — and main/ and health/ when the main process launches.

5.1.1 The cgroup id #

<cgroup-id> is the service name with every byte outside [A-Za-z0-9._-] percent-encoded as % plus two uppercase hex digits. Service names are already restricted to that set (§3.1), so in practice the id equals the name.

The encoding is a defensive guarantee that distinct names always map to distinct, cgroup-safe ids. % is not itself in the safe set, so it escapes to %25 and the encoding is prefix-free. That is what makes it injective, unlike a plain substitution of / for -, under which a/b and a-b would collide.

The id is internal. The name a user sees is unchanged.

5.1.2 Generations #

A cgroup whose processes survived SIGKILL cannot be removed — rmdir on it fails with EBUSY. peinit records the leak and moves the service to a fresh tree:

/sys/fs/cgroup/peinit/<cgroup-id>%gen<N>/

Old leaked trees persist until the next reboot.

Two conditions record a leak, and both are needed because they answer different questions. cgroup.events reporting populated at the post-kill deadline says live processes remain. rmdir returning EBUSY says the tree cannot be given up, which covers that case and the ones where nothing is running but the directory still will not go. Acting on only the first left the second reusing a tree that already existed and was not empty.

The generation advances once per tree, not once per leak recorded against it. A single failed start records two cleanup deadlines — one for hooks/, one for the service tree — and a tree cleanup can report main/, hooks/, health/ and the root separately; all of those are the same tree, so the first advances the generation and the rest find themselves already behind it. Leaks are still deduplicated by path and kind, so each is recorded once.

The separator is %gen, not .gen. . is in the safe set and service names permit it, so a service literally named app.gen1 would have shared a directory with generation 1 of a service named app. % self-escapes and the encoder only ever emits it followed by two uppercase hex digits, so %g cannot occur inside an encoded id — which makes the whole path injective, not just the id.

5.2 Pre-Start Evaluation

Peios / Peios technical reference / peinit / Starting a Service

Before anything is forked, peinit evaluates the service's conditions and asserts (§3.5). The definition comes from the in-memory cache, so this never touches the registry.

  1. If the service names a TTYPath and another service is holding that terminal, it goes to Skipped with cause TtyUnavailable and the start is abandoned (§11.6). Asked first because a terminal is a fact about the machine right now, and evaluating conditions ahead of it would fork a check helper for a start that was never going to happen.
  2. Conditions are evaluated. Any failure transitions the service to Skipped and abandons the start. Skipped satisfies dependents.
  3. If every condition passed and the service has asserts, they are evaluated. Any failure transitions the service to Failed with cause AssertionError and abandons the start.

Only when all three pass does the pre-exec sequence continue.

5.2.1 Where the transition happens #

For a fresh start from Inactive, the evaluation gates the transition: the service becomes Starting only after the checks pass, so a skipped service goes straight Inactive → Skipped.

For an activation that is already Starting — the start leg of a restart, for instance — the transition has already happened, and the evaluation gates further progress instead. A restart whose conditions no longer hold therefore passes through Starting on its way to Skipped, where a fresh start would not.

An assert failing takes the same route to Failed on both paths, and on both it propagates: dependents in the same graph context hear that the dependency failed (§7.3). This matters more on the restart leg than elsewhere, because an AssertionError is never restarted — where a running dependent is deliberately unaffected by a dependency crashing (§7.1), since a crash is transient and the restart policy handles it, a failed assert means the dependency is not coming back without intervention.

5.2.2 Two check kinds, two mechanisms #

registry: checks resolve against the in-memory model. Since a non-cacheable key is rejected when the definition is read, this evaluation never needs a live read.

Filesystem checks — path:, file:, directory: — run in a forked helper. peinit clones it with CLONE_PIDFD | CLONE_INTO_CGROUP into the service's checks/ sub-cgroup, exactly as it launches anything else. The helper stats the paths and writes the results to a non-blocking pipe; peinit watches the pipe and the helper's pidfd through epoll.

The terminal is re-checked when the helper's results come back, rather than the earlier answer being carried over : the helper ran in between, and a terminal can change hands while it did.

PreStartCheckTimeout bounds the helper, at 5 seconds by default. On expiry every check still outstanding is marked not satisfied — the fail-safe direction — and re-evaluated on that basis, so a condition skips the service and an assert fails it. peinit then kills the helper's cgroup and unregisters the result descriptor.

A helper that survives the kill, stuck in uninterruptible sleep, has its cgroup abandoned and recorded exactly as a leaked hook or health cgroup is (§5.7).

5.2.3 Caching #

The results are computed once, before the service's dependencies are started, and reused for the rest of that activation. A service that waits a long time on a dependency starts on the answer that was true when the wait began.

Filesystem checks are gathered in one helper run, from the conditions and the asserts together. There is only ever one run — the completion path evaluates both lists against the results it gets back, and there is no mechanism to ask for a second — so both lists' paths have to be stat'd before it. A path named in both is stat'd once.

A check with no result counts as not satisfied, which is the fail-safe direction for a timeout and is why the gather has to be complete: an omitted path is indistinguishable from a path that is not there.

5.3 The Pre-Exec Sequence

Peios / Peios technical reference / peinit / Starting a Service

Everything between "peinit decides to start service X" and "X's binary is running". The service is in Starting throughout.

5.3.1 Step 1: Arm the start timeout #

StartTimeout covers the entire remaining sequence — pre-hooks, fork, exec, and the readiness wait. A single deadline, measured from the operation's creation rather than from this point, bounds all of it.

Expiry aborts the start and kills the service's cgroup tree. The cause recorded depends on where the deadline landed : a timeout during pre-hooks is PreHookFailure; one during the readiness wait or the main process's execution is ReadinessTimeout.

5.3.2 Step 2: Provision runtime directories #

If the definition lists RuntimeDirectories, peinit creates each one under /run and applies its descriptor (§3.3) before anything else in the launch. Failure classifies as ParentSetupFailure.

5.3.3 Step 3: Run pre-exec hooks #

Each ExecStartPre command runs in sequence, forked into hooks/. A token is materialised for each at the point of use, from HookIdentity if set and Identity otherwise (§4.1); a failure there fails the hook and the service with PreHookFailure.

Any hook exiting non-zero kills the entire service cgroup tree — cleaning up whatever grandchildren the hook left — and fails the service with PreHookFailure.

When every hook has succeeded, peinit kills the hooks/ sub-cgroup before the main process starts, so a hook that forked something into the background does not become part of the service.

5.3.4 Step 4: Materialise the service token #

The main process's token, per §4.1. A failure means no child exists and the service fails with ParentSetupFailure.

If a later parent-side step fails before the fork, peinit closes the token descriptor exactly once. A failure to close is retained as cleanup evidence and does not change how the start is classified — the classification describes what went wrong with the start, and a failed close is a separate fact about the same failure.

5.3.5 Step 5: Create the cgroups and the error pipe #

The main/ and health/ sub-cgroups are created, and a pipe2(O_CLOEXEC) error pipe. The parent keeps the read end, non-blocking; the child will hold the write end.

The pipe is how the child reports a failure it hits after fork but before exec. If exec succeeds, the write end closes automatically — that is what O_CLOEXEC is for — and the parent reads EOF, which means setup succeeded. If setup fails, the child writes a structured error first.

The payload is exactly eight bytes, written with one write(2):

BytesContent
0–3little-endian u32, the child setup step identifier
4–7little-endian i32, a positive Linux errno

The child writes at most one payload — the first reportable failure — and then exits. The parent treats a non-EOF payload that is not exactly eight bytes, or carries an unknown step identifier, or an errno of zero or less, as malformed evidence and fails closed with PreExecFailure.

The step identifiers:

IdStep
1Close the read end of the error pipe
2Set the standard streams
3Reset the signal environment
4Install the service token
5Set the resource limits
6Set oom_score_adj
7Set the working directory
8reserved
9Confirm NOTIFY_SOCKET
10Inject stored descriptors
11Exec the binary
12Create a session
13Acquire the controlling terminal

Identifier 8 is reserved and never emitted : the environment is built in the parent and applied by execve, so there is no step in the child that could fail. Identifiers 12 and 13 are the two terminal steps, which occur only for a service with a TTYPath.

If pipe2 fails, no child exists and the service fails with ParentSetupFailure.

5.3.6 Step 6: Fork #

clone3(CLONE_PIDFD | CLONE_INTO_CGROUP), targeting main/. This does two things atomically: it returns a pidfd for the child, and it places the child directly into main/ at creation. There is no window in which the child exists without a pidfd, and none in which it runs or execs in peinit's own cgroup.

Placing the child at creation also avoids the alternative, which would be having the child write its own PID into cgroup.procs — something its post-installation token could not do.

A clone3 failure means no child exists: ParentSetupFailure, with EMFILE/ENFILE, EAGAIN and ENOMEM the usual causes.

peinit does not take the atomicity on trust. If clone3 reports success without setting the pidfd, or the pidfd cannot be made close-on-exec, peinit writes to cgroup.kill through the main/ directory descriptor before returning ParentSetupFailure — because by then a child does exist, running in the service's cgroup with the service's token, and the parent has no handle on it. Walking away would leave a process peinit cannot supervise, cannot stop, and has no record of, which the zero-delay cleanup that follows a ParentSetupFailure would then record as a leaked tree rather than kill.

Immediately after clone3 returns, in the parent and on either outcome, peinit closes the token descriptor and the main/ cgroup descriptor, exactly once each. The child relies on close-on-exec and its own exit instead. A failure to close is cleanup evidence and does not change how the start is classified.

5.3.7 Step 7: The parent after the fork #

  1. Close the write end of the error pipe.
  2. Register the read end with the event loop. peinit does not block on it — signals, control traffic, timers and log pipes stay serviceable while the child's setup is pending.
  3. When the read end becomes readable or hangs up, read it non-blocking:
    • Would block: leave the source registered, the job stays in pending setup.
    • EOF: exec succeeded. Record the pidfd on the job, emit peinit.job.started, and only then apply readiness side effects such as Simple/Alive activation.
    • Data: setup failed. Parse the step and errno, log the specific failure, fail the service with PreExecFailure.

While setup is pending the job is not Running, and no dependent that waits on this service's readiness is released.

5.3.8 Steps 8 and 9 #

The child's own path is §5.4. What follows exec is the readiness wait and the post-readiness work:

  • Simple with Readiness=Notify: wait for READY=1, then Active.
  • Simple with Readiness=Alive: Active as soon as exec succeeds.
  • Oneshot: wait for the exit. Success is Completed; without RemainAfterExit it then goes Inactive once dependents are released. A non-zero exit is Failed.

On readiness or successful exit, peinit runs ExecStartPost in sequence into hooks/, releases the service's dependents, kills hooks/, and — for a Simple service only — arms the watchdog if WatchdogTimeout is non-zero and the health check timer if HealthCheck is set.

A post-hook that fails is logged and does not fail the service. The state transition to Active or Completed happens before the post-hooks run, so a service is already Active while its post-hooks are executing.

5.3.9 Failures before the fork #

Steps 2, 4, 5 and 6 can all fail with no child in existence. peinit handles all four entirely in the parent: clean up whatever cgroups were created, fail the service with ParentSetupFailure, and return the error to the caller. These are system-level resource problems — file descriptor limits, PID limits, memory, cgroup filesystem errors — rather than anything about the service.

5.4 The Child Path

Peios / Peios technical reference / peinit / Starting a Service

Between clone3 returning in the child and execve, peinit runs a straight line of setup. The path is kept minimal — no logging, no complex library calls — because it runs after a fork, where very little is safe to do.

Each step reports through the error pipe with the identifier from §5.3 if it fails, then exits: _exit(126) for a setup failure, _exit(127) for a failed exec.

The steps, in the order the child runs them:

#StepId
1Close the read end of the error pipe1
2setsid() — only with a TTYPath12
3Set the standard streams2
4ioctl(TIOCSCTTY) — only with a TTYPath13
5Reset the signal environment3
6Set RLIMIT_NOFILE and RLIMIT_CORE5
7Set oom_score_adj6
8Install the KACS token, then close its descriptor4
9Change the working directory7
10Confirm NOTIFY_SOCKET is present in the environment9
11Inject stored descriptors from fd 3 upward10
12execve11

5.4.1 The terminal steps #

setsid() comes first, and its position is load-bearing in two directions. It has to precede the stream setup, because setsid() drops any controlling terminal the child inherited — doing it afterwards would throw away the terminal just attached. And it has to precede TIOCSCTTY, which requires a session leader that does not already own a controlling terminal.

TIOCSCTTY is passed a literal zero argument, which is what makes it unable to steal a terminal already owned by another session.

Without a TTYPath neither step runs and the service stays in peinit's session.

5.4.2 The standard streams #

Without a TTYPath: stdin from /dev/null, stdout and stderr onto the write ends of the service's output pipes, and every inherited pipe end that is no longer needed closed.

With a TTYPath: all three streams onto the opened terminal, and the /dev/null descriptor and both pipe pairs closed. A terminal-attached service's output is therefore not captured for logging — it goes to the terminal, which is the point of asking for one.

5.4.3 The signal environment #

peinit blocks every signal for its signalfd (§12.3) and the child inherits that mask across the fork. Step 5 empties the mask and resets every resettable disposition to SIG_DFL. A service starting with signals blocked, or with PID 1's handling in place, is one of the classic ways for a daemon to behave inexplicably.

5.4.4 oom_score_adj #

-1000 — OOM-immune — for an ErrorControl=Critical service, and 0 for everything else. A Critical service is one whose loss reboots the machine, so letting the OOM killer choose it would convert memory pressure into a reboot.

5.4.5 Resources before the token #

The resource limits and oom_score_adj are set while the child still holds peinit's credentials, and only then is the service token installed. Both are peinit's policy rather than authority the service holds: raising a hard limit or lowering oom_score_adj needs CAP_SYS_RESOURCE, which KACS projects from SeIncreaseQuotaPrivilege. Setting them after the install would make every Critical service, and every service with a raised limit, depend on that privilege. A RequiredPrivileges list that leaves it out would then stop the service from starting at all.

The step identifiers are unchanged by this order. They name steps, not positions.

5.4.6 The environment #

There is no step that sets the environment, which is why identifier 8 is reserved and never emitted. peinit builds the environment in the parent (§5.5) and hands it to execve as envp, so it arrives with the exec rather than being installed beforehand. Step 10 is a check rather than a set: it confirms NOTIFY_SOCKET is present in the prebuilt environment and fails with a synthetic EINVAL if it is not.

5.4.7 What the child does not inherit #

A service inherits only what peinit hands it: its standard streams and any descriptors injected from the fd store. Everything else peinit holds is created close-on-exec — the control socket and every accepted connection, the notification socket, the epoll instance, the signalfd, every timerfd, both pidfds, every pipe, and every stored descriptor until it is deliberately un-marked at injection.

The signal reset and the close-on-exec discipline together are what make a service start from a clean context rather than from PID 1's privileged one.

A descriptor opened through the Peios native file interface returns without close-on-exec set, so it does not get the discipline for free. peinit sets the flag by hand on each of those — the input devices it opens for the power button, the main/ cgroup directory descriptor, /dev/null, and the service and recovery consoles — immediately after opening them. The child duplicates its intended streams onto descriptors 0, 1 and 2 without close-on-exec, then explicitly closes the original descriptors. The streams survive exec; the original handles do not.

5.5 The Base Environment

Peios / Peios technical reference / peinit / Starting a Service

peinit constructs every service and hook process's environment from scratch, in four layers, lowest precedence first. Nothing is inherited : peinit's own startup environment holds TERM and nothing else (§2.1), and none of it is passed through.

5.5.1 Layer 1: the compiled-in base #

One variable:

VariableValue
PATH/sbin:/bin

Executables are addressed through the root-level StrataFS runtime views. Package storage paths under /usr are deliberately not on the default search path.

5.5.2 Layer 2: global environment variables #

Each value under Machine\System\Init\EnvVars\ becomes a variable: the value name is the variable name, the REG_SZ data is the value. An EnvVars\PATH overrides the compiled-in PATH; every other name adds.

A malformed entry — an empty name, or a name containing = — fails the whole layer, which at boot means recovery mode.

registryd does not receive this layer. The exemption is a trust rule, not an availability one. Write access to EnvVars\ is equivalent to compromising every service peinit starts: LD_PRELOAD, LD_LIBRARY_PATH and their relatives are not filtered, because the key's Security Descriptor is meant to be the control boundary. A key that could inject into the daemon that enforces who may write it would make that boundary self-referential.

The exemption is narrow, and matches on two things at once: the job's resolved identity is SYSTEM and its service name is registryd. A non-platform service that happens to be called registryd receives the ordinary layering. It matches on the job, so a hook of registryd's running under a non-SYSTEM HookIdentity would receive the layer; and it matches the resolved identity string, so a definition naming S-1-5-18 literally rather than SYSTEM would not be exempt.

registryd is launched in Phase 1, before EnvVars has been read at all, so the exemption is only observable on a restart.

5.5.3 Layer 3: the service's own Environment #

The definition's Environment entries, overriding both layers below.

5.5.4 Layer 4: protocol variables #

NOTIFY_SOCKET, always. LISTEN_FDS, LISTEN_FDNAMES and LISTEN_PID, only when descriptors are being injected — from the fd store for a service, or from a submission's attachments for a submitted job (§8.5).

These have the highest precedence. The first three are inserted after both configurable layers, and all four names are dropped from those layers before insertion, so a service cannot override NOTIFY_SOCKET and break its own notification protocol, and an EnvVars\LISTEN_FDS=3 cannot point an fd-store-less service's sd_listen_fds at whatever happens to sit at descriptor 3. The name filter is what protects the LISTEN_* set when it is not being set.

LISTEN_PID, which a conforming sd_listen_fds implementation checks against its own PID before trusting LISTEN_FDS, is appended by the child itself after the clone, because only then is the PID known; the name filter keeps a configured value from shadowing it.

5.5.5 What peinit does not set #

Not HOME, USER, LOGNAME, SHELL or TERM. Peios identity is a KACS token — a SID — rather than a passwd entry, so there is no canonical home directory or login shell to populate. A service that needs one supplies it through EnvVars\ or its own Environment; a submitted job's submitter supplies it in the submission's environment, which occupies layer 3 for a job.

5.5.6 Hooks and probes #

Hooks, health checks and reload commands are built through the same path, so they receive the identical environment: the same layers, the same NOTIFY_SOCKET, and the service's WorkingDirectory, LimitNOFILE, LimitCORE and RequiredPrivileges. They never receive LISTEN_FDS — stored descriptors go to the main process only.

5.5.7 When changes apply #

The global layer is a snapshot refreshed at boot and on reload-config, and both it and the per-service Environment take effect at a service's next start. Neither is applied to a running process.

Note

Write access to Machine\System\Init\EnvVars\ is equivalent to compromising every service on the system. peinit does not filter variable names, consistent with the registry's write-authority threat model — the key's descriptor is the boundary. The recommended default is SYSTEM full control and Administrators read-only.

5.6 Health Checks

Peios / Peios technical reference / peinit / Starting a Service

A watchdog tells peinit that a service is still ticking. A health check tells it that the service still works. They address different failures: a process can be alive and responsive to its own event loop while having lost its database connection, wedged in a bad state, or started returning errors to everyone.

5.6.1 Execution #

The health check command runs with the service's own token — never HookIdentity — so it checks the service's health from the service's own vantage point.

Each invocation runs in an ephemeral health/ sub-cgroup under the service's tree, as a child of peinit rather than of the service. When the check completes or times out, peinit kills the whole sub-cgroup, which cleans up anything the check spawned.

5.6.2 Overlap #

If the previous check is still running when the next interval fires, the new one is skipped and nothing is counted. A check exceeding HealthCheckTimeout has its sub-cgroup killed and is counted as a failure.

A launch failure — a token that could not be materialised, a fork that failed — is not counted. The invocation is recorded and the next interval schedules normally, and nothing about the service changes.

The distinction is between "the probe ran and said the service is unhealthy" and "the probe could not be run". Only the first is evidence about the service. Collapsing them meant a transient authd unavailability could kill a service outright: with HealthCheckRetries=1, a reasonable setting for a probe an operator trusts, one failed token materialisation exhausted the budget and restarted it.

5.6.3 Failure #

HealthCheckRetries consecutive failures mark the service unhealthy. An unhealthy service is restarted through the ordinary restart policy: RestartPolicy, exponential backoff and throttling all apply, exactly as for a crash. The failure count resets the moment a check succeeds.

Escalation kills the service's root cgroup rather than just main/, so it takes hooks and probes with it.

5.6.4 The flap constraint #

Restart throttling is what stops a service flapping — failing checks, restarting, passing initial checks, failing again. But it only works if the failure cycle is shorter than RestartWindow, because otherwise the service stays healthy long enough between failures to reset the restart counter, RestartMaxRetries is never reached, and it restarts forever.

So this relationship has to hold:

HealthCheckRetries × HealthCheckInterval < RestartWindow

It is enforced as an error rather than a warning, in both places a definition can arrive. At boot, a violating service is blocked with cause ValidationError and never started. On reload-config, it is a validation finding, and a finding rejects the entire reload.

The constraint applies only where a check will actually run, which means Simple services. A Oneshot cannot flap through health checks because it never runs one.

A Oneshot that declares a HealthCheck is still rejected — but for declaring one at all, not for its timing. That is the thing actually wrong with the definition; reporting it as timing arithmetic let an operator adjust RestartWindow, see the definition validate, and still have a HealthCheck that does nothing.

Note

Active health checks on ErrorControl=Critical services deserve caution. A false positive eventually exhausts the restart budget and reboots the machine. Critical services — registryd, authd, lpsd, eventd — are usually better served by a passive watchdog with a generous timeout: the service knows whether it is healthy and can stop sending keepalives when it decides it is not. peinit does not prevent health checks on Critical services and applies identical semantics to them, but the escalation path ends at a reboot.

5.7 Leaked Sub-Cgroups

Peios / Peios technical reference / peinit / Starting a Service

A process in uninterruptible kernel sleep — D-state, typically a hung NFS mount or a failing disk controller — does not die when it is SIGKILLed. Its cgroup cannot be removed while it is there.

peinit detects this through cgroup.events: after sending the kill it arms a post-kill deadline, 5 seconds by default, and checks whether populated is still 1 when the deadline fires.

5.7.1 What happens depends on which cgroup it is #

For the main process, a survivor is fatal to supervision: the service transitions to Abandoned with cause ProcessUnkillable and peinit stops supervising it (§6.1).

For a health check or a hook, it is not. Those are diagnostic and setup processes; they hold no service resources — no ports, no file locks, no database connections — so a stuck one does not make the service unmanageable. peinit orphans the sub-cgroup instead:

  1. Marks it leaked, recording the path, the kind and the time.
  2. Increments the service's cgroup generation, so the next start builds a fresh tree (§5.1).
  3. Carries on supervising the service normally.

A leaked pre-start check helper is treated the same way: its checks/ sub-cgroup is recorded and the generation bumped, which matters because otherwise the next start would build into a tree that still contains the unkillable process.

The leaked cgroup stays in the hierarchy until the next reboot.

A tree the service's next instance is already running in is not checked. A restart can relaunch before the post-kill deadline of the instance it replaced — the default restart delay is one second, the post-kill deadline five — and it relaunches into the same tree, since nothing has leaked yet to move the generation on. When the deadline fires, the tree is populated by the live instance, not by a survivor, so peinit drops the check rather than record a leak. A survivor of the killed instance that shares that tree goes unreported until the tree is next killed and checked.

5.7.2 Visibility #

Leaks are not silent. peinit both pushes one when it is detected and keeps it queryable afterwards.

The push happens once, on first detection — recording is idempotent, so a leak that is re-examined on a later cleanup pass is not announced again:

  • A peinit.cgroup.leaked event on the event stream, carrying the service, the sub-cgroup path (object.cgroup.path) and its kind (object.cgroup.type: service-tree, health, hooks or helper). When it was found is the event's own time.

  • A console line naming the same service, kind and path.

The pull side survives the moment of detection, for anyone who was not watching:

  • A status query includes a warnings array, one entry per leak, each an object with the sub-cgroup path, its kind, and the time of detection:

    {"path": "/sys/fs/cgroup/peinit/jellyfin/health", "type": "health", "detected_at": "2026-06-01T12:34:56.123456789Z"}
    
  • A start command on a service with leaks returns a warning in its acknowledgement, saying the service has leaked sub-cgroups from a previous generation and that this indicates an I/O problem needing investigation.

The type is the same vocabulary everywhere : health for a leaked health/ sub-cgroup, hooks for a leaked hooks/ one, helper for a pre-start check helper's checks/, and service_tree for a leaked service root — the last being the most serious, since it means the whole tree including main/ could not be reclaimed.

Whichever way it reaches you, what the leak means is the same: something underneath the service is not responding to the kernel, and no amount of restarting the service will fix it.

6.1 States

Peios / Peios technical reference / peinit / The State Machine

Every service is in exactly one state. There are ten.

StateProcess?Satisfies dependents?Meaning
InactiveNoNoNot started, or stopped cleanly and not set to restart.
StartingMaybeNoActivation in progress: checks, hooks, fork, or the readiness wait. The process may not exist yet.
ActiveYesYesRunning and ready.
ReloadingYesYesRe-reading configuration. Still satisfies dependents.
StoppingBrieflyNoSIGTERM sent, awaiting exit or SIGKILL escalation.
CompletedNoYesOneshot only. Exited successfully.
BackoffNoNoA restart is pending; waiting out the backoff delay.
FailedNoNoExited abnormally with the restart policy exhausted or absent.
AbandonedYes, unkillablyNoSIGKILL sent and processes survived. Supervision has stopped, the cgroup is leaked.
SkippedNoYesConditions were not met. The service does not apply.

6.1.1 Dependent satisfaction #

Exactly four states satisfy dependents: Active, Reloading, Completed and Skipped. Nothing else does, and a dependent blocked on a Requires target in any other state does not start.

Completed satisfies regardless of RemainAfterExit — a Oneshot without it passes through Completed to release its dependents on the way to Inactive, rather than skipping the state.

Skipped satisfies because a service whose conditions do not hold has succeeded by not needing to run. Treating it as a failure would make every conditional service a hazard to everything that depends on it.

Reloading satisfies because the process is still there and still serving; a reload is a service telling itself to re-read a file, not an outage. So a reload does not restart the RestartWindow clock (§6.4), clear a readiness level the service has published (§7.5), or count as its target going away to a BindsTo dependent (§7.1).

Backoff does not, and the distinction from Failed matters: a service in Backoff is going to start again, and its dependents wait rather than failing. It is the state that makes a restart something other than a transit through Failed. A dependent held this way looks exactly like a start held on a readiness level — Inactive, its start operation Pending (§8.2) — and is released when the service next reaches a dependent-satisfying state. It fails with dependency_failure only when the service gives up: its restart budget is exhausted, or it is stopped or its definition withdrawn while in Backoff.

6.1.2 Invariants #

  1. A service is in exactly one state at any moment. There is one state field and exactly one place in the implementation that assigns to it.
  2. Only the transitions in §6.2 are performed. The assignment is gated by a central whitelist, so an unlisted transition is not merely avoided by convention — it cannot be written.
  3. Only peinit transitions a service. The control socket produces operations; nothing outside peinit writes state.
  4. A service object is securable independently of its process token. The ServiceSecurity descriptor governs who may manage the service; the process token governs what the service may reach. Neither implies anything about the other.
  5. Readiness is per start generation. The generation increments on every transition into Starting, and a READY=1 carrying a stale generation is rejected — so a notification from a previous incarnation can never be mistaken for this one's.

6.2 Transitions

Peios / Peios technical reference / peinit / The State Machine

Every transition peinit performs. Anything not listed here is not performed.

FromToTrigger
InactiveStartingStart command, dependency resolution, or a timer trigger.
InactiveSkippedA condition check failed.
InactiveFailedValidation, assertion, cycle or dependency failure — before any process existed.
StartingActiveReadiness. Simple only: READY=1 received, or the process exists under Readiness=Alive.
StartingCompletedOneshot exited successfully. Without RemainAfterExit it passes through, releasing dependents, then goes Inactive.
StartingSkippedA condition failed after the activation had already entered Starting.
StartingFailedAn assert failed after entering Starting; or a timeout, hook failure, setup failure, or an exit before readiness.
StartingBackoffA startup failure with the restart policy allowing a retry and budget remaining.
StartingStoppingAn explicit stop cancelling an in-progress start. A restart on a Starting service is queued rather than cancelling.
StartingFailedThe shutdown wave SIGKILLed a starting service.
ActiveReloadingExecReload issued, or SIGHUP delivered.
ActiveStoppingExplicit stop, conflict eviction, a bound dependency stopping, or shutdown.
ActiveBackoffA crash, a watchdog timeout, or health check failure, with a restart allowed and budget remaining.
ActiveFailedThe same three, with RestartPolicy=Never or the budget exhausted.
ActiveInactiveA Simple service exited successfully and RestartPolicy is not Always. Cause CleanExit.
ActiveBackoffA Simple service exited successfully and RestartPolicy=Always. Cause CleanExitRestart.
ReloadingActiveReload resolved: READY=1, the detection window expiring, the extended wait expiring, or the reload command exiting — success or failure.
ReloadingStoppingThe same triggers as Active to Stopping. The reload is cancelled.
ReloadingBackoffThe main process exited during the reload, with a restart allowed.
ReloadingFailedThe same, with no restart available.
StoppingInactiveThe process exited after an explicit stop or a shutdown.
StoppingFailedThe process exited after a conflict eviction or a bound dependency stopping.
StoppingAbandonedSIGKILL sent and main/ still populated after the post-kill deadline.
StoppingStartingAn explicit start queued against a service that is still stopping. Cause ExplicitStart.
BackoffStartingThe backoff delay elapsed. Cause RestartPolicy.
BackoffInactiveAn explicit stop cancelled the pending restart.
BackoffSkippedThe backoff delay elapsed and the relaunch's pre-start checks skipped the service: its terminal is now held (§11.6), or a condition no longer holds.
BackoffFailedThe backoff delay elapsed and peinit could not execute the relaunch. Cause InternalError.
Starting, Active, Reloading, StoppingFailedpeinit could not carry out a step on the service and contained the error to it (§8.2). Cause InternalError.
CompletedInactiveRemainAfterExit=0 and dependents released; or an explicit stop; or shutdown clearing it.
CompletedStartingA start command or timer trigger re-running the Oneshot.
FailedStartingAn explicit start, a bound dependency recovering, or a timer trigger.
FailedInactiveA reset command clearing the state.
FailedAbandonedA service SIGKILLed by the shutdown wave whose cgroup stayed populated past the post-kill deadline.
AbandonedInactiveA reset command.
SkippedInactiveA reset command, or an explicit start clearing Skipped before it re-evaluates the conditions.

6.2.1 Things the table settles #

A restart never passes through Failed. A restart-eligible failure goes to Backoff, waits, and goes to Starting. Failed is reached only when there will be no retry: RestartPolicy=Never, an invalid policy for the cause, or an exhausted budget. This is why OnFailure (§6.3), which fires on entry to Failed, does not fire on each retry — only when the service finally fails out.

A clean exit is not a crash. A Simple service exiting zero goes to Inactive under cause CleanExit, consulting neither the restart policy nor the budget. It goes to Backoff only under RestartPolicy=Always, and then with the distinct cause CleanExitRestart, so status and events say plainly that the process succeeded and was restarted by policy rather than that anything went wrong.

A forced stop remembers why. Stopping to Failed carries the cause from the transition that started the stop — ConflictEviction or BindsToPropagation — rather than a generic failure. A service that lost a conflict and a service whose binding target went away are distinguishable afterwards, which is what makes bound-dependency recovery (§7.1) possible at all.

A restart detours through Inactive. The stop leg of an administrator's restart ends in Inactive, and the start leg begins from there. The two legs meet within one turn of the event loop, and peinit answers no request mid-turn, so no query ever sees the service Inactive in between.

A crash before readiness may be retried. A Simple process that exits before signalling readiness is a ProcessCrash from Starting, and is restart-eligible like any other, rather than a terminal startup failure.

A process can outlive its service's state. Most states are reached by the main process exiting, but not all of them: a watchdog timeout, a health check escalation and the post-kill give-up all move a service on while its process is still running. The exit that follows arrives for a service in Backoff, Abandoned or another state that no longer expects one.

peinit records that exit and performs no transition. The service keeps the state it had, any pending restart deadline stands, and the console reports it:

peinit: service <service> main process exited in state <state>; no action taken

This is deliberately not a fatal condition. A supervised service producing an unexpected sequence costs that service; it does not cost the machine. Treating it as a broken invariant of the runtime loop meant one crash-looping service dropped PID 1 into recovery mode and killed every session on the box.

6.2.2 Reset from Abandoned #

Resetting an Abandoned service re-checks its main/ sub-cgroup. If it has finally emptied, peinit cleans up the whole service tree — main/, hooks/, health/, then the root — and transitions to Inactive.

If it is still populated, peinit leaves the cgroup leaked, transitions to Inactive anyway, and returns this warning in the operation acknowledgement:

abandoned main cgroup for service <service> is still populated after
reset -- cgroup remains leaked; underlying D-state process requires
investigation

The warning is also written to the console, so a reset issued without reading the response still leaves a trace of the still-leaked cgroup.

The re-check targets main/. Note that the two paths into Abandoned probe different cgroups: an explicit stop checks main/, while the shutdown wave checks the service root.

6.3 Transition Causes

Peios / Peios technical reference / peinit / The State Machine

Every transition carries a cause recording why it happened. peinit tracks both the current state and the cause of the most recent transition, and the cause determines restart eligibility, OnFailure behaviour, and what an administrator is told.

6.3.1 The taxonomy #

CauseLeads toMeaning
ExplicitStartStartingAn administrator, an OnFailure handler, or a boot plan started it.
ExplicitStartInactiveAn explicit start on a Skipped service, clearing it so the conditions are re-evaluated.
DependencyStartStartingStarted to satisfy another service's dependency.
RestartPolicyStartingAn automatic restart after a backoff delay.
BindsToRecoveryStartingA bound dependency returned to Active.
TimerStartingA timer trigger fired.
ExplicitStopStoppingAn administrator requested a stop.
ExplicitReloadReloading, ActiveA reload was issued and resolved.
ExplicitResetInactiveAn administrator cleared Failed, Abandoned or Skipped.
ConflictEvictionStoppingA conflicting service started; this one lost.
BindsToPropagationStoppingA bound dependency stopped.
ShutdownWaveStopping, FailedThe system is shutting down. Active and Reloading services go to Stopping; Starting services go straight to Failed.
ProcessCrashFailed, BackoffThe main process exited unexpectedly.
CleanExitInactiveA Simple process exited successfully with a policy other than Always.
CleanExitRestartBackoffA Simple process exited successfully under RestartPolicy=Always.
ReadinessTimeoutFailed, BackoffStartTimeout expired before readiness.
WatchdogTimeoutFailed, BackoffA keepalive did not arrive in time.
HealthCheckFailureFailed, BackoffHealthCheckRetries consecutive failures.
PreHookFailureFailed, BackoffAn ExecStartPre hook exited non-zero, or its token failed, or the start timed out during hooks.
ParentSetupFailureFailed, BackoffA parent-side failure before the fork. No child was created.
PreExecFailureFailed, BackoffPost-fork setup failed before exec, reported through the error pipe.
DependencyFailureFailedA Requires dependency entered Failed.
RestartBudgetExhaustedFailedRestartMaxRetries reached.
CycleDetectedFailedThe service is part of a dependency cycle.
ValidationErrorFailedThe definition failed graph validation.
AssertionErrorFailedA start-time assert failed.
ConditionSkippedSkippedA start-time condition failed.
TtyUnavailableSkippedAnother service was holding the terminal this one names in TTYPath. §11.6
ProcessUnkillableAbandonedProcesses survived SIGKILL.
InternalErrorFailedpeinit could not carry out a step on the service — a relaunch, a job terminal, a setup status, a health probe, a deadline action. The service did nothing wrong; its operation fails with the internal_error result (§8.2).

6.3.2 Restart eligibility #

Causes fall into four classes, and the class decides whether the restart policy is consulted at all.

Restart-eligible. peinit consults RestartPolicy and the budget. If a restart is allowed and the budget holds, the service goes to Backoff and then to Starting; otherwise Failed.

ProcessCrash, WatchdogTimeout, HealthCheckFailure, ReadinessTimeout, PreHookFailure, PreExecFailure, ParentSetupFailure.

Note

The startup failures are restart-eligible because a transient problem during startup should get another go. A pre-hook that failed because a network mount was briefly unavailable deserves a retry rather than an immediate give-up.

Always-only. CleanExitRestart applies when a Simple service exits successfully and the policy is Always. It uses the same backoff and the same budget as a failure, which is what stops a daemon that exits cleanly in a tight loop from bypassing throttling entirely. It is never treated as a ProcessCrash, and both status and events make clear the process succeeded and was restarted only because the policy says so.

CleanExit is its non-restarting counterpart: straight to Inactive, consulting neither policy nor budget.

Budget-exempt. BindsToRecovery takes a service from Failed to Starting when its binding target returns. It is not subject to the policy or the budget, because the service did not fail on its own — it was stopped because its dependency went away.

Never-restart. peinit does not consult the policy at all: ExplicitStop, ExplicitReset, ShutdownWave, ConflictEviction, BindsToPropagation, ProcessUnkillable, RestartBudgetExhausted, ValidationError, CycleDetected, DependencyFailure, AssertionError, ConditionSkipped, TtyUnavailable, InternalError. Retrying cannot help with any of them.

6.3.3 OnFailure #

When a service enters Failed and its definition names an OnFailure service, peinit starts that service — with the exceptions below. OnFailure fires on entry to Failed, and Failed to Failed is not a transition, so it fires at most once per failure.

It does not fire for:

  • ShutdownWave — no new service starts during shutdown, so a fallback would be both impossible and pointless.
  • ValidationError, CycleDetected, DependencyFailure, AssertionError — these are definition or graph breakage rather than runtime degradation. A broken definition cannot meaningfully trigger a fallback, and the fallback would probably sit in the same broken graph.

It fires for everything else, including ProcessCrash, WatchdogTimeout, HealthCheckFailure, the startup failures, and RestartBudgetExhausted on a non-Critical service.

For a Critical service exhausting its budget, the reboot takes precedence and no fallback is started. The suppression asks whether a reboot is owed rather than inspecting ErrorControl itself, which is the same question §6.4 puts.

OnFailure is for graceful degradation — the main web interface fails, so start a minimal emergency endpoint. It is not for monitoring or alerting, which is eventd's job.

6.3.3.1 A handler that is already running #

If the named handler is already Active — or Reloading, on its way back to Active — the failure starts nothing. A running service has no state to transition into on being started again, and it is already doing the job the failure would have asked of it, so the start resolves as satisfied: no operation is created, and no loop-guard chain entry is recorded for a cascade that did not happen.

This is the same answer the control interface gives an administrator who starts an already-running service, and it is deliberately the same test. A handler in any other state is started normally — so a Oneshot handler that ran for an earlier failure runs again for a new one, and a handler that has since failed is restarted.

6.3.3.2 The loop guard #

An OnFailure handler can fail and carry its own OnFailure, so a misconfiguration where A's handler is B and B's is A could run forever. peinit bounds the chain originating from one failure two ways: it tracks the set of services already started as handlers for that failure and will not start one already in the set, and it will not follow the chain past a fixed depth of 16. When either trips, peinit records a peinit.on-failure.suppressed event naming which (outcome.reason cycle or max-depth), and stops.

A chain entry is retired two ways.

The handler stopped needing tracking. Completed, Inactive, Skipped or Abandoned — it ended, gave up, or became unkillable.

The handler arrived. It has been dependent-satisfying for its RestartWindow — the same window that resets a restart budget (§6.4). That is peinit's existing test for a service having genuinely taken over rather than merely come up, and it is what ends the originating failure: a later failure of that handler is a new one, starting a fresh chain.

The distinction matters in both directions. Without the second rule, a handler that runs healthily for days holds a slot forever, so a degradation path handing off through several healthy layers exhausts its own depth budget by working. But retiring on Active alone would break the guard entirely: Readiness=Alive reports Active the instant the process spawns, so two services naming each other as handlers would hand off forever, with no delay and no cost — an OnFailure start is not a restart and spends no restart budget. Only a handler that held is treated as having arrived.

6.3.4 The logging contract #

A transition into Failed, Skipped or Abandoned is written to the console, naming the service and its cause: what failed, and why. There is no per-transition event — a transition is visible through the peinit.job.* and peinit.operation.* events that carried it, not as a record of its own.

Cryptic failure messages are a defect. A reboot loop caused by a configuration error with an opaque message is the worst outcome the system has, and the cause taxonomy exists so that the "why" is never a guess.

6.4 Restart

Peios / Peios technical reference / peinit / The State Machine

When a restart-eligible cause occurs, or a Simple clean exit produces CleanExitRestart, peinit evaluates the restart policy. The outcome is either Failed, or Backoff followed by Starting.

evaluate_restart(service, cause):
    // 1. Policy.
    if cause is never-restart:
        return STAY_FAILED
    if cause == CleanExitRestart and policy != Always:
        return INVALID_CAUSE_FOR_POLICY
    if policy == Never:
        return STAY_FAILED
    if policy == OnFailure:
        // A termination counts as success only when the cause is a
        // process exit whose code is in SuccessExitCodes. Only
        // ProcessCrash and CleanExitRestart carry an exit code, and
        // CleanExitRestart was handled above. Every other eligible
        // cause has no exit code and is always a failure here.
        if cause == ProcessCrash and exit_code in success_exit_codes:
            return STAY_FAILED
    // A clean terminal logout is not a failure.
    if cause == CleanExitRestart and TTYPath is set:
        consecutive_failures = 0
        return RESTART_AFTER(min(restart_delay, 60))
    // Other Always exits fall through.

    // 2. Budget.
    if consecutive_failures >= restart_max_retries:
        cause = RestartBudgetExhausted
        if error_control == Critical:
            sync and reboot
        return STAY_FAILED

    // 3. Delay.
    delay = min(restart_delay << consecutive_failures, 60)

    // 4. Schedule.
    return RESTART_AFTER(delay)

RESTART_AFTER puts the service in Backoff for the delay; when it elapses the service transitions to Starting and the ordinary activation sequence begins, with its own fresh StartTimeout.

The elapsed backoff timer is consumed when the relaunch is admitted. If a pre-start check needs an asynchronous filesystem helper, the service remains in Backoff until that check finishes, retaining its terminal reservation. The pending check drives progress; the expired timer cannot admit another restart while it waits.

The relaunch runs the pre-start checks like any other start, and the machine can have changed while the service waited. If its terminal is now held (§11.6) or a condition no longer holds, the service goes from Backoff to Skipped, with the cause the check gives, and its pending operation completes as a skip. A relaunch that peinit itself cannot execute — its bookkeeping refuses the start — takes the service from Backoff to Failed under InternalError (§6.3), fails the operation waiting on the restart with the internal_error result, and reports both on the console; it does not cost the machine.

The exit code is available only when peinit observed a process exit, and it is carried into the evaluation wherever there was one — including the pre-readiness exit, where a Simple service exits before signalling READY=1. So SuccessExitCodes means the same thing on both sides of readiness: a service that legitimately concludes "nothing to do" during startup and exits with a listed code is not restarted.

Failures with no process exit behind them — a hook that never ran, a readiness deadline, a dependency failure — carry no code, and the SuccessExitCodes branch cannot apply to them. That is not a gap: a success code is a statement about how the service's own process ended.

6.4.1 The policies #

PolicyValueBehaviour
Never0Never restart. The service stays Failed.
OnFailure1Restart on a non-zero exit or a runtime failure. An exit matching SuccessExitCodes is not restarted.
Always2Restart on any failure regardless of exit code, and for a Simple service also on a successful clean exit.

For a Oneshot with RestartPolicy=Always, a successful exit is not restart-eligible. RestartPolicy governs the response to failures; a Oneshot that succeeds has done its job. It goes to Completed — and then Inactive without RemainAfterExit — whatever the policy says, and only a non-zero exit reaches the restart evaluation at all. Timer triggers are the mechanism for re-running a Oneshot on a schedule.

6.4.2 Backoff #

The delay doubles on each consecutive failure, starting from RestartDelay and capped at 60 seconds. The arithmetic is overflow-safe, so a large RestartDelay or a long failure run saturates at the cap rather than wrapping.

While a service is in Backoff it is down and does not satisfy dependents. An explicit start creates or merges into a deferred start operation, which honours the remaining delay rather than short-circuiting it; a stop cancels the pending restart and takes the service to Inactive.

For a terminal-attached service, a successful Always exit is a normal session logout. It resets the failure counter and uses the base delay (capped at 60 seconds), without spending or exhausting the crash budget. Failed exits and startup failures still use the ordinary budget and exponential backoff.

6.4.3 The budget, and when it resets #

consecutive_failures counts consecutive restart-eligible failures. It resets to zero only after the service has stayed Active for RestartWindow seconds. It is not a count of restarts within a trailing window, and the difference is what the mechanism turns on.

peinit stamps the moment a service becomes dependent-satisfying, and clears that stamp on any transition to a non-satisfying state — so a crash restarts the clock. The reset fires when the stamp plus RestartWindow is reached with the service still Active.

A service that recovers and stays Active longer than RestartWindow between crashes therefore never exhausts its budget: each crash starts from a counter of zero. Only failures recurring faster than the service can sustain a window of health accumulate.

Two other events also zero the counter, both of which mean the service is no longer in a failure run: a clean exit to Inactive, and an administrative reset. An explicit stop while in Backoff does not — the accrued failures are preserved.

Because the reset requires the service to be Active, a service that happens to be Reloading when its window boundary passes misses that reset and gets it on returning to Active.

6.4.4 Exhaustion #

Once RestartMaxRetries restarts have happened without the service sustaining a window of health, the next failure is not restarted: Failed with cause RestartBudgetExhausted. peinit then applies ErrorControl:

  • Normal — the service stays Failed.
  • Critical — peinit syncs the filesystems and reboots immediately. The reboot takes precedence over OnFailure.

The reboot does not depend on how the budget was exhausted. The paths that observe a terminal outcome for a running service — the main job ending, a health check failing or timing out, the watchdog expiring — raise it directly. Every other route reaches it through a check made once per runtime turn, which asks a single question of the service table: is a Critical service Failed with cause RestartBudgetExhausted? That covers a budget exhausted purely by startup failures — repeated ReadinessTimeout, PreHookFailure, ParentSetupFailure — where a service that can never get as far as running would otherwise have settled in Failed with neither escalation.

The OnFailure suppression asks the same question rather than inspecting ErrorControl itself, so the handler is skipped exactly when a reboot is owed and never on the strength of one that was not scheduled.

A failure during a shutdown reboots nothing (§12.2 step 2), and the handler is not started either — the machine is already going down.

6.5 Reload

Peios / Peios technical reference / peinit / The State Machine

Reload tells a service to re-read its configuration without restarting. peinit issues the reload, moves the service to Reloading, and resolves it one of three ways.

A failed reload never takes a running service out of Active. And reload never gets stuck: every path has a timeout.

6.5.1 Choosing a path #

ExecReload absent means SIGHUP to the main process. A signal:<NAME> value means that signal instead. Anything else is a command, forked into the service's hooks/ sub-cgroup under the service's own identity — never peinit's token, and HookIdentity does not apply.

6.5.2 The signal path #

There is no command exit to observe, so completion is inferred from the main process's own notifications.

start the detection window (2 seconds)

on READY=1, at any time:
    -> Active, "confirmed"

on RELOADING=1 within the window:
    cancel the window
    start the extended wait (StartTimeout)

on the window expiring with no RELOADING=1:
    -> Active, "advisory"

on the extended wait expiring with no READY=1:
    -> Active, "advisory"

The two-second window is a constant and is not configurable through the registry, directly or otherwise. It is not clamped to the operation's own deadline: a service configured with a short StartTimeout, which is a reasonable thing to do for something that starts fast, still gets its full two seconds to answer a reload. The reload operation's own lifetime already bounds the whole thing.

The extended-wait expiry means the service announced a reload and never finished one. That is what the detection protocol exists to catch — the service has wedged mid-reload or lost its handler — so it is reported rather than only returned:

peinit: service <service> signalled RELOADING=1 but never completed reload

and audited as a peinit.service.reload.timed-out event. The outcome is still carried in the operation's result for a wait=true caller, but a reload issued without waiting — the default — no longer resolves silently.

A detection window expiring is a different thing and stays quiet. A service that does not implement the handshake lets it expire on every reload, and reporting that would drown the case above.

6.5.3 The command path #

The command's exit gates failure; the main process's READY=1 gates confirmation.

on the command exiting non-zero:
    -> Active, "failed"

on the command exceeding StartTimeout:
    SIGKILL the hooks/ sub-cgroup, taking its descendants
    -> Active, "failed"

on the command exiting zero:
    -> Active, "confirmed" if the main process sent READY=1 during the
       reload, otherwise "advisory"

6.5.4 Auto-detection #

The protocol needs no per-service configuration. A service that implements the notification handshake — RELOADING=1 then READY=1 — gets real lifecycle tracking. One that does not gets a brief Reloading state that resolves itself when the detection window expires. Neither has to declare which it is.

6.5.5 Interruptions #

The main process crashes while Reloading. That is a ProcessCrash, and the restart policy is consulted: Reloading to Backoff if a restart is allowed and the budget holds, Reloading to Failed otherwise. Both reload timers are cancelled and any in-flight reload command is killed.

This is a different event from an external reload command exiting non-zero, which is the "failed" outcome above and leaves the main process — and the service — running.

A stop arrives while Reloading. peinit cancels the reload immediately: it drops the reload deadlines, kills any reload command's cgroup, and sends SIGTERM in the same turn, without waiting out the window or the extended wait. The service goes to Stopping and the reload operation is aborted.

6.6 The Watchdog and Timeout Extension

Peios / Peios technical reference / peinit / The State Machine

Two notification fields let a running service adjust the deadlines it is held to. Both are authenticated exactly as any other notification (§10.5), and both carry microseconds.

6.6.1 The watchdog #

WatchdogTimeout sets the interval peinit expects WATCHDOG=1 pings at. Zero, the default, disables it. Missing a ping is a WatchdogTimeout cause and takes the ordinary restart path.

A service may change the interval at runtime by sending WATCHDOG_USEC=<value>:

  • A value greater than zero updates the interval and re-arms immediately — the current timer is cancelled and a fresh one starts from the moment the message was received, rather than the new interval applying only from the next ping.
  • A value of zero disables the watchdog entirely, equivalent to WatchdogTimeout=0.

The runtime value does not persist. On a restart the interval reverts to the definition's WatchdogTimeout converted to microseconds, and if that is zero the watchdog starts disabled whatever the previous incarnation had set.

WATCHDOG_USEC is honoured only while the service is Active. A service that sends it while still Starting — before its own READY=1 — is ignored and gets the definition's value.

Note

Runtime watchdog updates suit a service whose phases have genuinely different latency. A database engine might want a tight five-second watchdog in normal operation and sixty seconds during a compaction pass. The service knows its own phases better than whoever wrote its definition does.

6.6.2 Timeout extension #

A service may ask for more time during a start, stop or reload by sending EXTEND_TIMEOUT_USEC=<value>.

peinit sets the current phase's deadline to expire that many microseconds from now. The extension replaces the deadline rather than adding to it — each message sets an absolute deadline computed from its own arrival — and may be sent repeatedly.

Because it replaces, a small value shortens the remaining time rather than being ignored, and a value of zero sets the deadline to now.

6.6.2.1 The caps #

The extended deadline cannot exceed four times the phase's base timeout:

PhaseBaseCeiling
StartingStartTimeoutStartTimeout × 4
StoppingStopTimeoutStopTimeout × 4
ReloadingStartTimeoutStartTimeout × 4

A value beyond the cap is clamped, not rejected — the message succeeds and the deadline becomes the maximum permitted. Because the cap is anchored to when the operation started rather than to the previous deadline, repeated messages cannot creep past it.

During shutdown an additional cap applies: the deadline cannot exceed the time remaining in the global ShutdownTimeout, and where both caps apply the stricter wins.

6.6.2.2 Where it does not apply #

A message arriving while the service is in a non-transitional state — Active, Completed, Failed — is ignored. There is no deadline to extend.

During shutdown, as at any other time, an extension moves only a deadline the service already has. A service still in a transition when the shutdown begins — starting, reloading, or stopping under an explicit stop — extends that transition's deadline, and one already stopping keeps its stop deadline, extension included, into its wave (§12.2). A service that is Active until its stop wave reaches it has no deadline to extend before then, so a request it sends earlier is ignored like any other in a non-transitional state. A service that expects its stop to take longer sends the request once its stop has begun.

Note

Timeout extension is for a service doing variable-duration work in a transition — replaying a write-ahead log on startup, say. It sends periodic messages to prove it is making progress; if it stops sending, the last deadline fires and peinit escalates as usual. The fourfold cap is what stops a buggy service extending forever.

7.1 Relationships

Peios / Peios technical reference / peinit / Dependencies

Four relationship types. Each says something different about how two services interact at start, at stop, and on failure.

7.1.1 Requires #

A hard dependency. If A Requires B:

  • Start. B has to reach a dependent-satisfying state before A starts. If B is not running, peinit starts it with cause DependencyStart. If B enters Failed, A goes to Failed with cause DependencyFailure without attempting to start.
  • Stop. Stopping B does not stop A. This is a start-ordering constraint, not a runtime coupling.
  • Runtime failure. If B crashes while A is Active, A is unaffected and keeps running. B's own restart policy handles B.
  • Missing target. A goes to Failed with DependencyFailure, detected at graph validation.

7.1.2 Wants #

A soft dependency. If A Wants B, peinit starts B before A — with cause DependencyStart — provided B exists and is not disabled. If B fails to start, or does not exist at all, A starts anyway. There is no stop or failure effect in either direction.

The waiting rule is where the difference from Requires actually lives: a dependent blocked on a Requires target waits for it to reach a satisfying state, while a dependent blocked on a Wants target waits only for it to reach any terminal state, satisfying or not. That is what makes Wants ordering rather than dependency — "start this first if you can, but I will work without it".

7.1.3 BindsTo #

A runtime coupling. If A BindsTo B:

  • Start. Identical to Requires.
  • Stop. If B stops for any reason — explicit stop, conflict, crash, shutdown — A stops too, transitioning to Stopping with cause BindsToPropagation.
  • Recovery. When B returns to Active, peinit automatically restarts anything sitting in Failed with cause BindsToPropagation from B's stop. This is reactive rather than polled: peinit watches for the transition into Active from a non-satisfying state, and reacts to it. These restarts do not consume the restart budget — the dependent never failed on its own, it was stopped because its binding target went away.

BindsTo implies Requires. A definition may list both for clarity, and if it does, the BindsTo semantics apply.

7.1.4 Conflicts #

Mutual exclusion. Starting A while B is Active creates a stop operation for B — source ConflictResolution, cause ConflictEviction — and A does not start until B has left every state in which it could still be running.

Conflicts are symmetric. If A declares Conflicts = ["B"], starting either one stops the other; B does not have to declare it reciprocally, and peinit scans both a starting service's own conflicts and everything that declares a conflict against it.

If both A and B are boot-triggered and they conflict, graph validation detects an unresolvable conflict and fails both with ValidationError.

A missing conflict target is silently dropped — there is nothing to conflict with.

Note

Conflicts is for real mutual exclusion: two services binding the same port, or two implementations of one role where exactly one must run. It is not a way to solve resource contention that has a better answer.

7.1.5 Ordering and self-reference #

Dependencies imply start ordering, and — except for BindsTo — imply nothing about stopping. peinit starts dependencies before dependents; during shutdown it reverses the same graph and stops dependents before dependencies (§12.2), derived from the one graph rather than from a separate stop-ordering configuration.

A service may not name itself in any dependency field. peinit rejects a self-reference at graph validation as a CycleDetected failure, which is what it is — a cycle of length one.

7.2 Graph Validation

Peios / Peios technical reference / peinit / Dependencies

peinit validates the dependency graph before executing it. Validation runs once per graph build — at boot, on an administrator-requested on-demand start's transitive closure, and on a reload-config — and is never incremental.

7.2.1 Cycles #

peinit topologically sorts the graph; if the sort fails, a cycle exists. Detection returns all cycles rather than stopping at the first: each detected cycle's members are removed and the search re-run until the graph is clean.

Every service in a cycle is failed with cause CycleDetected, and the cycle path is logged so an administrator can see what to break.

If any service in a cycle is Critical, peinit downgrades to Safe mode (§2.6) rather than rebooting. The cycle is a configuration error, and rebooting would find it again.

7.2.2 Missing targets #

RelationshipA missing target means
RequiresThe dependent is failed with DependencyFailure.
BindsToThe same — treated as a missing Requires.
WantsSilently dropped.
ConflictsSilently dropped.

This holds in every mode. Safe mode drops a hard dependency on a service it excluded — that is its own rule (§2.6), and without it excluding a service would fail everything downstream and Safe mode could start almost nothing. Only that case drops. A target missing from the registry or disabled by an administrator is a configuration error rather than a Safe mode exclusion, and blocks the dependent in Safe mode exactly as in Full.

7.2.3 Validation errors #

A service that hits one is failed with cause ValidationError and never started:

  • The flap constraint, HealthCheckRetries × HealthCheckInterval < RestartWindow (§5.6).
  • An invalid timer calendar expression (§9.1). This one is checked across every definition, not only those in the graph.
  • Two boot-triggered services that conflict with each other. Both are failed. Safe mode applies if either is Critical.

7.2.4 Validation warnings #

Logged, and do not prevent boot:

  • A service using Readiness=Alive that something else depends on hard. Alive readiness means the process exists, which is no guarantee it is functional, so anything waiting on it is waiting on the wrong thing.
  • Services that need a role no service fills — in practice authn, needed by every service whose Identity is not SYSTEM. Warned about rather than failed, because a missing hard dependency rejects the whole reload transaction, and that would prevent reloading the very definition that installs the missing provider. See §7.6.

7.2.5 Multiple findings #

A service can attract more than one finding in one pass — being both in a cycle and missing a Requires target, say. The runtime state records a single primary cause, chosen by precedence:

  1. CycleDetected
  2. ValidationError
  3. DependencyFailure

The precedence affects only which cause is stored. Every other finding for the service is retained beside it, in discovery order, and the operation's failure message enumerates all of them:

CycleDetected: dependency cycle a -> b -> a (also: ValidationError:
health check interval exceeds the restart window) [2 findings]

The primary comes first and unqualified, so a service with one finding reads exactly as it always did. A higher-precedence finding arriving later demotes the previous primary rather than deleting it — breaking the cycle should not be what it takes to discover the second fault.

Every finding is also emitted as its own peinit.graph.validation.failed KMES event, carrying graph.phase boot and the finding as outcome.reason. The console lines are for whoever is watching the boot; the events are the account that survives it.

HardDependencyBlocked — blocked because a dependency is blocked — gets its own reason, hard-dependency-blocked, rather than being reported as a missing dependency, which would claim the target does not exist when it does. It has no reload-path equivalent, because reload rejects wholesale instead of propagating a block.

The reload path behaves differently, because its consequence is different. Validation there accumulates every finding, encodes each as its own event under graph.phase reload-config, and then rejects the entire reload — the previous generation stays live and the findings return to the caller (§10.4). Boot marks individual services and continues; reload reports everything and changes nothing.

Both use the same event type deliberately: one consumer filter catches validation problems in either regime, and phase says which.

7.2.6 On-demand starts #

Before dispatching an administrator-requested on-demand start, peinit runs the shared definition validator over the services the start plan would activate. An unschedulable health check, an invalid health-check restart window or an invalid timer schedule rejects the plan with INVALID_STATE; no service from that plan is launched. This includes definitions reached through dependencies.

Definitions outside the plan, and dependencies whose running state already satisfies the request, do not make it fail. Missing or disabled hard dependencies retain the ordinary blocked-plan handling; dependency cycles retain the existing cycle refusal. Conflicting boot triggers are not themselves a reason to reject an explicit start.

This check also applies to an administrator restart when the service has no running process and the request builds an on-demand start plan. A queued administrator operation is checked when promoted into such a plan; rejection fails that operation with an invalid_state reason and leaves other queued operations eligible for promotion. It does not terminate the supervisor loop.

Internal starts from restart policy, timers, BindsTo recovery, OnFailure and TTY release retain their existing closure and cycle checks, without this additional definition-validation refusal, including when a queued internal operation is promoted. In particular, an invalid nonboot OnFailure target must not roll back the source service's transition to Failed. The stop/start leg of a restart of a running service and a deferred Backoff restart also retain their existing internal start behavior. Boot and reload-config validation remain as described above.

7.3 Graph Execution

Peios / Peios technical reference / peinit / Dependencies

Executing a graph means starting the services whose dependencies are all satisfied, and doing it again each time something becomes satisfying, until nothing is left.

execute_graph(graph, max_parallel):
    ready = services with no unsatisfied dependencies
    in_flight = 0

    while ready is not empty or in_flight > 0:
        while ready is not empty and in_flight < max_parallel:
            begin_start(ready.dequeue())
            in_flight += 1

        match wait_for_event():
            ServiceSatisfied(s):
                in_flight -= 1
                for each dependent of s:
                    if all its dependencies are satisfied:
                        ready.enqueue(dependent)

            ServiceFailed(s):
                in_flight -= 1
                propagate_failure(s)

MaxParallelStarts bounds the concurrency, counted per context as the members currently running.

The events are not only member completions. A dependency naming a readiness level is settled against the live service table rather than the graph's own bookkeeping, and the arrival of a LEVEL= — or the publisher leaving a dependent-satisfying state — re-runs the release for every context holding an edge on that service.

7.3.1 Execution contexts #

A graph execution is a retained object, not a transient loop. peinit holds a context carrying its members, their dependencies and their status, and there are two kinds: one boot context built from the boot plan, and an on-demand context per explicit start, built from that service's validated transitive closure.

Both use the same scheduler, the same satisfaction rules, the same failure propagation and the same parallelism, but they are distinct runtime objects — which matters because they can overlap.

An operation is associated with every context that created or adopted it. One operation can belong to more than one active on-demand context: two administrators starting different services that share a dependency both end up merged into the same already-starting operation for it, and both contexts need to hear how it turns out.

So when a pre-start outcome is terminal for an operation, peinit dispatches the corresponding graph event once per associated context. An operation with no associated context completes or fails normally, and its waiters are notified, but no graph event is dispatched.

A member whose dependent resolves without ever needing it is pruned — a dormant sub-tree is cancelled rather than started, so an on-demand start that turns out not to need half its closure does not start that half.

A context is retired once every member has reached a terminal status — satisfied, failed or pruned — which is exactly when no further graph event can be dispatched through it. Its operation associations go with it, except where an operation is also associated with a context that is still live.

Retirement happens on the operation-maintenance sweep rather than at the moment the last member goes terminal. The callers of a terminal outcome walk the graph events it produced and release the contexts those events name, so dropping a context inline would pull it out from under the rest of the same turn's work.

Without this a context and its associations persisted for the lifetime of the process. The memory was the smaller half: dispatching a terminal outcome walks every context associated with the operation, so on a long-lived machine where services are started regularly the terminal path grew steadily slower, in PID 1's single thread.

7.3.2 Failure propagation #

When a service enters Failed during graph execution:

  1. Everything that Requires or BindsTo it transitions to Failed with cause DependencyFailure.
  2. Everything that Wants it is unaffected and starts normally.
  3. Propagation is transitive: if A requires B and B requires C, and C fails, then B fails and then A fails.

7.3.3 On-demand start #

Starting a service explicitly resolves its dependencies first:

  1. Collect the transitive Requires and BindsTo closure.
  2. Collect the transitive Wants closure, best-effort.
  3. Validate the sub-graph — cycles, missing targets.
  4. Resolve conflicts, stopping whatever has to stop.
  5. Start the sub-graph with the same parallel algorithm. Dependencies started this way carry cause DependencyStart and operation source DependencyPropagation.

A service already in a satisfying state — Active, Completed or Skipped — is not restarted. Its dependency is already met.

A disabled hard-dependency target blocks the dependent, on both paths. Disabled suppresses automatic activation and the service may still be started explicitly (§3.2) — but nobody started this one explicitly. Something that requires it did, and its administrator took it out of service deliberately, so starting it to satisfy someone else's dependency would defeat the flag by a route its description does not consider.

Starting the disabled service itself still works. That is the escape hatch, and it is unchanged.

A disabled Wants target is skipped rather than blocking anything: a soft dependency is advisory, so there is nothing to fail.

7.3.4 Shutdown ordering #

Shutdown reverses the graph: services with no dependents stop first, services that everything depends on stop last, derived by reverse topological sort from the same edges.

Only hard dependencies — Requires and BindsTo — contribute to the ordering. A Wants dependent may therefore be stopped after its target.

The ordering is entirely emergent from what the definitions declare. There is no floor and no pinning, so where the TCB services end up in the wave order depends on their declared dependencies being right.

7.4 The Boot and On-Demand Paths

Peios / Peios technical reference / peinit / Dependencies

The two ways a graph gets built differ in more than scope, and the differences are worth having in one place.

BootOn-demand
MembersEvery boot-triggered root plus its closureOne requested service plus its closure
A missing hard-dependency targetBlocks the dependentBlocks the dependent
A disabled hard-dependency targetBlocks the dependentBlocks the dependent
A validation findingThe service is failed, others continueThe start fails
Multiple findings on one serviceOnly the highest-precedence one is reported—
ConflictsTwo boot-triggered conflicting services fail bothThe conflicting service is evicted
ContextOne boot contextOne context per explicit start
Failure of the whole buildRecovery modeAn error to the caller

7.4.1 Where a definition error lands #

The three places a bad definition can be caught behave differently, and which one catches it depends on what kind of wrong it is:

  • Decoding. A definition that does not parse — an invalid name, a malformed trigger, an unclosed quote, a registry: check naming an uncacheable key, a duplicate field — is caught when the registry is read. It fails that service with ValidationError and the rest continue, at boot and on reload alike (§3.2).
  • Graph validation. A definition that parses but does not fit — a cycle, a missing target, a flap-constraint violation, an invalid calendar expression — is caught here, and fails that service at boot or the whole reload on a reload-config (§7.2).
  • Start. A definition that parses and fits but whose preconditions do not hold — a failed assert, an unresolvable identity, a missing binary — is caught when the service actually starts, and fails that activation (§5.2, §5.3).

The dividing line between the first two is whether the problem is visible in one definition on its own. Decoding sees one key at a time; validation is the first place that can see two definitions together.

7.5 Readiness Levels

Peios / Peios technical reference / peinit / Dependencies

network-online.target on systemd is famously meaningless, because "online" has no single definition. A service needing a default route and a service needing only a loopback address want different things, and one target cannot mean both.

peinit does not have an online target. A service says what it is waiting for, in the publisher's own vocabulary:

Requires = ["netd:routed"]
Wants    = ["timed:synchronised"]

netd:routed waits for netd to be active and to have published the level routed. netd on its own is unchanged — the ordinary "that service must be active" dependency.

7.5.1 The syntax #

<service>:<level>, split on the first colon. It rides on the existing Requires, Wants and BindsTo fields rather than a field of its own, because a level dependency is a service dependency with a stricter predicate: netd:routed subsumes netd. Each relationship keeps exactly the semantics it already has, applied to the stricter test.

It cannot collide with a service name. A service name may contain only letters, digits, ., _ and -, so a colon never appears in one, and no existing definition can accidentally become a level dependency.

A trailing colon — "netd:" — reads as a plain dependency on netd. It is a typo, and treating it as a request for the empty level would produce a condition nothing could ever satisfy.

7.5.2 Through a role #

The service side of the syntax may be a role instead of a name:

Requires = ["network:routed"]

network is filled by whichever service declares Provides = ["network"] — netd, on a shipped image — and peinit rewrites the entry to netd:routed before anything reads it, exactly as §7.6 resolves a role. The level rides across unchanged. Everything downstream — validation, the graph, svctl status — sees a level dependency on a service, and never learns a role was involved.

This is the form a definition should use for the network. The role is "the executor of Peios Network Policy's interface layer", and the levels are not the executor's own words but the specification's: PNP defines readiness as link, addressed and routed, and whichever service fills the role publishes those. A definition that says network:routed survives netd being replaced; one that says netd:routed does not.

Several services may fill a role; the entry then resolves to one level dependency per provider, and a Requires waits for all of them. A role no service fills is left as written and fails validation as the missing hard dependency it is, naming the role the definition wrote.

7.5.3 How a level gets there #

The service publishes it on the notification channel it already has, as LEVEL= beside READY=1 and STATUS=:

LEVEL=link
LEVEL=addressed
LEVEL=routed

All LEVEL lines in one datagram form one set: peinit atomically replaces the sender's previous set with the distinct nonempty values, before releasing any dependents, including when the same datagram carries READY=1. A datagram with no LEVEL lines leaves the set unchanged.

LEVEL= on its own clears the set. Empty values mixed with nonempty ones add no member; they do not clear other lines in the same datagram. peinit records the set against the sending service, and holds any dependent whose requested level is absent.

There is no subscription. peinit does not connect to netd's socket or timed's, does not need to find them before they exist, and carries no knowledge of their wire formats — which matters, because PID 1 is the one process on the machine that cannot afford to fail to start.

7.5.4 Levels match set membership #

Requires = ["netd:addressed"] is satisfied when the active publisher's set contains addressed. Every member is an opaque, case-sensitive name; commas do not split a name into several levels.

peinit has no ordering over another daemon's vocabulary. A publisher that sends only LEVEL=routed does not satisfy an addressed dependent. netd knows which conditions imply others, so when routed it publishes link, addressed and routed together. A service should depend on the condition it needs, and the publisher reports every condition that holds.

7.5.5 A level is never stale #

peinit clears a service's entire level set whenever that service leaves a state that satisfies dependents. A level is a claim about a process that is currently making it, and one that outlived its process would hold a dependent open on a promise nobody was keeping — the exact failure the mechanism exists to prevent.

Because peinit is the process that notices a service exiting, this needs no timeout and no liveness check.

7.5.6 How the gate holds #

A level is settled at dispatch, not at planning. The plan decides what to start; whether a level is met is a live question — it can arrive after the target's own start completed (netd is active long before DHCP finishes) and be retracted while a dependent is still waiting — so the execution graph re-checks it against the service table every time it considers releasing the dependent, and holds the start until the answer is yes.

That check runs even when there is nothing to start. An already-active target is not part of the start plan, but the condition on it still is: the graph keeps a level edge to a service it is not starting, which is what makes Requires = ["netd:routed"] hold when netd is up and merely not routed yet.

Two events re-open the question. A level-set update arriving from the publisher releases every dependent whose requested level is now present — this is the only thing that can open a Requires or BindsTo gate, which take the exact level and nothing else. And the publisher leaving a state that satisfies dependents releases Wants waiters: a soft dependency waits only while someone is running who could still publish the level, and proceeds once nobody is — which keeps Wants failure-tolerant, the property that defines it.

A held start does not time out, and its StartTimeout runs from the moment the level arrives, not from when the start was asked for. Requires = ["netd:no-such-level"] holds the dependent indefinitely: the service stays inactive and the start operation stays pending, visible in svctl status as an operation that has not completed. That is the declared semantics, not a hang — the condition was never met, so the start never happened.

7.5.7 What happens on a drop #

Nothing, in this version. If a level falls away after a dependent has already started, the dependent keeps running and is not told.

That is a deliberate limit, not an oversight. The notification channel is one-way by specification (PSPU §4.16), so peinit has no way to tell a running service anything. A service that needs to react to a level changing while it runs should subscribe to the publisher directly — which is what resolvd does with netd today, and what any consumer that cares that much will want anyway, since it will need the detail rather than just the fact.

peinit's contribution is the start gate: the moment when the service does not exist yet and so cannot subscribe to anything at all.

7.5.8 Published levels #

PublisherLevels
netd, as the role networklink → addressed → routed, defined by Peios Network Policy
timedunsynchronised, settling, synchronised, spike

Any service may publish a level; nothing about the mechanism is specific to these two. The vocabulary is the publisher's own — or, where the publisher fills a role a specification defines, the specification's — and needs no registration.

7.6 Derived Dependencies

Peios / Peios technical reference / peinit / Dependencies

Most dependencies are declared. One is not.

A service whose Identity is anything but SYSTEM cannot start until the authority is listening, because peinit has to ask it for a token before there is a process at all (§4.3). That is a hard dependency in every sense Requires means one.

It used to be nobody's. peinit hardcodes the socket the authority listens on, so the requirement was real on every boot, but the ordering existed only if a definition happened to declare it — and all three LocalService definitions in the tree did not. Three of three is the number that makes it a defect in the mechanism rather than in three definitions: the requirement is created by Identity, so Identity is what should produce the edge.

7.6.1 Provides #

A service names the roles it fills:

Provides = ["authn"]

A role is a virtual name — a capability, not a service. Virtual and real names share one namespace, exactly as they do for packages (PSPU §5.4): a dependency on authn is satisfied by a service literally called authn, or by any service providing it.

The indirection is the point. peinit knows it needs an authority and knows which socket to speak to, but the authority's name is registry data and none of peinit's business — an image may ship a different one. So the derived edge names the role, and peinit resolves it to whoever declares it.

Provides entries use the service-name grammar and never carry a readiness level. netd:routed names a condition on a service, and a role is not a service, so there would be nothing for the level to qualify; a level on a Provides entry is rejected rather than ignored.

The other side of a dependency may name a role, with or without a level. Requires = ["network:routed"] is rewritten to one entry per service providing network, the level carried across, before the definition set becomes a service table — the same pass, at boot and on every reload, that derives the edges below. Requires, Wants and BindsTo are rewritten; a real service name is never mistaken for a role, a provider never gains a dependency on its own role, and a role no service fills is left as written for validation to report. See §7.5 for why the network is declared this way.

7.6.2 What peinit derives #

Before a definition set becomes a service table — at boot and on every reload, through the same code, because an edge that held until the first reload would be worse than no edge — peinit adds a Requires on each provider of authn to every service that needs the authority to launch.

A service needs it when its Identity is not SYSTEM, or when it has hooks and its HookIdentity is not SYSTEM. The second half matters: a SYSTEM service whose hooks run as LocalService materialises a token for them, and a definition that needs the authority for half its launch needs the ordering for all of it.

Three rules keep the derivation from doing damage:

  • A provider never gains a dependency on its own role. An authority declaring a non-SYSTEM identity would otherwise require itself, and a service that requires itself never starts.
  • An edge already declared is not added again, whether it was declared as Requires or BindsTo, and including one carrying a level. Requires = ["authd:ready"] already orders against the authority, and more strictly; adding a plain edge beside it would be a second edge to the same service saying less. BindsTo requires its target as well as binding to it, so the same holds for it.
  • Nothing is derived when no service fills the role. See below.

Because the result is an ordinary Requires, it needs no new kind of edge and behaves exactly like a declared one everywhere else — in graph validation, in graph execution, at shutdown, and in svctl status, where it appears as the dependency it is rather than as a hidden ordering.

Several services MAY fill one role; each becomes its own Requires, so the dependent is ordered after all of them. For authn this cannot arise — PGSS Logon §2.1 requires at most one authority on a running system — and where a future role allows several, ordering after all of them is the conservative reading and needs nothing new. peinit deliberately does not offer "wait for any one of them": that would change the release rule from every edge satisfied to every group has one satisfied, and would contradict Requires's failure rule, under which a target entering Failed fails the dependent.

7.6.3 When nothing fills the role #

An image whose services need a token and which ships no authority is broken. A reload of it produces a validation warning naming every service that cannot start, and succeeds anyway.

It deliberately does not invent an edge to a name no service answers to. That would be a missing hard dependency, and a missing hard dependency rejects an entire reload transaction — so an operator could not reload the definition that installs the authority. The image would be unrecoverable by the one mechanism meant to recover it.

The individual launches still fail, loudly, where the fault actually is: at the service that asked for a token and could not have one.

7.6.4 A role is not an online target #

§7.5 opens by rejecting network-online.target, and the objection there does not reach this mechanism. It is worth being explicit about why, because the two look alike from a distance.

The complaint about network-online.target is that "online" has no single definition: a service needing a default route and a service needing a loopback address want different things, and one name cannot mean both. The abstraction is over a condition, and the condition is ambiguous.

authn abstracts over no condition at all. It means "whoever answers ServiceAttest on the logon socket" — a protocol role, specified by PGSS Logon §2.19, which §2.1 says MUST have at most one occupant. There is exactly one thing it can mean, and what a dependent gets is the same thing it would get by naming the service directly.

network:routed passes the same test from the other direction. The role network is a position too — the executor of Peios Network Policy's interface layer — and routed is not peinit's word for "online" but the specification's word for one exact condition, a default route. The ambiguity that sinks network-online.target is absent because the condition is named, not the target.

The rule the two share: peinit does not invent vocabulary for conditions inside another service. A readiness level is the publisher's own word for its own state, or its specification's; a role is a name for a position in a protocol peinit already speaks. Neither is a target.

8.1 Jobs

Peios / Peios technical reference / peinit / Jobs and Operations

A job is one supervised process execution. Every fork peinit performs is a job: a service's main binary, a pre-exec hook, a post-exec hook, a reload command, a health check invocation, a submitted job (§8.5).

Jobs are the observable unit of what actually ran. Services are definitions carrying identity, policy and configuration; jobs are instances. A restart creates a new job.

8.1.1 Lifecycle #

Created --> Running --> Completed
   |           |
   |           +------> Failed
   |           |
   |           +------> Abandoned
   +------------------> Failed
StateMeaning
CreatedThe job object exists but exec has not succeeded. The process may not have been forked, or it may be in pending post-fork setup with the error pipe unresolved.
RunningExec succeeded and the process is alive.
CompletedThe process exited successfully — code 0, or one in SuccessExitCodes.
FailedThe process failed, or peinit classified the job failed before the fork because parent setup failed.
AbandonedThe process survived SIGKILL.

Job states are simpler than service states because they describe a process rather than a policy. A service has Starting, Reloading and Backoff because those are decisions; a job is running, or it finished, or it is stuck.

8.1.2 Fields #

Job {
    id:                 GUID       // UUIDv7
    service:            string?    // null for a submitted job
    job_type:           enum       // ServiceMain, PreExecHook, PostExecHook,
                                   // ReloadHook, HealthCheck, Submitted
    state:              enum
    pid:                u32?
    pidfd:              fd?
    resolved_identity:  string     // the resolved service, hook or submitter identity
    token_summary:      object     // the resulting SID, groups, privileges
    image_path:         string
    arguments:          string[]
    created_at_ns:      u64
    started_at_ns:      u64?
    ended_at_ns:        u64?
    exit_code:          i32?
    exit_signal:        i32?
    failure_cause:      string?
    cgroup_id:          string
    cgroup_generation:  u32
    activation_generation: u32
    operation_id:       GUID?      // null for a submitted job
    hook_index:         u32?
}

The rules that govern when the nullable fields are populated are what make a job record trustworthy:

  • id is assigned before the fork, so a job that never forks still has an identity.
  • pid and pidfd land on the record only once exec success is confirmed by EOF on the error pipe. Until then they are held in pending setup state and the job is Created.
  • An exit observed during that window is held, not applied. peinit resolves a reaped child to a job by PID, and inside this window no job carries the PID yet — a short-lived process can be gone before the error pipe is read. The exit is kept against its PID and replayed once the job has been started and can receive it. Discarding it would be unrecoverable: the job would be started against a PID that is already dead and reaped, and no second SIGCHLD is coming, so it would stay Running for ever. A reaped PID that matches no job and no pending setup is genuinely not peinit's — PID 1 reaps orphans it never started — and is ignored.
  • A setup failure takes the job straight from Created to Failed. ended_at_ns records the classification time, failure_cause records what went wrong, and pid, pidfd, started_at_ns, exit_code and exit_signal all stay null. There was no process to have a PID or an exit status.
  • exit_code and exit_signal are populated only when peinit observed an exit — never both, since a process either exits or is killed.
  • For an Abandoned job, ended_at_ns records when peinit stopped supervising, and the exit fields stay null. Nothing exited.

resolved_identity is the identity string — SYSTEM, LocalService, a SID — that was resolved for the execution. For a submitted job it is the job identity's user SID, since no name was ever involved (§8.5). token_summary is what the resulting token actually contains. They are separate because they can differ, and the identity field exposed in status views and job events is the former.

8.1.3 Retention #

peinit tracks active jobs in memory. When a job reaches a terminal state it emits a structured event carrying the full record and then drops the job. There is no job history in peinit, and no structure that could hold one.

eventd is the historian. It consumes those events from the KMES kernel ring buffer, and a query for a service's past jobs is a query to eventd.

A submitted job is the one partial exception: its record is dropped like any other, but the jobs system keeps its own entry for a further 60 seconds so that a submitter polling for the outcome can still read it (§8.5). That entry is a retained answer, not a history.

8.1.4 The launch queues are hints #

A job awaiting launch is queued by id, in one of six queues — service main, pre-start hook, post-start hook, control, health check and submitted. The queues hold ids; the store holds the records, and the store is the only truth.

They can therefore disagree. Any path that finishes a job in Created drops the record, and a path that does not also clear the queue leaves an id behind pointing at nothing. Requiring every such path to remember was a poor trade: there are around a dozen of them, nothing enforced it, and the cost of one omission was that peinit's runtime loop ended — which ends PID 1.

So a drain that meets an id with no record discards it and moves on to the next, rather than treating it as an error. A live entry behind a stale one still launches, so one bad id cannot stall a queue. This is why the ordering rule is worth stating plainly: keeping the queue in step is a tidiness obligation, not a correctness one.

Discarded entries are counted and reported on the work pump's turn, so a bookkeeping fault shows up as a number rather than disappearing. A count that is not zero means some path finished a job without clearing its queue entry, and that path is a defect even though it is no longer fatal.

8.1.5 Ownership #

ConcernOwner
Restart policy, dependencies, health check schedule, ErrorControlService
Current state — Active, Failed, …Service
PID, pidfd, exit code, exit signalJob
Execution timestampsJob
Identity and tokenJob
cgroup assignmentJob
Log correlationJob
Submitter, job descriptor, progress, retentionSubmitted job entry (§8.5)

A service tracks its current main job's identifier, and a status query returns it.

8.2 Operations

Peios / Peios technical reference / peinit / Jobs and Operations

An operation is a requested state machine action on a service, as a first-class object. Control commands do not mutate state directly: every one creates an operation that is validated, queued, resolved against whatever else is in flight, and executed by the event loop.

Operations exist because peinit serves concurrent callers — administrative tools, automated triggers, other services. Without them, two commands arriving together collide with whatever behaviour falls out; with them, the resolution is explicit and observable.

8.2.1 Lifecycle #

Pending --> Running --> Completed
  |           |
  +-> Merged  +-------> Failed
  |           |
  +-> Cancelled +-----> Aborted
  |
  +-> Failed
StateMeaning
PendingValidated and queued, waiting on a precondition.
RunningExecuting.
CompletedThe goal was reached. Start: Active for Simple, Completed or Inactive for Oneshot. Stop: the service is no longer running. Reload: the reload resolved.
FailedThe goal was not reached — or the operation's maximum lifetime expired while it was still Pending (a start peinit is holding on a dependency has no lifetime, §7.5) — or peinit could not execute it (below).
MergedMerged into an existing identical operation, whose identifier is recorded.
CancelledTerminated while Pending. It never executed.
AbortedTerminated while Running.

An operation can also fail because peinit could not execute it: the matrix admitted it and then the boundary found nothing to act on — no current main job, a transition the state machine does not permit. That is peinit's fault rather than the service's, and it is contained to the one operation. The operation fails with the result internal_error: <cause>, the service keeps the state it had, a [FAILED] console line names the service, the operation and the cause (peinit: service <service>: <type> operation <id> failed before it began: <cause>), and the runtime loop carries on. A caller waiting on the operation is answered INTERNAL_ERROR (§10.1).

The same rule covers every path that is about one service — a job terminal, a setup status, a health probe, a lifecycle deadline, a notify datagram, an event the ring refuses. An internal error there fails that service under InternalError (§6.3): its job is retired, its operation fails with internal_error: <step>: <cause>, a [FAILED] line peinit: service <service>: internal error at <step>: <cause> and a peinit.internal-error.contained event (§8.4) announce it, and the runtime loop carries on. Only an error about supervision itself — the wait, event source registration, the listeners, the event ring refusing even the small peinit.event.dropped, shutdown finalisation — ends the loop and enters recovery (§2.8). A bug peinit can attribute to one service costs that service; it does not cost the machine its control socket.

A lifecycle deadline that raised is removed once its error is contained, so it is announced once, not again on every timer turn. One that peinit cannot remove is retried at most once a second, and a retry that raises the same error and changes nothing is not announced again.

Cancelled and Aborted are the same idea at different points: never ran versus was running. Why it happened is a property of the event, not of the state.

Two things reach Aborted. A conflicting operation superseding this one, and a Restart whose target's definition is withdrawn while its stop leg is draining — the stop still finishes, but there is nothing to start, so the operation is aborted with the reason definition_removed_during_restart_stop_leg (§3.8).

A start that is aborted or cancelled takes its readiness deadline with it: the start is no longer waiting for READY=1, and ending its main process is the superseding operation's job. A readiness deadline whose operation has ended by the time it comes due is dropped, not acted on.

8.2.2 Fields #

Operation {
    id:               GUID
    operation_type:   enum    // Start, Stop, Restart, Reload, Reset
    service:          string
    state:            enum
    created_at_ns:    u64
    started_at_ns:    u64?
    completed_at_ns:  u64?
    source:           enum
    caller:           token_summary?   // admin-initiated only
    result:           string?
    merged_into:      GUID?
}

8.2.3 Sources #

Why peinit created the operation:

SourceMeaning
AdminA control client asked for it.
BootThe Phase 2 boot plan.
ShutdownThe shutdown lifecycle.
DependencyPropagationA start operation created one for an unsatisfied dependency.
RestartPolicyA restart policy generated a start.
TimerA timer trigger fired.
BindsToRecoveryA bound target returned to Active.
BindsToPropagationA bound target stopped.
ConflictResolutionA conflict evicted the running service.
OnFailureA failed service's fallback handler.
TtyReleaseA service released a terminal another service was waiting for.

Graceful shutdown creates a Stop operation with source Shutdown for each participant in its stop waves (§12.2). All are requested when the plan is fixed; each starts when its wave becomes eligible. These records observe the shutdown planner. They do not enter the ordinary control queue or replace an existing administrator operation, whose retained stopping deadline still applies. A later-wave observation may fail its Pending lifetime without authorising an early signal or cancelling the service's eventual stop.

A participant that exits before its wave can complete its observation without ever starting it; a crash fails it. Once stopping, reaching Inactive completes it, while a stop timeout, global timeout or forced reboot fails any observation still live. A failed observation is not completed again when the process is subsequently reaped.

8.2.4 The types #

Start creates a job for the target. Unsatisfied dependencies produce their own start operations with source DependencyPropagation. It completes when the service reaches Active, Completed or Inactive as appropriate, or Skipped when pre-start conditions do not hold.

Stop sends SIGTERM, arms StopTimeout, escalates to SIGKILL. It completes when the service reaches Inactive, or Failed after a conflict eviction or bound-dependency propagation.

Restart is a stop then a start, tracked under one identifier across both phases, and the type stays Restart throughout for observability.

Reload issues the reload command or signal (§6.5) and completes when the reload resolves. Unlike the other lifecycle commands it defaults to not waiting — the caller gets the identifier immediately.

Reset clears Failed, Abandoned or Skipped, taking the service to Inactive. It is synchronous.

8.2.5 Timeouts #

A start, reload or reset inherits the target's StartTimeout as its maximum lifetime; a stop inherits StopTimeout. A restart has two legs, each enforced against its own timeout, with the sum as the overall lifetime. A start the graph is holding has no lifetime while every unsatisfied dependency it waits on is a fact with no clock of its own — a readiness level not yet published (§7.5), a target in Backoff (§6.1), or a target itself held only on such facts; it stays Pending for as long as those facts are undecided, and a caller waiting on it waits with it. A dependency that is itself starting is bounded by that service's StartTimeout, and a dependent waiting on one keeps its own lifetime, measured from creation as below. When a hold ends, the released start's lifetime runs from the release. Queue time behind another operation is not a hold and still counts.

This also applies to an on-demand start whose Requires or Wants target is already in Backoff. The request is accepted and held; it does not bypass the target's remaining restart delay. Repeated requests merge into the pending operations. The target's deferred start receives its lifetime when the restart deadline releases it. If the target is stopped instead, hard dependents fail and soft dependents may proceed.

For a transitive hold, the required release point can precede the dependent's own dispatch: when its dependency becomes clocked, the dependent must receive its full lifetime even if it is still Pending, waiting for that dependency to finish. Its later dispatch must not restart that lifetime again.

The clock starts at creation, including queue time — except across a hold (§7.5), during which it does not run. From the caller's point of view they have been waiting since they sent the command, not since peinit got round to it. A start that sits Pending behind a stop for longer than StartTimeout fails without ever running.

8.2.6 Retention #

Pending and Running operations are held in memory. A terminal operation is emitted as an event and dropped after a grace period of 60 seconds — long enough for a polling client to collect the result.

peinit keeps no operation history, for the same reason it keeps no job history. eventd is the historian.

8.3 Conflict Resolution

Peios / Peios technical reference / peinit / Jobs and Operations

When a new operation is requested and one for the same service is already Pending or Running, peinit resolves the two.

8.3.1 Merging #

An operation of the same type merges. The new caller receives the existing operation's identifier, and from their point of view their request is in progress — they neither know nor need to know that it merged.

ExistingNewResolution
StartStartMerge
StopStopMerge
ReloadReloadMerge
RestartStartMerge — a restart already includes a start

Restart is not mergeable with itself. A second restart while one is in progress is queued. The one place that does not apply is a restart deferred by a backoff (§10.3): it is Pending with nothing running for a second one to queue behind, so a second restart in Backoff merges into it.

8.3.2 Cross-type #

ExistingNewResolution
Start (Pending)StopCancel the start, create the stop
Start (Running)StopAbort the start, create the stop
Start (Pending)RestartCancel the start, queue the restart
Start (Running)RestartQueue the restart
Stop (either)StartQueue the start
Stop (either)RestartQueue the restart
Restart (Pending)StopCancel the restart, create the stop
Restart (Running)StopAbort the restart, create the stop
Restart (either)RestartQueue
Reload (Pending)StopCancel the reload, create the stop
Reload (Running)StopAbort the reload, create the stop
Reload (Pending)RestartCancel the reload, create the restart
Reload (Running)RestartAbort the reload, create the restart
Anything (either)ResetReject

Combinations outside this table are rejected: a new Reload while a Start, Stop or Restart is active. A new Start while a Reload is active is not rejected — a reloading service is already where a start would take it, so the command is answered with the service's status and creates no operation, before conflict resolution is reached at all.

8.3.3 The principles #

  1. Stop wins over start. An explicit stop always takes priority. The administrator said stop, so stop; a queued start can follow.
  2. Later supersedes earlier. Start then immediately stop means the stop wins and the start is cancelled, recorded as superseded.
  3. Merging is transparent. The merged caller gets the original identifier and blocks on the original operation's outcome.

Reset is rejected outright while anything is in flight, because reset means "clear a terminal state" and nothing in flight has one.

8.3.4 Dependency propagation #

When a start executes against a service with unsatisfied dependencies, peinit creates start operations for them:

  • Requires — source DependencyPropagation. If one fails, the parent operation fails with DependencyFailure.
  • Wants — source DependencyPropagation. If one fails, the parent continues.
  • BindsTo recovery — source BindsToRecovery, created when a bound target returns to Active, for dependents sitting in Failed with cause BindsToPropagation.

Dependency-created operations follow the same resolution rules as administrator-created ones. If a dependency is already starting because something else also depends on it, the operations merge — and both graph execution contexts are then associated with the one operation (§7.3).

BindsToRecovery restarts are not subject to the restart budget. They are created because a dependency returned, not because anything failed.

8.3.5 Restart policy and timers #

A restart-eligible failure creates a start operation with source RestartPolicy once the backoff delay elapses. It goes through the ordinary validation and resolution: if an administrator has already sent a stop, or the budget is exhausted, it is rejected.

A timer firing creates an operation based on the service's current state:

TypeStateAction
OneshotInactive, Completed, FailedCreate a start, source Timer.
OneshotActive, StartingSet the pending flag. One catch-up run, no operation.
SimpleInactive, FailedCreate a start, source Timer.
SimpleActive, StartingNo-op. The firing is recorded.

The Oneshot catch-up creates its start when the current run completes. Multiple missed firings collapse into one pending run.

8.3.6 Boot and shutdown are not operations #

Boot and shutdown are modes peinit enters, which then generate per-service operations. There is no "shutdown operation" to observe or cancel. Boot-generated starts use source Boot.

Shutdown-generated stops use source Shutdown, and they are the one exception to the tables above. They are created outside conflict resolution: a Shutdown Stop does not merge into, cancel or abort an administrator's Stop or Restart already in flight, but exists beside it, so that operation keeps its identity and its retained timing (§12.2).

8.4 Event Emission

Peios / Peios technical reference / peinit / Jobs and Operations

peinit emits a structured event at every job and operation lifecycle transition, and for its own records. All of them go into the KMES kernel ring buffer through kmes_emit, encoded as msgpack per the KMES event-record format (Peios Kernel TRM §2).

There is no event socket. Structured events are not sent to eventd over any connection. eventd consumes them from the ring buffer, which is why they survive eventd being down, being restarted, or not existing yet. The only thing peinit sends eventd over a socket is service output (§11.4), which is a different path with different guarantees.

8.4.1 Names and shape #

Every type peinit writes is under its platform root, peinit, and every payload is a nested map whose fields sit at their catalogue paths (PGSS §6.4): the job is object.job.guid, the service object.service.name, the client that asked subject.token.sid, the result outcome.success. A value peinit does not have is an absent key, never nil; SIDs are written in binary (bin.sid), job and operation identifiers as binary GUIDs (bin.guid) that read back as the text svctl shows, enumerated values in kebab-case (service-main, explicit-start), and times as nanoseconds since the Unix epoch. peinit's fragment, /usr/share/evman/peinit.evman, describes every type and field, and the Events reference generates its pages from it.

No event carries a principal's name, peinit's own log wording, or Rust debug output. What a person reads is on the console; what a consumer queries is in the fields.

8.4.2 The emission policy #

Before peinit builds an event that is not essential, it asks the emission policy, Machine\Generic\Events (PGSS §6.9): a type switched off is never built, let alone written. With nothing set, a standard type is on and a verbose type is off. peinit opens one view of the policy for the life of PID 1; it needs no registry to open, decides by tier until the registry is readable, and applies a committed change to the next decision.

Four types are essential and never ask: peinit.boot.downgraded, peinit.recovery.entered, peinit.critical-service.failed and peinit.event.dropped. Each is rare, and each is the only record of what it describes. The verbose types are peinit.job.created, peinit.job.status.reported, peinit.operation.started, peinit.operation.merged, peinit.graph.operation.ended, peinit.config.reload.deferred, peinit.notify.status.reported and peinit.notify.progress.reported; everything else is standard. Setting Enabled = 1 on Machine\Generic\Events\peinit turns all of them on.

8.4.3 Job events #

peinit.job.created — the job object exists, before its process does. Carries the job, its type and state, its service and the activation generation it serves, the operation it serves, the executable, and the token it will run as.

peinit.job.started — exec succeeded. Adds the process ID (object.process.pid) and the cgroup (object.cgroup.path).

peinit.job.ended — the process exited or was killed, or the job ended without starting. Carries the whole record: the terminal state, outcome.success (true exactly for completed) with the failure cause as outcome.detail, the exit code or signal, the executable and arguments, the times it was created and started, its duration, and its cgroup and cgroup generation.

The token a job runs as is on each once peinit holds it: its user and group SIDs, and its present and enabled privileges as flags. A service's token is minted as its process is set up, so its peinit.job.created carries at most the user SID of a well-known identity.

peinit.job.ended cuts object.job.arguments to 32 KiB of whole arguments; object.job.arguments-truncated and object.job.arguments-count say so. An event the ring refuses outright is dropped, counted, and replaced by a peinit.event.dropped naming its type, its length, its service and job, and the error number, and a [ WARN ] line reports it; only the ring refusing that small event is fatal (§8.2). The count of drops is the console's, and the trail's is the number of those records. peinit.internal-error.contained records an error peinit contained to one service (§8.2): the step, the service, the job and whether the service was failed. The error's text stays on the console and in the failed operation's result.

A submitted job's lifecycle rides the same three events with no service, no activation generation and no operation: the keys are absent. Two more are its own:

peinit.job.status.reported — the job reported STATUS or PROGRESS. Carries the job, the submitter's SID, and notify.status and notify.progress.* as retained after the datagram. Written at most once per job per second; a datagram inside the window updates the view and writes nothing (§8.5).

peinit.job.output.dropped — the submitter's output sink stopped draining and peinit began dropping its copy of the job's lines. Once per job, on the first drop; the record in eventd is unaffected (§11.1).

8.4.4 Operation events #

Every operation event carries the operation (object.operation.guid), its type, source and state after the transition, and its service. An operation a client's command created also carries the client as subject.token.sid; one peinit created itself has no subject.

EventAdds
peinit.operation.requested—
peinit.operation.started—
peinit.operation.endedoutcome.success, outcome.detail, object.operation.duration
peinit.operation.mergedobject.operation.merged-into.guid

peinit.operation.ended is one type for the four ways an operation can end other than merging — completed, failed, cancelled while pending, aborted while running — and object.operation.state says which. outcome.success is true exactly for completed. A merged operation has no outcome of its own: the one it merged into decides it.

object.operation.duration is measured from the request, not from the start of execution — the same reasoning as the operation timeout — and is absent on a cancelled operation, which never ran. What a caller waited is what matters, and queue time is part of it.

The operation's result, in the words the control interface gives a client (inactive, startup killed by shutdown), is outcome.detail, whichever way it ended. The control interface's operation view calls the same text result or error (PSPU §4).

8.4.5 Ordering #

When one runtime step produces several lifecycle events, peinit emits them in causal order before committing the retained state for that step. For a terminal pre-start graph dispatch, the terminal event for the operation whose outcome satisfied or failed the graph input precedes the events for the operations that dispatch releases, which preserve graph dispatch order.

Every operation in a graph context is requested when the context is built rather than when its turn comes, so what a release emits is peinit.operation.started.

8.4.6 Peinit's own records #

peinit's own records go through the same path:

  • peinit.on-failure.suppressed when the fallback chain guard trips;
  • peinit.graph.validation.failed and peinit.graph.validation.warned for validation findings, one per finding, and peinit.boot.downgraded for each finding that forced Safe mode;
  • peinit.config.reload.applied for an explicit reload-config and for the reload that ends the boot window, and peinit.config.reload.deferred for a reload during it (§3.7);
  • peinit.notify.rejected for a refused notification datagram, and peinit.fd-store.rejected for a refused descriptor;
  • peinit.notify.status.reported, .errno.reported, .exit-status.reported, .stopping.reported and .progress.reported for the event-emitting notification fields, the last at most once a second for each activation (§10.5);
  • peinit.cgroup.leaked the first time a sub-cgroup is found still populated after its post-kill deadline (§5.7);
  • peinit.graph.operation.ended for a graph member's terminal outcome;
  • peinit.recovery.entered for the reason peinit dropped to a recovery shell (§2.8);
  • peinit.critical-service.failed and peinit.service.abandoned from the shutdown path (§12);
  • peinit.service.reload.timed-out for a reload a service started and never confirmed (§6.5).

Records are events rather than logs, and that distinction is the point: the ring buffer persists from the moment PKM loads, so a recovery during Phase 1 is captured before the registry exists, let alone eventd.

8.4.7 Access decisions #

peinit writes no event of its own for a refused command. Every access check it makes goes through KACS AccessCheck, and KACS records the decision, as kacs.audit.access.checked, when the descriptor's SACL asks for it (PGSS §6.7). peinit's compiled-in descriptors — the default service descriptor, the control descriptor and a submitted job's default — each carry a SACL auditing every refusal, for everyone, so a denial is recorded unless a descriptor says otherwise (§4.6, §13.4). A refusal also leaves a debugging line on the console, written only at peios.quiet=0.

So that the record says what was decided on, each check passes KACS an audit context naming its object, which the kernel copies into the record beside fields.attestation.userspace:

CheckedContextThe record carries
A service{kind: "service", service: {name}}object.kind service, object.service.name
A job, on either socket{kind: "job", job: {guid}}, the GUID in binaryobject.kind job, object.job.guid
The control door{kind: "peinit-system"}object.kind peinit-system

The rights in access.requested and access.granted are decoded against object.kind: the SERVICE_*, JOB_* and SYSTEM_* tables (§4.6, §8.5, §10.2). job-list checks every job it might list against that job's own descriptor, so each job it leaves out is one more decision for the job's SACL.

peinit holds SeAuditPrivilege enabled for this: without it KMES refuses its events, and the access-check path writes no record (§13.2).

8.5 Submitted Jobs

Peios / Peios technical reference / peinit / Jobs and Operations

A submitted job is a supervised process whose definition arrived on the jobs socket rather than from a service definition. It has no name, no policy and no operation: something asked for it, peinit ran it under an identity the asker was entitled to confer, and it is managed by whoever the job's own Security Descriptor admits. The protocol is PSPU §7; this chapter is how peinit implements its side.

Two shapes drive the design, and one mechanism serves both. A task — a backup, a migration — is submitted by a broker on behalf of the principal it is serving, runs to completion and reports progress on the way. A session — a per-logon host process — is submitted with a freshly issued logon token and one end of a socketpair, and lives until something stops it. Nothing in peinit distinguishes them.

8.5.1 Two doors #

The control socket is an administrator's door: its commands change the state of the system's services, and its descriptor admits the principals entitled to do that. Submitting a job is a different permission held by a different population, so it has a different socket, /run/services/peinit/jobs.sock (§10.7), with a different descriptor. The filesystem says who may submit and who may administer, separately, and peinit carries no policy of its own about either: being able to connect to the jobs socket is the permission to submit, and peinit performs no access check before a submit.

What a caller may do to a job is a third question, answered by the job's own descriptor. A submitter manages its jobs on the jobs socket; an administrator observes and stops them from the control socket with job-list, job-status and job-stop (§10.2). Both doors check the same descriptor with the same rights.

8.5.2 Two identities #

Every submission involves two identities, and peinit keeps them apart.

The submitter is the connection identity: the peer token captured once, at accept, exactly as the control socket captures it (§10.1). It is recorded on the job, it is what the quota counts against, and it is the owner of the job's descriptor. A submitter that connects while impersonating is recorded as the principal it was impersonating.

The job identity is what the process runs as. It is established in exactly one of two ways, and the boundary that does it — LinuxJobIdentityProvider — has no third:

  • No token attached. peinit opens the primary token of the connecting process through the kernel's handle on it. The handle comes from SO_PEERPIDFD at accept; the token comes from peios_token_open_process on that pidfd with QUERY | DUPLICATE. The job then has the identity a child of the submitter would have had — its primary token, with no impersonation inherited — so a submitter that was impersonating when it connected produces a job running as itself while being recorded as a submitter under the impersonated identity. A submitter that wants the impersonated identity on the job attaches it.
  • A token attached. The kernel delivered a token with the submit message through KACS_SCM_TOKEN, having run the impersonation gates with the submitter as installer and recorded on the token the lower of the socket's level and the source's own (Peios Kernel TRM §3.5). peinit uses that token as it arrived, at QUERY | IMPERSONATE | DUPLICATE.

Either way the source is duplicated to a primary token at its own impersonation level, with TOKEN_ALL_ACCESS, and the duplicate is what the child installs. The level carries through: a job started from an Impersonation-level token cannot convey Delegation, and a submitter that needs it — for a session that will authenticate onward on its user's behalf — sets its own jobs connection to Delegation before attaching, because the attach is clamped to the socket's level.

peinit refuses, with BAD_TOKEN, a source whose impersonation level is below Impersonation — an Identification-level token cannot pass an access check and an Anonymous one is not an identity to run a process as — and a source it cannot duplicate. It refuses nothing because of who the token names. Whether the submitter may run a job as that identity was the kernel's decision, made when the token was conveyed, and a second decision inside peinit could only disagree with the first.

There is no identity field in the definition, and no route by which a plain descriptor passed with SCM_RIGHTS becomes a job identity. A descriptor passed that way is not gated by the kernel — whoever holds an fd can pass it — and it would be the one path worth attacking.

The prepared token is a raw descriptor held on the job's entry until the launch takes its own duplicate of it (materialize_prepared_token, §4.1). A submission refused after the token was prepared closes it on the way out.

8.5.3 The submission #

submit runs in this order, and a refusal at any step leaves nothing behind — no record, no event, no quota consumed, and every descriptor the message carried closed:

  1. Parse and validate the definition against the attachment count (below). A malformed definition is INVALID_ARGUMENTS before any identity work is done.
  2. Refuse during shutdown with INVALID_STATE. The other four commands keep working while the system goes down.
  3. Establish the job identity, as above.
  4. Check the quota. peinit counts the submitter's live jobs — those not yet terminal — by submitter SID against Machine\System\Init\MaxJobsPerSubmitter, default 64. SYSTEM (S-1-5-18) is exempt. At the bound the answer is QUOTA_EXCEEDED. A job stops counting the moment it is terminal, not when its retained record is dropped.
  5. Build the job's descriptor: the one the submission supplied in SDDL, used as given, or the default (below).
  6. Create the record, the entry and the launch queue entry, in one SupervisorWork transaction.

The job is two things in peinit's state. The job record is the same JobRecord every fork gets (§8.1), with job_type: Submitted, no service, no operation, the job identity's user SID as resolved_identity, and a cgroup id under /sys/fs/cgroup/peinit/jobs/ named by the job's GUID. The entry is what the jobs system holds beside it: the submitter, the job identity's user SID and logon session, the definition, the descriptor, the prepared token, the attached descriptors with their names, the output sink, readiness and the retained STATUS and PROGRESS, any stop in progress, and once the job is terminal its outcome and retention deadline. Records are dropped when terminal, as every job's is; entries outlive them for the grace period, which is how a terminal job still answers status.

The submit is answered when the job leaves created — on exec confirmation or on launch failure — and not before. The submitter's connection holds a pending wait until then, reads nothing further, and is not idle. A running job's answer carries a duplicate of the job's pidfd as SCM_RIGHTS; a failed launch is still "status": "ok", with a terminal view whose cause says why. A pidfd that cannot be duplicated is answered INTERNAL_ERROR rather than a view with the handle silently missing — the submitter was promised one, and status gives it another chance.

8.5.3.1 The definition #

FieldDefaultValidation
image_pathrequiredAbsolute, non-empty, no NUL. Not checked for existence: execve decides that, as the job's identity.
arguments[]Strings without NUL.
environment{}Keys non-empty, no =, no NUL; values no NUL.
working_directory/Absolute.
description""
timeout0, no limitNon-negative integer, seconds.
stop_timeout10Positive integer, seconds.
readinessnonenone or notify.
readiness_timeout30Non-negative integer, seconds.
success_exit_codes[]Integers 0–255.
descriptors[]One name per injected descriptor; non-empty, no :.
outputfalseRequires at least one attached descriptor.
security_descriptorthe defaultSDDL with an owner and a DACL.

descriptors has to match the attachments exactly: its length plus one if output is set equals the number of descriptors on the message. The combined size of arguments and environment is bounded at 2 MiB, below what execve accepts. Nothing here restarts, depends, probes or schedules; a program that needs any of that is a service.

8.5.3.2 The descriptor #

Unless the submission supplied one, the descriptor is built by PeiosSystemAccessChecker from the submitter SID: owner and group the submitter; a DACL granting JOB_ALL_ACCESS to the submitter, to SYSTEM and to Administrators; nothing else; and a SACL auditing every refusal, for everyone, so a denied command on the job is recorded (§8.4). The job identity is granted nothing. A process cannot, by virtue of running as U, see or stop a job that runs as U — a submitter that wants the principal to see its own session says so in the descriptor it supplies.

A supplied descriptor is used as given, with no default entries added — its SACL included, so one with none has its denials recorded by no one. A submitter that omits itself has locked itself out of its own job, and peinit does not prevent that. The descriptor is fixed at submission.

RightBit
JOB_QUERY0x0001
JOB_STOP0x0002
JOB_SIGNAL0x0004
JOB_ALL_ACCESS0x0007

The generic mapping is read → JOB_QUERY, write and execute → JOB_STOP | JOB_SIGNAL, all → JOB_ALL_ACCESS. Every command on either door checks against this descriptor, the submitter included, naming the job to KACS in the check's audit context; a denial is answered ACCESS_DENIED, and the descriptor's SACL decides whether KACS records it (§8.4).

8.5.4 Launch #

Queued submissions launch from the work pump, one per pump step, through the ordinary child path (§5.3, §5.4) with three differences. The token source is LaunchTokenSource::Prepared: the launch takes its own duplicate of the prepared token through materialize_prepared_token, and the entry's copy is closed once the launch has it, whatever the outcome. The attached descriptors ride as inherited descriptors, placed from 3 upward with close-on-exec cleared and LISTEN_FDS, LISTEN_FDNAMES and LISTEN_PID set, exactly as the fd store hands descriptors back to a service (§10.6). And the environment is built as a service's is — the compiled-in base, the global layer, then the submission's environment in the place a definition's own variables occupy, then the protocol variables last, so a submitter cannot override NOTIFY_SOCKET or the LISTEN_* set (§5.5).

The cgroup is /sys/fs/cgroup/peinit/jobs/<guid>, a tree of its own rather than a service's.

Exec confirmation and failure arrive through the setup pipe as for any job. A parent-side failure — no token, no cgroup, no fork — takes the record from Created straight to Failed with cause parent_setup_failure; a child-side failure between fork and exec, or a failed exec, is pre_exec_failure. Both close everything the entry still held and try to remove the cgroup. On success the record is Running, the runtime registers the job's pipes with origin jobs/<guid>, and adopts the output sink if there is one (§11.1).

A stop on a job still queued cancels it: the launch entry is removed, the record fails before start with the stop's cause, and the answer is the terminal view. A stop that lands while setup is pending kills the cgroup and lets setup completion find the stop already recorded.

8.5.5 While it runs #

A submitted job speaks the notification socket like a service, with one difference in how it is believed: peinit routes a datagram to a service or to a submitted job by a pure PID lookup first — does any service's current main job carry this PID — and only then verifies the sender against that job's pidfd. A submitted job has no activation generation and no replacement, so the generation step does not apply. The sender is verified exactly once whichever way it was routed (§10.5).

What the fields do to a submitted job:

FieldEffect
READY=1For a notify job not yet ready: ready becomes true and the readiness timeout is disarmed. Otherwise ignored.
STATUS=Retained as status_text.
PROGRESS=Parsed as §4.19's three forms; retained as progress. A value outside the forms, T of zero or N above T is dropped, never repaired; the rest of the datagram is applied.
PROGRESS_UNIT=bytes, items or percent, retained; anything else dropped.
STOPPING=1Recorded, so a later stop sends no termination signal.
Everything elseIgnored: a job has no reload, no watchdog, no fd store.

A datagram that changed status_text, progress or the unit is due a peinit.job.status.reported event carrying the retained values (a verbose type, written only when the emission policy turns it on) — but at most one per job per second, measured from the last event emitted for that job, so a sender updating a thousand times a second produces one event a second while the view is current on every query.

Deadlines are held on the entry and folded into the one lifecycle deadline timer the event loop arms, as SubmittedJob kinds:

DeadlineDueAction
Timeoutstarted_at + timeoutBegin a stop with cause timeout.
ReadinessTimeoutstarted_at + readiness_timeout, for a notify job not yet readyBegin a stop with cause readiness_timeout.
StopKillstop requested + stop_timeoutSIGKILL the job's cgroup.
PostKillkill + PostKillTimeoutAbandon (below), or re-arm if the cgroup is empty but the exit not yet reaped.
CgroupCleanupterminal + PostKillTimeoutRetry removing a cgroup that was busy when the job ended.

A job being stopped has only its stop deadline; a deadline that fires for a job that has already ended does nothing.

8.5.6 Stopping #

Every stop — a submitter's stop, an administrator's job-stop, a timeout, shutdown — goes through begin_submitted_stop: record the cause and the kill deadline, then SIGTERM the main process through its pidfd unless the job has already sent STOPPING=1. A second stop on a job already stopping changes nothing and does not restart the deadline; a stop on a terminal job is a no-op answered with the unchanged view.

At the kill deadline the cgroup is killed. At the post-kill deadline, a cgroup still populated means the process survived SIGKILL: the record is abandoned, the cause becomes process_unkillable, the cgroup is leaked and reported as peinit.cgroup.leaked, naming the job rather than a service, and supervision stops.

signal is the raw mechanism, deliberately: one signal, by number, to the main process through its pidfd, on a running job only. SIGKILL this way leaves whatever the job spawned and produces a failed job with exit_signal set and a null cause. A submitter that wants the job ended uses stop.

8.5.7 Ending #

On reap, exit code 0 or one in success_exit_codes completes the job; any other code, or a signal, fails it. Then whatever the job left in its cgroup is killed and the cgroup removed; a busy cgroup schedules one CgroupCleanup retry, and a cgroup still busy after that is reported leaked. The prepared token, any attached descriptor the launch never took, and any sink the runtime never adopted are closed. The sink the runtime did adopt closes when the job's last pipe closes.

cause records what peinit decided, and stays null when the process ended of its own accord. A stop whose process handled SIGTERM and exited 0 is completed with cause explicit_stop: peinit asked, the process agreed, and both facts are recorded. A stopped job that died to the signal is failed with exit_signal set and the same cause.

The record is dropped at once, as every terminal job's is (§8.1), and peinit.job.ended carries it. The entry is retained for 60 seconds so a submitter polling for the outcome finds it, and then purged by operation maintenance; status on a purged or never-existent identifier is UNKNOWN_JOB either way. A terminal job whose cgroup cleanup is still pending is not purged until the retry has run.

8.5.8 Shutdown #

When a shutdown begins, every live submitted job is stopped at once, with cause shutdown and no ordering between them or against the service waves — a job has no dependencies to order by. A job still queued for launch is cancelled with the same cause. The shutdown is not finished while a live job remains: a job's reap advances shutdown progress exactly as a service's does, and the global timeout kills every live job's cgroup along with the remaining services (§12.2). submit is refused throughout; the other four commands keep answering, because a submitter watching its jobs go down has a reason to keep looking.

8.5.9 Clients #

svctl job list|status|stop are the control socket's job commands, and svctl job submit|wait|signal speak the jobs socket, submitting as the caller's own primary token with --fd NAME=FD for descriptors and --output for a sink. libpeinit exposes the same two halves: peinit_job_status, peinit_job_list and peinit_job_stop on the control client, and <peinit/jobs.h> — a peinit_jobs_t with peinit_job_submit, peinit_jobs_status, peinit_jobs_wait, peinit_jobs_stop and peinit_jobs_signal — with peinit_response_take_pidfd to claim the handle a submit returned.

8.6 What a Caller Sees

Peios / Peios technical reference / peinit / Jobs and Operations

Operations are how a caller observes something taking effect. This article covers what the model looks like from the outside; the wire contract itself is PSPU §4.

8.6.1 Every lifecycle command returns an identifier #

A command that creates, merges into, queues, cancels, clears or executes an operation returns that operation's identifier. A caller can then poll it, or block on it.

Two cases return no identifier, because no operation exists: a command whose target is already in the state it asks for, and one that has no effect at all — a stop on an Inactive service. Both return the service's status instead of an acknowledgement, which is the honest answer, and which also says what else the caller may do to the service (§10.2).

Commands that never create an operation — status, list, operation-status — have their own shapes. So do the three job commands, which never create an operation either: a submitted job has no state machine to contend for, so job-stop acts on it directly and answers with the job view.

8.6.2 Waiting #

Lifecycle commands block by default until the operation is terminal. The exception is reload, which returns immediately unless asked otherwise, because a reload's outcome is often advisory and a caller usually wants the identifier rather than the wait.

A waiting connection is not idle and is never closed by the idle timeout, however long the operation runs. It is bounded by the operation's own lifetime instead.

8.6.3 Merging is invisible #

A caller whose command merged receives the surviving operation's identifier and blocks on that operation's outcome. Nothing tells them they merged, because there is nothing they could usefully do about it. The consequence worth knowing is that the identifier a caller gets back may be older than their request, and its created_at will be earlier than when they sent it — which is exactly right, because that is when the work they are waiting on actually began.

8.6.4 What an operation's result carries #

A completed stop or reset carries the resulting service state; a completed start carries the readiness that satisfied it — alive readiness: process started, notify readiness: READY=1, or the exit code for a Oneshot — rather than a state name. A failed one carries the failure reason; one that peinit could not execute (§8.2) carries internal_error: <cause>. A merged one carries the survivor's identifier. A cancelled or aborted one carries why.

For a reload, the result also determines the reload's mode — whether the service confirmed the reload with a READY=1, whether peinit is merely assuming it happened, or whether it outright failed.

8.6.5 The job view #

A submitted job is reported in one shape on both sockets, the job view of PSPU §7.7: identifier, type of submitted, state, cause, submitter and identity SIDs, logon session, description, image path, pid while there is a process, ready for a notify job, exit code or signal, the retained status_text and progress, and the three timestamps. Every inapplicable field is present and null, and pid is null once the job is terminal — there is no process to name. Abandoned is the exception, and the reason for it: an abandoned job's processes survived SIGKILL and are still running, so its pid is kept.

What a submitter sees of its job on the jobs socket is exactly what an administrator sees of it in job-status, but for one field the control socket adds: granted, the job rights that caller holds on it (§10.2). And job-list is a list of the same views, filtered by JOB_QUERY on each (§10.2). A job-stop with wait registers a wait on the connection like a lifecycle command's, answered with the terminal view — or UNKNOWN_JOB, if the record was purged before the answer could be sent.

8.7 Submission and Progress Examples

Peios / Peios technical reference / peinit / Jobs and Operations

This reference expands the submitted-job operator workflow. Job implementation is in §8.5; transport is in §10.7 and the submission protocol is PSPU §7.

8.7.1 What a job runs as #

There is no identity field. A submitter cannot name an identity for its job; it can only run the job as an identity it already holds:

  • Nothing attached — the job runs as the submitting process's own primary token, exactly as a child of the submitter would. This is the common case for a tool or a service running something as itself.
  • A token attached — the submitter attaches a token to the submit message, and the kernel (through KACS) gates the attach: it verifies the submitter could act as that identity, at that impersonation level, at that moment. peinit turns the attached token into the job's primary token. This is how backupd runs a backup as its client, or a logon service starts a session as the user who just authenticated — the same impersonation machinery, with the kernel as the only judge. A job whose session must convey Delegation needs the submitter to set its jobs connection to Delegation level first; a job started at Impersonation cannot be raised later.

A token below Impersonation level, or one peinit cannot duplicate, is refused with BAD_TOKEN. peinit never refuses a token because of who it names — that decision was the kernel's.

Two identities are recorded on every job, and the view shows both: the submitter (who asked — the identity the connection was made under) and the identity (who it runs as), plus the identity's logon session.

8.7.2 The definition #

A submission carries the whole job in one message: image_path (absolute; required), arguments, environment, working_directory (default /), description, timeout (seconds; 0, the default, is no limit), stop_timeout (default 10 s), readiness (none or notify, to wait for READY=1), readiness_timeout (default 30 s), success_exit_codes, a list of named descriptors to hand the job, an output flag, and optionally a security_descriptor for the job itself. There are deliberately no policy fields — no restart, dependencies, health check, or trigger. A program that needs those is a service.

peinit does not check that the program exists or is executable when it accepts the submission; that is decided at execve, by the job's own token, and a program that cannot be run becomes a failed job rather than a refused submission. The submit is answered only once the job has left created — the process confirmed its exec, or starting it failed. A readiness: notify job is answered when it is running, not when it is ready; a submitter that wants to block until readiness follows with a wait.

Descriptors a submitter attaches are handed to the job from descriptor 3 with LISTEN_FDS, LISTEN_FDNAMES and LISTEN_PID set — the same convention a service uses to adopt stored descriptors, so a program written for one adopts them from the other unchanged. This is what makes a session job possible: the submitter keeps one end of a socketpair and attaches the other.

8.7.3 The job view #

Whether you ask on the jobs socket or with job status, you get the same view: the job id, type (submitted), state and cause, submitter and identity SIDs, logon_session, description, image_path, pid (while running), ready (null for a readiness: none job, otherwise whether READY=1 has arrived), status_text, progress, the exit code or signal, and the created_at/started_at/ended_at timestamps.

status_text and progress come from the job itself, through the same sd_notify socket a service uses. A job sends STATUS=Backing up /data/media for a human-readable line and PROGRESS= in one of three forms:

FormMeaningShown as
PROGRESS=NCounting, with no end in sight.A rising count.
PROGRESS=N/Counting towards an end that exists but is not yet known.A rising count, awaiting a bound.
PROGRESS=N/TN of T.A bar.

PROGRESS_UNIT=bytes|items|percent says what the numbers are. peinit keeps the latest accepted values and emits a peinit.job.status.reported event when they change, rate-limited to at most one per job per second so a chatty job cannot flood the event stream — the view always has the latest value regardless.

9.1 Calendar Expressions

Peios / Peios technical reference / peinit / Timers

A timer schedule is a calendar expression in systemd's OnCalendar format. The grammar below is what peinit parses.

DayOfWeek Year-Month-Day Hour:Minute:Second Timezone

Every field is optional, and which fields are present is worked out positionally from their shape.

FieldFormDefault if omitted
DayOfWeekweekday namesany day
DateYear-Month-Day*-*-*
TimeHour:Minute:Second00:00:00
Secondthe :Second of Time:00
Timezonean IANA namesystem-local

A time-only expression such as 02:00:00 implies the date *-*-*. Hour:Minute alone defaults the seconds to 00.

Years range 0–9999, months 1–12, days 1–31.

9.1.1 Component syntax #

Every numeric component, and the weekday, accepts:

  • Wildcard — * matches anything.
  • List — comma-separated values: 1,15.
  • Range — two values around .., inclusive: Mon..Fri, 8..17. A reversed range is a parse error rather than a wrap-around.
  • Repetition — a value or range suffixed with / and a step. value/step matches the value and every multiple of the step above it, so 0/15 in the minute field is 0, 15, 30 and 45. start..end/step walks from start to end inclusive.

Weekdays are English names, case-insensitive, abbreviated or full, and accept lists and ranges. Mon and Monday are the same day, as are Tue, Tues and Tuesday.

9.1.2 Last day of the month #

A ~ in place of the - between Month and Day counts the day from the end of the month: ~01 is the last day, ~02 the second-to-last, and so on. *-*~01 is the last day of every month; *-02~03 is the third-to-last day of February.

Repetition combines with it, and steps in the day-of-month direction — which means it walks the offset downwards, towards the end of the month. Mon *-05~07/1 covers the last seven days of May, and combined with Mon resolves to exactly one day: the last Monday in May.

Wildcards, ranges and lists are also accepted after ~, so ~* matches every day and ~03..07 the third- to seventh-to-last.

9.1.3 Named shortcuts #

ShortcutEquivalent
minutely*-*-* *:*:00
hourly*-*-* *:00:00
daily*-*-* 00:00:00
weeklyMon *-*-* 00:00:00
monthly*-*-01 00:00:00
quarterly*-01,04,07,10-01 00:00:00
semiannually*-01,07-01 00:00:00
yearly, annually*-01-01 00:00:00

Shortcut names are case-insensitive and accept a trailing timezone, so daily UTC is valid.

9.1.4 Timezones #

Timezone specifiers are IANA database names — Europe/London, US/Eastern, UTC. An expression with no timezone is interpreted in system-local time.

A name is validated when the expression is parsed and resolved again at evaluation. An unrecognised zone is a hard parse error, not a silent fallback to UTC.

9.1.5 Daylight saving #

  • Spring forward, where the clock skips an hour: a scheduled time falling inside the skipped interval does not fire. peinit moves on to the next scheduled second, then the next date. A schedule of *-03-31 01:30 Europe/London skips 2024 entirely.
  • Fall back, where an hour repeats: a scheduled time inside the repeated interval fires exactly once, on the first occurrence. On re-arming, the same civil time resolves to the same instant, which is not later than the firing that just happened, so the timer advances to the next day rather than firing again.

9.1.6 Precision #

Second-level. Unlike systemd, peinit does not accept fractional seconds — service scheduling has no use for finer granularity, and dropping it keeps the parser simpler. A fraction anywhere in the time component is a parse error.

9.1.7 Examples #

ExpressionMeaning
*-*-* 02:00:00Every day at 2am, system-local.
Mon *-*-* 00:00:00Every Monday at midnight.
*-*-1,15 12:00:00The 1st and 15th, at noon.
*-*~01 00:00:00The last day of each month, at midnight.
Mon..Fri *-*-* 09:00:00Weekdays at 9am.
*-*-* *:00/15:00Every fifteen minutes.
*-*-* 02:00:00 Europe/LondonEvery day at 2am London time.

Note

An expression that parses but can never match — *-02-30, or a fixed year already past — is not rejected at parse time. Computing its next occurrence walks forward a day at a time for ten years and then gives up, reporting no future occurrence. The horizon is there because walking the full year range would turn an unsatisfiable schedule into a boot that hangs.

9.2 Evaluation and Arming

Peios / Peios technical reference / peinit / Timers

A timer is a trigger, not a service type. A service with a timer:<schedule> trigger is an ordinary Simple or Oneshot service that peinit starts on a schedule.

9.2.1 Arming #

At boot, once the service graph is loaded, and whenever timer configuration changes, peinit computes the next firing time for every active trigger and arms a timerfd for it. Each trigger gets its own descriptor and its own computation.

A disabled service gets neither a registration nor a firing.

A schedule that fails to parse, or whose next occurrence cannot be computed, fails that trigger. Every other timer arms normally, and what did not arm is reported to the console and in the service's status. This matches how graph validation already treats an invalid schedule (§7.4), so the outcome no longer depends on which of the two caught it.

The next-occurrence search looks ten years ahead and then gives up. A schedule can parse and still match nothing — *-02-30, or a fixed year already past — and the horizon turns that into a prompt error against the one service rather than a very long walk. Ten years clears the sparsest schedule that is genuinely meaningful: *-02-29 skips a century year not divisible by 400, so it can run eight years dry.

9.2.2 Firing #

handle_timer(service, trigger):
    // 1. Decide what the firing means, from the service's state.
    match (service.type, service.state):
        (Oneshot, Active | Starting):
            service.pending_timer = true      // at most one
        (Simple,  Active | Starting):
            record the firing; no action
        (Oneshot, Inactive | Completed | Failed):
            create_operation(Start, service, source = Timer)
        (Simple,  Inactive | Failed):
            create_operation(Start, service, source = Timer)

    // 2. Record when it fired.
    write the last-run timestamp to the registry   // asynchronously

    // 3. Re-arm.
    next = next_occurrence(trigger.schedule, now) + random(0, TimerJitter)
    arm an absolute CLOCK_REALTIME timerfd for next

Every other state — Backoff, Stopping, Reloading, Abandoned, Skipped — records the firing and does nothing.

A timer can outlive its service until the next reload. A service whose definition is deleted while it runs carries on, and is discarded when it stops, and nothing re-plans the timers then. A firing for a service no longer known does nothing, records no last run, and is not armed again; the next reload leaves it out. It is never an error: an error out of a firing takes PID 1 into recovery.

No last run is recorded for a service whose definition has been deleted, whether it is still running or already gone: there is no key to record it in, and writing one would make the key again.

The last-run write happens in a forked child so that the event loop never waits on the registry — which matters because the registry is served by registryd, a service peinit supervises, so a synchronous write would let a wedged registryd stall PID 1.

The parent returns immediately with the child's pid, and remembers it. When the child is reaped, peinit matches its exit status back to the write; a failure is reported:

peinit warning: recording the last run of timer <schedule> for service
<service> failed; it will run catch-up again after a reboot

The write stays best-effort — nothing is retried and nothing is failed over it — but a persistent timer whose timestamp never lands runs its catch-up on every boot, and that is otherwise a symptom with no thread to pull. The outstanding-write table is bounded, so a child that somehow escapes reaping cannot grow it.

9.2.3 Oneshot pending runs #

A Oneshot that fires while it is already running sets a flag rather than queueing an operation. When it next reaches Inactive or Completed, peinit immediately creates a start operation and clears the flag.

Multiple firings during one run collapse into a single pending run. There is no queue, and the flag is per service rather than per trigger — a service with three timers that all fire during one long run still gets exactly one catch-up.

9.2.4 Multiple triggers #

Triggers on one service are independent: each has its own timerfd, its own next-firing computation, and its own last-run history. Only the Oneshot pending flag is shared.

9.2.5 Reporting #

status reports every trigger of a service that is not disabled, as it stands: the occurrence armed, the time it will fire with its jitter, and when it last fired; or, for a trigger that did not arm, why not (PSPU §4.14). list carries the soonest firing of each service.

The figures are the armed ones, given to the supervisor whenever a trigger is armed: at boot, on a reload that adds or changes the timer (an unchanged timer keeps its armed deadline), after each firing and after a clock change. A client cannot compute them itself, because the jitter is drawn at random when the trigger is armed.

When a trigger last fired is held in memory as well as the registry. It is seeded at boot from the recorded timestamp of a persistent trigger, set by each firing, including a boot catch-up, and kept across a reload for the same service and schedule. A definition deleted and made again between two reloads is the same service and schedule, and keeps it too. A non-persistent trigger, which records nothing (§9.3), reports only its firings since peinit started.

9.3 Persistence

Peios / Peios technical reference / peinit / Timers

TimerPersistent, on by default, controls whether a run missed across a reboot is caught up.

9.3.1 Where history lives #

Last-run timestamps are REG_QWORD values in the registry, written after a firing.

A service with a single timer trigger stores its timestamp as LastTimerRun on the service's own key:

Machine\System\Services\<name>\LastTimerRun

A service with multiple triggers stores one per trigger under a subkey, named by the schedule string:

Machine\System\Services\<name>\TimerState\<encoded-schedule>

A schedule contains characters — spaces, :, * — that are not valid LCS value names, so the name is the schedule with every character outside [A-Za-z0-9._-] percent-encoded, with uppercase hex digits. This is the same encoding used for cgroup ids (§5.1). The schedule *-*-* 02:00:00 is stored as:

%2A-%2A-%2A%2002%3A00%3A00

Two identical schedule strings on one service encode to the same name and therefore share one timestamp.

Timer firings are infrequent, so the write cost is negligible.

9.3.2 Catching up #

On boot, for each persistent trigger:

  1. Read the last-run timestamp.
  2. Compute the next scheduled firing after it.
  3. If that time has already passed, at least one run was missed: fire once, immediately.
  4. Compute the next future occurrence normally.

Catch-up is always a single run however many were missed. A daily timer that missed five days fires once on the next boot, not five times.

A trigger with no history at all is treated the same way, so its first boot produces one catch-up firing.

TimerPersistent=0 ignores history entirely — peinit does not even read the registry for that trigger, and computes the next occurrence from now.

9.3.3 When the timestamp is written #

The timestamp is written after the timer fires and the start is initiated, not after the service finishes. A service that crashes mid-run is not re-triggered on the next boot: the run was attempted, not missed.

A configuration reload keeps an already-armed timer when its service name, schedule string, TimerPersistent, TimerJitter, and timestamp storage layout are unchanged. Its scheduled occurrence, jittered firing time, and last firing are retained, even if the occurrence has passed while its jitter is still pending. Unrelated configuration edits and a timer's own LastTimerRun writes therefore cannot postpone that firing.

New or changed timers are armed from the current time with no catch-up, whatever TimerPersistent says. Removed or disabled triggers are unregistered. History is consulted at boot only.

Note

Timestamps for a multi-trigger service are keyed by schedule string, so changing a schedule orphans its history and the new schedule produces one spurious catch-up run. A stable trigger identifier would fix it; schedule changes are rare and one extra run is the whole cost. A single-trigger service is unaffected, since its timestamp lives at a fixed name.

9.4 Jitter and Clocks

Peios / Peios technical reference / peinit / Timers

9.4.1 Jitter #

TimerJitter, zero by default, adds a random delay to each firing. peinit draws a uniformly random whole number of seconds from zero to TimerJitter inclusive, from the kernel's random source, and adds it to the computed occurrence.

The delay is recomputed on every firing, so a daily timer with TimerJitter=900 fires at a different moment between 00:00 and 00:15 each day. With TimerJitter=0 no randomness is consulted at all.

Jitter is applied after the calendar expression is evaluated and is only ever added, so a timer never fires early — only late.

The boot catch-up firing is not jittered. It fires immediately, and jitter applies from the next armed occurrence onward.

9.4.2 Which clock #

The split is the point of this section, and it is not cosmetic.

Calendar timers are wall-clock schedules, so they are armed as absolute CLOCK_REALTIME timers: timerfd_settime with TFD_TIMER_ABSTIME | TFD_TIMER_CANCEL_ON_SET. CANCEL_ON_SET makes the descriptor's read return ECANCELED whenever the realtime clock is discontinuously changed — an NTP step, a manual set. peinit recomputes the next occurrence against the new wall clock and re-arms, which is what keeps *-*-* 02:00:00 anchored to 02:00 across clock corrections.

Interval timers are genuine relative durations and use CLOCK_MONOTONIC: the watchdog, health check intervals and timeouts, restart backoff, and the Start, Stop and Reload phase timeouts. "Wait thirty seconds" means thirty elapsed seconds regardless of what happens to the wall clock. They are not armed with CANCEL_ON_SET, correctly — a monotonic timer has no reason to be cancelled by a realtime set.

These are not separate descriptors. Every interval deadline is aggregated onto one monotonic timerfd armed to the earliest of them.

Last-run timestamps are recorded on CLOCK_REALTIME, since they record when a timer actually fired in wall-clock terms. The same firing passes a monotonic timestamp into the operation machinery, because operation timing is elapsed time.

9.4.3 Clock events #

  • A realtime step at runtime. The armed timer is cancelled; peinit recomputes against the new wall clock and re-arms. A backward step pushes the next firing later; a forward step that crosses an occurrence fires it once. If the step lands inside a jitter window, the firing happens at the un-jittered scheduled time — later than the schedule, never earlier.
  • Suspend and resume. An absolute deadline that elapsed while suspended fires once on resume. The expiration count is ignored, so a long suspend produces one firing, not one per occurrence.
  • A missed occurrence within one uptime. Fire once, then compute the next future occurrence. peinit never replays every occurrence that elapsed during a gap — the same rule as the cross-reboot catch-up.
  • A backward jump across a boot. If the last-run timestamp is in the future relative to the current wall clock at boot, peinit treats the history as unknown and fires the catch-up immediately. This check is boot-time only; there is no runtime equivalent.
  • A wrong clock at boot. A system that boots with a badly wrong clock and has NTP correct it later may fire a persistent catch-up spuriously or not at all. The runtime half is covered by CANCEL_ON_SET — once NTP corrects the clock, armed calendar timers are cancelled and recomputed — but the boot-time catch-up decision has already been made by then. Short of NTP-aware rescheduling, this remains an edge.

Note

The calendar parser deserves heavy testing. Time parsing is a rich source of edge cases: month boundaries, leap years, last-day-of-month arithmetic, DST transitions and timezone database updates are all fertile ground for subtle errors, and every one of them is a bug that only appears on a particular day of a particular year.

10.1 The Control Socket

Peios / Peios technical reference / peinit / The Control Interface

peinit serves every runtime command on a Unix stream socket at /run/services/peinit/control.sock, created during Phase 1 infrastructure setup and existing for the lifetime of the system. Its wire protocol is specified in PSPU §4; this chapter is how peinit implements its side. A second socket beside it, the jobs socket, is the door for submitting jobs rather than administering services; it is §10.7.

10.1.1 Creation and protection #

The socket is created with SOCK_CLOEXEC | SOCK_NONBLOCK and a listen backlog of 32, and unlinked when peinit drops it. Accepted connections come from accept4 with both flags, so no connection descriptor is ever inherited by a service.

peinit sets no POSIX mode bits on the socket, on the notification socket, or on anything else it creates. Under KACS, mode bits are not what governs access — a Security Descriptor is — so setting them would be inert.

What governs access is the descriptor on the socket inode, which the kernel checks at connect() before any peer identity is established. peinit creates /run/services/peinit/ and stamps the control socket explicitly, after binding and before anything can connect :

O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)

Every authenticated principal may connect. FW is the file-write generic right, which is what connect() on a pathname socket needs. Reaching the socket grants nothing on its own: every command is checked against the control descriptor or the target service's descriptor, with the token captured at accept (§10.2). Admitting only SYSTEM and Administrators would make that check moot for everyone else, since a descriptor that grants a principal something could never be reached by them.

The notification socket is stamped explicitly too, with a descriptor of its own that grants SYSTEM and the Service group rather than Administrators (§10.5). The jobs socket carries a broader descriptor again, because reaching it is a different permission (§10.7).

10.1.2 Connections #

peinit accepts a connection, obtains the peer's token, and only then admits it against the connection limits:

KeyDefaultMeaning
Machine\System\Init\MaxControlConnectionsPerUser16Concurrent connections one caller may hold, counted by the user SID of its token. SYSTEM is exempt.
Machine\System\Init\MaxControlConnections256Concurrent connections in all: a bound on peinit's descriptors, set well above what callers reach.
Machine\System\Init\MaxRequestSize65536Maximum request size, in bytes.
Machine\System\Init\ConnectionTimeout30Seconds before an idle connection is closed.

The per-user limit is what keeps one caller from taking the connections everyone shares: with every authenticated principal able to connect, a single overall limit would let anyone fill it and lock administrators out. SYSTEM is exempt, as it is from MaxJobsPerSubmitter, because what it does is the machine's.

A connection over either limit is closed at the socket level, before any request is read and without a response — there is no error code for it, because there is no protocol state in which to deliver one. A peer whose token cannot be obtained is closed the same way.

The jobs socket has its own three, under Machine\System\Init\: MaxJobsConnections (64), MaxJobMessageSize (32768) and JobsConnectionTimeout (30), plus a fourth bound that is the submitter's rather than the connection's, MaxJobsPerSubmitter (64). All four are read at boot and on reload-config, and are listed with the other operational keys in the registry key reference.

10.1.3 The peer token #

The token is read once, when the connection is accepted, through the kernel's peer-token socket option (getsockopt(SOL_KACS, KACS_SO_PEER_TOKEN); the Peios Kernel TRM §3.5). It is the identity the peer thread was acting under when it connected, so a peer that was impersonating is captured as the impersonated identity — which is what makes access decisions reflect the identity a client is actually operating under rather than its underlying service identity.

Because it is captured once, a peer that changes identity mid-connection is still evaluated against the identity it connected with.

10.1.4 Idle and waiting #

A connection is idle only when it has nothing in flight. One blocked on a wait=true operation, on a job-stop with wait, or with output still buffered, is never idle and is never closed by ConnectionTimeout — it stays open until the operation or job resolves, bounded by the operation's own timeout rather than the connection's. A job wait has no timeout of its own; it is bounded by the job.

A wait is answered when its operation reaches a terminal state, with the lifecycle acknowledgement carrying the service's state at that moment. Two cases are answered differently. An operation that failed because peinit could not execute it (§8.2) is answered with an INTERNAL_ERROR whose message carries the operation's result, not with an ok carrying an unchanged state. And an operation whose service has been discarded by the time it resolves — a stop that drained a definition-removed entry (§10.3) — is answered with the state the service last had, since there is no entry left to ask. Neither answer costs the connection or the listener.

peinit handles one frame per readiness turn, and processes no further frames from a connection while a wait is pending on it. Pipelined requests are therefore serialised behind a wait.

10.1.5 Timestamps #

Every timestamp peinit puts on the wire is derived by projecting a monotonic event stamp through the current offset between the realtime and monotonic clocks. Elapsed-time decisions stay monotonic; only the presentation is wall-clock.

10.2 Dispatch and Authorisation

Peios / Peios technical reference / peinit / The Control Interface

A parsed command runs a fixed sequence before it does anything.

  1. The shutdown gate. If peinit is shutting down, everything except status, list, operation-status, boot, job-status, job-list and job-stop is rejected. boot stays open because, like the service queries, it changes nothing. The three job commands stay open because a shutdown stops every submitted job anyway (§12.2), and an administrator watching that happen, or wanting a job gone sooner than the shutdown will manage, has a reason to ask. The gate runs before the access check, so during shutdown a caller who would have been denied is told the command is invalid for the current state rather than that they lack the right.
  2. Resolve the target. A command naming no definition, and no addressable definition-removed entry, returns UNKNOWN_SERVICE; one naming no submitted job peinit holds returns UNKNOWN_JOB. peinit does not synthesise a descriptor for something that does not exist.
  3. AccessCheck. The caller's token, the target's descriptor, the generic mapping, the right the command requires (§4.6, §4.7), and an audit context naming the target: the service, the job, or the control door itself (§8.4). For a job the descriptor is the job's own and the mapping is the job mapping (§8.5).
  4. On denial, return ACCESS_DENIED. The decision is recorded by KACS, as kacs.audit.access.checked naming the caller, the target and the access requested and granted, whenever the target's descriptor has a SACL that asks for it; every descriptor peinit builds itself audits every refusal. peinit writes no event of its own. Silent denial is not acceptable; a denial an administrator cannot see is indistinguishable from a bug, which is why the defaults audit, and why a descriptor written to the registry should keep a SACL that does (§4.6).
  5. On grant, classify the command against the service's state (§10.3) and act.

10.2.1 Rights #

CommandRight
startSERVICE_START
stopSERVICE_STOP
restartSERVICE_START and SERVICE_STOP, requested as one mask
reloadSERVICE_INTERROGATE
resetSERVICE_STOP
statusSERVICE_QUERY_STATUS
listFiltered per service by SERVICE_QUERY_STATUS
operation-statusSERVICE_QUERY_STATUS on the target service
job-statusJOB_QUERY on the job
job-listFiltered per job by JOB_QUERY
job-stopJOB_STOP on the job
shutdownSYSTEM_SHUTDOWN
reload-configSYSTEM_RELOAD_CONFIG
bootSYSTEM_QUERY_STATUS on the control descriptor, which by default every authenticated principal holds (§4.7)

operation-status resolves the operation before it checks the right, so an unknown identifier is reported as unknown regardless of who asked. If the operation's service has since been discarded (§3.8), the right is checked against the ServiceSecurity the service had when the operation was created, which peinit records on the operation; a retained operation therefore stays queryable for its retention window (a2) by exactly the callers who could have queried the service.

job-status and job-stop resolve the job first too, and for them that is the whole story: UNKNOWN_JOB for an identifier that names nothing, ACCESS_DENIED for one that names a job the caller may not touch. A job identifier is unguessable, so a caller that names one was told it by something that had it, and there is no existence to hide. job-stop creates no operation; it stops the job directly (§8.5) and, with wait, registers a job wait on the connection answered by the terminal view.

10.2.2 What the caller may do #

A status response carries granted, the service rights the caller holds on the service, so that a client can offer only the commands that will be allowed rather than learn the rest from ACCESS_DENIED (PSPU §4.14). peinit finds them with step 3's AccessCheck — the caller's token, the service's ServiceSecurity, the service mapping — asking for MAXIMUM_ALLOWED instead of one right, once the query itself has been allowed, and reports the service rights among what it grants. A lifecycle command that has nothing to do answers with the status (§10.3), and that carries granted for its caller too.

Each job view on the control socket carries granted the same way: the job rights the caller holds, from the AccessCheck against the job's own descriptor with the job mapping (§8.5). job-stop with wait finds them when it is taken and keeps them on the connection's wait, because the terminal view is written by the run loop with no caller's token to hand; the descriptor is fixed at submission, so they are the same as at the answer. The jobs socket's view has no granted.

The MAXIMUM_ALLOWED check is a question, not a command. A right it does not grant is left out of granted; nothing is refused, and peinit answers the command as it would have without it.

10.2.3 Filtering #

list checks every service and partitions the result. Services the caller can query are returned; services it cannot are omitted, and the denials are left to each service's SACL rather than shown to the caller. A caller with no query rights anywhere gets an empty list and a successful response, not a denial . The filtering keeps a service's state from whoever may not query it; it does not hide that the service exists, which step 2 above already answers for any command that names one (§4.6).

Definition-removed services are listed, and the list entry does not say so. A status query on one does.

job-list is filtered the same way. The four request filters — submitter, identity, logon_session, state — narrow the candidates first; each survivor is then checked for JOB_QUERY against its own descriptor, each denial is the job's SACL's to record, and the response does not say whether a filter or a right removed an entry. Terminal jobs still within their retention are listed; a caller that wants only live ones filters by state.

10.3 The Command × State Matrix

Peios / Peios technical reference / peinit / The Control Interface

A command sent to a service in an unexpected state gets an answer, not a silent no-op. What the answer is depends on the pair.

InactiveStartingActiveReloadingStoppingCompletedBackoffFailedAbandonedSkipped
start StartMERGEALREADYALREADYQUEUEStartDEFERStartERRORStart
stop NOOPCancel+StopStopStopMERGEClearClearNOOPERRORNOOP
restart StartQUEUERestartRestartQUEUEStartRestartStartERRORStart
reload ERRORERRORReloadMERGEERRORERRORERRORERRORERRORERROR
reset NOOPERRORERRORERRORERRORERRORERRORClearClearClear
status OKOKOKOKOKOKOKOKOKOK

ALREADY — the service is already where the command would take it and no operation of that type is in flight. peinit returns the current status rather than an error.

MERGE — an operation of that type is already running. The command merges into it; the caller receives that operation's identifier and, if waiting, blocks on its outcome.

DEFER — create a Pending start operation but do not execute it until the existing backoff deadline expires. A deferred start already present is merged into.

QUEUE — the operation is queued Pending and executes after the current one completes.

NOOP — the command has no effect. peinit returns the status.

ERROR — the command is invalid for the state.

Clear — reset to Inactive.

Cancel — abort the current operation, then proceed.

10.3.1 The Backoff column #

Backoff is the interesting one, because the service is down with an automatic restart already pending.

  • start creates or merges into a deferred start operation and honours the remaining delay. It does not short-circuit the backoff. If the automatic restart later becomes due, it merges into the administrator's operation, so the identifier the caller holds is the one that executes.
  • stop cancels both the pending restart and any deferred start, and the service goes Inactive. A subsequent automatic restart is refused, because the service is no longer in Backoff.
  • restart replaces the automatic restart with the administrator's. A Restart operation with source admin is queued Pending and honours the remaining delay exactly as a deferred start does; when the delay expires it is that operation — keeping its type, for observability — that executes, so the identifier the caller holds is the one that runs. A deferred start already pending is cancelled with reason superseded_by_restart, and a second restart merges into the pending one rather than queueing behind it. Nothing is sent to the control boundary, because there is no process for it to act on.
  • reload and reset are invalid: there is no process to reload, and no terminal state to clear.

10.3.2 The Skipped column #

start and restart clear Skipped before they run. A Skipped service is not in a state a start can proceed from — the state machine permits Skipped -> Inactive and nothing else — so the activation performs that transition first, then re-evaluates the conditions from scratch. Both outcomes are possible: the precondition that was missing at boot may now hold, in which case the service starts; or it may still not, in which case the service is skipped again, for whatever reason applies now.

The clear is reported like any other transition, so a console watching the service sees it leave Skipped rather than appearing to jump: peinit: service X left skipped: ExplicitStart, before the start's own lines.

reset also clears Skipped, and differs only in stopping there.

10.3.3 The Abandoned column #

Every lifecycle command is invalid on an Abandoned service except reset, which clears it (§6.2). Nothing else is meaningful while processes that ignored SIGKILL are still in the cgroup.

10.3.4 Definition-removed services #

Independently of state, a service whose definition has been removed (§3.8) rejects start, restart and reload with UNKNOWN_SERVICE, accepts stop, and reports its state on status.

The accepted stop drains the instance with the cached StopTimeout, and the exit that ends it also discards the entry. A caller waiting on that stop is answered after the discard, with the state the service last had — Inactive, cause ExplicitStop — rather than with a lookup of an entry that no longer exists. A status sent after that is UNKNOWN_SERVICE, as for any service peinit does not know.

10.4 reload-config

Peios / Peios technical reference / peinit / The Control Interface

reload-config takes a fresh snapshot of the configuration from the registry. It does not live-update anything.

It is also the path a registry change notification takes: any drained watch event triggers the same full reload, rather than a targeted re-read of whatever changed.

10.4.1 Atomicity #

peinit reads everything first. Every registry read happens before any mutation, so a read failure returns an error with nothing touched. It then builds and validates a complete new graph in memory, and only swaps it in if validation succeeds.

If validation fails, the previous generation stays live and the findings are returned to the caller. This is where the reload path differs sharply from boot: boot marks individual services Failed and carries on, because it has to produce a running system; reload rejects the whole thing, because it has a running system already and a half-applied configuration would be worse than the one in place.

An explicit reload-config writes one peinit.config.reload.applied event either way: outcome.success, and on success the counts of what it added, updated, restored, marked removed, discarded and found undecodable; on failure why, in words, and one peinit.graph.validation.failed per finding under phase reload-config (§8.4). A reload a registry watch starts writes its findings and warnings but no peinit.config.reload.applied.

Decoding does not split: reload decodes per key exactly as boot does. A key that will not decode fails that one service with ValidationError — a placeholder entry, marked definition-removed, that status reports and nothing can start — and every other definition in the batch loads; a dependent of the failed key fails through ordinary propagation when it is next started. A service that is running when its key stops decoding keeps running with its definition marked removed, as if the key had been deleted. The reload succeeds, lists the keys in summary.undecodable, and names each key, the offending field and the problem in undecodable; each is also a peinit.graph.validation.failed event with outcome.reason validation-error under phase reload-config. A key whose name is not a service name (§3.1) is undecodable in the same way, reported under its raw name with field: name. Repairing the key restores the service on the next reload.

10.4.2 What changes #

  • Every service definition is re-read.
  • A new dependency graph is built and validated.
  • Running services are unaffected and continue on their activation generation.
  • New definitions take effect at the next start, restart, or trigger.
  • New services become available for start immediately.
  • Timer triggers are re-evaluated. Unchanged calendar timers retain their armed occurrence and jittered deadline (§9.3); new or changed timers are armed from the current time with no catch-up.

A reload also refreshes things that are not service definitions: the control descriptor, the four control socket limits, the log configuration, the shutdown settings, the global environment layer, and the eventd log socket path. It also prunes the fd stores of services that no longer exist.

10.4.3 Removals and the compiled-in service #

A definition that has disappeared is handled by §3.8 — discarded if nothing is running, marked definition-removed if something is.

registryd is exempt. Its compiled-in definition survives a reload that does not mention it, and its provenance survives a registry entry that shadows it. peinit started registryd before the registry existed, and a reload finding no definition for it cannot conclude that it should stop being managed.

10.5 The Notification Socket

Peios / Peios technical reference / peinit / The Control Interface

peinit binds one Unix datagram socket for service notifications, by default at /run/services/peinit/notify.sock. Its path is an implementation detail: services receive it through NOTIFY_SOCKET and nothing hardcodes it. The kernel command line can override it with peios.notifysocket=. peinit creates the socket's directory only when it lies under /run; an overriding path anywhere else must have a parent that already exists. /run/services/peinit itself is created whichever path is in use.

There is one socket, not one per service, and the bind unlinks any stale path first.

10.5.1 The descriptor #

O:SYG:SYD:(A;;GA;;;SY)(A;;FW;;;S-1-5-6)

S-1-5-6 is the Service group, which every token minted for a service logon carries.

It is written out rather than as its SU alias, and that is deliberate rather than an oversight. The alias table lives in libpeios, which is a separately versioned package that nothing here declares a minimum version of — so an image pairing a peinit that writes SU with a libpeios that predates the alias parses this descriptor, fails, and takes PID 1 into recovery before a console exists to say why. That is not hypothetical: it is what happened the first time this descriptor was written. A literal SID has been understood by every version there has ever been. That is the right grantee because it is the population with something to say here: membership follows from having been started as a service rather than from which account a process runs under, so an ordinary user process cannot acquire it however it was launched.

FW is the file-write generic right, which is what a write to a pathname socket needs — the same reasoning as the jobs socket (§10.7). Administrators are absent, unlike the control socket: an administrator has no business asserting that a service is ready.

The descriptor is installed by path, immediately after bind — before the socket is made non-blocking, given SO_PASSCRED, or polled.

It is by path because the fd form does not work on a socket. KACS exposes no fd-targeted descriptor call of its own: fd_set_sd is the path call with the target as dirfd, an empty path and AT_EMPTY_PATH, and the kernel answers EOPNOTSUPP for a socket. Stamping by fd was tried here and took PID 1 into recovery.

That leaves a window between bind and the stamp in which the socket carries what it inherited. It is safe for the same reason it is on the control socket: the parent grants services traverse and nothing else and carries no inheritable ACE, so the inherited descriptor is narrower than the one being installed. The failure direction is refusal, not exposure.

The parent directory /run/services/peinit carries a descriptor of its own, shared with the control socket that lands in it later in boot: SYSTEM and Administrators in full, and S-1-5-6 traverse only. One constant serves both because ensure_directory re-stamps a directory that already exists — two call sites installing different descriptors on one path would silently leave whichever ran last, and since the notify socket is bound first, that would be the control socket's, quietly removing the traverse services need to reach notify.sock at all.

The protocol itself is PSPU §4. What follows is how peinit decides whether to believe a datagram.

10.5.2 Authenticating a sender #

SO_PASSCRED is enabled on the socket, so every datagram arrives with a kernel-attested SCM_CREDENTIALS control message. A datagram without one is rejected outright. Descriptors for the fd store arrive alongside, as SCM_RIGHTS.

Authentication then runs five steps, and each closes a hole the previous one leaves:

  1. Find the sender. Scan for the current service-main job whose PID equals the sender's. Only a main job is ever a candidate, which is what makes NotifyAccess=Main the only mode there is — a hook or a health check cannot notify on a service's behalf. If no service's main job carries the PID, the live submitted jobs (§8.5) are scanned instead. The routing question is asked purely by PID, before either door verifies anything, so that a sender is verified against its pidfd exactly once whichever way it is routed.
  2. The job is Running. A job still in pending setup has not exec'd.
  3. The job has a pidfd.
  4. The pidfd still refers to that PID. This is the step that matters: PID matching alone is racy, because a PID can be recycled between the sender writing and peinit reading. The pidfd was obtained atomically at fork, so verifying the PID against it is what makes the match sound rather than probable.
  5. The generation matches. A job whose activation generation is not the service's current one is a previous incarnation, and its notifications are rejected. This is invariant 5 of §6.1 in force: a READY=1 from the process that just crashed cannot mark its replacement ready. A submitted job has no generation and no replacement, so for it the check stops at step 4.

Anything that fails is dropped and recorded as a peinit.notify.rejected event carrying the sender's PID (subject.process.pid), the sender's service, job and activation when peinit could attribute it, and the reason as one of a closed set of words in outcome.reason — unauthenticated-sender, generation-mismatch, malformed-line and so on (§8.4).

The UID and GID in the credentials are parsed and never used. They are not policy inputs, and peinit does not consult them for anything — identity on Peios is a token, and the token here is established by which job the sender is, not by what UID it claims.

10.5.3 Applying a datagram #

A datagram may carry several newline-separated lines, and peinit applies every one. Parsing happens before application and is all-or-nothing: if any line is malformed, nothing from that datagram is applied, and any descriptors it carried are dropped and closed. Partial application of an ambiguous service-control message is structurally impossible rather than merely avoided.

A rejection is recorded after authentication, so the event can name the service where one could be established.

10.5.4 What the fields do #

Most are handled elsewhere: READY=1 and RELOADING=1 in §6.5, WATCHDOG=1 and WATCHDOG_USEC and EXTEND_TIMEOUT_USEC in §6.6, STOPPING=1 in §12.2, and the fd store fields in §10.6. What each does to a submitted job — and which are ignored for one, since a job has no reload, watchdog or store — is in §8.5.

Six are event-emitting. STATUS=, ERRNO= and EXIT_STATUS= are authenticated and then emitted as KMES events — peinit.notify.status.reported, peinit.notify.errno.reported, peinit.notify.exit-status.reported — whose payloads carry the sender as subject.service.name, subject.job.guid, subject.job.activation-generation and subject.operation.guid, alongside the value: notify.status; notify.errno, the number the service sent, negated as every errno field is; notify.exit-status. A value that is not a number is left out and the event still written. They take the same path as job and operation events, not a forward to eventd. peinit.notify.status.reported is a verbose type, written only when the emission policy turns it on (§8.4).

STOPPING=1 emits peinit.notify.stopping.reported, carrying the same attribution and no value. It is there because the field's only effect is the absence of an action — peinit suppresses the SIGTERM (§12.2) — and an absence cannot be inferred from what happened afterwards. Without the event, a service that was stopping and correctly received no SIGTERM looks identical to one that should have received it and did not.

PROGRESS= and PROGRESS_UNIT= emit peinit.notify.progress.reported (a verbose type), carrying the same attribution and the progress as retained after the datagram — notify.progress.current, .total, .bounded and .unit, the fields of a submitted job's peinit.job.status.reported. A datagram carrying either yields one event, not one per line, and at most one a second for each activation: a datagram inside the second updates what a status query reports and emits nothing, so a service updating a thousand times a second produces one event a second.

READY=1 and RELOADING=1 emit nothing, deliberately: both are observable through the state transitions they cause.

STATUS= is additionally stored on the service's runtime state and exposed as status_text in a status query. It is cleared to null at the start of every activation generation, in the same step that increments the generation, so a status string cannot survive a restart and describe a process that no longer exists.

PROGRESS= and PROGRESS_UNIT= are stored the same way and exposed together as progress in a status query, in the form a submitted job's view gives it: {current, total, bounded, unit}, or null until the service sends a PROGRESS=. Each is parsed as for a submitted job (§8.5): a value outside PROGRESS='s three forms, a T of zero, an N above T, or a unit other than bytes, items or percent is dropped, never repaired, and the rest of the datagram is applied. Each is replaced only by a datagram that carries it. Both are cleared with status_text, in the same step.

ERRNO= and EXIT_STATUS= are not stored. They are emitted and otherwise not retained.

10.5.5 Bounds #

A datagram is read into a fixed 64 KiB buffer, and the control message buffer is sized for 64 descriptors. Both MSG_TRUNC and MSG_CTRUNC are inspected on the way in, and a datagram that overran either buffer is refused as truncated rather than acted on in part: a larger datagram is not applied from its surviving prefix, and a send carrying more than sixty-four descriptors is not applied from the ones that fitted.

10.6 The Fd Store

Peios / Peios technical reference / peinit / The Control Interface

The fd store lets a service keep file descriptors across its own restart. It pushes them to peinit, peinit holds them, and the new process gets them back. That is what lets a stateful daemon — a web server holding a listening socket, say — restart without dropping connections it has already accepted.

FdStoreMax in the definition sets the maximum number of descriptors peinit will hold. It defaults to 0, which disables the store : most services do not need it, and holding descriptors on behalf of a service that will never ask for them back is pure cost.

10.6.1 Storing #

When an authenticated datagram carries FDSTORE=1 with descriptors attached:

  1. If FdStoreMax is 0, peinit logs the rejection and closes the descriptor.
  2. If the store already holds FdStoreMax entries, peinit logs the rejection and closes the descriptor. The existing store is not modified — a full store does not evict.
  3. FDNAME=<name> names it; an absent or empty name means stored.
  4. FDPOLL=0 marks the descriptor exempt from poll monitoring.

Either rejection emits a peinit.fd-store.rejected event carrying the service, the descriptor's name and the reason (disabled or full), so a service whose descriptors are being silently dropped can find out why.

Several descriptors may share a name. One FDSTORE=1 carrying N descriptors creates N entries under the one name, each independently subject to the limit — so the first few fit and the overflow is rejected and closed.

peinit does not monitor stored descriptors. The FDPOLL flag is recorded and nothing reads it, so no stored descriptor is evicted for becoming invalid.

10.6.2 Removing #

FDSTOREREMOVE=1 with FDNAME=<name> removes every descriptor of that name and closes them. A name matching nothing is a no-op rather than an error.

FDSTOREREMOVE=1 without a name aborts the whole fd-store step for that datagram — so a datagram carrying both an unnamed remove and an FDSTORE=1 performs neither, and the attached descriptors are dropped and closed.

10.6.3 Injecting #

When a service restarts, peinit injects the stored descriptors during the child's pre-exec path (§5.4):

  1. They are placed consecutively from SD_LISTEN_FDS_START — descriptor 3 — upward, with close-on-exec cleared: the only sanctioned exception to the close-on-exec discipline.
  2. LISTEN_FDS is set to the count.
  3. LISTEN_FDNAMES is set to a colon-separated list of names in the same order as the descriptor numbers.
  4. The store is cleared. peinit no longer holds them.

All three variables are omitted entirely when the store is empty. LISTEN_PID, which a conforming client checks against its own PID before trusting LISTEN_FDS, is appended by the child itself after the clone, since only then is the PID known ; a value for it from either configurable environment layer is discarded so that it cannot shadow the real one (§5.5).

A submitted job receives its attached descriptors by exactly this path (§8.5).

A listening socket that comes back this way still conveys the identity captured when the previous instance called listen() on it. peinit does not touch that : a service that wants connecting clients to see the instance actually accepting restamps the returned listeners itself (peios_socket_restamp, the Peios Kernel TRM §3.5) before it starts accepting.

Injection happens for the main process only. Hooks and health checks never receive stored descriptors.

The store is cleared on a successful injection, at the top of the started-launch handling. A launch that fails does not clear it, so the descriptors survive a failed attempt and are available to the next one.

10.6.4 Clearing #

The store is cleared, and its descriptors closed, when:

  • The service is stopped explicitly — by an administrator, or by shutdown. The distinction peinit draws is the operation's type and source together: an administrator's stop clears, and a restart-policy-sourced stop does not. The service is not coming back from an explicit stop, so the descriptors are no longer useful.
  • The definition is removed and its entry finally discarded (§3.8), immediately if nothing was running and on the instance's exit otherwise.

It survives an automatic restart — crash, restart policy, new start — which is the entire point. The descriptors persist through exactly the restart the service did not choose and cannot prepare for.

Note

The LISTEN_FDS and LISTEN_FDNAMES convention is systemd's, so software already written to receive descriptors that way works unmodified. peinit's own C client library exposes the notification helpers but no descriptor-passing helper, so reaching the store from that library means constructing the control message by hand.

10.7 The Jobs Socket

Peios / Peios technical reference / peinit / The Control Interface

peinit serves job submission on a second Unix socket at /run/services/peinit/jobs.sock, created in Phase 1 infrastructure setup beside the control socket (§2.3) and existing for the lifetime of the system. Its wire protocol is PSPU §7; this article is how peinit implements the transport. What a submission is — identities, the definition, the job's life — is §8.5.

10.7.1 Why a second socket, and why sequenced packets #

The control socket's descriptor admits the principals allowed to change the system's services. Submitting a job is a different permission held by a different population, and putting the two on one socket would force one descriptor to serve both. Two sockets let the filesystem say who may submit and who may administer, separately, with no policy inside peinit.

The socket is SOCK_SEQPACKET rather than a stream because a message on it carries more than bytes: the token the job is to run as, and the descriptors it is to be given, as ancillary data. The kernel ties ancillary data to the record it was sent with, and a sequenced-packet socket makes one send one message and one identity, so peinit never has to decide which request an attached token or descriptor belongs to. The cost is that a message has a maximum size and cannot be split; a definition is small.

10.7.2 Creation and protection #

The listener is created with SOCK_CLOEXEC | SOCK_NONBLOCK and a backlog of 32; a stale path is unlinked before the bind, and connections are accepted with accept4 under the same flags. After binding, and before anything can connect, peinit stamps the socket inode :

O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)

FW is the file-write generic right, which is what a Unix connect() on a pathname socket needs, so every authenticated principal may connect and SYSTEM and Administrators may additionally change the descriptor. Being able to connect is the permission to submit. The kernel checks this descriptor at connect(), and peinit performs no access check of its own before a submit ; an administrator who wants a narrower or wider population changes the descriptor on the socket, not anything in peinit. What a submitter may then do to a job is decided by the job's own descriptor (§8.5), never by this one.

A failure to bind or stamp the socket sends peinit to recovery, as the control socket's does: a system whose jobs door is missing is administrable, but one whose door exists with the wrong descriptor is either unreachable or open to everything, and neither is a state to boot into.

10.7.3 Connections #

The listener is one event source of the runtime loop; each accepted connection is another, keyed by its descriptor. On accept peinit captures the peer's identity once — the peer token, as the control socket does (§10.1), and the peer's process handle through SO_PEERPIDFD — and only then admits the connection against the limit :

KeyDefaultMeaning
Machine\System\Init\MaxJobsConnections64Concurrent connections.
Machine\System\Init\MaxJobMessageSize32768Maximum message content, in bytes. Half of KMES MaxEventSize's default, so a job whose record fills a message still has its peinit.job.ended fit the ring (§8.4).
Machine\System\Init\JobsConnectionTimeout30Seconds before an idle connection is closed.
Machine\System\Init\MaxJobsPerSubmitter64Live jobs one submitter SID may hold; SYSTEM exempt. See §8.5.

A connection over the limit, or one whose peer cannot be identified, is closed at the socket level without a response. The pidfd is held for the life of the connection, because a submit with no token attached opens the job identity through it (§8.5).

A connection carries an identity and nothing else. Closing it does not affect any job submitted on it: a job belongs to its submitter's identity, and a submitter that reconnects finds its jobs where it left them.

10.7.4 Messages #

Every message is received with room for one attached token and for 64 descriptors, the output sink included. One token is all a message can carry: the kernel refuses a second KACS_SCM_TOKEN control message at send time, so the "more than one attached token" refusal in PSPU §7.5 is enforced before peinit ever sees the record. Two truncations are told apart. Content the kernel truncated (MSG_TRUNC) is REQUEST_TOO_LARGE, and that closes the connection, since the transport has lost a record. Ancillary data the kernel could not fit (MSG_CTRUNC) is INVALID_ARGUMENTS, with the connection kept: the content is intact, but a request whose attachments were partly lost does not describe the job the submitter meant, and peinit does not act on it. Every descriptor a message carried that is not handed to a job or adopted as a sink is closed, on every path.

A response is one compact JSON object, no terminator. A submit answered with a running job carries a duplicate of the job's pidfd as SCM_RIGHTS on the response record; nothing else carries ancillary data.

Only REQUEST_TOO_LARGE closes the connection after an error. Every other error is answered and the connection kept.

10.7.5 The turn #

peinit handles one message per readiness turn on a connection, and reads nothing from a connection while a wait is pending on it, so pipelined messages serialise behind a wait. A turn either answers at once, or records a pending wait on the connection state, or does both in the case of a refusal. Three waits exist:

WaitSet byAnswered when
SubmitsubmitThe job leaves created: exec confirmed, or the launch failed.
WaitwaitThe condition holds — terminal, or ready-or-terminal.
Stopstop with waitThe job is terminal.

Waits are flushed after the supervisor's work has been committed each turn, and again whenever a job's state moves — a launch, a reap, a cancelled stop. The flush answers every satisfied wait with the job view at that moment, and for a Submit whose job is running, the pidfd. An answer that cannot be built — a job whose record was purged before the flush, or a pidfd that cannot be duplicated — is that connection's error record, UNKNOWN_JOB or INTERNAL_ERROR; it never aborts the flush of every other wait.

A connection with a pending wait, or with a response still queued, is not idle and is never closed by JobsConnectionTimeout. A wait has no timeout of its own; it is bounded by the job, and a submitter that needs a bounded wait polls status or waits on the pidfd it was given.

10.7.6 Commands #

submit, status, wait, stop and signal, exactly as PSPU §7.8 defines them and §8.5 implements them. Every one but submit names a job and is checked against that job's descriptor with the connection's token; a denial is answered ACCESS_DENIED, and recorded by KACS as kacs.audit.access.checked naming the job when the job's descriptor audits it, as a default descriptor does (§8.4).

10.7.7 Idle and shutdown #

Idle connections are closed before and after each wait of the event loop, and the next idle deadline is folded into the loop's wait timeout. During shutdown the socket stays open: submit is refused with INVALID_STATE, and the other four commands keep answering (§12.2).

10.7.8 Timestamps #

As on the control socket (§10.1): monotonic stamps projected through the current realtime offset at the moment of answering.

10.8 The Boot Query

Peios / Peios technical reference / peinit / The Control Interface

boot tells a client how the machine booted, so that a settings app or an administrator at a shell can see that it is in Safe mode, why, and how many more failed boots stand between it and recovery. The wire shape is PSPU §4.15. This page is where peinit gets each answer.

It takes no arguments and changes nothing. It is checked against the control descriptor for SYSTEM_QUERY_STATUS, which the default grants every authenticated principal (§4.7), and it is answered during shutdown (§10.2).

10.8.1 Mode and reason #

mode is the mode Phase 2 booted in, after any downgrade: full or safe (§2.6). A Recovery boot never reaches the runtime — peinit runs the recovery shell instead (§2.8) — so there is no control socket to ask, and boot never answers recovery.

reason says why.

reasonWhen
normalA Full boot.
requestedA Safe boot asked for with peios.safemode=1.
safe_mode_downgradeA Full boot that Phase 2 downgraded to Safe: a Critical service in a dependency cycle, or in an unresolvable conflict, at boot (§2.6).

For a downgrade, downgrade carries every finding that forced it, in the words the console line and the peinit.boot.downgraded event use — all of them, not the first. peinit keeps them from the boot plan, the one place they survive the Safe-mode rebuild.

10.8.2 Attempts #

attempts is the boot attempt count (§2.7) as Phase 1 checked it against the recovery threshold: the value in /.peinit/boot-attempts before this boot's increment, or 0 if the increment could not be written, which is how Phase 1 treats a counter it cannot advance. It is the count at the start of this boot, so it does not drop to 0 when this boot is confirmed; confirmed says that.

max_attempts is the threshold: peios.bootattempts=N, or 3 when the command line does not set it. 0 means the check is off.

10.8.3 Confirmation #

A boot is confirmed when it has counted as a success: every Critical service has held a dependent-satisfying state for BootSuccessGrace seconds without a break, and peinit has reset the counter to 0 (§2.5). grace_seconds is that grace. The other fields say where the boot stands.

confirmedwaiting_onconfirms_atconfirm_errorMeaning
falsethe Critical services not holdingnullnullThe boot is waiting for them.
falseemptya timenullEvery Critical service holds; if they keep holding, the boot counts then.
trueemptynullnullThe boot has counted, and the counter is 0.
falseemptynullwhyThe grace ended but the counter could not be reset. The next boot will count this one as a failure.

confirmed is true only once the reset has been written. A grace that ends with the write failing — a full disk, say — is not a confirmed boot: the file still holds this boot's increment, and the next boot will count it, whatever its services did. The failed reset is not otherwise reported, so this is where an administrator finds it.

A confirmed boot stays confirmed. A Critical service that fails later is handled by the ordinary Critical failure path — restart budget, then reboot (§2.6) — not by un-counting the boot.

A boot with no Critical services holds from the moment Phase 2 runs, so confirms_at is known at once.

10.9 Control Client Examples

Peios / Peios technical reference / peinit / The Control Interface

These examples connect the svctl operator guide to the control interface. PSPU §4 owns the wire contract; the socket and dispatch mechanisms are in §10.1 and §10.2.

10.9.1 How the control interface works #

The socket speaks newline-delimited JSON: one request object per line, one response object per line. A request and its success response look like this:

{"command": "start", "service": "jellyfin", "wait": true}
{"status": "ok", "operation_id": "a1b2c3d4-...", "service": "jellyfin", "state": "active", "cause": "explicit_start", "warnings": []}

Two properties are worth knowing even if you only ever use svctl:

  • Every command is access-controlled. By default, anyone who is signed in may connect. When you connect, peinit captures your token from the kernel and runs AccessCheck against the target service's descriptor for every command. There is no "trust localhost," no override. Who may do what is the subject of Who can manage a service.
  • Lifecycle commands create operations. A start/stop/restart/reload/reset returns an operation_id — a GUID you can poll. Conflict resolution between concurrent commands happens at the operation layer, which is why two simultaneous starts merge instead of colliding.

10.9.2 Reading status #

status returns the full picture for one service:

{
    "status": "ok",
    "service": "jellyfin",
    "display_name": "Jellyfin",
    "description": "Media server for the living room.",
    "state": "active",
    "cause": "explicit_start",
    "status_text": "Listening on port 8096",
    "progress": null,
    "current_job": {"id": "a1b2...", "type": "service_main", "pid": 1234, "started_at": "...", "identity": "jellyfin-svc"},
    "current_operation": {"id": "e5f6...", "type": "start", "source": "admin"},
    "health": "healthy",
    "uptime_seconds": 86400,
    "definition_removed": false,
    "warnings": [],
    "timers": [],
    "granted": ["query_status", "start", "stop", "interrogate"]
}
  • display_name and description are the definition's, or null.
  • state and cause are the lifecycle pair — read them together.
  • status_text is the latest STATUS= string the service sent via sd_notify (null if never sent; cleared on each restart).
  • progress is how far the service last said it had got, from PROGRESS= and PROGRESS_UNIT=: {"current": 3, "total": 10, "bounded": true, "unit": "items"}, with total and unit null when it didn't say them. null if it never sent one; cleared on each restart.
  • current_job and current_operation are the job and operation GUIDs, or null.
  • health is healthy, unhealthy, unknown, or null (no health check).
  • definition_removed is true when the definition was deleted but an instance is still draining.
  • warnings lists leaked sub-cgroups and other operator-relevant notices.
  • timers has one entry for each timer trigger, described below.
  • granted is what you may do to this service, as peinit checks it: the rights the service's permissions give you, out of query_status, start, stop and interrogate (reload). A program shows a control only where its right is listed, rather than offering one that will be refused.

Each entry in timers describes one schedule as peinit has it armed:

{"schedule": "*-*-* 02:00:00",
 "scheduled_at": "2026-06-02T02:00:00.000000000Z",
 "fires_at": "2026-06-02T02:07:12.000000000Z",
 "last_fired_at": "2026-06-01T02:03:40.000000000Z",
 "not_armed": null}
  • scheduled_at is the schedule's next occurrence.
  • fires_at is when the timer will actually fire. It is later than scheduled_at by the random delay TimerJitter drew for this firing.
  • last_fired_at is when it last fired, or null if it has not fired since peinit started and has no recorded run.
  • not_armed is set only for a schedule peinit refused, such as one that never comes round. It gives the reason, and the times are then null.

Times are in UTC. svctl status prints them to the second:

$ svctl status logrotate
logrotate: inactive
cause: clean_exit
timers:
  *-*-* 02:00:00
    next: 2026-06-02T02:00:00Z, firing at 2026-06-02T02:07:12Z after jitter
    last fired: 2026-06-01T02:03:40Z

list returns a compact summary of every service you can query — services you lack SERVICE_QUERY_STATUS on are simply omitted, not denied:

{"status": "ok", "services": [
    {"service": "jellyfin", "state": "active", "cause": "explicit_start", "health": "healthy", "next_timer_at": null},
    {"service": "logrotate", "state": "inactive", "cause": "clean_exit", "health": null, "next_timer_at": "2026-06-02T02:07:12.000000000Z"}
]}

next_timer_at is the soonest fires_at of the service's timers. svctl list shows it in a NEXT TIMER column.

operation-status returns one operation by GUID; an unknown or expired GUID is the UNKNOWN_OPERATION error. (Operations are dropped after a short retention grace once terminal — long enough for a polling client to read the result, not forever.)

job status returns one submitted job's view under "job", and job list returns the views you can query under "jobs":

{"status": "ok", "job": {
    "id": "5f2a...", "type": "submitted", "state": "running", "cause": null,
    "submitter": "S-1-5-21-...-1001", "identity": "S-1-5-21-...-1001", "logon_session": 999,
    "description": "nightly backup", "image_path": "/usr/bin/backup", "pid": 4521, "ready": null,
    "exit_code": null, "exit_signal": null,
    "status_text": "Backing up /data/media", "progress": {"current": 3, "total": 5, "bounded": true, "unit": "items"},
    "created_at": "...", "started_at": "...", "ended_at": null
}}

The fields are explained in Jobs and operations. A job is retained for 60 seconds after it ends; after that its GUID is the UNKNOWN_JOB error. Like list, job list omits the jobs you lack JOB_QUERY on rather than denying them.

11.1 Wiring

Peios / Peios technical reference / peinit / Output Handling

peinit is not a logging system — eventd stores, indexes and queries logs. But peinit holds the pipes at birth. It decides where a service's output goes, and it has to cover the window before eventd exists.

11.1.1 The pipes #

Before exec, peinit creates a pipe for stdout and one for stderr with pipe2(O_CLOEXEC). The child's streams are redirected onto the write ends; peinit keeps the read ends and watches them with epoll.

The blocking discipline is asymmetric, deliberately :

  • The parent's read ends are non-blocking. PID 1 cannot afford to block on a read.
  • The child's write ends keep ordinary blocking semantics.

That second half is what makes backpressure work. When a service produces faster than peinit consumes, the kernel pipe buffer fills and the service's own write() blocks, so the service slows down. Making the write ends non-blocking would turn that into EAGAIN failures in the service, converting a flow-control mechanism into an error the service has to handle.

The child's stdin is redirected to /dev/null. peinit provides no interactive input channel; a service needing input obtains it explicitly — through a socket, or a stored descriptor — never through inherited stdin.

The epoll instance itself is close-on-exec and is never inherited.

11.1.2 Tagging #

peinit reads output line by line and tags each line with:

  • the origin,
  • the stream — stdout or stderr,
  • a CLOCK_REALTIME timestamp,
  • the job's identifier.

The origin is the service name for a main process. For a hook it is the service name, the hook kind and its index — jellyfin/ExecStartPre[0]. Reload commands are <service>/ExecReload and health checks <service>/HealthCheck. A submitted job, which has no service, is jobs/<guid>.

Health check output is captured, which is usually the only way to find out why a health check failed.

11.1.3 The output sink #

A submitted job's output is recorded like any other's; that is unconditional, and eventd is always the primary destination. A submitter may additionally ask for a copy by attaching a descriptor with output: true (§8.5). When the job's pipes are registered, the runtime adopts that descriptor as the job's sink: it is put in non-blocking mode, and every line read from either pipe is written to it as the line followed by a newline, untagged, in the order the lines were read. The two streams interleave; the sink gets what the job wrote and nothing else.

The copy is best-effort and the record is not. A write that would block drops the line for the sink only, counts it, and on the first drop for that job emits one peinit.job.output.dropped event — one per job, however many lines follow — so a submitter that sees a gap can learn it caused one. Any other write failure closes the sink at once. The sink is otherwise closed when the job's last pipe closes, after the last line has gone to it, and a sink attached to a job that never got pipes is closed without a write.

11.1.4 Terminal-attached services #

A service with a TTYPath has all three streams on its terminal, and both pipe pairs are closed in the child (§5.4). Its output is not captured at all — it goes to the terminal, which is what asking for one means.

peinit opens the terminal with read data, write data and read attributes. The last matters because the descriptor is inherited by every process the service starts, and a KACS handle answers fstat only if that right is on it: without it a session can read and write its terminal but not name it — ttyname() fails, tty reports "not a tty", and a program choosing TERM cannot tell a serial line from a virtual console. isatty() is an ioctl and works either way, which is why the gap is easy to miss.

11.2 The Pre-Eventd Buffer

Peios / Peios technical reference / peinit / Output Handling

Before eventd starts there is nowhere to send service output: eventd's log socket does not exist until eventd binds it. peinit buffers in memory until then. The buffer is bounded, and drops the oldest entries when it fills.

KeyDefaultMinimumMeaning
Machine\System\Init\PreEventdBuffer10485764096Bytes of output retained before eventd exists.

A value below the minimum keeps the default and logs a warning, rather than being honoured or failing the boot (§11.3). Zero is the case that matters: a zero-capacity buffer rejects every record, so honouring it would silently discard the whole pre-eventd window.

The value is read from the registry at boot and refreshed on reload, and the buffer adopts the new capacity each time. Lowering it takes effect immediately, dropping the oldest entries until the contents fit — the same end of the buffer a steady-state overrun drops.

Dropping the oldest rather than the newest is the right way round for this buffer: the point of the window is the boot that is happening now, and the most recent output is what explains where it got to.

Note

In practice the only services that run before eventd are registryd, lpsd, authd and eudev. The first three are Peios-owned with controlled output. eudev is the real overrun risk, being verbose about device enumeration. A crash loop's output is bounded by the restart budget.

11.2.1 Audit records are not logs #

peinit's own audit records — access denials, critical failures, recovery mode entry, graph errors, security-relevant transitions — are events rather than logs. They go into the KMES kernel ring buffer, which persists from the moment PKM loads: before eventd, before the registry, before Phase 2.

So there is no pre-eventd buffer for them, and no window in which they could be lost. eventd picks them up from the ring buffer when it attaches, wherever in the boot they were emitted.

That distinction is the reason the two paths exist at all. Logs are best-effort and lossy by design; audit events are neither.

11.3 Flood Protection

Peios / Peios technical reference / peinit / Output Handling

A noisy service cannot be allowed to starve the event loop. Three bounds apply.

KeyDefaultMinimumMeaning
Machine\System\Init\MaxLogLineLength8192256Bytes per line before truncation.
Machine\System\Init\MaxLogBufferPerService655364096Bytes buffered per service pipe before backpressure.
Machine\System\Init\LogReadBytesPerEvent16384512Bytes drained from one pipe per readable event.

A value below the minimum is not honoured and does not fail the boot: the compiled-in default is used and a warning is logged naming the key, the configured value and what was used instead. The same rule covers PreEventdBuffer (§11.2), and it is the rule peinit already applies to the equivalent kernel command-line knobs — a typo in a logging knob must not decide how the machine boots, and must not be silent either.

MaxLogBufferPerService's minimum is the one with an external cause: F_SETPIPE_SZ will not go below one page and rounds up regardless, so a smaller number in the registry would describe a pipe that does not exist. The other minimums are engineering judgement — the point below which the mechanism stops working rather than working differently.

A line exceeding MaxLogLineLength is truncated and marked [truncated], with the content trimmed so that content and marker together come to exactly the limit. Everything up to the next newline is then suppressed rather than emitted as a second line.

MaxLogBufferPerService is applied as the pipe's own capacity through F_SETPIPE_SZ , which is a literal reading of "bytes buffered per service pipe before backpressure" — the kernel does the buffering and the bound is where it belongs.

LogReadBytesPerEvent bounds one readable event rather than one loop iteration. A turn with several ready pipes reads up to the budget from each, bounded overall by how many events one epoll wait returns.

11.3.1 Where loss is allowed and where it is not #

At the pipe stage, peinit does not drop. It keeps reading within its budget and appends every complete line; only EAGAIN, end of file, or an error stops the read. Backpressure through the pipe is the flow-control mechanism between a service and peinit, and dropping there would replace it with silence.

Downstream of the pipe, delivery is loss-tolerant by design. The pre-eventd buffer drops its oldest entries when full, and eventd's datagram socket may drop records under load. The no-silent-drop guarantee covers reading the pipe, not delivery to eventd.

Audit events are exempt from all of it. They go through KMES.

11.3.2 Event loop fairness #

No source may starve the loop. Signals are handled at the highest priority — SIGCHLD reaping and shutdown handling take precedence over everything else in every iteration — with the shutdown deadline timer next and every other source below that, ties broken by arrival order.

The power button shares the top priority with signals , on the reasoning that someone physically pressing it is asking for the same class of attention.

11.4 The eventd Handoff

Peios / Peios technical reference / peinit / Output Handling

When eventd reaches Active, peinit switches from buffering to forwarding.

  1. Start sending to eventd's log datagram socket, at the path from Machine\System\eventd\LogSocketPath.
  2. Replay the pre-eventd buffer, oldest first, preserving each line's timestamp and metadata. The replay is best-effort: these are datagrams on a loss-tolerant socket, so some may be dropped, and peinit does not block waiting to deliver them.
  3. Switch to real-time forwarding — new output sent as it arrives.
  4. Clear the buffer.

From there peinit is a relay: read from the pipes, tag each line, forward. Audit events continue to flow as KMES events, entirely separately.

11.4.1 Broker authentication #

eventd's log socket denies FILE_WRITE_DATA to the Service logon group before allowing SYSTEM. peinit's bootstrap token is SYSTEM without that group; every phase-2 service token carries it, including a token whose user is SYSTEM. peinit is therefore the only service-output broker.

peinit does not attach a KACS token per log datagram. The socket descriptor authenticates the broker once at send admission, and peinit derives each origin from the pipe it is draining. Avoiding ancillary token fds also preserves mixed-origin batching and the hot-path reuse described below.

11.4.2 Datagram framing and the record #

peinit sends a msgpack array of one or more records in each datagram. It takes the largest ordered prefix that fits the smaller of the PSPU portable ceiling of 262144 encoded bytes and the log socket's send buffer. It deliberately does not read eventd's larger local ceiling: fixing the producer boundary avoids registry work and means an eventd configuration change cannot invalidate peinit's batches. peinit asks for a send buffer large enough for the portable ceiling when it connects, but the kernel caps what it grants, so the socket's own limit is the one that finally bounds a batch. A successful datagram advances the replay buffer by the whole array, while a failed datagram advances it by nothing. The one exception is a batch the socket refuses as too large: a single record that cannot fit one datagram is discarded rather than replayed forever, and peinit announces the discard on the console.

Each record is a msgpack map :

KeyTypeContent
originstringThe service name, or the hook identifier.
is_errorboolTrue for stderr.
messagestringThe line.
timestampuintNanoseconds, wall clock.
job_idbinThe job's 16-byte GUID in PCDS binary layout. Omitted when absent.

The map has four or five entries depending on whether a job identifier applies.

PSPU §3.7 requires PCDS GUID layout: Data1, Data2 and Data3 are little-endian; the final eight bytes (Data4) retain their byte order. The payload is still exactly 16 bytes.

Note

peinit 0.0.12 source sends RFC-order UUID bytes instead. The PCDS layout above requires the log job-ID encoding fix; check the installed peinit revision before relying on that encoding.

11.4.3 Lossy delivery #

eventd's log socket is a non-blocking Unix datagram socket. If eventd cannot drain it fast enough its SO_RCVBUF fills and the kernel drops further datagrams silently — log ingestion deliberately exerts no backpressure on senders.

peinit therefore keeps no unbounded outbound write buffer. It sends bounded arrays of records and accepts that some datagrams may be dropped. It never blocks on a send, and never lets pending records grow without bound.

peinit creates one non-blocking datagram socket when forwarding begins, connects it to eventd, and reuses both that socket and its encoding allocation. It does not create a socket or allocate a fresh output buffer per record. When eventd becomes inactive, the connection is discarded so the next Active transition connects to the newly bound socket.

A drop and a transport failure are handled differently, and the difference is what keeps the lossy design working.

A drop is EAGAIN, EWOULDBLOCK or ENOBUFS — the receive buffer is full and the datagram did not fit. That is the designed outcome under load, so the connection stands and forwarding continues. Live records in the dropped datagram are gone; peinit counts them and sends the next batch. Replaying the pre-eventd buffer is the one case where waiting beats dropping — those records are already held and the buffer is bounded, so the flush stops and the next turn tries again.

A transport failure is anything else: the socket path gone, the peer refusing. peinit clears the connection, keeps the unsent records in the pre-eventd buffer in order, and the end of the turn re-establishes forwarding by replaying them.

Treating a drop as a transport failure made peinit oscillate between forwarding and buffering under exactly the load the lossy socket exists to absorb, and replay records eventd may already have held — so the busier eventd got, the more work peinit made for it.

11.4.4 When eventd goes away #

peinit supervises eventd like any other service, so it sees the exit directly. It re-enables the pre-eventd buffer, and when eventd restarts and reaches Active the handoff repeats.

There is a log gap between eventd crashing and restarting, bounded by the buffer size. Events are unaffected: they land in the KMES ring buffer regardless of eventd's state, and eventd resumes consuming from the last persisted sequence when it comes back.

11.5 Console Output

Peios / Peios technical reference / peinit / Output Handling

peinit writes its own operational messages to /dev/console :

  • Phase 1 progress — mount results, registryd starting.
  • Phase 2 progress — services starting and failing, dependency errors.
  • Shutdown progress.
  • Recovery mode entry.
  • Critical service failures.

Service output is never echoed to the console. The console is for peinit's own messages; a service that wants a terminal asks for one with TTYPath.

11.5.1 The line format #

Every line carries a fixed-width outcome tag, then the component name, then the message:

[      ] peinit: phase1 starting
[  OK  ] peinit: service authd started
[ SKIP ] peinit: service resolvd skipped: DependencyFailure
[ WARN ] peinit warning: calendar timer not armed: Malformed
[FAILED] peinit: service netd failed to launch: token materialisation failed
[ CRIT ] peinit: entering recovery: BootAttempts(3)

The tag column is the point of the format. An operator scanning a boot reads down one column rather than reading sentences, and a failure stops looking exactly like the twenty lines around it.

TagMeans
[ ]Progress with no outcome yet. Holds the column while receding.
[ OK ]It worked.
[ SKIP ]Deliberately not done.
[ WARN ]Wrong, but the boot continues.
[FAILED]It did not work.
[ CRIT ]The machine is about to be lost.

An [ OK ] line reads "mounted", never "mounting": the tag already claims the outcome, so the verb has to agree with it.

11.5.1.1 Tag and severity are different axes #

The tag is presentation. Severity decides who is allowed to print. They correlate but neither determines the other — a Normal service failing and a Critical service failing both render [FAILED] while carrying different severities — and collapsing them would quietly change what peios.quiet=2 suppresses.

11.5.1.2 Colour #

Tags are coloured with ANSI SGR escapes: green for OK, yellow for SKIP and WARN, red for FAILED, and white-on-red for CRIT. Untagged progress is never coloured.

Colour is decided once, from the kernel command line, following the same rule systemd uses: on unless the command line says TERM=dumb. peinit is PID 1 and has no inherited TERM to consult, so the command line is the only place an operator can say a console cannot render escapes. Deliberately not a peios.* token — that set is kept small on purpose, and TERM=dumb is a spelling people already know.

Colour changes the bytes and never the layout, so a serial log and a virtual terminal agree about where the message starts.

11.5.1.3 The stage banner #

Before Phase 1 does any work, peinit prints a banner naming the stage and the boot mode:

  ══════════════════════════════════════════════════════════════
   peinit · real root · PID 1 · Full boot                 v0.0.1
  ══════════════════════════════════════════════════════════════

It is punctuation rather than decoration. prelude prints the matching one when the initramfs takes over, so the two mark the handover from kernel to initramfs to real root — the one thing a boot log otherwise never says out loud.

The mode is the one this boot starts in: Full boot, Safe mode, or RECOVERY MODE. A later downgrade to Safe is announced by its own message rather than by reprinting a banner, because a second banner would read as a second stage.

Naming the mode is why the kernel command line is read at the very top of run_init, before any Phase 1 work. That reverses an earlier ordering which assumed mounting the virtual filesystems is what makes /proc/cmdline readable. It is not: the mount step reads /proc/self/mountinfo first, and /proc is provided by the initramfs. One consequence — a machine with both an unreadable command line and another Phase 1 fault now reports the command line as its recovery reason.

The banner carries ordinary status severity, so peios.quiet=2 drops it along with every other kind of progress.

11.5.1.4 Who else writes to this console #

The format is shared by specification, not by shared code. peinit, prelude and the hook scripts each implement it separately, because prelude is a size-critical initramfs PID 1 with no dependencies and the hooks are shell. Changing a tag word or a width means changing it in all three.

Everything peinit runs that reaches the console is in this format, including output peinit did not write itself. Autorun scripts are captured, not left to inherit peinit's streams, and their lines are relayed under the script's own file name :

[      ] 10-apply-seeds.sh: applied 20 file(s), 92 key(s); 0 failed
[  OK  ] peinit: ran 1 autorun script(s)

Relayed lines always carry the blank tag. peinit does not know whether a given line of somebody else's output is good news — the producer knows and peinit does not, and guessing from prose is how a rename silently turns an error green. The producer's outcome is reported separately, from its exit code, which peinit does know.

Relayed lines are sanitised first. A script sharing the console could otherwise steer it with an escape sequence, and peinit's own output shares the device. Three rules apply:

  • Control characters become ?.
  • A line is cut at 512 characters.
  • A producer is capped at 200 lines, with the remainder counted.

The only lines on a normal boot that are not in this format come from outside peinit's reach entirely: the firmware's own messages before any kernel runs, and a service that owns the terminal through TTYPath writing to it directly.

11.5.2 Severity and quiet #

Each message carries a severity, and peios.quiet (§2.6) decides what that means:

  • At 0, everything is written.
  • At 1, the default, peinit stays out of a terminal held as the controlling terminal of a running service, except to announce loss of the system. Terminals are matched by device rather than by path, since /dev/console and /dev/ttyS<n> can be the same device; where the device cannot be determined peinit assumes the terminal is held.
  • At 2, ordinary progress is dropped everywhere while errors still get through.

Suppressed messages are discarded rather than buffered for later.

Shutdown progress carries ordinary status severity, so peios.quiet=2 suppresses it along with every other kind of progress.

The autorun step in Phase 1 (§2.3) bypasses the policy entirely, on the grounds that a script running that early and going wrong is worth interrupting anything for. Its lines are still tagged.

11.5.3 Phase 1 registryd failures #

One deliberate exception to "service output is never echoed to the console". peinit gives every service capture pipes and drains them in the runtime loop — but Phase 1 has no loop, so a registryd that printed exactly why it could not serve was reported to the operator as nothing but registryd readiness timeout expired before READY=1, with the reason sitting unread in a pipe.

When Phase 1 registryd fails to become ready, or says READY=1 and then cannot serve, peinit now drains those pipes and relays what it found :

[FAILED] peinit: registryd failed; what it said follows
[      ] registryd(stderr): open hive Machine: permission denied

Failure path only. On a normal boot the descriptors are retained for the runtime, which drains them into the pre-eventd buffer and on to eventd like any other service's output — so registryd's startup lines are queryable with evctl and are not on the console.

The drain is safe to run from PID 1 because the read ends are non-blocking: a registryd that is alive and silent yields EAGAIN rather than hanging the boot.

Note

peios.quiet governs peinit only. What the kernel prints is governed separately by loglevel, which shipped images set to 4. See Boot and boot modes.

11.6 Terminal Ownership

Peios / Peios technical reference / peinit / Output Handling

A TTYPath (§3.2) names a device that has exactly one owner at a time. Two services writing to the same terminal do not fail; they interleave. On a console that produces a login prompt with a progress line through the middle of it, and a password echoed into whatever redrew last. So peinit hands the device to one service and makes the others wait.

11.6.1 Who is holding it #

A service holds its terminal in every state where it may still have a process on the device:

HoldingReleased
Starting, Active, Reloading, Stopping, BackoffInactive, Completed, Failed, Skipped, Abandoned

The two emphasised entries are the ones worth stating outright.

Backoff holds. A service between restart attempts is coming back, and handing its console to somebody else during the gap produces two owners a second later. This is deliberately not the "settled" rule used by boot:settled (§2.5), which also treats Backoff as still moving, but reaches it from the opposite direction.

Abandoned releases , and that one is a compromise. The process survived SIGKILL (§5.7) and may well still be writing. But peinit has no remaining way to reclaim the device from it, so treating it as held would mean the terminal is lost for the rest of the boot — and a console nobody can log in on is worse than a console with a stuck process on it.

Nothing is stored. The holder is derived from the service table, so there is no claim to leak, to double-release, or to fall out of step with the state machine.

11.6.2 Losing the terminal #

Every start path — boot, deferred, on-demand, restart, and the completion leg after a filesystem check helper (§5.2) — asks whether the terminal is free before it asks anything else. If another service holds it, the start ends there: the service goes to Skipped with cause TtyUnavailable (§6.3), its operation completes, and its dependents are satisfied exactly as any other skip satisfies them.

Skipped rather than Failed, because nothing is wrong. Nothing is written to the console about it either — it is the mechanism working, and the message would land on the very terminal whose new owner is at that moment drawing on it.

The check runs ahead of the service's own conditions and asserts, which is why a service queued on a busy terminal never forks a check helper.

The graph paths — the Phase 2 boot plan, and any start that brings dependencies with it — ask twice. Their pre-start check runs for every service released together before any of them has moved, so two boot-plan services naming one terminal both pass it. The question is then asked again at the moment each one would enter Starting, and by then the first holds the device: the second is skipped there, with the same cause and the same completed operation, rather than launched into a terminal that already has a session.

A relaunch from Backoff asks too, and a service whose terminal was taken while it waited goes from Backoff to Skipped (§6.2).

11.6.3 Winning the terminal #

TTYPrecedence decides who wins when several services want the same device at the same moment. Higher wins; equal precedence breaks on service name , so an operator who states no preference still gets the same machine on every boot rather than whichever definition the registry happened to enumerate first. The default is 0.

Precedence is consulted among simultaneous candidates only. A service that already holds a terminal is never preempted by a higher-precedence latecomer : taking a live terminal away from a process that is using it would lose whatever the operator was in the middle of typing.

Two places have simultaneous candidates. Deferred starts (§2.5) are dispatched highest-precedence first, so the winner takes the device and the rest are skipped. Handovers, below, pick one winner outright.

Within the Phase 2 boot plan the order is the dependency graph's, not precedence's — a boot-plan terminal service is being written over by peinit's own progress output anyway, which is what boot:settled exists to avoid.

11.6.4 Handing it on #

A service that names a terminal and carries the tty:released trigger (§3.4) is started when that terminal comes free. peinit watches every transition out of a holding state, so it does not matter whether the holder exited cleanly, crashed, was stopped by an administrator or was evicted by a conflict — all four free the device , and the waiter is wanted after a crash more than after a clean exit, not less.

Which terminal was released is recorded on the transition, not looked up afterwards, and that is load-bearing rather than tidy. A service whose definition has been removed loses its table entry the moment it reaches a state that does not retain one (§3.8) — the same moment it releases its terminal. Asking afterwards which TTYPath it had gets nothing.

That is not a corner case. It is precisely what a service that retires itself does: remove its own definition so it never runs again, then exit. First-boot setup is exactly that, and before the release rode on the transition it ended by telling the operator they could log in, on a console it had just vacated and nobody had been given.

One winner per release, chosen by precedence. Starting the whole queue would have every loser immediately skipped again by the rule above: the same outcome, reached noisily, with a skipped-looking service for each.

Two services are never offered a terminal:

  • The one whose exit freed it. It has the strongest claim on its own console, so without this exclusion the highest-precedence holder restarts itself on every exit — a restart policy with no budget, and one no waiter behind it ever gets past. Relaunching a service that stopped is RestartPolicy's job (§6.4).
  • One whose definition has been removed (§3.8), since it cannot be started at all.

A waiter is usually sitting in Skipped when its turn comes, from the moment it lost the terminal. The handover starts it as an explicit start would, which is the transition that takes a service out of Skipped (§6.2) and re-evaluates its conditions.

11.6.5 Why not Conflicts, or a dependency #

Conflicts (§7.1) fails both services when they meet in one boot. The services that want a terminal are the interactive ones — a login prompt, an installer, a first-boot setup flow — so that answer is a machine with no way into it.

A dependency says the wrong thing twice. A login prompt does not need first-boot setup to have run; it needs the console to be free, and a setup flow that failed frees it just as well as one that succeeded. And naming the holder would mean rewriting every waiter each time the set of things that might take the console changed.

What is left is a queue on a device, which is what this is.

11.6.6 Fixed system-console endpoints #

When the LCS boundary reads service definitions, /dev/console in TTYPath is resolved from the last entry in /sys/class/tty/console/active. A tty0 endpoint is pinned to /dev/tty1; an explicit /dev/tty0 is also pinned to /dev/tty1. The selected system-console endpoint is cached for the boot so registry reloads and foreground VT changes do not move existing sessions. Resolution failure makes the affected definition undecodable, rather than silently opening a moving alias. Arbitration and process launch use the same resolved path.

On terminal main-process exit the runtime kills the exiting job's cgroup before releasing the terminal or scheduling its successor. Descendants belong to the session that ended, not the next login. Other terminal services and ordinary daemon lifecycle semantics are unaffected. An error killing the cgroup fails terminal-event processing; a new prompt must not be scheduled through that error.

11.7 Service Output Flow

Peios / Peios technical reference / peinit / Output Handling

These notes collect the output-flow explanation behind finding service logs. The preceding sections define the precise transport and limits. Service-output delivery is best-effort; KMES events use a separate finite in-memory ring, and eventd supplies persistence.

11.7.1 Capturing output #

When peinit forks a service it creates pipes for stdout and stderr, redirects the child's streams to the write ends, and keeps the read ends — monitoring them in its event loop. (The child's stdin is /dev/null; see The execution environment.) It reads output line by line and tags each line with:

  • the service name,
  • the stream (stdout or stderr),
  • a timestamp,
  • the job GUID.

The job GUID is the important one: it is what lets a later query return exactly one execution's output, never interleaved with the run before or after it. Hook output is captured too, labelled with the hook (e.g. jellyfin/ExecStartPre[0]), and so is health-check output — invaluable for diagnosing why a check failed. A submitted job's output is captured the same way, labelled jobs/<id>; its submitter cannot opt out of that, but can ask for a copy by attaching an output sink, to which peinit writes each line as it reads it. The copy is best-effort — lines the sink cannot take are dropped for the sink only, while forwarding to eventd continues independently, and reported once per job as an output.dropped event — and the sink never slows the job or peinit.

11.7.2 The pre-eventd buffer #

Before eventd is running there is nowhere to send logs — its socket does not exist yet. peinit therefore buffers captured output in a fixed-size in-memory pre-eventd buffer (default 1 MB); when it fills, the oldest entries are dropped.

In practice the only services that run before eventd are registryd, lpsd, authd, and eudev. The first three are Peios-owned with controlled output; eudev is the real overrun risk (verbose device enumeration), and the restart budget naturally bounds output from a crash loop.

11.7.3 Flood protection #

A noisy service must never starve peinit's event loop. Several limits enforce that:

Key (Machine\System\Init\)DefaultLimits
MaxLogLineLength8192Bytes per line; longer lines are truncated with a [truncated] marker.
MaxLogBufferPerService65536Bytes buffered per service pipe before back-pressure.

Two mechanisms back these up:

  • A per-readable-event budget. LogReadBytesPerEvent bounds the bytes drained from one pipe per readable event. A loop turn with several ready pipes may read up to that budget from each; see §11.3.
  • Back-pressure, not dropping. If a service writes faster than peinit reads, the kernel pipe buffer fills and the service's own write() calls block. This is deliberate: the service slows down. While reading the pipe, peinit does not drop output — the no-silent-drop guarantee covers reading the pipe. It is only downstream (the pre-eventd buffer, and eventd's datagram socket) that delivery becomes lossy.

Note

Signals are never starved. In every loop iteration, child reaping (SIGCHLD) and shutdown handling take priority over all other event sources, including log reading. A flood of service output can never delay peinit noticing that a service died or that the system is shutting down.

11.7.4 The eventd handoff #

When eventd reaches Active, peinit switches from buffering to forwarding:

  1. It begins sending to eventd's log datagram socket (path from Machine\System\eventd\LogSocketPath).
  2. It replays the pre-eventd buffer, oldest first, preserving each line's timestamp and metadata. This replay is best-effort — the records are datagrams on a loss-tolerant socket and some may be dropped under load; peinit never blocks waiting to deliver them.
  3. It switches to real-time forwarding — new output goes out as it arrives.
  4. It clears the buffer.

From then on peinit is a pipe relay: read lines, tag them, and forward the largest ordered batch that fits the smaller of the portable 262144-byte ceiling and the log socket’s send buffer. peinit does not read eventd’s larger local ceiling; see §11.4. It keeps one connected socket and reuses its encoding buffer, so normal forwarding does not create a socket or output allocation for every line. The log socket is non-blocking and loss-tolerant — if eventd cannot drain it fast enough the kernel drops further datagrams silently, because log ingestion must never exert back-pressure on the whole system through PID 1. peinit keeps no unbounded outbound buffer and never blocks on a send to eventd.

If eventd itself crashes after starting, peinit (which supervises it like any service) notices, re-enables the pre-eventd buffer, and repeats the handoff when eventd comes back. There can be a log gap across the restart, bounded by the output buffer. Structured events use the independent KMES ring; eventd can resume reading the events that remain there. The ring can overwrite old events and does not survive reboot, so eventd supplies persistence rather than KMES. See KMES failure modes.

12.1 Triggers

Peios / Peios technical reference / peinit / Shutdown

Four paths initiate a shutdown.

12.1.1 The control socket #

A shutdown command naming a type, gated on SYSTEM_SHUTDOWN against peinit's control descriptor (§4.7):

TypeEffect
poweroffStop everything, unmount, power off.
rebootStop everything, unmount, reboot.
haltStop everything, unmount, halt — the CPU stops, the system stays powered.

12.1.2 Signals #

SignalMeaning
SIGINTReboot. The kernel sends it on Ctrl+Alt+Del.
SIGTERMPoweroff. PID 1 cannot be killed by it but may choose to act on it.
SIGPWRPoweroff. The compatibility path for environments that surface power failure or a power-button policy as a signal.

Three SIGINTs within five seconds force an immediate shutdown: no graceful stop, no ordering, SIGKILL every service cgroup, sync, reboot. The window is a sliding five seconds and the press is recorded before the already-shutting-down check, so three presses still force even after a graceful reboot has begun. That is the point — someone pressing it three times has decided the graceful path is not working.

12.1.3 The power button #

An EV_KEY / KEY_POWER press from a readable /dev/input/event* device is a graceful poweroff. Only a press — value 1 — initiates. Releases, key repeats, other keys and other event types are ignored.

The path is fail-soft throughout: a missing /dev/input, a device that cannot be opened or registered, and a registered descriptor that later fails to read are all survivable, and a failing descriptor is removed from the event loop so repeated failures cannot spin PID 1. Losing it degrades only direct power-button handling; the socket and signal paths remain.

It is deliberately minimal. It is not a power-management policy engine and does not replace a future daemon that would translate richer policy into control socket commands.

12.1.4 Critical service failure #

A Critical service entering Failed with its restart budget exhausted means peinit syncs the filesystems and reboots immediately. This is not a graceful shutdown: there is no stop ordering, no seed save, and no unmount. The system is in an undefined state and the fastest path to a defined one is a reboot.

12.2 The Graceful Sequence

Peios / Peios technical reference / peinit / Shutdown

12.2.1 Step 1: Enter the shutdown state #

peinit sets an internal flag. While it is set, no new service starts. Control commands other than status, list, operation-status, job-status, job-list and job-stop are rejected as invalid for the current state.

Every live submitted job (§8.5) is stopped at this point too: SIGTERM to each — unless it has already sent STOPPING=1 — with cause shutdown, its own stop_timeout to the kill, and no ordering between jobs or against the service waves, because a job has no dependencies to order by. A job still queued for launch is cancelled with the same cause. submit is refused for the rest of the shutdown; status, wait, stop and signal keep answering, as do the control socket's job-status, job-list and job-stop.

Timer triggers are disarmed. The timerfds stay armed and keep firing — peinit gates the handler rather than the fd, matching how every other event source is handled — but a firing during the shutdown window is classified as a no-op and nothing is started or queued.

The gate matters more than the rule reads. The states a timer firing would start from are Inactive, Completed and Failed, precisely the states step 3 classifies as not participating, so a firing would create a service after the plan was fixed: in no wave, not waited for, and not reached by the global-timeout sweep, still holding files open when peinit reaches the unmount step.

12.2.2 Step 2: Suspend Critical failure semantics #

A Critical service failing during shutdown is recorded but does not trigger a reboot. The system is already going down, and rebooting from here would loop.

12.2.3 Step 3: Classify #

Completed services — Oneshots with RemainAfterExit — have no process and are transitioned to Inactive, releasing the dependency relationships they were holding so their dependents can be stopped cleanly.

The rest are classified for stop eligibility:

  • Active and Reloading are graceful-stop eligible and enter the waves.
  • Stopping services are already on a stop path. They join the waves for ordering and timeout purposes, but do not receive another SIGTERM.
  • Starting services are not eligible. peinit cancels the startup, SIGKILLs the service cgroup if one exists, and transitions them to Failed with cause ShutdownWave while post-kill verification is pending. A cgroup still populated after the post-kill timeout takes the service to Abandoned with cause ProcessUnkillable and the cgroup is leaked. A Starting service whose job never forked skips the check and goes straight to Failed.
  • Inactive, Failed, Skipped, Backoff and Abandoned do not participate. Abandoned cgroups stay leaked and shutdown continues.

12.2.4 Step 4: Stop in reverse dependency order #

peinit builds the reverse dependency graph over hard dependencies and stops in waves:

  1. Each eligible service receives SIGTERM. Already-stopping ones do not.
  2. Each has StopTimeout to exit.
  3. On expiry, SIGKILL to the service's entire cgroup.
  4. No service is stopped until everything depending on it has stopped.

The compiled-in Phase 1 service, registryd, stops in a wave of its own after every other participant, though no definition declares a dependency on it. Every service defined in the registry rests on it, as it started before any of them; stopped by the graph alone it would land in the first wave and go while the services still running needed it. The rule follows the definition's compiled-in provenance, not its name.

A participant the service table no longer has counts as stopped. A service whose definition is withdrawn while it runs is kept until it stops and then discarded (first-boot setup withdraws its own service this way), and the plan fixed at step 3 still names it. Waiting for it would hold its wave open for good.

A service that sent STOPPING=1 does not receive a SIGTERM at all: it has already said it is shutting down, and peinit goes straight to the stop deadline. The notification is acknowledged as a peinit.notify.stopping.reported event (§10.5), which is what makes the suppressed SIGTERM legible afterwards rather than looking like one peinit failed to send.

12.2.4.1 Observing the waves #

Every stop-wave participant receives its own Shutdown-sourced Stop record when the shutdown plan is fixed. Its requested event records the intent; its started event records when the participant's wave becomes eligible, including when an existing stop is adopted without another SIGTERM. The records and their identifiers are visible through the ordinary operation queries and peinit.operation.* events.

A service that exits cleanly before its wave completes its Pending observation without a started event. A crash before the wave fails it. For a service already stopping, reaching Inactive completes the observation; a stop timeout, global timeout, or forced reboot fails it. Post-kill abandonment does not produce a second terminal result for an observation already failed at the kill. Starting services retain their existing startup-failure records, and Completed services cleared without a process do not receive stop observations.

An administrator Stop or Restart already in flight keeps its identity and its original timing evidence. The shutdown observation is separate; it neither supersedes that record nor queues a second control command.

12.2.4.2 Timing an already-stopping service #

peinit does not reset an already-Stopping service's clock, either when shutdown begins or when its wave becomes eligible. It uses the timing evidence retained from the stop path that put the service in Stopping:

  • If a Stop operation, or a Restart executing its stop leg, is in flight, that operation's retained timing governs.
  • Otherwise peinit uses service-level evidence, which carries the cause that initiated the transition — ExplicitStop, ShutdownWave, ConflictEviction or BindsToPropagation.

peinit requires the evidence to be present, to belong to a service actually in Stopping, to carry a cause matching the service's current cause, to name one of those four causes, and not to describe a deadline earlier than its own start.

Evidence that is missing, stale or ambiguous fails closed rather than being guessed at. A stop that began before the shutdown has already spent part of its own clock, and inventing a fresh budget here would hand the service a second full StopTimeout.

Failing closed costs that one participant, not the shutdown. A service whose evidence cannot substantiate a deadline is given no graceful period at all: its deadline is already due, so the timeout scan escalates it to SIGKILL on its wave, and the console says which way the evidence was unusable:

peinit: shutdown cannot substantiate a stop timeout for <service>
(<reason>); killing it without a graceful period

Every other participant keeps its full StopTimeout, and the shutdown runs to completion.

An operation whose lifetime expires while it is still Pending — a later-wave stop waiting for its dependencies — fails that operation and its waiters. It does not authorise signalling the service before its wave is eligible. Shutdown owns the signalling; the operation object is an observation of it.

12.2.5 Step 5: Global timeout #

KeyDefaultMeaning
Machine\System\Boot\ShutdownTimeout90Seconds for the whole sequence.
Machine\System\Boot\PostKillTimeout5Seconds for a cgroup to drain after SIGKILL.

On expiry, every remaining participant's cgroup is killed, anything that does not drain within the post-kill timeout is marked Abandoned with its cgroup leaked, and shutdown continues regardless. Live submitted jobs are killed in the same sweep, whatever phase their stop had reached, and abandoned on the same terms.

The global timeout fires once. What its sweep killed is then governed by the post-kill timeout alone: sweeping again would kill the survivors again and push their post-kill checks back each time, so nothing would ever be abandoned and the sequence would never reach step 6.

The console says what the sequence was waiting for when the timeout fired, and, after any later timeout turn that leaves it still waiting, what remains: participants with their state and wave, pending stop and post-kill checks, and live jobs.

peinit: shutdown global timeout expired waiting for lpsd (Active, wave 1), job <id>
peinit: shutdown still waiting for post-kill check of lpsd

Submitted jobs are named on the console as they are stopped, killed and abandoned: peinit: shutdown stopping job <id>, killing job <id> and abandoned job <id>.

The sequence does not move to step 6 while a submitted job is live. A job's reap advances shutdown progress exactly as a service's does, so a session that exits promptly on SIGTERM costs the shutdown nothing, and one that does not costs it stop_timeout and the post-kill grace.

12.2.6 Step 6: Save the random seed #

After every service has stopped and before any unmount, peinit writes a fresh seed to /var/state/peinit/random-seed for the next boot: 512 bytes from the kernel CSPRNG on this machine, never copied from an image, protected so that only SYSTEM-equivalent authority can read or replace it.

The write is crash-conscious: a temporary file on the same filesystem, written, flushed, atomically renamed over the old seed, and the containing directory flushed. A failure is recorded and shutdown continues.

The forced and Critical-reboot paths skip it, along with the unmount step, and go directly to sync and the final action.

12.2.7 Step 7: Unmount #

  1. Snapshot the mount table first, from /proc/self/mountinfo.
  2. Attempt to unmount every remaining non-root mount in the namespace — not only what peinit mounted, so the Phase 1 set is covered by construction.
  3. Process in descending path depth, so children go before parents.
  4. A mount point already gone is a successful no-op.
  5. On failure, attempt a read-only remount. If that fails too, record it and continue.
  6. The root is never unmounted, but is remounted read-only at the end. A failure there is recorded and does not stop step 8.

12.2.8 Step 8: Sync and the final action #

This step is irreversible. Cleanup failures retained from steps 6 and 7 affect diagnostics only and never block it.

  1. sync(). Called on all three paths — graceful, forced and Critical reboot.
  2. reboot(2) with RB_POWER_OFF, RB_AUTOBOOT or RB_HALT_SYSTEM.
  3. If reboot(2) returns, the final action failed. peinit enters a minimal failed-shutdown state, keeps PID 1 alive, records the failure, and retries the same action no more than once a second. It does not restart services and does not enter recovery mode — finalisation has begun and there is nothing to go back to.

12.2.9 Shutdown during boot #

A shutdown requested while Phase 2 is still running takes effect immediately, through the same classification: Starting services are SIGKILLed, services that reached Active are stopped gracefully, and the boot is abandoned.

12.3 Signals

Peios / Peios technical reference / peinit / Shutdown

PID 1 handles every signal through a signalfd. All signals are blocked and read from the event loop. So peinit installs no signal handlers of its own, and nothing peinit writes has to be async-signal-safe.

/proc/1/status does report two caught signals, SIGSEGV and SIGBUS. They are the Rust runtime's stack-overflow guard, which runs on an alternate stack and touches nothing peinit owns, so the reasoning above is unaffected.

12.3.1 Setup #

peinit builds a mask containing every blockable signal in the supported range — SIGKILL and SIGSTOP are not blockable and are never delivered through a signalfd — and installs it with rt_sigprocmask(SIG_BLOCK, ...) before entering the main event loop. It then creates the descriptor with signalfd4(-1, mask, ...), using the same mask, with SFD_CLOEXEC | SFD_NONBLOCK.

If any part of that fails — blocking, creating the descriptor, retaining it, registering it with the event loop — peinit fails closed. There is no fallback to asynchronous handlers, because a PID 1 with handlers installed where it expected a signalfd is a PID 1 whose assumptions about what can interrupt it are wrong.

The mask is inherited across fork, which is why every child resets it before exec (§5.4).

12.3.2 The signals #

SignalBehaviour
SIGCHLDReap children with waitpid. Match them to tracked jobs; also reap orphans belonging to nothing.
SIGINTReboot. Three within five seconds forces one.
SIGTERMPoweroff.
SIGPWRPoweroff.
SIGHUPIgnored. PID 1 has no controlling terminal.
SIGPIPEIgnored. A broken pipe on the control socket cannot be allowed to kill PID 1.

Everything else is ignored. The kernel protects PID 1 from fatal signals, so no signal can kill it.

12.3.3 Reaping #

peinit reaps with waitpid(-1, ..., WNOHANG) in a drain loop, and normalises the wait status before any service or job policy sees it:

  • An exited child carries its exact exit code, 0–255.
  • A signalled child carries the terminating signal number and whether the core-dump bit was set.
  • A stopped or continued status is invalid on this path, because peinit never asks for them. Observing one fails closed rather than being interpreted.

That last rule matters more than it looks. A stopped child reported as an exit would be read as a service that had terminated, and peinit would act on a process that is merely paused.

As PID 1, peinit also reaps processes nobody is tracking — orphans reparented to it — and reports them as untracked rather than trying to attribute them to a service.

12.4 Finalisation

Peios / Peios technical reference / peinit / Shutdown

Three shutdown paths reach the kernel, and they do different amounts of work on the way.

PathStop wavesSeed saveUnmountSyncFinal action
GracefulYesYesYesYesYes
Forced — three SIGINTsNo, SIGKILL everythingNoNoYesYes
Critical service failureNoNoNoYesYes

The two abrupt paths are required to reach sync() and the final kernel action with minimal additional work, and skipping the seed and the unmounts is what "minimal" means. A machine that is rebooting because its audit daemon died has nothing to gain from a tidy unmount and something to lose from the time it takes.

12.4.1 What survives a failure #

Steps 6 and 7 — the seed and the unmounts — retain their failures as evidence rather than acting on them. Every one is recorded, none of them blocks step 8, and none of them enters recovery mode. By this point there is nothing to recover to: services have stopped and the filesystems are on their way down.

12.4.2 If the final action returns #

reboot(2) does not return on success. If it does, the final action failed, and peinit enters a minimal failed-shutdown state:

  • PID 1 stays alive. It has to — PID 1 exiting panics the kernel.
  • The failure is recorded.
  • The same action is retried, no more than once a second.
  • Services are not restarted, and recovery mode is not entered.

There is no way back from here. Finalisation has begun, the services are gone and the filesystems are read-only; the only correct behaviour is to keep trying the one thing that would end it.

RB_HALT_SYSTEM does not return either, so this state is only reachable for a genuine failure rather than for the halt case.

12.4.3 A note on the mount table #

The unmount step re-reads /proc/self/mountinfo when checking whether a mount point that returned ENOENT or EINVAL is really gone. Depth ordering puts the depth-one mounts last, alphabetically — /dev, /proc, /run, /sys — so /proc is unmounted before /run and /sys are attempted. Neither of those answers ENOENT or EINVAL in practice — /sys unmounts, and /run either unmounts or is busy — so the check never runs for them. It is a latent hazard rather than a live one: if either ever did answer that way, the check could not read the file it needs, and the shutdown would record a cleanup failure and attempt a pointless read-only remount.

13.1 Trust Boundaries

Peios / Peios technical reference / peinit / Security

peinit sits at two.

13.1.1 Kernel to peinit #

peinit is the first userspace process, and the kernel gives it a SYSTEM token — S-1-5-18, every privilege. This is the root of trust for all userspace identity on the system.

peinit does not drop that token and does not authenticate to anything. Its identity is axiomatic: there is no authority above it in userspace that could vouch for it, and the kernel handing it the boot token is the vouching.

13.1.2 peinit to services #

peinit creates service processes with specific identities and reduced privileges. The trust runs one way. peinit trusts the kernel because it has no alternative; services trust peinit because peinit gave them their identity. Services do not trust each other — KACS mediates every access between them, and nothing peinit does creates a relationship between two services beyond the ordering their definitions asked for.

13.1.3 The TCB #

peinit is part of the Trusted Computing Base, alongside the kernel, KACS, LCS, KMES, registryd, authd, lpsd and eventd. A compromise of any of them compromises the system.

That list is not decoration. It is why registryd is exempt from the global environment layer (§5.5) — a component in the TCB cannot be configurable by a mechanism it is itself the enforcement point for — and it is why eventd is Critical, since a TCB whose audit trail can be silently stopped is not one.

13.1.4 Filesystem enforcement #

KACS enforces Security Descriptors on filesystem access. peinit's Phase 1 descriptor seeding (§2.3) exists precisely because of it: a freshly mounted tmpfs carries no descriptor, and under DENY_MISSING every inode on it would be unreachable to everything — including peinit — until something stamps a descriptor it can inherit from.

That enforcement has no bypass. There is no owner exemption, no privilege that overrides a missing descriptor, and no root escape , which is what makes the seeding step fatal on failure rather than advisory.

FACS extends the model further, and until it lands the filesystem layer still relies partly on conventional trust — correct packaging, controlled binary paths — for the objects nothing has stamped.

13.2 peinit's Privileges

Peios / Peios technical reference / peinit / Security

peinit runs as SYSTEM with every privilege for the lifetime of the system. What it actually exercises is narrower.

Privilege or capabilityUsed for
SeCreateTokenPrivilegeMinting SYSTEM tokens for platform services during bootstrap, before authd exists (§4.2).
SeTcbPrivilegeRequesting tokens from authd on a service's behalf, and installing a primary token on a child whose identity differs from peinit's own — a service's, or a submitted job's.
SeAuditPrivilegeWriting events: KMES refuses an emit from a token without it enabled. And recording access decisions: an AccessCheck peinit runs writes the kacs.audit.access.checked record its descriptor's SACL asks for only while the caller holds it enabled (§8.4).
Token duplicationTurning a job identity into the primary token a submitted job runs as (§8.5): opening the submitter's own primary through its pidfd, or taking the token the kernel attached, and duplicating either to a primary. SYSTEM holds full access on every token's default descriptor, which is what makes the duplicate permitted.
Process creationFork and exec, inherent to PID 1.
cgroup managementCreating and destroying trees under /sys/fs/cgroup/peinit/.
Signal deliverySIGTERM and SIGKILL to managed processes.
Mount operationsThe Phase 1 virtual filesystems.

peinit verifies the three privileges at startup, before Phase 1 does anything that needs them, and fails to recovery naming which is missing :

peinit: required privileges are not held: Token("peinit is missing
required privilege(s): SeCreateTokenPrivilege")

Present and enabled — a privilege the token carries but has not enabled is not usable, and would fail identically to being absent. SeAuditPrivilege is the exception: held but disabled, peinit enables it before checking, rather than run with no events and no record of who was refused what. The boot SYSTEM token PID 1 runs as holds every privilege enabled, so on a token the kernel made the check passes and changes nothing.

The check earns its place by what it replaces. A missing SeCreateTokenPrivilege used to surface as an EPERM from the first token mint, which is the first service start — registryd, in Phase 1 step 6. So a peinit that could not mint tokens entered recovery reporting what read as a registryd problem, and nothing anywhere named the privilege.

SeImpersonatePrivilege is not required and not checked, even though it appears in the requirement list peinit was specified against. It is not used. peinit passes the peer's token descriptor to AccessCheck directly rather than impersonating the caller and evaluating as them, so the privilege that would be needed to impersonate is not needed at all. The same holds on the jobs socket: peinit never impersonates a submitter, and the identity a job runs as is one the kernel already verified the submitter could convey.

For non-platform services peinit creates no tokens. It installs the ones authd minted. It mints only for the SYSTEM platform services it starts during bootstrap, before authd is available.

Note

Minting is interim. The intended model derives a service's token from peinit's own handle through a KACS duplicate-with-additions operation — kernel-attested as a descendant of peinit's real token, and needing no minting privilege. That would let peinit drop SeCreateTokenPrivilege entirely, which is the point: the privilege to mint arbitrary tokens is the strongest thing peinit holds and the one it has the least use for.

13.3 The Attack Surface

Peios / Peios technical reference / peinit / Security

SurfaceReachable byControlsProtected by
Control socketEvery authenticated principal, by defaultService lifecycle, shutdown, observing and stopping jobsThe peer token and AccessCheck against the target's descriptor for every command; a per-user connection limit
Jobs socketEvery authenticated principal, by defaultRunning a program under supervision as an identity the submitter holds; managing the jobs it submittedThe socket inode's descriptor, which is the whole of the submission permission; the kernel's gating of the attached token; the job's own descriptor for every later command; a per-submitter quota
Notification socketAnything that can connectService readiness, watchdog, stored descriptorsThe socket inode's descriptor, then PID matching verified through a pidfd, plus the start generation
Registry keysAnything with registry accessDefinitions, triggers, configurationRegistry key descriptors, enforced by LCS
Machine\System\Init\EnvVars\Anything with registry accessThe environment of every serviceThat key's descriptor, and nothing else — variable names are not filtered
Service cgroupspeinitProcess tracking and clean killOwnership of the hierarchy
Phase 1 mountspeinitVirtual filesystem availabilityHardcoded, no external input
/lcl/policy/autorun.dWhatever can write itArbitrary code as SYSTEM in early bootThe directory's descriptor
Boot attempt counterWhatever can write /.peinit/Recovery mode entryThat file's descriptor

13.3.1 What the socket descriptors mean in practice #

The control socket admits every authenticated principal (§10.1), and what each may then do is decided per command by the control descriptor and each service's own. The ACCESS_DENIED path and the KACS audit record it produces are therefore reachable by anyone who can log on: a caller can make as many kacs.audit.access.checked records as it has refused commands to send, because the default SACLs ask for one per refusal; a descriptor whose SACL audits less makes fewer. So is peinit's request parser, which bounds every request by MaxRequestSize before parsing it. What one caller can hold open is bounded by MaxControlConnectionsPerUser, so no one caller can fill the connections everyone shares.

The notification socket is stamped for SYSTEM and for the Service group S-1-5-6 (§10.5) — every process started as a service, and nothing else. Administrators are deliberately absent from it: an administrator has no business asserting that a service is ready, and the two sockets have different populations however alike their paths look.

A connection to the notification socket from any other principal is refused at connect(), by the filesystem, before peinit ever obtains a peer token — which means no audit event records it.

The jobs socket is deliberately wider: FILE_WRITE_DATA for Authenticated Users, full control for SYSTEM and Administrators (§10.7). Connecting is the submission permission, and peinit adds no check of its own behind it. What bounds the population that can reach it is that descriptor, and what bounds what any one submitter can do is the quota and the rule that a job runs only as an identity the kernel verified the submitter held. The ACCESS_DENIED path is reachable here — it is how a submitter learns it may not touch another submitter's job — and every such denial is the job's SACL's to record: under a job's default descriptor, KACS records each as kacs.audit.access.checked naming the job (§8.4).

These moved together with §4.3, and had to. A service resolved to a non-SYSTEM identity could not have reached the notification socket to report that it had started, so making identity real while the socket stayed SYSTEM-only would have broken every non-SYSTEM service's readiness protocol at once. What makes the pair work is that the tokens peinit mints for itself now carry S-1-5-6 as well, so a SYSTEM service and a LocalService one are on the same footing here.

13.3.2 The autorun directory #

The Phase 1 autorun step (§2.3) executes every file in /lcl/policy/autorun.d as SYSTEM, before path provisioning and before any service. It is fail-open by design, so nothing about a script going wrong stops the boot.

Its only protection is the descriptor on that directory. Anything that can write there executes as SYSTEM at the earliest point in userspace that exists.

13.4 Security Invariants

Peios / Peios technical reference / peinit / Security

Properties peinit does not violate.

1. peinit does not grant privileges it was not asked to grant. RequiredPrivileges is subtractive. peinit removes privileges and never adds one — there is no code path that constructs anything but a removal (§4.5).

2. peinit does not bypass AccessCheck for control operations. Every control command reaches a check against the appropriate descriptor. There is no backdoor, no override flag, and no "trust localhost". Every command on a submitted job, on either socket, is checked against the job's own descriptor — the submitter included, and the job identity excluded unless the descriptor says otherwise (§8.5).

3. peinit does not expose one service's state to another without access control. list returns only what the caller may query, and status is checked per service; job-list and job-status are checked per job (§10.2).

4. No access denial is silent by default. Every check names its object to KACS in an audit context, and every descriptor peinit builds itself — the default service descriptor, the control descriptor, a submitted job's default — carries a SACL auditing every refusal, so a failed AccessCheck is recorded by KACS as kacs.audit.access.checked with the caller, the object and the access requested and granted (§8.4). A descriptor written to the registry or supplied with a job is honoured as it is, SACL included; whoever may write one is trusted with that object's audit policy (§4.6). The MAXIMUM_ALLOWED check that fills a response's granted refuses nothing (§10.2).

5. The control descriptor, the ServiceSecurity descriptors and the per-job descriptors are the only policy inputs for runtime access control. peinit consults no configuration file, no environment variable and no hardcoded principal list. The only inputs to AccessCheck are those descriptors — the first two sourced from the registry with a compiled-in default, the third fixed at submission from the submitter's SID or the SDDL it supplied. Who may submit is not a policy input at all: it is the jobs socket's file descriptor, decided by the filesystem before peinit sees the connection.

6. peinit does not share its SYSTEM token. It opens its own token query-only, as a template, and never installs it on a child. Even an Identity=SYSTEM service gets a separately minted token.

7. peinit does not drop its SYSTEM identity. PID 1 runs as SYSTEM for the lifetime of the system. Only the forked child installs a token; peinit never installs one on itself.

8. Identity is deterministic, and the dangerous case is never implicit. Every service runs with a known identity. SYSTEM has to be declared explicitly; an absent or empty Identity means LocalService.

The declaration logic upholds the eighth: an empty value resolves to LocalService, and SYSTEM is reached only by naming it. What a service actually receives depends on materialisation, and the two can no longer diverge: the authd client that returned a minted SYSTEM token for whatever it was asked for is gone, and a token whose user SID is not the one the declared identity predicts fails the launch rather than being reported under the name that was asked for (§4.3).

9. A submitted job runs only as an identity the kernel verified its submitter could convey. There is no identity field, no route from a plain descriptor to a job identity, and no token opened through a PID or descriptor the submitter named. The job identity is the connecting process's own primary — reached through the kernel's handle on that process — or the token the kernel attached to the message after running the impersonation gates with the submitter as installer. peinit never re-derives that decision and never refuses a token for who it names (§8.5).

14.1 Losing the Registry

Peios / Peios technical reference / peinit / Failure Modes

peinit depends on the registry once, hard, and then never again in the same way. The difference between those two situations is most of what this section is about.

14.1.1 During Phase 1 #

registryd failing to start, failing readiness, or failing the schema-version probe sends peinit to recovery mode immediately. There is no Phase 2 without a registry and nothing useful to degrade to. The counter is incremented, so a persistently broken registryd burns boot attempts even though every one of them fails the same way.

The recovery shell reached from this failure does not have registryd running, and the offline tools (§2.8) are what an administrator has. A Phase 1 failure earlier than the registryd step is different: recovery starts a registryd of its own there (§2.8).

14.1.2 During Phase 2 #

A registry read that fails or times out during the definition read sends peinit to recovery, for the same reason: there is no graph to boot.

14.1.3 At runtime #

This is where the design pays off. peinit holds a complete in-memory model and does not read the registry during normal supervision, so registryd going away does not stop peinit supervising anything. Services keep running, restarts keep working, the control socket keeps answering, and timers keep firing.

What stops working is anything that needs new configuration: reload-config fails. Change notifications stop arriving, and a timer's last-run timestamp cannot be written — so a persistent timer may produce a spurious catch-up on the next boot.

registryd itself is a Critical service , so its failure takes the ordinary Critical path: restart budget, then sync and reboot. peinit does not have to handle a permanently absent registryd at runtime, because the system reboots first.

14.1.4 A definition that will not decode #

One definition that fails to decode fails that definition (§3.2). At boot the key is marked Failed with cause ValidationError and every other service starts normally; on a reload the key is failed the same way and every other definition loads. One malformed key no longer stops unrelated definitions in the same batch from taking effect, and a boot has no previous generation to fall back to anyway, so refusing everything would mean not booting at all over a single malformed key — which is what it used to do.

14.2 Losing a Dependency

Peios / Peios technical reference / peinit / Failure Modes

14.2.1 authd #

Without authd, no non-SYSTEM service can obtain a token, so every such start fails with ParentSetupFailure. Platform services are unaffected, since they never take that path.

authd is Critical , so its own failure eventually reboots the system rather than leaving it in a state where no user-facing service can start.

14.2.2 eventd #

peinit supervises eventd like anything else, and eventd is Critical. While it is down, peinit re-enables the pre-eventd buffer (§11.2) and keeps collecting output; when eventd returns the handoff repeats.

There is a log gap bounded by the buffer size. There is no event gap — audit events land in the KMES ring buffer regardless of eventd's state, and eventd resumes from the last persisted sequence. A submitter's output sink (§11.1) is fed regardless, since it is written from the pipe read rather than from the forwarder.

14.2.3 A bound dependency #

A service that BindsTo something is stopped when that something stops, with cause BindsToPropagation, and restarted when it returns, with cause BindsToRecovery and no charge against the restart budget (§7.1). This is the one dependency relationship that recovers by itself.

A Requires dependency crashing does not affect a running dependent at all. The dependent was ordered after it, not coupled to it.

14.2.4 A dependency that never becomes satisfying #

A dependent blocked on a hard dependency waits until its own operation lifetime expires, and then fails. The timeout is measured from when the operation was created rather than from when it started running, so a service queued behind a slow dependency can exhaust its StartTimeout without ever having attempted to start — which is the honest answer, in that the caller really has been waiting that long.

14.3 Unkillable Processes

Peios / Peios technical reference / peinit / Failure Modes

A process in uninterruptible kernel sleep does not respond to SIGKILL. Nothing peinit can do will make it exit, and the design consequence is that peinit stops trying rather than blocking on it.

Detection is uniform: after sending the kill, arm a post-kill deadline (5 seconds by default) and check whether the cgroup still reports as populated when it fires.

For a service stop, the post-kill deadline belongs to that stop operation, including the stop leg of a Restart. When the main exits and the stop ends, the deadline is cancelled. A retained obsolete deadline is ignored before probing or removing the cgroup: the same pathname may already belong to the next instance. A later stop that escalates to SIGKILL gets its own full PostKillTimeout; it cannot inherit the earlier instance's remaining grace. Re-recording a deadline for the same stop does not extend it.

What follows depends on which cgroup it was:

CgroupConsequence
main/The service goes to Abandoned with cause ProcessUnkillable. Supervision stops; the cgroup is leaked.
health/ or hooks/The sub-cgroup is orphaned and recorded as a leak. The service carries on normally.
checks/The sub-cgroup is recorded as a leak of type helper, as any other leaked sub-cgroup is (§5.7).

The distinction is about what the stuck process is holding. A main process holds the service's ports, locks and connections, so a service whose main process cannot be killed cannot be restarted into a working state. A health check or a hook holds nothing, so a stuck one is a nuisance rather than a blocker.

14.3.1 What Abandoned means #

peinit has given up. The service is not restarted, does not satisfy dependents, and every lifecycle command against it is invalid except reset (§10.3).

A reset re-checks main/. If it has finally emptied — the I/O that was hung completed, or the device came back — peinit cleans up the whole tree and the service returns to Inactive. If it is still populated, the service returns to Inactive anyway, the cgroup stays leaked, and both the acknowledgement and the console carry a warning saying so.

14.3.2 When the process finally dies #

Abandoning a service does not close its main job. The process is still there, so there is still something to reap, and peinit keeps the job open against the possibility that it eventually exits — which it often does, once the I/O it was blocked on completes.

That exit arrives for a service already in Abandoned, a state that has stopped expecting a process. peinit records it and does nothing else: no transition, no restart, no operation update. The service stays Abandoned, the stop stays failed , and the console reports the exit:

peinit: service <service> main process exited in state Abandoned; no action taken

The exit is news about the process, not a retroactive success for the stop that gave up on it. Recovering the service still takes a reset, which is also what cleans up the cgroup the process was holding.

The same handling covers a late exit in any state that no longer expects a process — see §6.2.

14.3.3 The generational escape #

A leaked cgroup cannot be removed, so the next start would collide with it. Recording a leak increments the service's cgroup generation, and the next start builds a fresh tree at %gen<N> (§5.1). Old trees persist until reboot.

That is what lets a service recover from a leak at all: peinit cannot clean up the old tree, so it stops trying to and uses a new one.

14.3.4 What it actually means #

Every path here has the same underlying cause. Something below the service — a hung mount, a failing controller, a device that stopped answering — is not responding to the kernel, and no amount of restarting the service will change that. The leak record exists to say so, because the alternative is a service that mysteriously will not restart.

14.4 Resource Exhaustion

Peios / Peios technical reference / peinit / Failure Modes

peinit is single-threaded PID 1, so most resource pressure reaches it as a failure at a syscall rather than as slowness.

14.4.1 Descriptors #

peinit holds a descriptor per supervised process (a pidfd), two per supervised process for output pipes, one per armed timer, one per control connection, two per jobs connection (the socket and the peer's pidfd), one per submitted job's output sink, plus the sockets, the epoll instance and the signalfd. Control connections are bounded in all by MaxControlConnections (256), which is set for descriptors rather than for fairness: what keeps one caller from taking them is MaxControlConnectionsPerUser (§10.1). A submission also holds its prepared token and attached descriptors from acceptance until the launch takes them, which is bounded by MaxJobsPerSubmitter times the descriptor cap per message.

EMFILE or ENFILE from pipe2 or clone3 fails the start with ParentSetupFailure — a restart-eligible cause, so a service that failed because the system was momentarily out of descriptors gets another go.

The pre-start check helper's result descriptor and pidfd are both unregistered from the event loop and closed, so a service using filesystem conditions costs no descriptors beyond the start itself.

14.4.2 Processes #

EAGAIN from clone3 — the PID limit — is also ParentSetupFailure and restart-eligible.

The one path that forks outside the launch machinery is the timer last-run write, one child per firing of a persistent timer (§9.2). The children _exit as soon as the write returns and peinit matches each one's exit status back to its write, so a failure is reported rather than discarded — but the fork itself is real and happens on every firing.

It is a fork of PID 1, so the child briefly inherits everything peinit holds: every pidfd, the epoll instance, both sockets, every service's pipe read end. It uses none of them and exits immediately. Driving the write through an asynchronous LCS path instead would remove the fork entirely; there is no such path today.

14.4.3 Memory #

ENOMEM from clone3 behaves like the others.

peinit's own memory is bounded by design in the places that could otherwise grow without limit: the pre-eventd buffer has a fixed size and drops its oldest, there is no outbound queue for log delivery, terminal jobs and operations are dropped rather than retained, and neither has a history structure.

Nor do the two structures that once did. A graph execution context and its operation associations are retired on the maintenance turn after every member reaches a terminal state (§7.3), so a boot or an on-demand start costs nothing lasting. And an OnFailure chain entry is cleared once its handler has held for a RestartWindow (§6.3), so a resident handler does not permanently occupy a slot of that failure's depth budget.

14.4.4 Disk #

A full root filesystem shows up in three places. The boot attempt counter cannot be written, which peinit treats as a counter of zero and continues — a failure to record an attempt is not itself worth escalating. The random seed cannot be saved at shutdown, which is recorded and does not block the shutdown. And registryd cannot write, which is registryd's problem and reaches peinit as a Critical service failing.

14.4.5 The OOM killer #

An ErrorControl=Critical service is marked OOM-immune, with oom_score_adj at -1000; everything else is left at the default (§5.4). A Critical service is one whose loss reboots the machine, so letting the OOM killer pick it would turn memory pressure into a reboot.

peinit itself is PID 1 and the kernel will not choose it.

14.5 Power Loss and Corruption

Peios / Peios technical reference / peinit / Failure Modes

14.5.1 What peinit writes #

Three things, and all three are written to survive an unexpected loss of power:

WhatWhereHow
Boot attempt counter/.peinit/boot-attemptsA plain integer, rewritten each boot.
Local machine ID/lcl/etc/machine-idTemporary file, flush, rename.
Random seed/var/state/peinit/random-seedTemporary file on the same filesystem, flush, atomic rename, directory flush.

The seed and the machine ID are atomic in the sense that matters: a reader sees either the old value or the new one, never a partial write.

14.5.2 Reading a damaged file #

The three behave differently on finding something they cannot use, and the differences track how bad each situation is.

The counter is the strictest. Absent means zero. But unreadable, empty, non-decimal, carrying trailing data, or overflowing all send peinit to recovery mode. A counter that cannot be trusted cannot escalate, and the whole point of it is escalation — so failing to read it is treated as though it had already escalated.

The machine ID is regenerated. Absent, empty, all zeroes, the wrong length, not hexadecimal, or missing its trailing newline all produce a fresh identifier, recorded as a warning. It is an opaque install identifier, not a security principal, and a new one is a smaller problem than no boot. An unreadable file or an unwritable path is fail-soft too — the boot continues on an ephemeral identifier, recorded as a warning. Only a failure to generate one sends peinit to recovery, because at that point there is no identifier to continue with (§2.3).

The seed is entirely fail-soft. Absent, empty, oversized, or unrestorable all continue the boot. An empty or oversized seed is recorded as an error on the way past (§2.3); none of them stops the boot. A system with no entropy cache still boots; it starts with less entropy, which is the image builder's problem to solve with a hardware or virtio RNG.

14.5.3 The registry #

peinit does not write service state to the registry, apart from timer last-run timestamps. Everything else about a service's runtime state lives in memory and is rebuilt from the definitions on the next boot, which means a power loss cannot leave peinit's own state inconsistent — there is no state on disk to be inconsistent with.

Registry consistency across power loss is loregd's concern, and its recovery paths are what the recovery shell offers (§2.8).

14.5.4 Timers across a loss #

A persistent timer's last-run timestamp is written after the firing and after the start is initiated, not after the service completes (§9.3). A power loss mid-run therefore does not re-trigger on the next boot — the run was attempted, not missed.

A power loss between the firing and the write does re-trigger, which is the right way round: one extra run is better than a silently skipped one.

14.5.5 Shutdown that never finishes #

If power is lost during the graceful sequence, the effect depends on how far it got. Before step 7, the filesystems are still mounted read-write and the next boot behaves like any unclean shutdown. After step 7, the root has been remounted read-only and everything else unmounted, so there is nothing outstanding to lose.

The counter was incremented at the start of the boot that is now ending, and a successful boot resets it during that boot, once the success grace period has been held. So a power loss during the shutdown of a boot that succeeded leaves the counter at zero; one during the shutdown of a boot that never reached success leaves it advanced. Enough of those in a row reach the recovery threshold, which is the intended behaviour: a machine that keeps failing to reach a successful boot is a machine an administrator should be looking at.

Appendix A Registry Key Reference

Peios / Peios technical reference / peinit

Every registry key peinit reads or writes. Semantics are in the sections referenced.

A.1 Service definitions #

KeyPurposeDefined in
Machine\System\Services\Parent key. Each child key is one service.§3.2
Machine\System\Services\SchemaVersionSchema version guard. REG_DWORD, currently 1. Created by peinit if absent.§2.3, §3.2
Machine\System\Services\ServiceSecurityThe descriptor inherited by definitions that carry none. REG_BINARY.§4.6
Machine\System\Services\<name>One service definition.§3.2
Machine\System\Services\<name>\ProvidesRoles this service fills, which dependencies may name in place of the service. REG_MULTI_SZ. Read at boot and on reload.§7.6
Machine\System\Services\<name>\LastTimerRunLast-run timestamp for a single-trigger persistent timer. REG_QWORD, written by peinit.§9.3
Machine\System\Services\<name>\TimerState\Per-trigger timestamps for a multi-trigger service. Each value is named by the percent-encoded schedule and holds a REG_QWORD.§9.3

A.2 Boot configuration #

KeyTypeDefaultPurposeDefined in
Machine\System\Boot\MaxParallelStartsdword, > 010Services starting concurrently during boot. Zero, a type mismatch, or a malformed payload is invalid and enters recovery.§2.5
Machine\System\Boot\BootSuccessGracedword30Seconds every Critical service has to hold a dependent-satisfying state before the boot counts as successful.§2.5
Machine\System\Boot\SettleTimeoutdword5Seconds to wait for the boot set to settle before starting boot:settled services regardless. Zero is legal.§2.5
Machine\System\Boot\ShutdownTimeoutdword90Seconds for the entire graceful shutdown.§12.2
Machine\System\Boot\PostKillTimeoutdword5Seconds for a cgroup to drain after SIGKILL before it is treated as stuck. Bounds one service's final stop, where ShutdownTimeout bounds the sequence.§12.2

A.3 Operational parameters #

KeyTypeDefaultPurposeDefined in
Machine\System\Init\ControlSecuritybinarySYSTEM and Administrators, every right; Authenticated Users, SYSTEM_QUERY_STATUSThe descriptor for system-level control operations and the boot query.§4.7
Machine\System\Init\MaxControlConnectionsPerUserdword16Concurrent control socket connections one caller's user SID may hold. SYSTEM is exempt.§10.1
Machine\System\Init\MaxControlConnectionsdword256Concurrent control socket connections in all.§10.1
Machine\System\Init\MaxRequestSizedword65536Maximum control request size, in bytes.§10.1
Machine\System\Init\ConnectionTimeoutdword30Seconds before an idle control connection is closed.§10.1
Machine\System\Init\MaxJobsConnectionsdword64Concurrent jobs socket connections.§10.7
Machine\System\Init\MaxJobMessageSizedword32768Maximum jobs socket message content, in bytes.§10.7
Machine\System\Init\JobsConnectionTimeoutdword30Seconds before an idle jobs connection is closed.§10.7
Machine\System\Init\MaxJobsPerSubmitterdword64Live submitted jobs one submitter SID may hold. SYSTEM is exempt.§8.5
Machine\System\Init\MaxLogLineLengthdword8192Bytes per output line before truncation. Minimum 256; below that the default is used and a warning logged.§11.3
Machine\System\Init\MaxLogBufferPerServicedword65536Pipe capacity per service, applied with F_SETPIPE_SZ. Minimum 4096 (one page, the kernel's own floor).§11.3
Machine\System\Init\LogReadBytesPerEventdword16384Bytes drained from one output pipe per readable event. Minimum 512.§11.3
Machine\System\Init\PreEventdBufferdword1048576Bytes of output retained before eventd is available. Applied at boot and on reload. Minimum 4096.§11.2
Machine\System\Init\EnvVars\parent keyemptyVariables injected into every service but registryd. Value name is the variable name; REG_SZ data is the value. Its descriptor is security-critical.§5.5
Machine\System\Init\ProvisionedPaths\parent keyemptyBoot-time path provisioning entries.§2.4
Machine\System\Init\ProvisionedPaths\<name>\Kindstring—directory or file. Required.§2.4
Machine\System\Init\ProvisionedPaths\<name>\Pathstring—The absolute path. Required.§2.4
Machine\System\Init\ProvisionedPaths\<name>\Securitybinarybuilt-inThe descriptor to apply.§2.4
Machine\System\Init\ProvisionedPaths\<name>\Requireddword0If 1, a failure enters recovery before Phase 2.§2.4

A.4 Other subsystems #

KeyTypePurposeDefined in
Machine\System\eventd\LogSocketPathstringWhere peinit forwards service output.§11.4

A.5 Watches #

peinit subscribes to Machine\System\Services\ and Machine\System\Init\ at boot. Any drained event triggers a full reload-config, which also covers the OVERFLOW case (§3.7).

Appendix B Constants and Paths

Peios / Peios technical reference / peinit

Values compiled into peinit, which no registry key changes.

B.1 Filesystem paths #

PathPurpose
/usr/bin/peinit2Where peinit is installed in package storage.
/bin/peinit2The runtime path the kernel init= names.
/sbin/registrydThe compiled-in registryd image path.
/var/state/loregd/Machine.hiveThe machine hive, passed to registryd.
/var/state/loregd/Users.hiveThe users hive, passed to registryd.
/.peinit/peinit's own state directory on the root filesystem.
/.peinit/boot-attemptsThe boot attempt counter.
/lcl/etc/machine-idThe local machine identifier.
/var/state/peinit/random-seedThe persisted entropy seed.
/lcl/policy/autorun.dPhase 1 autorun scripts.
/run/services/peinit/control.sockThe control socket.
/run/services/peinit/notify.sockThe notification socket, by default.
/run/services/peinit/jobs.sockThe jobs socket.
/sys/fs/cgroup/peinit/The root of every service cgroup tree.
/sys/fs/cgroup/peinit/jobs/<guid>A submitted job's cgroup.
/dev/rtc, /dev/rtc0The hardware clock, in that order of preference.
/dev/consoleWhere peinit writes its own messages.
/bin/recsh, /bin/shThe recovery shell, in that order of preference.

B.2 Timeouts and limits #

ConstantValueMeaning
registryd setup timeout30 sProcess setup, driven synchronously in Phase 1.
registryd readiness timeout30 sWaiting for READY=1 in Phase 1.
Reload detection window2 sWaiting for RELOADING=1 after a reload signal. Bounded above by the operation deadline.
Restart delay cap60 sThe ceiling on exponential backoff.
Timeout extension cap×4The multiple of a phase's base timeout an extension may reach.
Operation retention60 sHow long a terminal operation is kept before being dropped.
Submitted job retention60 sHow long a terminal submitted job's entry answers status before being dropped.
peinit.job.status.reported interval1 sThe least spacing between two peinit.job.status.reported events for one job.
Submitted job stop_timeout10 sThe default, per submission.
Submitted job readiness_timeout30 sThe default, per submission.
Submitted job argv and environment2 MiBThe bound on a definition's arguments and environment together.
Boot attempt threshold3The default, overridden by peios.bootattempts=.
OnFailure chain depth16The maximum handler chain from one originating failure.
Pre-eventd buffer1 MiBThe compiled-in buffer capacity.
Notification datagram64 KiBThe receive buffer size.
Descriptors per datagram64The control message buffer capacity.
Random seed512 bytesWritten at shutdown. Restoring accepts 1–4096 bytes.
Control listen backlog32
Jobs listen backlog32
Descriptors per jobs message64Including an output sink.
Tokens per jobs message1
First injected descriptor3Where stored descriptors are placed.
Final action retry1 sThe minimum interval between reboot(2) retries.

B.3 Environment #

VariableValue
PATH/sbin:/bin — the compiled-in base for every service.

The recovery shell additionally receives TERM=linux and HOME=/.

B.4 Kernel command line #

ParameterEffect
peios.safemode=1Force Safe mode.
peios.recovery=1Force recovery mode.
peios.bootattempts=NThe recovery threshold; 0 disables the check.
peios.quiet=NConsole verbosity: 0, 1 or 2.
peios.notifysocket=PATHOverride the notification socket path.

B.5 Descriptors peinit applies #

WhereSDDL
/dev/shm, /run, /sys/fs/cgroup after mountingO:SYG:SYD:(A;OICI;GA;;;SY)(A;OICI;GA;;;BA)(A;CI;GRGX;;;WD)(A;OICIIO;GA;;;S-1-3-0)
/dev/null, /dev/zero, /dev/full, /dev/random, /dev/urandom, /dev/tty, /dev/ptmx (DACL only)D:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FRFW;;;WD)
A provisioned path with no SecurityO:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FR;;;BU)
A service's /run/<name> runtime directoryO:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;GA;;;<service SID>)
The random seed fileO:SYG:SYD:(A;;GA;;;SY)
The default ServiceSecurityO:SYG:BAD:(A;;0x000F;;;SY)(A;;0x000F;;;BA)(A;;0x0001;;;AU)
The default ControlSecurityO:SYG:BAD:(A;;0x0007;;;SY)(A;;0x0007;;;BA)(A;;0x0004;;;AU)
The control socket inodeO:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)
The jobs socket inodeO:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)
A submitted job with no security_descriptorOwner and group the submitter; (A;;0x0007;;;<submitter>)(A;;0x0007;;;SY)(A;;0x0007;;;BA)

B.6 Access rights #

RightBit
SERVICE_QUERY_STATUS0x0001
SERVICE_START0x0002
SERVICE_STOP0x0004
SERVICE_INTERROGATE0x0008
SERVICE_ALL_ACCESS0x000F
SYSTEM_SHUTDOWN0x0001
SYSTEM_RELOAD_CONFIG0x0002
SYSTEM_QUERY_STATUS0x0004
JOB_QUERY0x0001
JOB_STOP0x0002
JOB_SIGNAL0x0004
JOB_ALL_ACCESS0x0007

B.7 Identifiers #

Every GUID peinit generates — jobs, operations — is UUIDv7, so identifiers sort by creation time.

Peios Learn — documentation for the Peios project.

Built with Trail.