# Process runtime reference --- # Process runtime reference _Peios / Developing for Peios / Process runtime reference_ > Developer contracts for process creation, program replacement, termination, child collection, orphan adoption, and per-thread operations. Use this reference when implementing a launcher, supervisor, threading library, or other code that creates and controls processes. For inspecting or stopping a running program, start with [Task Manager](/peios/using-peios/threads-and-processes/task-manager.md) and [Process lifecycle](/peios/using-peios/threads-and-processes/process-lifecycle.md). - [Process creation](/peios/developing-for-peios/process-runtime-reference/process-creation-reference.md): `fork`, `vfork`, `clone`/`clone3`, sharing flags, `exec`, `posix_spawn`, and pidfds. - [Process lifecycle](/peios/developing-for-peios/process-runtime-reference/process-lifecycle-reference.md): termination, exit-status encoding, the wait family, and subreaper adoption. - [Thread operations](/peios/developing-for-peios/process-runtime-reference/thread-operations-reference.md): TIDs, thread-exit notification, and thread-local storage. These calls also interact with Peios security. See [Token lifecycle](/peios/security-fundamentals/tokens/lifecycle.md) for identity inheritance, [the process security descriptor](/peios/security-fundamentals/process-integrity-protection/the-process-security-descriptor.md) for process rights, and [the two-check rule](/peios/security-fundamentals/process-integrity-protection/the-two-check-rule.md) for cross-process access. --- # Process creation reference _Peios / Developing for Peios / Process runtime reference_ > The precise contract for fork, vfork, clone and clone3, the clone sharing flags, the exec family, posix_spawn, and the pidfd calls. This is the detailed counterpart to [Creating processes](/peios/using-peios/threads-and-processes/creating-processes.md). That page explains the ideas; this one gives the exact contract — what each call does, what a new process inherits, and what each `clone` sharing flag controls. ## fork `fork()` creates a new process by duplicating the caller. The child is a near-identical copy that runs independently from the point of the call. The child **inherits**: - a private copy of the parent's memory, made lazily (copy-on-write), so the duplication is cheap and changes on either side stay invisible to the other - copies of the parent's open file descriptors — each refers to the same open file description, so file offset, status flags, and signal-driven-I/O settings are shared between parent and child - open directory streams and open message-queue descriptors - open-file-description locks (`flock`, OFD locks) through those shared descriptors - the parent's signal dispositions (the handler set for each signal) - the file-creation mask (umask), the current directory, and the root directory - resource limits and the timer-slack value - the parent's identity — it begins acting on the same token (see [Token lifecycle](/peios/security-fundamentals/tokens/lifecycle.md)) - the parent's mitigations and protection, carried on its [PSB](/peios/using-peios/threads-and-processes/the-process-security-block.md) The child does **not** inherit: - the parent's PID — it receives a new PID and a new Process GUID - memory locks (`mlock`, `mlockall`) - memory regions marked `MADV_DONTFORK` (absent in the child) or `MADV_WIPEONFORK` (present but zeroed) - process resource usage and CPU-time counters — reset to zero - pending signals — the child's set starts empty - semaphore adjustments (semadj) - process-associated record locks (POSIX `fcntl` locks — distinct from the OFD and `flock` locks above, which are shared through the inherited descriptors) - timers (`setitimer`, `alarm`, `timer_create`) - outstanding asynchronous I/O - directory-change notifications (`dnotify`) - the `PR_SET_PDEATHSIG` parent-death-signal setting (reset) - I/O-port access permissions (`ioperm`) The child's termination signal — what its parent is notified with when it ends — is `SIGCHLD`. The child is created with a **single thread** — the one that called `fork()`. If the parent had other threads, they do not exist in the child. ## vfork `vfork()` is a lightweight variant of `fork()` for the one case where the child will immediately replace itself with another program. It differs in two ways: - the caller is **suspended** from the call until the child either `exec`s or exits; - the child runs in the **parent's memory** — no copy is made — until that point. Because the two share memory and the parent is frozen, the child must do almost nothing first. The contract: the child must not return from the function that called `vfork()`, must not modify any variable other than the one holding the return value, and must not call any other function before a successful `exec` or `_exit`. Doing otherwise is undefined behaviour. `vfork` is an optimisation for the create-then-replace pattern. Ordinary code should use `posix_spawn` (below) rather than reach for it directly. ## clone and clone3 `clone()` is the general creation call. `fork` and thread creation are both `clone` with particular flag sets; `clone` exposes the choice directly. The caller gives a set of **flags** controlling what the new task shares with the creator, a stack for the new task, and locations for thread-ID and thread-local-storage bookkeeping. `clone3()` is the modern form, taking a structure instead of positional arguments so it can grow new fields over time without changing the call: | Field | Purpose | |---|---| | `flags` | The sharing flags (below). | | `pidfd` | Where to store a pidfd for the new child. | | `child_tid` / `parent_tid` | Where to record the new thread's ID, in the child's and the parent's memory. | | `exit_signal` | The signal delivered to the parent when the child ends. | | `stack` / `stack_size` | The new task's stack and its size. | | `tls` | The new task's thread-local-storage area. | | `set_tid` / `set_tid_size` | Request specific thread IDs for the new task — a privileged operation (see below). | | `cgroup` | Place the child directly into a resource group at creation (see resource management). | The structure is **size-versioned**: the caller passes the size it knows, the kernel reads up to the size it knows, and any unknown trailing fields must be zero. This is how new fields are added without breaking existing programs. Requesting a specific thread ID (`set_tid`) is a **privileged** operation — it exists for checkpoint-and-restore tools that must recreate a process with its original ID. The privilege required follows the [capability model](/peios/advanced-peios/linux-compatibility/dac-neutralization-and-capabilities.md). ## clone sharing flags Each flag makes the new task **share** something with its creator instead of getting its own copy. The flags that build threads and control creation: | Flag | Effect | |---|---| | `CLONE_VM` | Share memory rather than taking a private copy. | | `CLONE_FS` | Share filesystem context — current directory, root directory, file-creation mask. | | `CLONE_FILES` | Share the open-descriptor table, so opening or closing a descriptor in one is seen by the other. | | `CLONE_SIGHAND` | Share the table of signal handlers. Requires `CLONE_VM`. | | `CLONE_THREAD` | Put the new task in the same process (thread group) as the creator. Requires `CLONE_SIGHAND`. | | `CLONE_SETTLS` | Set the new task's thread-local-storage area. | | `CLONE_SYSVSEM` | Share the System V semaphore-adjustment list. | | `CLONE_IO` | Share the I/O context, so the two are accounted as one for disk I/O. | | `CLONE_PIDFD` | Return a pidfd for the new child. | | `CLONE_PARENT` | Make the new task a sibling of the creator — its parent becomes the creator's parent, which is also what is signalled when the task ends. An init process (PID 1) cannot use this flag, as it would create unreapable zombies. | | `CLONE_PARENT_SETTID` / `CLONE_CHILD_SETTID` | Record the new task's ID in the parent's / child's memory. | | `CLONE_CHILD_CLEARTID` | Clear the recorded thread ID and wake a waiter when the task exits — the mechanism behind joining a thread. | | `CLONE_UNTRACED` | Prevent a tracer from forcing tracing onto the new child. | | `CLONE_VFORK` | Suspend the creator until the child `exec`s or exits (the `vfork` behaviour). | A **thread** is `CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGHAND | CLONE_THREAD` (plus TLS and thread-ID bookkeeping) — everything shared, same process. A `fork` is the opposite end, with almost nothing shared. Being in one thread group has consequences beyond shared memory. Every thread returns the same PID from `getpid` (the thread-group ID), and signal **actions** are process-wide — an unhandled fatal signal delivered to any thread ends them all — though each thread keeps its own signal mask. A thread that ends does not notify the thread that created it and cannot be collected with a wait call; only once every thread in the group has ended is the process's parent sent `SIGCHLD`. The new thread shares the creator's parent, and `CLONE_THREAD` requires `CLONE_SIGHAND` (and hence `CLONE_VM`). Two further flag groups select **separation** rather than sharing, each documented where it belongs: - The `CLONE_NEW*` flags create **namespaces** — separate views of system resources (mounts, process IDs, networking, and more). Only `CLONE_NEWNS` is available to an ordinary process: a private mount table is an object its creator owns, described in the Peios Kernel TRM (§3.13). Every other namespace type needs `SeTcbPrivilege`, which is what `CAP_SYS_ADMIN` means on Peios. - `CLONE_INTO_CGROUP` places the new process into a **resource group** at creation, covered under resource management. `CLONE_DETACHED` is a historical flag with no effect and is ignored. ## The exec family An `exec` replaces the program running in the current process with a different one. The process keeps its PID, its Process GUID, and its identity — only the program changes (see [Token lifecycle](/peios/security-fundamentals/tokens/lifecycle.md) for the identity detail). | Call | What it does | |---|---| | `execve` | Replace the current program with the one at a given path. | | `execveat` | Replace it with a program named relative to an open directory, or — with `AT_EMPTY_PATH` — by an open file descriptor directly. `AT_SYMLINK_NOFOLLOW` refuses a final symbolic link. | | `fexecve` | Replace it with the program referred to by an open file descriptor (a thin wrapper over `execveat`). | ### What survives an exec A successful `exec` keeps the process itself but replaces the program it runs. Preserved across the call: - the PID, the parent PID, and the Process GUID - the process's identity (its token), its process group and session, and its controlling terminal - open file descriptors — except those marked close-on-exec, which are closed - the current directory, root directory, file-creation mask, and resource limits - pending signals, and the dispositions of signals that were ignored or left at their default Reset or discarded: - all memory — the program's mappings, stack, heap, and data are replaced; memory locks are dropped and `MADV_*` region markings are gone - every thread except the one calling `exec` — the new program starts single-threaded - handlers for caught signals — reset to the default; the alternate signal stack is dropped - attached System V shared memory, POSIX shared-memory mappings, open POSIX message-queue descriptors and named semaphores, and in-process synchronisation objects (mutexes, condition variables) - POSIX timers, outstanding asynchronous I/O, open directory streams, and registered exit handlers - the floating-point environment, and the process name (set to the new program) What an `exec` does to a process's identity when the program file is marked to run as another principal is **not** the Linux set-user-ID model — Peios handles that through the token, covered in [setuid and uid0](/peios/advanced-peios/linux-compatibility/setuid-and-uid0.md). Whether a program is allowed to run at all is a separate, execution-policy question — covered under binary signing — not part of these calls. ## posix_spawn `posix_spawn()` is the standard library function for the create-then-replace pattern: it makes a new process and runs a named program in it with a single call, with controls for arranging the child's file descriptors and a few attributes first. It is built on the calls above and is the recommended way to launch a program, in preference to assembling `fork`/`vfork` and `exec` by hand. ## Process handles (the pidfd family) A **pidfd** is a file descriptor referring to one specific process — the reliable handle introduced in [Creating processes](/peios/using-peios/threads-and-processes/creating-processes.md). `CLONE_PIDFD` hands one back at creation; a few calls work with them afterwards: | Call | What it does | |---|---| | `pidfd_open` | Obtain a pidfd for a process that already exists, given its PID. | | `pidfd_send_signal` | Send a signal to the process through its pidfd — with no risk of the PID having been recycled for a different process in between. | | `pidfd_getfd` | Duplicate one of the target process's open file descriptors into the caller. | `pidfd_getfd` reaches into another process, so it is not something any process may do to any other: it is gated by the right to act on the target, governed by that process's [security descriptor](/peios/using-peios/threads-and-processes/the-process-security-block.md). ## See also - [Creating processes](/peios/using-peios/threads-and-processes/creating-processes.md) — the conceptual counterpart to this page. - [Process lifecycle reference](/peios/developing-for-peios/process-runtime-reference/process-lifecycle-reference.md) — ending a process and collecting its result. - [Thread operations reference](/peios/developing-for-peios/process-runtime-reference/thread-operations-reference.md) — per-thread calls: TIDs, exit notification, TLS. --- # Process lifecycle reference _Peios / Developing for Peios / Process runtime reference_ > The precise contract for ending a process and collecting its result — exit, _exit and exit_group, the exit-status encoding, and the wait family. This is the detailed counterpart to [Process lifecycle](/peios/using-peios/threads-and-processes/process-lifecycle.md). That page explains how a process ends and is cleaned up; this one gives the exact calls — how a process terminates, what its exit status encodes, and how a parent collects it. ## Ending a process A process can end itself in a few ways, differing in how much tidying happens first: | Call | What it does | |---|---| | `exit()` | The orderly end. Runs every handler registered with `atexit`/`on_exit` (in reverse order of registration), flushes and closes the standard I/O streams, removes temporary files, then terminates. | | `_exit()` / `_Exit()` | The immediate end. Terminates at once — no registered handlers run and the standard I/O streams are not flushed — though open file descriptors are still closed. Used when a process must stop without running cleanup, such as a child after a failed `exec`. | | `exit_group()` | Ends the **whole process** — every thread in it — not just the calling one. This is what ordinary termination uses: a normal `exit()`, and a return from the program's entry point, both end the entire process. | Returning from the program's entry point is equivalent to calling `exit()` with the returned value. At the lowest level the raw `_exit` system call ends only the calling thread; the library `_exit` and ordinary termination route through `exit_group` so the whole process ends. Ending one thread while the rest of the process keeps running is the unusual case. A process can also be ended **from outside**, by a signal rather than by a call of its own — covered under signals. Either way, it leaves an exit status behind. ## The exit status When a process ends it leaves a small **exit status** for its parent to read. It records one of two outcomes: - **Ended on its own** — carries an exit code: the low 8 bits of the value passed to `exit`/`_exit` (so 0–255, with 0 conventionally meaning success). - **Ended by a signal** — carries the number of the signal that ended it, plus a flag for whether the process produced a core dump. The status is a packed value, read not directly but through a set of macros: | Macro | Tells you | |---|---| | `WIFEXITED` | the process ended on its own — if so, `WEXITSTATUS` gives its exit code | | `WIFSIGNALED` | the process was ended by a signal — if so, `WTERMSIG` gives the signal number and `WCOREDUMP` whether it dumped core | | `WIFSTOPPED` | the process was **stopped** (not ended) by a signal — `WSTOPSIG` gives which | | `WIFCONTINUED` | the process was resumed by a continue signal | ## Collecting the result A parent collects a finished child's status — and in doing so clears the [zombie](/peios/using-peios/threads-and-processes/process-lifecycle.md) — with one of the wait calls. They differ in which child they target and how much they report: | Call | What it offers | |---|---| | `wait` | Wait for **any** child to end; return its PID and status. The simplest form. | | `waitpid` | Wait for a chosen target (see below). Options adjust the behaviour: `WNOHANG` returns immediately if nothing is ready (a poll rather than a wait), and `WUNTRACED`/`WCONTINUED` also report children that have stopped or resumed, not only ended. | | `waitid` | The most precise form. It can target a child by a **pidfd** (`P_PIDFD`) — the reliable handle from [Creating processes](/peios/using-peios/threads-and-processes/creating-processes.md) — as well as by PID, by process group, or any child. It reports through a structured record (which child, and what happened: ended, killed, core-dumped, stopped, continued). Which kinds of change to wait for are chosen explicitly — `WEXITED` for children that have ended, `WSTOPPED` for those stopped by a signal, `WCONTINUED` for those resumed — and its `WNOWAIT` option peeks at the result while leaving the child collectable again later. | | `wait4` | Like `waitpid`, and additionally returns the child's **resource usage** — processor time consumed, peak memory, and so on. | `waitpid`'s target is chosen by a single number: a positive value waits for that exact PID; `-1` for any child; `0` for any child in the caller's own process group; and a value below `-1` for any child in the process group whose ID is its absolute value. Two further options matter for threads and other `clone`-created children, which may not notify their parent with `SIGCHLD` the way an ordinary child does: `__WCLONE` waits only for such clone children, and `__WALL` waits for every child regardless of type. ## Orphan adoption and subreapers PID 1 is the catch-all, but a process can arrange to catch orphans within its own subtree. By marking itself a **subreaper** (with `prctl(PR_SET_CHILD_SUBREAPER)`), a process becomes the one that adopts any orphan descended from it — its grandchildren and below — instead of letting them reparent all the way up to PID 1. Service managers and container managers use this to keep responsibility for everything they launched: when something deep in their subtree loses its immediate parent, it reparents to the manager, which still learns when it finally ends. ## See also - [Process lifecycle](/peios/using-peios/threads-and-processes/process-lifecycle.md) — the conceptual counterpart to this page. - [Process creation reference](/peios/developing-for-peios/process-runtime-reference/process-creation-reference.md) — fork, clone, exec, and the pidfd calls. --- # Thread operations reference _Peios / Developing for Peios / Process runtime reference_ > The per-thread operations a program uses directly — reading a thread's ID, the exit-notification mechanism behind thread joins, and thread-local storage. This is the thread-level companion to [the process and thread model](/peios/using-peios/threads-and-processes/the-process-and-thread-model.md) — the operations that act on an individual thread rather than on the process as a whole. ## Thread IDs Every thread has its own **thread ID** (TID), returned by `gettid`. Within a multi-threaded process all threads share one PID — the value `getpid` returns, which is really the **thread-group ID** (TGID) — but each thread's TID is unique. In a single-threaded process the TID, PID, and TGID are all the same number. The TID is how the system targets one specific thread rather than the whole process: sending a signal to a single thread, setting one thread's CPU affinity, or addressing it under `/proc`. It names a thread the way a PID names a process — and, like a PID, it is a reference, not an identity. What a thread is acting as is its token, never its TID. The TID is also distinct from the opaque handle a threading library returns when it creates a thread; that is a library-level identifier, not the kernel's TID. ## Thread-exit notification A thread can have the kernel notify waiters when it exits — the mechanism threading libraries build thread join on. `set_tid_address` sets a **clear-tid address** for the calling thread (and returns its TID). When that thread later terminates, if it shares memory with other threads, the kernel writes 0 to the integer at that address and wakes one thread waiting on it (a single futex wake). A thread waiting for another to finish blocks on that address; the kernel clears it and wakes the waiter at the exact moment the other thread ends. This is the same machinery as the `CLONE_CHILD_CLEARTID` flag in [process creation](/peios/developing-for-peios/process-runtime-reference/process-creation-reference.md) — that flag arranges the clear-tid address at thread creation, where `set_tid_address` sets it afterwards. ## Thread-local storage **Thread-local storage** (TLS) gives each thread its own private copy of a variable — a value that is per-thread rather than shared across the process, even though the threads share the same memory. The per-thread data lives in an ordinary block of memory; what makes it per-thread is a register pointing at this thread's block, so the same code reaches a different copy depending on which thread is running. The kernel's part is maintaining that per-thread pointer. It is set when the thread is created (the `CLONE_SETTLS` flag carries the address) and, on x86-64, a running thread can change it through `arch_prctl` with `ARCH_SET_FS` (the older `set_thread_area` serves the same purpose on 32-bit). Everything above that — how thread-local variables are laid out and reached in code — is handled by the compiler and the threading library. ## See also - [The process and thread model](/peios/using-peios/threads-and-processes/the-process-and-thread-model.md) — the conceptual counterpart to this page. - [Process creation reference](/peios/developing-for-peios/process-runtime-reference/process-creation-reference.md) — the clone flags that create threads and arrange TLS and clear-tid addresses.