# Peios System Protocols Userspace > The foundational userspace protocols — public, stable contracts between the components a Peios system is built from, which third parties may implement. --- # 1.1 Scope _Peios / Advanced Peios / PSPU / Introduction_ > What PSPU is — the protocols and interchange formats the foundational userspace components speak to each other — and why they are not conformance requirements. This document defines the **Peios System Protocols Userspace (PSPU)**: the protocols and interchange formats by which the foundational userspace components of a Peios system agree with one another. An interface belongs in this document when both of these hold: - it is a contract between userspace parties, at least one of which is a component the system is built from rather than an application running on it; and - the interface is public — a third party is expected to implement one side of it. The parties need not exist at the same moment. A live protocol has two processes in conversation; an interchange format has a producer and a consumer that never meet, and the artifact between them carries the contract. Both are in scope, because what makes something belong here is that two independently written parties must agree on it. ## 1.1.1 These protocols are not conformance requirements A system that does not offer a protocol in this document is still Peios. The components that speak these protocols are one answer to a problem, not the definition of the platform; a system that solves the same problem with different components conforms exactly as well. They are specified because they are *public* even so. A third party writing a component to plug into one side of one of these protocols needs the contract written down, and needs it to stay put. What they are not is a bar anyone must clear. For each interface, this document covers: - for a live protocol: the channel, its direction, which party connects to which, message framing and encoding, the messages exchanged, and the shape of a conversation - for an interchange format: the layout of the artifact, how it is identified and versioned, and how a consumer validates one it receives - how a party announces itself or is identified, and how its counterpart establishes what it is and what it may speak for - the rules under which the format may be extended - what each party must declare about itself, and what its counterpart validates rather than believes - the conformance requirements for each role This document does not cover: - Standards a system MUST implement to be Peios — defined in PGSS - Protocols spoken across the kernel boundary — defined in PSPK - The binary structures these interfaces carry — defined in PCDS - How a component stores its data, reaches the answers it gives, or produces the artifacts it emits — its own design - Which counterparts a system is configured to trust, and how that configuration is expressed — the consuming component's own design - Administering a component's contents — its own design The fourth of those is the point of the whole document. A component is asked a question and gives an answer, or is asked for an artifact and produces one; how it arrives there is exactly what different components exist to do differently. ## 1.1.2 Stability Publication here is a commitment that the contract is written down and will not change out from under an implementation. Each specification states its own rules for extending its wire or file format; those rules are the supported way for an interface to grow. --- # 1.2 Conventions _Peios / Advanced Peios / PSPU / Introduction_ > The normative keywords PSPU uses, and the shared conventions it inherits from the PCSA conventions book. The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as described in RFC 2119. Text set off as a note is informative, not normative. Everything else — roles, byte order, sizes, layout tables, notation, strings, timestamps, citation, and the external standards this anthology depends on — is defined in the Conventions book and is not restated here. PSPU departs from none of it. Where a chapter needs a convention of its own, that chapter states it. --- # 2.1 Scope and Roles _Peios / Advanced Peios / PSPU / Principal Source Interface_ > What PSI specifies — how an authentication authority federates identity to separate processes that hold it — and its two roles. This chapter specifies the **Principal Source Interface (PSI)**: the protocol by which an authentication authority federates identity to separate processes that hold it. Two roles participate. The **authority** is the process that mints tokens and creates logon sessions. It is the party asking. On PSI it listens; it never dials out (§2.3). It is also the party that speaks PGSS Logon to its clients, and PSI exists so that it can answer them. The **source**, in full a *principal source*, is a process that is authoritative for some set of principals: it verifies their credentials and says who they are. The source role is publicly implementable: any process an authority has been configured to accept MAY register as a source, and a third party writing one for a directory, a hardware token service or an identity provider is the case this chapter is written for. A conforming source is the subject of the source obligations in §2.21. A source is not a *store*, necessarily. A local source owns its bytes; a directory-backed source owns nothing and forwards the question. The interface deliberately does not distinguish them, which is why the term is "source" rather than "store". What a source is emphatically **not** is a component of the authority. It runs as a separate process, at lower trust, and it cannot mint anything (§2.4). This chapter covers: - the channel, its direction, and why sources connect inward (§2.3, §2.6) - message framing, the conversation identifier, and the rules under which the format may be extended (§2.7) - registration: how a source announces itself, how the authority establishes what it is, and what domain it may speak for (§2.8 to §2.10) - the relayed interrogation, and its relationship to PGSS Logon (§2.5, §2.12) - assertion and refusal, the terminal messages of a source conversation (§2.13) - querying a source outside a logon, so that an authority can serve PGSS Logon's identity lookup (§2.15, §2.16) - what a source must declare about itself before an authority may cache its answers (§2.8, §2.17) - scope: what a source may claim about identity, separately about membership, and separately again about POSIX identifiers (§2.18 to §2.20) - the obligations binding on each role (§2.21) This chapter does not cover: - The logon protocol itself, specified in PGSS. - Tokens, SIDs, sessions and privileges — described in the Peios Kernel TRM, with SIDs, security descriptors and claim attributes specified in PCDS. - How a source stores identity, or verifies a credential. - Derivation — what a token ends up containing — which is the authority's, applying local policy (PGSS §2.1). - Which sources a machine trusts, and how that is configured. - Which identifier range a source is given, and how that is configured. The *rules* the assignment must satisfy are §2.20. - Administering a source's contents. The third of those is the point of the whole interface. A source is asked a question and gives an answer; how it reaches the answer is exactly what different sources exist to do differently. ## 2.1.1 PSI is not a conformance requirement PGSS Logon is a Peios Generic System Standard: a system that does not offer it is not Peios. **PSI is not.** It is the interface an authority uses to reach the processes that know who exists, and a system running entirely different authentication infrastructure is still Peios. It is specified because it is a *public* interface even so. A third party writing a principal source needs the contract written down, and needs it to be stable. What it is not is a bar anyone must clear. > [!NOTE] > The distinction shows up in what happens when you disagree with each > document. Disagreeing with PGSS Logon means shipping something that is > not Peios. Disagreeing with this chapter means shipping an authority > that federates differently, or does not federate at all, which is a > design choice nobody will dispute. --- # 2.2 Terminology _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Terms this chapter borrows unchanged from the Peios Kernel TRM, PCDS and PGSS Logon. Terms defined in the Peios Kernel TRM (token, logon session, privilege), in PCDS (SID, security descriptor, claim attribute) and in PGSS (authority, client, principal, conversation, round, credential material, prompt, derivation) are used here with the same meaning and are not redefined. **Source.** A process that is authoritative for some set of principals: it verifies their credentials and says who they are. Called a *principal source* in full. **Registration.** The exchange in which a source announces itself and the authority decides whether to accept it. Precedes any conversation. **Domain.** The SID namespace a source is authoritative for. Every principal a source may assert lives under it. See §2.10. **Source conversation.** One logon's exchange between the authority and a source, distinguished from other concurrent ones by a conversation identifier. Not to be confused with a PGSS Logon conversation, which is between a client and the authority; one of each exists per logon. **Assertion.** A source's terminal message stating who a principal is. The only successful outcome a source can produce. **Originator.** The verified identity of the process that requested a logon, as established by the authority from the client's connection. Relayed to the source, which cannot learn it any other way. **Service SID.** A SID derived from a service's name, placed in that service's token by the init system, and unforgeable by anything else. How a source's identity is established (§2.9). **Membership scope.** The constraint on which groups a source may assert. Separate from **identity scope**, which constrains whose identity it may assert at all. Sections 2.18 to 2.20 exist because these are different questions with different answers. **Relative identifier.** In this chapter, a POSIX identifier as a source states it: an offset within the range the authority assigned that source, never an absolute number (§2.20). Where the SID sense is meant — the last sub-authority of a SID — the text says so. --- # 2.3 Sources Dial In _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The authority never dials a source; sources connect inward. The most consequential shape decision in the chapter, and what it buys. **The authority listens. Sources connect to it.** The authority never initiates a connection to a source. This is the most consequential shape decision in this chapter, and it is worth being explicit about what it buys. ## 2.3.1 Why the direction matters The authority holds the privilege to mint tokens. It is the most privileged userspace process on the system. An authority that dialled out would need, in its configuration, a list of paths to connect to — and a process holding that privilege having a configurable list of things to go and talk to is a liability out of proportion to the convenience. Because sources connect inward, the authority's sockets are `accept()`-only. It never opens an outbound connection to anything, for any reason. ## 2.3.2 What follows **Restart is the source's problem.** A source whose connection drops reconnects. The authority does not retry, does not queue, and does not track sources it has not heard from. A source that has gone away is simply not registered. **A source is not required to exist.** An authority with no registered sources cannot authenticate anybody, and that is a coherent state rather than an error — it means no identity has been made available to it yet. **Ordering is the init system's problem.** The authority must be listening before a source can register, and a service that depends on authentication must start after a source has. Expressing that is a service-ordering question, not a protocol one, and this chapter says nothing about it. > [!NOTE] > A source SHOULD report itself ready only once its registration has > been *acknowledged*, so that "the source is running" and "the source > can authenticate" are the same statement to anything ordered after it. > This is what makes the `Registered` acknowledgement load-bearing > despite carrying almost nothing. --- # 2.4 Assert, Never Mint _Peios / Advanced Peios / PSPU / Principal Source Interface_ > A source says who somebody is and nothing else — the structural reason a compromised source cannot grant itself privilege. A source says **who somebody is**. It cannot say anything else, and the protocol is built so that this is structural rather than a rule anyone has to remember. ## 2.4.1 The success terminal PGSS Logon's success terminal is `AccessGranted`, carrying a session identifier and a token descriptor. If PSI reused it, sources would be minting sessions. PSI's success terminal is `Assertion` (§2.13), which carries an identity: a SID, a canonical name, and group memberships. **There is no session identifier and no descriptor to attach a token to.** A source could not mint one if it wanted to, because there is no message in which to say so. That is the whole of the mechanism. No capability check, no trust level, no configuration flag — a source cannot mint because the protocol gives it no way to express minting. ## 2.4.2 What the authority keeps Everything else: - **Derivation.** What the token actually contains — its privileges, its integrity level, its derived group memberships, its projected identifiers — is the authority's, applying local policy (PGSS §2.1). - **Session creation.** The logon session, and the record of which source vouched for it. - **Validation.** Every SID a source sends is bytes until the authority has checked it (§2.13). - **Scope enforcement.** What a source is permitted to claim (§2.18 to §2.20). - **Peer verification.** On every connection it accepts. - **Rate and round limits**, and the policing of what a source may ask a client for (§2.12). ## 2.4.3 Why a compromised source is bounded A source that is entirely compromised can lie about the principals in its own domain. It cannot mint a token, cannot elevate anyone's privileges, cannot claim identities outside its domain (§2.18), and — unless configured otherwise — cannot assert memberships outside it either (§2.19). That bound is the reason for the process boundary. It is not that sources are expected to be malicious; it is that a source is the component parsing credentials from the outside world, and therefore the one most likely to be wrong. --- # 2.5 A Superset of PGSS Logon _Peios / Advanced Peios / PSPU / Principal Source Interface_ > A source is an authority for its own slice of the world, which is why most of PSI is PGSS Logon — and where the two deliberately diverge. A principal source *is* an authentication authority for its slice of the world. The authority that federates is an authority over authorities. Once that is seen, most of PSI writes itself. ## 2.5.1 The relationship PSI's interrogation phase is **PGSS Logon's, with identical message bodies**. `CredentialRequest` and `CredentialResponse` carry exactly the bytes PGSS §2.8 defines, and the authority relays them nearly verbatim in both directions. The consequences are worth stating plainly: - **The source decides what to ask for.** Not the authority. The authority does not know what credentials a source requires, and does not need to. - **The authority becomes a relay** in the interrogation phase. It is a shorter path than synthesising its own prompts, not a longer one. - **Adding a credential type is a change to sources**, not to the authority and not to clients, which already render what they are given (PGSS §2.3). - **A source could be tested in isolation** by pointing a PGSS Logon client at it, for the interrogation phase at least. ## 2.5.2 Where they diverge, deliberately Three differences, each for a stated reason. **The success terminal.** `Assertion` rather than `AccessGranted`, so that a source cannot mint. See §2.4. This is the divergence that matters. **Multiplexing.** PGSS Logon is one conversation per connection; the connection *is* the conversation. PSI carries many concurrent logons over one long-lived connection, so its header adds a conversation identifier (§2.7). The alternative — serialising every logon behind one connection — would make any slow logon a system-wide login stall. **Distinct magic.** `PPSI` rather than `PGSL`. Two protocols this similar sharing a codec is a cross-protocol hazard: a socket plugged into the wrong daemon would *partially* work, which is far worse than failing outright. The magic makes it a hard error on byte zero. The full accounting of what is shared, what is added and what differs is §2.C. ## 2.5.3 "Just relaying" is loose The authority is a relay in the interrogation phase only, and even there it is not passive. It polices what a source may ask a client for (§2.12), it validates what a source asserts (§2.13), and it enforces scope (§2.18 to §2.20). Everything before and after the interrogation is entirely its own. --- # 2.6 The Channel _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The PSI socket, its access control, why connections are long-lived, and what happens when one fails. ## 2.6.1 Socket An authority that federates over PSI MUST listen on a `SOCK_STREAM` Unix domain socket. Unlike PGSS Logon's path, this one is **not normative**. PSI is not a conformance requirement (§2.1), and an authority that offers it may put it where it likes provided its sources are told. Mainline's is `/run/psi.sock`. ## 2.6.2 Access control The socket SHOULD carry a security descriptor. It is **DoS protection and nothing more**, and an implementation MUST be written as though it were absent. The reason is that the socket cannot be the boundary. What establishes a source's identity is the peer's token (§2.9), which is checked on every connection. A descriptor that kept casual traffic away would be a convenience; a descriptor *relied upon* would be a second, weaker access control that someone will eventually assume is doing the work. An authority MUST therefore bound the number of unregistered connections it will hold open, and the time it will wait for a registration, independently of any descriptor. > [!NOTE] > An unauthorised peer is refused at the identity check, which is cheap, > and the connection cap bounds what it can occupy in the meantime. That > is the whole of what the descriptor's absence costs. ## 2.6.3 Long-lived connections A source's connection persists for the life of the source and carries every logon routed to it. An authority MUST bound the number of registered sources and the number of concurrent conversations per source. A source MUST bound the conversations it will track, and MUST NOT depend on the authority's bookkeeping to do it — a source that trusted the authority's limit would be trusting a bound it cannot verify. ## 2.6.4 Failure A framing error is **fatal to the connection**, not to a conversation. Once a message has failed to parse there is no way to know where the next one starts, so both parties MUST tear the connection down rather than attempt resynchronisation. A failed write is likewise fatal. A partial write desynchronises the stream just as a bad frame does, and treating it as a per-conversation error would leave a corrupt connection in use. Ordinary semantic failures — an unknown principal, a bad credential, a refused logon — are **not** connection failures. They are `Refusal` messages (§2.13) and the connection continues. --- # 2.7 Message Framing _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The 20-byte header — the first twelve bytes are PGSS Logon's, unchanged — plus conversation identifiers, size limits and SIDs on the wire. ## 2.7.1 Header Every message begins with a 20-byte header: | Offset | Size | Field | Value | |---|---|---|---| | 0 | 4 | `magic` | `PPSI` (`50 50 53 49`) | | 4 | 2 | `version` | `1` | | 6 | 2 | `msg_type` | See §2.A | | 8 | 4 | `total_len` | Header plus body, in bytes | | 12 | 8 | `conversation` | See below | The first twelve bytes are PGSS Logon's header, unchanged and at the same offsets. `total_len` in particular sits where PGSS §2.6 puts it, which is what lets one transport implementation frame either protocol off a stream. ## 2.7.2 Magic `PPSI`, checked on every message, fatal to the connection when wrong. This matters more here than it would for an unrelated protocol, because PSI and PGSS Logon **share message bodies** (§2.5). A `CredentialRequest` from one is byte-identical to the other's. Without distinct magic, a socket plugged into the wrong daemon would decode several fields correctly before going wrong — which is the failure mode hardest to diagnose and easiest to miss. ## 2.7.3 Conversation identifier The `conversation` field distinguishes concurrent logons on one connection. - **Conversation `0` is reserved** for connection-level messages: `Register`, `Registered` and `Changed` (§2.17). It MUST NOT be used for a logon or a query. - Logon and query conversations use identifiers from 1 upward, drawn from one space. - The **authority allocates** them. A source MUST NOT invent one, and MUST reply on the identifier it was given. - An identifier is unique among *live* conversations on one connection. An authority MAY reuse one after a conversation has reached a terminal state. A source MUST reject a message on a conversation it does not know, and MUST NOT treat it as opening a new one. Only `Authenticate` (§2.11), `Query` (§2.15) and `EnumerateSource` (§2.16) open one, and an authority MUST NOT open one with an identifier already live. Rejecting means declining to act on it. A source MUST NOT reply on a conversation it does not know: an authority MAY have reused the identifier after a terminal state, so a reply could arrive as a second terminal message for a conversation that has already ended. Discarding it, and recording that it happened, is the whole of the obligation. A source MUST likewise refuse an `Authenticate`, `Query` or `EnumerateSource` arriving on conversation `0`, which is reserved. ## 2.7.4 Size limit A message MUST NOT exceed **81920 bytes** — larger than PGSS Logon's ceiling, because a PSI message wraps one. ## 2.7.5 Message direction The high bit of `msg_type` marks a message sent by **the source**. This follows PGSS Logon's convention that the bit marks the authority for the matter at hand: on this interface the source is the authority for its own principals, and the logon authority is the one asking. ## 2.7.6 Encoding PSI shares its codec with PGSS Logon, and PGSS §2.6's encoding rules apply here unchanged: little-endian multi-byte integers; UTF-8 strings, length-prefixed and never NUL-terminated; length-framed structures and array elements, skipped to their declared end; `u32` element counts on arrays, with stated maxima binding on encoder and decoder alike. ### 2.7.6.1 SIDs on the wire SIDs are carried as **opaque bytes**, in the binary self-relative form PCDS specifies, never as text. They are opaque *to the codec*, which has no business knowing what a SID is. They are emphatically not opaque to the authority, which MUST validate every SID it receives before treating it as identity (§2.13). The obligation to check sits in the process that mints tokens, not in the layer that moves bytes. A SID MUST NOT exceed 68 bytes — the eight-byte prelude plus fifteen sub-authorities, which is the most the encoding's one-byte count admits. ## 2.7.7 Body extensibility The extensibility rules of PGSS §2.6 apply unchanged: fields are appended only, a new field is optional with a safe default, and a new enumeration value is a breaking change requiring a version bump. The one exception is the capability bitmask of §2.8, for the reason given in §2.B. One PSI-specific application deserves stating. `Authenticate` (§2.11) **nests** a whole `LogonStart` inside its own length frame rather than inlining its fields. The obvious encoding — `LogonStart`'s fields, then PSI's — is wrong: `LogonStart` belongs to PGSS Logon and grows on PGSS Logon's schedule, so a field appended there would silently displace the field after it. Nesting lets the two evolve independently. The same reasoning applies to every shared body PSI carries, and §2.C lists them. --- # 2.8 Registration _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The first message on every connection — the source's name, domain, capabilities, TTL, batch limit and the identifier range it is granted. A connection opens with registration. Nothing else may precede it. ## 2.8.1 Register `msg_type` = `0x8001`. Source to authority, on conversation `0`. | Field | Encoding | Limit | |---|---|---| | `source_name` | string | 32 bytes | | `domain` | length-framed bytes (SID) | 68 bytes | | `capabilities` | `u32` | §2.B | | `entry_ttl` | `u32` | seconds | | `max_batch` | `u32` | 64 | ### 2.8.1.1 source_name What the source calls itself. Bounded at 32 bytes because it becomes the authentication-package name on every session the source authenticates — so a token's provenance answers *which source vouched for this?* rather than merely *the authority minted it*. The name is a **claim**. It MUST be cross-checked against the identity the authority established for itself (§2.9), and it MUST NOT be used for anything else. A mismatch MUST be refused rather than quietly corrected: a service registering under another's name is worth failing on, not normalising. ### 2.8.1.2 domain The SID namespace this source is authoritative for (§2.10). ### 2.8.1.3 capabilities What the source can do beyond authenticating. | Bit | Name | Meaning | |---|---|---| | 0 | `QUERIES` | Answers `Query` (§2.15). | | 1 | `ENUMERATES` | Answers `EnumerateSource` (§2.16). | | 2 | `MEMBERS` | Can produce a group's membership. | | 3 | `PUSHES_CHANGES` | Sends `Changed` (§2.17). | An authority MUST NOT send a message a source did not declare it answers, and MUST NOT set a field bit gating a capability the source did not declare. A source declaring nothing authenticates and does nothing else, which is what a source predating these fields is saying by omission — and is the only reading that keeps such a source working. A source that declares no `QUERIES` cannot be asked about its principals outside a logon, so they are unresolvable through PGSS Logon's identity channel: they can sign in and will appear as bare numbers everywhere else. That is a coherent configuration and this chapter permits it, but it is almost never what an administrator intended, and an authority SHOULD report it where one will see it — as it reports a source registering with no identifier range. This is a declaration of capability, not of willingness. A source declaring `QUERIES` may still answer `Refused` to any particular question (§2.15); a source declaring `ENUMERATES` may still refuse a cursor it can no longer honour (§2.16). ### 2.8.1.4 entry_ttl How long, in seconds, the authority may hold an answer from this source before asking again. **Zero means do not cache.** A source declaring neither `PUSHES_CHANGES` nor a non-zero `entry_ttl` has said its answers must not be held at all, and an authority MUST honour that — see §2.17, where the reasoning for reading silence that way is set out. `entry_ttl` and `PUSHES_CHANGES` are not exclusive, and a source declaring the second SHOULD declare a non-zero first as well. See §2.17: the TTL is the backstop against a notification that was never sent. ### 2.8.1.5 max_batch The largest number of keys the source will accept in one `Query` (§2.15). Zero means one. An authority MUST NOT exceed it, and MUST NOT send more than 64 keys whatever the source declared: an encoder MUST NOT declare more, and a decoder MUST read a larger declaration as 64. A source MUST still validate what it receives. The field is a hint the authority is required to respect, not a guarantee about what will arrive. ## 2.8.2 Registered `msg_type` = `0x0001`. Authority to source, on conversation `0`. | Field | Encoding | |---|---| | `unix_id_base` | `u32` | | `unix_id_count` | `u32` | Load-bearing beyond its contents. A source SHOULD report itself ready only once it has received this, so that anything ordered after the source finds a system that can actually authenticate rather than merely a process that exists (§2.3). ### 2.8.2.1 unix_id_base, unix_id_count The POSIX identifier range the authority has assigned this source (§2.20). A base of **0** means no range was assigned, and every principal the source asserts will project as unmapped. **Informational.** A source counts within its range and asserts relative identifiers; the authority applies the base. A source MUST NOT apply it — see §2.20, where the reasoning is set out in full. It is sent so that a source's administration tools can show an operator the identifier a principal will really project to, rather than the relative number the source stores. Without it that arithmetic falls to the operator. An authority that predates these fields sends neither, and a source MUST read their absence as *no range assigned* — which is what such an authority means, since it has no ranges to assign. ## 2.8.3 Rules 1. A connection MUST open with `Register` on conversation `0`. An authority MUST refuse any connection that opens otherwise. 2. An authority MUST bound the time it waits for the opening `Register`, and close the connection on expiry. A peer that connects and says nothing MUST NOT be able to hold resources indefinitely. 3. An authority MUST NOT admit two sources under one name at the same time. 4. An authority MUST send `Registered` only after the source is routable, so that a logon racing the acknowledgement cannot find a source that is registered but not yet reachable. 5. A source MUST NOT send any other message before receiving `Registered`. 6. An authority SHOULD report, where an administrator will see it, that a source registered with no identifier range — its principals will all project as unmapped, and the cause is a configuration omission rather than anything the source did. --- # 2.9 Establishing What a Source Is _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The authority establishes a connecting source's identity from the kernel, never from what the source says, and checks it against an allowlist. The authority MUST establish a connecting source's identity **for itself**, from the kernel, and MUST NOT take it from anything the source sends. ## 2.9.1 A source proves nothing The mechanism worth recommending is that a source proves nothing at all, because the init system already did. Where the init system places a **service SID** in each service's token — a SID derived from the service's name, which only the init system can mint — the authority can: 1. take its list of permitted source names from its own configuration; 2. derive the service SID each of those names implies; 3. read the connecting peer's token and ask which of those SIDs it carries. The resulting identity is assembled entirely from the authority's configuration and the kernel. **Nothing is contributed by the process on the other end.** There is no shared secret, nothing to provision, nothing to rotate, and nothing to steal — the derivation is a pure function of a name that only the init system can act on. > [!NOTE] > Mainline derives `S-1-5-80- name>`, matching what the init system places in every service token. > The derivation exists so that platform services — all running as > SYSTEM — remain distinguishable from one another, which is exactly > this problem. ## 2.9.2 What is deliberately not checked **That the peer is SYSTEM.** The service SID subsumes it: the user SID could never distinguish one platform service from another, since they all run as the same principal. Requiring SYSTEM as well would needlessly forbid a future source running under a lesser account, which is a direction worth keeping open. ## 2.9.3 The allowlist An authority MUST NOT accept a source it has not been configured to accept. An empty configuration MUST mean **no source may register**, not *any source may*. An allowlist that fails open is not an allowlist. The visible cost is that a system configured with no sources cannot authenticate anyone, which is the correct way for that mistake to present — loudly, at the first logon attempt, rather than silently at the first compromise. > [!NOTE] > Mainline's allowlist is a registry key whose *subkey names* are the > permitted source names, so enumerating that key answers "what may > assert identity on this machine?" exactly, with no second list > anywhere to drift out of step. --- # 2.10 The Domain Claim _Peios / Advanced Peios / PSPU / Principal Source Interface_ > A source declares the domain it is authoritative for — how the claim is checked, why domains must be disjoint, and what it does not prove. A source declares the domain it is authoritative for. Every principal it may assert lives under it (§2.18). ## 2.10.1 It is a claim A source generates or is given its own domain, so **nothing about the SID can prove the claim is honest**. What gives it weight is entirely what the authority does with it. An authority MUST apply all of the following. ### 2.10.1.1 1. A domain MUST be declared A source that declares no domain MUST be refused. There would be nothing to confine its assertions to, which is the whole purpose of collecting one. > [!NOTE] > The field is appended to a message that existed before it, so a source > built against an older shape decodes as declaring *nothing* rather > than as malformed. That is why the refusal is stated as its own rule: > the diagnosis should name the real problem. ### 2.10.1.2 2. The shape MUST be checked A domain MUST be a well-formed, locally-issued domain SID: revision 1, the NT authority (`5`), the non-unique prefix `21`, and three further sub-authorities — `S-1-5-21-A-B-C`, four sub-authorities in total. The shape is the entire check, and it is enough. A source cannot claim `S-1-5-32` (BUILTIN), or the NT authority's well-known range, or `S-1-1-0`, because none of them has that shape. **There is no list of forbidden domains to keep in step with the SID catalogue** — the permitted shape excludes every one of them by construction. ### 2.10.1.3 3. Domains MUST be disjoint No two concurrently registered sources may claim the same domain. Two authorities for one namespace means whichever answers first decides who a name belongs to, and the other's principals become impersonable by the first. ### 2.10.1.4 4. A source MUST NOT change domain A source that re-registers MUST declare what it declared before. An authority MUST refuse a change. This is the check that survives a source restarting, and it is worth its cost: every other check passes for a source that is killed and comes back compromised. It still holds the right service SID, its new domain is still a claimable shape, and with itself deregistered there is nothing left to collide with. An authority MAY hold this record only for its own lifetime. Persisting it means the authority writing state, which is a larger commitment than the guarantee justifies. ### 2.10.1.5 5. An administrator MAY pin An authority SHOULD allow an administrator to configure the exact domain a named source must declare, and MUST refuse a source declaring anything else when one is configured. A configured pin that cannot be parsed MUST NOT be treated as absent. Absence means *no pin*; an unparseable value means an administrator tried to apply the control and got it wrong, and silently downgrading that to "unconstrained" removes the control at the moment it was being applied. An authority MUST fail towards refusing the source. ## 2.10.2 What remains uncovered With no pin configured, and another source not currently registered, a compromised source could declare *that* source's domain and assert its identities. Disjointness catches it only while both are registered. This is stated rather than solved. Closing it requires someone to write the pin down, and an authority cannot invent that authority for itself — writing it automatically would mean the process holding the token-minting privilege also holding a configuration write handle, which is a worse trade than the gap it closes. --- # 2.11 Authenticate _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The client's LogonStart nested whole and routed to a source, with the conversation limits that bound what follows. `msg_type` = `0x0002`. Authority to source. **Opens a conversation.** | Field | Encoding | Limit | |---|---|---| | `start` | nested `LogonStart`, length-framed | PGSS §2.7 | | `originator` | length-framed bytes (SID) | 68 bytes | ## 2.11.1 start The client's `LogonStart`, nested whole (§2.7). Its fields and their meanings are PGSS §2.7's, unchanged — including that `identifier` is an unverified claim and `supported_credential_types` binds what may be prompted for. ## 2.11.2 originator The **verified** identity of the process that requested this logon, taken by the authority from the client's connected socket and never from a message body. A source cannot learn this for itself: it is not party to the client's connection, and there is nothing it could ask. The authority relays it because a source may legitimately refuse a logon on the strength of it — an account restricted to console logons needs to know what asked — and that decision needs a trustworthy input. A source MUST treat `originator` as established fact and MUST NOT treat any other field of this message the same way. ## 2.11.3 Routing Before sending `Authenticate`, an authority MUST decide **which single source** answers. The credential MUST NOT be offered to more than one source. Trying each in turn *with the password* hands every source the credentials of every other source's users, including on typos — the failure PAM stacking exemplifies (§2.D). Resolution therefore happens on the identifier, before any credential exists. Asking several sources "do you own this name?" is a resolution step with no secret in it and is permitted; offering them the answer is not. An authority SHOULD resolve a qualified name to its owning source and MUST NOT fall back to another source when the owning one is unreachable. A name that can fall through lets anyone who can break a network choose which authority answers for a principal. ## 2.11.4 Conversation limits An authority MUST bound the conversations it opens against one source. A source MUST bound what it will track, and MUST refuse beyond its own limit with `AuthorityUnavailable` (§2.13) rather than dropping the conversation silently. --- # 2.12 The Relayed Interrogation _Peios / Advanced Peios / PSPU / Principal Source Interface_ > CredentialRequest and CredentialResponse carrying PGSS Logon bodies verbatim — what the authority polices on the way through, and what it never inspects. Two messages, both carrying PGSS Logon bodies verbatim. ## 2.12.1 CredentialRequest `msg_type` = `0x8002`. Source to authority. Body is PGSS §2.8's `CredentialRequest`, byte-for-byte. The source decides what to ask for, in what order, and over how many rounds. The authority relays it to the client. ## 2.12.2 CredentialResponse `msg_type` = `0x0003`. Authority to source. Body is PGSS §2.8's `CredentialResponse`, byte-for-byte. ## 2.12.3 What the authority polices The authority relays, but does not relay *anything*. **An authority MUST refuse to relay a prompt whose credential type is absent from the client's `supported_credential_types`.** PGSS §2.8 makes this the authority's obligation towards the client, and it holds however the authority reached the prompt — a prompt originating in a source is still the authority's to police. Relaying it would force the client to hard-fail, and a client that guessed instead might echo a secret to the screen. The authority MUST terminate the logon instead. An authority MUST also enforce its own round and time limits on the relayed exchange (PGSS §2.3), independently of any the source applies. A source that never terminates a conversation MUST NOT be able to hold a client's logon open indefinitely. ## 2.12.4 Credential handling The obligations of PGSS §2.12 bind both parties on this leg as they do on the client's. Credential material reaching a source has been decoded and re-encoded once more than it would have been without federation, and every buffer it passed through on the way is one the obligation covers. ## 2.12.5 What the authority does not do It does not interpret prompts, rewrite messages, reorder anything, or synthesise a request of its own. A source's prompt reaches the client as the source wrote it, which is the property that makes adding a credential type a change to sources alone. --- # 2.13 Assertion and Refusal _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The two terminal messages a source may send — and what an assertion pointedly does not contain: no session, no token, no privileges. Exactly one terminal message ends a source conversation. ## 2.13.1 Assertion `msg_type` = `0x8003`. Source to authority. **The only successful outcome a source can produce.** | Field | Encoding | Limit | |---|---|---| | `user_sid` | length-framed bytes (SID) | 68 bytes | | `canonical_name` | string | 256 bytes | | `groups` | array of group entries | 128 | | `unix_id` | `u32` | §2.20 | | `primary_group` | length-framed bytes (SID) | 68 bytes | | `profile` | length-framed structure (PGSS §2.9) | | | `claims` | array of claim entries | 64 | Note what is absent: no session, no token, no privileges, no integrity level. A source has no way to express them (§2.4). Every field after `groups` is optional in the way §2.7 requires: a source that does not write one has said nothing about it, and the authority substitutes the default named below rather than failing. ### 2.13.1.1 canonical_name The source's own spelling of the principal's name. A client may have typed `JACK`; this is what the principal is actually called. Carrying it is what makes case-insensitive matching safe: the authority records the canonical form rather than whatever was typed, so a session's records do not vary with a caller's shift key. A source MUST NOT assert a name that PGSS §2.15 forbids — one carrying a reserved character, a byte outside the printable ASCII range, or a leading or trailing space. The authority MUST refuse one anyway (§2.21), because a name from a source reaches a `passwd`-format record and an audit line, and by then the damage is the reader's to do. The obligation is on what a source **asserts**, not on what it creates. A source that validates a name when an administrator adds it, and not when it reads one back from storage, has enforced nothing against a store it did not itself write. ### 2.13.1.2 groups Each entry is a separate length-framed structure: | Field | Encoding | Limit | |---|---|---| | `sid` | length-framed bytes (SID) | 68 bytes | | `unix_id` | `u32` | §2.20 | **A SID and a number. No attributes.** A source asserts *which* groups a principal belongs to. Whether a group entry is enabled, owner-marked, or deny-only is a decision about how to build a token, and building tokens is the authority's (§2.4). A source saying "this principal is an administrator" is identity; a source saying "and mark that group deny-only" would be reaching into derivation. The per-entry framing is what allowed `unix_id` to be added here without breaking a decoder that predates it, and it will allow the next field the same way. A `unix_id` of **0** means the source does not number this group — the honest answer for a group it does not own. A source naming a well-known group is stating a membership, not claiming authority over what that group projects to; see §2.20. ### 2.13.1.3 unix_id The principal's POSIX identifier, **relative to the range the authority assigned this source** (§2.20). Zero means the source has no number for this principal. A source MUST NOT apply its own base. It counts within its range and the authority rebases; a source that added the base itself would have it added twice. ### 2.13.1.4 primary_group Which of the principal's groups projects to the POSIX group id, and becomes the default group of objects the token creates. Empty means the source did not say, and the authority chooses. It need not appear in `groups`. The authority is required to place it on the token regardless (§2.21), because a token's primary group must be a group the token carries — so naming a group here **is** a membership claim, and it is subject to membership scope exactly as a listed group is (§2.19). That applies to a primary group the **source asserted**. Where the field is empty and the authority substitutes one of its own, the substituted value is the authority's choice and MUST NOT be tested against the source's membership scope. Testing it would deny every logon from a source that declined to name a primary group, on the strength of a claim that source never made — and an authority's own default is very unlikely to be a sibling of the principal's domain, so the test all but always fails. > [!NOTE] > A source that made this the one field escaping the scope check would > have found a route to `BUILTIN\Administrators` that the group array > denies it. The obligation in §2.21 to apply membership scope to the > primary group closes that. ### 2.13.1.5 profile PGSS Logon's profile structure (PGSS §2.9), relayed onward to the client unchanged. It is not identity, it decides no access, and the authority does not interpret it. The one thing the authority does check is the one PGSS §2.9 requires of it: `home` and `shell`, when non-empty, are absolute paths. That obligation binds the authority towards its client whatever the value's provenance, so a relayed profile is not exempt from it. ### 2.13.1.6 claims Named, typed attributes fed to conditional ACE evaluation, in the claim attribute format PCDS §5.9 specifies. Each entry is a separate length-framed structure: | Field | Encoding | Limit | |---|---|---| | `name` | string | 255 bytes | | `flags` | `u32` | PCDS §5.9 | | `value_type` | `u32` | PCDS §5.9 | | `values` | array of length-framed values | 64 | A claim is the one field here that is a **trusted input to access decisions** rather than a statement of identity: a conditional ACE can turn a claim into a grant. Which claim names a source may assert is therefore the same kind of question as which groups it may assert, and belongs with membership scope (§2.19). An authority MUST reject an assertion carrying a claim it cannot carry to a token — an unsupported value type, a name containing an interior NUL, a value exceeding its limit — rather than dropping the claim. The reasoning is rule 3 below: a dropped claim signs the principal in against a policy nobody stated. ### 2.13.1.7 What the authority MUST do with an assertion 1. **Validate every SID** with a structural check before treating it as identity. Bytes from another process are bytes until checked, and the check belongs in the process that mints tokens rather than in the codec that moved them (§2.7). This includes SIDs carried *inside* a claim value. 2. **Enforce identity scope** (§2.18), **membership scope** (§2.19) — including over `primary_group` — and **numeric scope** (§2.20). 3. **Fail the logon** on a malformed group SID, an unusable claim, an invalid `canonical_name`, or a `unix_id` outside the source's range, rather than dropping it. Dropping would sign the principal in with authority the source did not state — a confusing way to be wrong at best, and for a claim, a silent change of the policy that will be applied to them. A source that cannot encode a SID is broken. 4. **Drop a logon SID from the asserted groups**, loudly. No source is authoritative for one: the kernel mints them per session, and this session's did not exist when the source answered. A source asserting one is either buggy or reaching for a *different* session's SID, which would forge membership of somebody else's logon. 5. **Drop duplicates**, first mention winning. A source asserting a group the authority also derives is redundant, not wrong. Rules 4 and 5 drop rather than refuse because the token still ends up correct, and refusing would punish a principal for a source's defect without making anything safer. Rule 3 refuses because the token would *not* end up correct. **Rules 4 and 5 run before rule 2.** A logon SID is by construction outside the principal's domain, so an authority that applied membership scope first would refuse the logon that rule 4 says to survive by dropping. Duplicates are the same shape of problem. The drops are about what a source should never have sent; the scope tests are about what it is permitted to claim, and only what survives the first is subject to the second. An authority MUST also place `primary_group` on the token even when the source did not list it among `groups`, since a token's primary group must be a group the token carries. ## 2.13.2 Refusal `msg_type` = `0x8004`. Source to authority. | Field | Encoding | Limit | |---|---|---| | `denial` | `u32` | PGSS §2.B | | `reason` | string | 512 bytes | Reuses PGSS Logon's denial vocabulary rather than inventing a parallel one, so that relaying a refusal outward needs no lossy translation. A source MUST NOT distinguish an unknown principal from a bad credential — by code, by reason, or by timing (PGSS §2.10, §2.12). The obligation is the source's here, because the source is where the distinction exists to be leaked. > [!NOTE] > A principal requiring no credential is a separate matter, and cannot > be hidden: a conversation that reaches `Assertion` with no round at > all says that the named principal exists and needs nothing. That is > inherent in permitting a zero-round logon rather than a defect in this > rule, and it is the reason a passwordless principal is a > configuration decision rather than a convenience. An authority MAY relay `reason` to the client and MAY replace it. It MUST NOT relay a reason that reveals a distinction the source was required not to make. --- # 2.14 Abandon _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Telling a source a conversation will not continue, and why the protocol would leak state without it. `msg_type` = `0x0004`. Authority to source. No fields beyond the header. Tells a source that a conversation will not continue: the client hung up, a limit was reached, or the authority terminated the logon for reasons of its own. ## 2.14.1 Why it exists Without it, a client that disconnects mid-prompt leaves the source holding conversation state forever. A source cannot detect this for itself — it is not party to the client's connection — so the authority has to say. ## 2.14.2 Rules 1. An authority MUST send `Abandon` for any conversation it opened that will not reach a terminal state, unless the connection itself is being torn down. 2. A source MUST discard all state for the conversation on receipt, and MUST NOT reply. 3. `Abandon` is not a terminal message *from* the source, and no `Assertion` or `Refusal` follows it. If one arrives anyway, the authority MUST ignore it — the conversation is gone, and a late answer to an abandoned question is at best stale. 4. A source that receives `Abandon` for a conversation it does not know MUST ignore it. It has already cleaned up, which is the outcome the message wanted. > [!NOTE] > The natural implementation is to send `Abandon` from whatever owns the > conversation's lifetime, on any path that does not reach a terminal > state — including error paths. An authority that sends it only on the > tidy path will leak conversations exactly when things are already > going wrong. --- # 2.15 Query _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Asking a source about a principal outside a logon — batching, relative identifiers, and the outcomes a query can return. An authority serving PGSS Logon's identity lookup must be able to ask a source about a principal outside a logon: to render a name for a SID, or a POSIX record for a number. A query is a conversation like any other. The authority allocates the identifier, `Query` opens it, and one terminal message closes it. ## 2.15.1 Query `msg_type` = `0x0005`. Authority to source. Opens a conversation. | Field | Encoding | Limit | |---|---|---| | `fields` | `u32` | PGSS §2.B | | `keys` | array of key entries | 64 | A key entry is a length-framed structure: | Field | Encoding | Limit | |---|---|---| | `key_type` | `u8` | §2.B | | `name` | string | 256 bytes | | `sid` | length-framed bytes (SID) | 68 bytes | | `relative_id` | `u32` | | | `kind` | `u8` | PGSS §2.B | Exactly one of `name`, `sid` and `relative_id` is meaningful, selected by `key_type`. An encoder MUST leave the others empty or zero. ### 2.15.1.1 Keys are never absolute | Value | Name | |---|---| | 1 | `Name` | | 2 | `Sid` | | 3 | `RelativeId` | **There is no key type carrying an absolute POSIX identifier**, and this is the load-bearing property of the message. PGSS Logon's lookup accepts one, because that is what `getpwuid` hands a name resolver. The authority resolves it: it locates the range containing the number, subtracts the base, and asks the owning source by relative identifier. A source is therefore never asked an absolute number, exactly as it never asserts one during a logon (§2.20). The reason is not that an absolute number would let a source escape its range — it could not, because the authority refuses a relative identifier at or past the count before adding anything to it (§2.20). The reason is that the arithmetic must exist in exactly one place. A source asked an absolute number would have to subtract its own base to answer, which is the operation §2.20 forbids it, and an authority that asked would have taught it that its stored numbers and the system's are the same numbers. Every subsequent bug in that source would be an off-by-a-base. > [!NOTE] > This is also why the authority is the only party that *can* answer a > `getpwuid`. See PGSS §2.13. ### 2.15.1.2 Batching `keys` is an array so that an authority may ask several questions in one exchange. An authority MAY send a single key, and one that always does is conforming. The array is here from the outset because adding it later would break every source written against a single-key message. A source with a cheap local store gains little; a source backed by a remote directory gains the difference between one query and a hundred. A source MUST answer every key it is sent, in order, and MUST NOT reorder, merge or omit results. A source that cannot serve a whole batch MUST refuse the conversation rather than answer part of it. ## 2.15.2 QueryResult `msg_type` = `0x8005`. Source to authority. Terminal. | Field | Encoding | Limit | |---|---|---| | `results` | array of result entries | 64 | One result per key, in the order the keys were sent. A result entry is a length-framed structure: | Field | Encoding | Limit | |---|---|---| | `outcome` | `u8` | §2.B | | `sid` | length-framed bytes (SID) | 68 bytes | | `canonical_name` | string | 256 bytes | | `kind` | `u8` | PGSS §2.B | | `present` | `u32` | PGSS §2.B | | `withheld` | array of withheld entries | 32 | | `values` | array of length-framed values | 32 | Everything from `present` onward is PGSS §2.16's structure, unchanged — the same reuse of message bodies as the interrogation phase (§2.5), and for the same reason: an authority relaying a source's answer outward should not have to translate it. Where `outcome` is not `Found`, everything after it MUST be empty or zero. ### 2.15.2.1 canonical_name, not qualified A source returns **its own spelling** of the name, as it does in an `Assertion` (§2.13). It does not qualify it. Qualification names which source answered, and a source cannot know what it is called in another authority's search order. PGSS Logon requires a qualified name on the way out (PGSS §2.15); producing it is the authority's. ### 2.15.2.2 Identifiers are relative Every identifier in a result — a `UNIX_ID` field, a reference's `unix_id` — is **relative**, exactly as in an `Assertion` (§2.20). A source MUST NOT apply a base, and the authority rebases before the number leaves it. ### 2.15.2.3 Outcomes A source may send only: | Value | Name | Meaning | |---|---|---| | 1 | `Found` | The source holds this object. | | 2 | `NotFound` | It does not. | | 4 | `Refused` | It holds it and will not say so. | A source MUST NOT send `Unavailable`: it is answering, so nothing was unavailable to it. The authority produces that outcome when a source does *not* answer (PGSS §2.18), and a source claiming it would let a working source be recorded as a broken one. A source MUST NOT send `Malformed` in a result. A message it cannot parse is a `Refusal` for the whole conversation (§2.13). > [!NOTE] > `Refused` is how a source declines to expose a principal it holds — > which §2.21 has always permitted. It is distinct from `NotFound` > because the authority may consult another source on `NotFound`, and > must not on `Refused`: the object was found, and the answer was no. A `Refused` result is about the object, not about the caller, and an authority MUST NOT relay it outward as PGSS Logon's `Refused` outcome, which is reserved for a caller that may not make the request (PGSS §2.18). What a source declining to expose an object means to a client is the authority's to decide, and it is not "you lack permission". A source that declared no `QUERIES` is a third case again. It has not answered and cannot be asked, so an authority MUST NOT record it as having answered `NotFound`: a source that was never consulted is not evidence that an object does not exist, and PGSS §2.18 forbids reporting `NotFound` on the strength of one. It contributes nothing to the search, and §2.8 says what an authority should do about the configuration that produced it. ## 2.15.3 Scope An authority MUST apply identity confinement (§2.18), membership scope (§2.19) and numeric scope (§2.20) to a `QueryResult` exactly as to an `Assertion`, and MUST validate every SID in one structurally before using it — including the SIDs of references inside a `PRIMARY_GROUP`, `GROUPS` or `MEMBERS` value, not only the `sid` of the result itself. "Exactly as to an `Assertion`" is meant literally, and it is the sentence an implementation is most likely to satisfy by halves. An authority that confines the object a result names, while relaying the group references beside it unchecked, has left the whole of membership scope unenforced on this channel — and a source with no permission to assert a foreign membership can then report one through a name lookup that it could not report through a logon. A query is not a weaker channel than a logon. A source that could name a principal outside its domain here would be able to make `ls -l` display another source's principals as its own — and, worse, could then be believed the next time something compared that name to a SID. --- # 2.16 Enumeration _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Paging through a source's principals or a large group's members, with cursors that belong to the source. `Query` asks about objects the authority can already name. Enumeration asks a source to produce them: to fill a POSIX `passwd` or `group` table, or to page through a group whose membership will not fit in one answer. **A source is never required to enumerate.** A source that declines is fully conforming, and this section exists as much to make declining safe as to make enumerating possible. ## 2.16.1 EnumerateSource `msg_type` = `0x0006`. Authority to source. Opens a conversation. | Field | Encoding | Limit | |---|---|---| | `kind` | `u8` | PGSS §2.B | | `fields` | `u32` | PGSS §2.B | | `of` | length-framed key entry (§2.15) | | | `cursor` | length-framed bytes | 256 bytes | `kind` MUST NOT be `Any`. An empty `of` enumerates every object of `kind` the source holds. A non-empty `of` MUST name a group, and enumerates that group's members. `cursor` is empty on the first request, and otherwise carries the `next` from the source's immediately preceding reply. ## 2.16.2 EnumerateResult `msg_type` = `0x8006`. Source to authority. Terminal. | Field | Encoding | Limit | |---|---|---| | `outcome` | `u8` | §2.B | | `entries` | array of result entries (§2.15) | 256 | | `next` | length-framed bytes | 256 bytes | An empty `next` ends the enumeration. A non-empty `next` means there is more, **even where `entries` is empty**. A source MUST size a page against the smaller of this chapter's message ceiling and PGSS Logon's, because the authority re-encodes what it returns into the latter (§2.A). Neither the 256-entry bound nor the message ceiling here prevents a page nobody can deliver. A source that will not enumerate replies `Refused`, with `entries` and `next` empty. An authority MUST record it as a source that did not contribute, and MUST NOT retry it for the remainder of that enumeration — across pages as well as within one (PGSS §2.17). `Refused` and an empty `Found` are **different answers and MUST NOT be conflated**. `Found` with `entries` and `next` both empty says *there are none*: the group exists and has no recorded members, or the source holds no objects of that kind. `Refused` says *this source is not answering*. A source that returns an empty `Found` where it means the second has told the authority a falsehood it cannot detect, and the authority will go on asking it — the non-retry rule above has nothing to attach to. The cases most often got wrong, all of which are `Refused` and not an empty `Found`: a group whose membership the source will not expose, a key that names an object the source does not hold, a key that names a principal where a group was required, and a cursor the source can no longer honour. ## 2.16.3 Cursors belong to the source A cursor is **opaque to the authority**. The authority MUST NOT construct, parse or modify one; it relays what it was given. A source MAY encode anything into a cursor, and MAY refuse one it no longer honours — a store rewritten underneath a half-finished walk is the ordinary case, not an exceptional one. A refused cursor is `Refused`, and the authority MUST NOT restart the enumeration on the source's behalf. > [!NOTE] > Restarting would turn a store edit during a `getent passwd` into an > unbounded loop over a source that never finishes. Reporting an > incomplete enumeration is the honest outcome, and PGSS §2.17 requires > the authority to say so. A source MUST NOT assume a cursor comes back on the same conversation, or on the same connection, and MUST NOT hold per-cursor state it is unwilling to discard. The authority's own cursor, the one it hands its client, is its to construct — PGSS §2.17 requires it to reject one it did not issue, which it can only do for a cursor it made. A source's cursor travels inside it, not as it. ## 2.16.4 Why members are here and not only in Query `MEMBERS` is a field of `Query` (§2.15), so the common case — a small group, whose members fit alongside the rest of the record — costs one exchange. A group whose membership will not fit is reported through PGSS Logon's `TooLarge` (PGSS §2.16), and this is where the caller is sent. Paging a membership through the mechanism that already pages is cheaper than a third message, and considerably cheaper than the alternative of a partial member list, which is a wrong answer rather than a smaller one. ## 2.16.5 Enumeration is not existence An authority MUST NOT use enumeration to determine whether a principal exists, and MUST NOT infer from a source declining to enumerate that the source holds nothing. A directory-backed source able to answer any single question while quite unable to answer all of them is the expected case, not a degraded one. --- # 2.17 Change Notification _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The unsolicited message that lets an authority cache name lookups safely, and what it deliberately does not carry. An authority answering a name lookup for every process on the system will cache. This is the message that lets it. ## 2.17.1 Changed `msg_type` = `0x8007`. Source to authority, on conversation `0`. Unsolicited, and never answered. | Field | Encoding | Limit | |---|---|---| | `scope` | `u8` | §2.B | | `sid` | length-framed bytes (SID) | 68 bytes | | `scope` | Name | Meaning | |---|---|---| | 1 | `All` | Everything this source holds may have changed. | | 2 | `Object` | The object named by `sid` may have changed. | `sid` is meaningful only for `Object`, and MUST be within the source's declared domain (§2.18). `Object` scope MUST be used for a **deletion** as well as a change, and for a **creation** — an authority may be holding a cached `NotFound` for a name that now exists. ## 2.17.2 Obligations A source declaring `PUSHES_CHANGES` (§2.8) MUST send `Changed` before, or at the same time as, the altered answer becomes observable through `Query`. Sending it afterwards leaves a window in which the authority's cache and the source disagree while both believe themselves current — which is indistinguishable, from the authority's side, from the notification never arriving. A source MAY send `All` where it could have sent `Object`. Over-invalidation costs a query; under-invalidation costs correctness. An authority MUST accept `Changed` at any time after `Registered`, including while conversations are open on the same connection, and MUST NOT reply to it. An authority MUST treat the loss of a source's connection as `All` for that source. It does not know what changed while it was not listening. A source declaring `PUSHES_CHANGES` SHOULD declare a non-zero `entry_ttl` as well (§2.8). The two are not alternatives: the TTL is the backstop against a notification that was never sent, or was sent and failed to write. Declaring `PUSHES_CHANGES` with a TTL of zero means a single lost notification leaves the authority holding a stale answer until the connection drops — and a source that tolerated a failed `Changed` write without tearing the connection down would have made that outcome reachable, which is one of the reasons §2.6 makes a failed write fatal. ## 2.17.3 Sources that do not push A source that does not declare `PUSHES_CHANGES` is conforming, and many cannot: a remote directory has no way to tell this machine that an account was renamed. Such a source declares `entry_ttl` instead (§2.8), and an authority MUST NOT hold its answers beyond it. A source declaring **neither** has said it cannot support caching, and an authority MUST NOT cache its answers at all. That is the safe reading of silence, and it is what a source predating this message says by omission. > [!NOTE] > The alternative default — cache anything not explicitly forbidden — > would make a source written against an older revision silently serve > stale identity, which is the one class of staleness that decides > access. An authority that does not cache at all satisfies this section trivially, and is conforming. The obligations here bind what an authority may hold, not whether it must hold anything. ## 2.17.4 What this does not carry `Changed` says that something changed. It does not say what it changed to. Carrying the new value would make this a second, unsolicited path by which a source could assert identity — one arriving outside any conversation, with no key to check it against, and no logon in progress to refuse. An authority that believed it would have accepted an identity assertion it never asked for. The authority discards what it holds and asks again through `Query`, where every scope rule in §2.18 to §2.20 applies. --- # 2.18 Identity Confinement _Peios / Advanced Peios / PSPU / Principal Source Interface_ > An assertion outside the source's registered domain is refused and the conversation terminated. No configuration lifts this. **A source may assert only principals within its declared domain (§2.10). No configuration lifts this.** An authority MUST refuse an `Assertion` whose `user_sid` does not lie within the domain the source registered for, and MUST terminate the logon. It MUST apply the same test to a `QueryResult` (§2.15) and to the `sid` of a `Changed` (§2.17). ## 2.18.1 Containment A principal SID lies within a domain when it is the domain's SID plus **exactly one** relative identifier. Exactly one, deliberately. `S-1-5-21-A-B-C-1000-1` is not a principal of `S-1-5-21-A-B-C`, and admitting it would let a source that owns one domain mint names in a nested namespace nobody agreed it owned. A domain SID is likewise not a principal of itself. ## 2.18.2 Why nothing lifts it A source that could assert identities outside its domain could hand out **another authority's principals** to anyone who satisfied *its* credential check. The concrete case: a local source holds no domain credential. If it were unconfined, it could produce a domain administrator's identity for anyone who knew a *local* password. The domain's own authority would never be consulted and would have no way to know. Confinement keeps a compromised source at "authority over its own domain" — which is what it already was — rather than "authority over everyone". This is why identity scope and membership scope (§2.19) are separate settings rather than one. They are different questions, and only one of them has a legitimate exception. --- # 2.19 Membership Scope _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Group membership is the one legitimate exception to confinement — the default, the exception, and why it never extends to identity. Group membership is the question with a legitimate exception, and it needs one. ## 2.19.1 The default By default, an authority MUST refuse an `Assertion` carrying a group outside the domain of the principal being asserted. A directory vouches for its own users and its own groups, and nothing else. Without this rule, a directory-backed source could declare its users members of `BUILTIN\Administrators` — making a remote authority the arbiter of who administers this machine. The test is *relative*: are the group and the principal in the same domain? It needs no configuration and no knowledge of which domain belongs to whom, which is why membership scope is enforceable before anything else about scope is settled. ## 2.19.2 The exception An authority SHOULD allow a source to be configured as permitted to assert **memberships** outside the asserted principal's domain. A local source needs it. Local group membership of *any* principal is a local decision: "`CORP\Domain Admins` is in `BUILTIN\Administrators`" is a record this machine keeps, not something a domain controller asserts at it. Without the exception, a local source could not express the one thing it is most authoritative about. An authority MAY grant this to more than one source. Nothing about it is exclusive. ## 2.19.3 Memberships only, never identity A source holding this permission **remains fully confined on identity** (§2.18). The two are separate because the risks are not symmetric. A source asserting a foreign *membership* is making a claim about what a principal may do **on this machine**, which is a local matter and is the local source's business. A source asserting a foreign *identity* is claiming to be the authority for somebody else's principal, which is never anyone's business but that authority's. > [!NOTE] > Specifying the permission narrowly — memberships only — from the > outset is what allowed identity confinement to be switched on later > without changing what the setting means. A permission defined as "this > source is trusted" would have had to be redefined, and every existing > configuration reinterpreted, the day identity scope arrived. ## 2.19.4 The primary group is a membership `primary_group` (§2.13) is subject to this section exactly as a listed group is, and an authority MUST apply the test to it. It would otherwise be a way round: the authority is required to place the primary group on the token whether or not the source listed it, so a source that named `BUILTIN\Administrators` there and nowhere else would obtain a membership the group array denies it. An authority that adds an unlisted `primary_group` to the membership set MUST do so **before** applying this section, not after. Adding it afterwards reintroduces exactly the route the rule closes. This binds a primary group the **source asserted**. A default the authority substituted for an empty field is not the source's claim and MUST NOT be tested against the source's scope (§2.13); an authority that tested its own default would refuse every logon from a source that simply left the field empty. ## 2.19.5 Claims are the same question, unanswered A claim (§2.13) is a trusted input to conditional ACE evaluation, so asserting one can produce a grant just as asserting a membership can. *Which claim names a source may assert* is therefore the same shape of control as this section — and it is **not yet specified**. The gap is stated rather than papered over. An authority federating to a source it does not fully trust should consider claims as it considers foreign memberships, and a future revision is expected to define the control here. > [!NOTE] > Active Directory splits this in a way worth borrowing: claim *type > definitions* are forest-level configuration while claim *values* are > per-user attributes. The equivalent split — the machine decides which > claims exist, the source supplies values for them — is the obvious > shape for the control, and is why this is a gap in configuration > rather than a flaw in the message format. --- # 2.20 Numeric Scope _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Every unix_id a source asserts is rebased into the range it was granted, refused rather than clamped when out of range. **A source's POSIX identifiers are relative. The authority assigns the range and applies the base; a source is told its range but MUST NOT act on it.** Every `unix_id` in an `Assertion` — the principal's and each group's — and every identifier in a `QueryResult` is an offset within a range the authority assigned that source. The authority adds the base before the number reaches a token or a caller. This is the numeric counterpart of identity confinement (§2.18). That section stops a source naming principals outside its namespace; this one stops it *numbering* them outside its namespace. ## 2.20.1 The range An authority MUST assign each source a range, as a **base** and a **count**, spanning `[base, base + count)`. > [!NOTE] > Mainline configures both per source in the registry, as `UnixIDBase` > and `UnixIDCount` alongside the domain pin (§2.10). How the assignment > is expressed is the authority's own design; what it must satisfy is > this section. An authority MUST reserve a band below every source's base for identifiers of its own — well-known SIDs, service SIDs, confinement SIDs — none of which come from any directory. The band has to be generous, because the last of those categories has no bound. ## 2.20.2 Rebasing Given a relative identifier `r` from a source with range `(base, count)`, the authority computes `base + r`, and MUST refuse to produce a number at all when: - `r` is **0**. Zero is not an identifier; it is how a source says it has no number for something. It MUST NOT become `base`. - `r` is **at or past `count`**. The source has reached outside the range it was given. A number the authority declines to produce projects as *unmapped*, which the Peios Kernel TRM §3.10.1 defines. ## 2.20.3 Refuse, never clamp An out-of-range identifier MUST be refused. It MUST NOT be clamped to the top of the range, and it MUST NOT be reduced modulo the count. Both alternatives look like graceful degradation and are worse than a refusal: - **Clamping** puts two principals on one number, so a filesystem cannot tell them apart. - **Wrapping** lands inside somebody else's range, so one source's principal projects as another source's. The range is a boundary, not an offset, and the count is what makes it one. ## 2.20.4 Two numbers a source can never reach Because the base sits above the reserved band and a relative identifier cannot escape the count: 1. **uid 0.** It belongs to the authority's own table and is not reachable by adding a base to anything a source can send. 2. **Another source's numbers.** Whatever a source asserts, arithmetic confines it to its own range. Neither depends on the source behaving. They hold because of what the source is *able to express*. ## 2.20.5 Well-known SIDs are not the source's to number An authority MUST use its own identifier for any SID in its reserved band, and MUST ignore whatever `unix_id` a source sent alongside it. A source asserting `BUILTIN\Administrators` is stating a membership. It is not claiming authority over what that group projects to, and honouring a relative identifier there would place a well-known group *inside that source's range* — where a second source could number something else identically. ## 2.20.6 The authority tells the source its range A source is told its base and count at registration (§2.8). This is informational and exists so an administration tool can show an operator the identifier a principal will really project to, rather than the relative number on disk. **A source MUST NOT apply the base to what it asserts.** It is told the range so it can explain itself, not so it can do the arithmetic. A source that applied its own base would have it applied twice. > [!NOTE] > Keeping the base out of the source's stored records is what makes > rebasing a configuration change rather than a data migration. An > administrator moving a source's range edits two configuration values; > nothing the source persists has to be rewritten, because nothing it > persists was ever absolute. Disclosure costs nothing that matters. What confines a source is not ignorance of the base but the authority's refusal to accept a relative identifier at or past the count — a check the authority performs on every number it rebases, whatever the source knows. ## 2.20.7 Uniqueness within a source SIDs are one namespace; POSIX user and group identifiers are two. A source MUST therefore allocate from a **single counter across every kind of object it holds**, so that a number issued to a principal is never issued again to a group. An authority cannot check this — it sees one assertion at a time — so it is stated as an obligation on the source (§2.21) rather than as something enforced. > [!NOTE] > The simplest way to satisfy it is to make an object's identifier its > relative identifier within the domain, which is already unique across > principals and groups. That has the incidental benefit that one object > has one number rather than two unrelated ones. --- # 2.21 Conformance _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Every requirement of this chapter collected by role, for an authority federating over PSI and for a source serving it. A conforming implementation MUST satisfy every requirement in this chapter. This section collects them by role. ## 2.21.1 Authority obligations An authority federating over PSI MUST satisfy all of the following. ### 2.21.1.1 Channel 1. Listen; never dial out to a source (§2.3). 2. Never rely on the socket's descriptor as the access control, and bound unregistered connections and registration time independently of it (§2.6). 3. Bound registered sources, and conversations per source (§2.6). 4. Tear down the connection on a framing error or a failed write, rather than attempting resynchronisation (§2.6). ### 2.21.1.2 Registration 5. Require `Register` on conversation `0` as the first message (§2.8). 6. Establish the source's identity from the peer's token, never from `source_name`, and refuse a mismatch rather than correcting it (§2.8, §2.9). 7. Accept only configured sources, treating an empty configuration as *no source may register* (§2.9). 8. Refuse a source that declares no domain (§2.10). 9. Refuse a domain that is not a well-formed locally-issued domain SID (§2.10). 10. Refuse a domain another registered source claims (§2.10). 11. Refuse a source declaring a different domain from the one it declared before (§2.10). 12. Refuse a source whose declared domain contradicts a configured pin, and treat an unparseable pin as refusing rather than as absent (§2.10). 13. Send `Registered` only once the source is routable (§2.8). 14. Report, where an administrator will see it, a source registering with no identifier range, and a source registering without `QUERIES` (§2.8). ### 2.21.1.3 Conversations 15. Allocate conversation identifiers, never accepting one from a source, never opening one with an identifier already live, and reserve `0` for registration (§2.7). 16. Route each logon to exactly one source, resolved before any credential is collected (§2.11). 17. Never fall back to another source when the owning source is unreachable (§2.11). 18. Relay the verified `originator`, taken from the client's socket (§2.11). 19. Refuse to relay a prompt for a credential type the client did not advertise (§2.12). 20. Enforce its own round and time limits on the relayed exchange (§2.12). 21. Send `Abandon` for any conversation that will not reach a terminal state (§2.14). ### 2.21.1.4 Assertions 22. Validate every SID structurally before treating it as identity, including SIDs carried inside claim values (§2.13). 23. Drop an asserted logon SID, and drop duplicates first-mention-wins, **before** applying any scope test (§2.13). 24. Enforce identity confinement, with no configuration lifting it, on an `Assertion`, on a `QueryResult`, and on the `sid` of a `Changed` (§2.18). 25. Enforce membership scope, subject only to a per-source permission covering memberships alone, and apply it to a **source-asserted** `primary_group` as well as to listed groups — promoting an unlisted one into the membership set before the test (§2.19, §2.13). 26. Never apply membership scope to a `primary_group` it substituted itself for an empty field (§2.13, §2.19). 27. Fail the logon on a malformed group SID, an unusable claim, a `canonical_name` PGSS §2.15 forbids, or an out-of-range identifier (§2.13). 28. Place `primary_group` on the token even when the source did not list it among `groups` (§2.13). 29. Perform derivation itself, and never accept privileges, integrity, or a token from a source (§2.4). 30. Never relay a refusal reason revealing a distinction the source was required not to make (§2.13). ### 2.21.1.5 Identifiers 31. Apply the source's base to every relative identifier it accepts, and never accept one already rebased (§2.20). 32. Refuse a relative identifier of `0` or one at or past the source's count, rather than clamping or wrapping it (§2.20). 33. Reserve a band of identifiers below every source's base for its own, and use its own value for any SID within that band regardless of what the source sent (§2.20). ### 2.21.1.6 Queries 34. Never send a message a source did not declare it answers, and never set a field bit gating a capability it did not declare (§2.8). 35. Never send more keys in one `Query` than the source's `max_batch`, nor more than 64 whatever it declared (§2.8). 36. Resolve an absolute POSIX identifier to a source and a relative identifier itself, and never send an absolute one to a source (§2.15). 37. Apply identity confinement, membership scope and numeric scope to a `QueryResult` exactly as to an `Assertion`, and validate every SID in one structurally — including those of references inside a value (§2.15). 38. Rebase every identifier in a result, under the rules of §2.20 (§2.15). 39. Qualify a source's `canonical_name` itself; never require a source to (§2.15). 40. Consult no further source on a `Refused` result, treat only `NotFound` as leave to continue, and never relay a source's `Refused` outward as PGSS Logon's `Refused` outcome (§2.15). 41. Never record a source it may not ask — one that declared no `QUERIES` — as having answered `NotFound` (§2.15). 42. Relay cursors opaquely, never construct or modify a source's, and never restart an enumeration on a source's behalf (§2.16). 43. Record a source that declined or could not be reached, and not retry it for the remainder of that enumeration, across pages as well as within one (§2.16). 44. Never infer from a source declining to enumerate that it holds nothing (§2.16). ### 2.21.1.7 Caching 45. Not cache a source's answers at all unless it declared `PUSHES_CHANGES` or a non-zero `entry_ttl` (§2.8, §2.17). 46. Not hold an answer beyond a declared `entry_ttl` (§2.8). 47. Accept `Changed` at any time after `Registered`, and never reply to it (§2.17). 48. Re-read through `Query` after an invalidation, and never take a new value from `Changed` (§2.17). 49. Treat the loss of a source's connection as `All` for that source (§2.17). An authority that holds nothing satisfies 45 to 49 trivially. ## 2.21.2 Source obligations A principal source MUST satisfy all of the following. ### 2.21.2.1 Connection 1. Connect to the authority; never listen for it (§2.3). 2. Open with `Register` on conversation `0`, carrying its name and its domain (§2.8). 3. Send nothing else before receiving `Registered` (§2.8). 4. Report itself ready — to an init system or equivalent — only after `Registered` (§2.3). 5. Declare the same domain on every registration, for the life of the machine's configuration (§2.10). 6. Tear down the connection on a framing error or a failed write, on every path including an unsolicited `Changed` (§2.6, §2.17). ### 2.21.2.2 Conversations 7. Reply on the conversation identifier it was given, and never invent one (§2.7). 8. Decline to act on a message on a conversation it does not know, and never treat it as opening one — without replying on it, since the identifier may since have been reused (§2.7). 9. Refuse an `Authenticate`, `Query` or `EnumerateSource` arriving on conversation `0` (§2.7). 10. Bound the conversations it tracks itself, rather than relying on the authority's limit (§2.6). 11. Refuse beyond that bound with `AuthorityUnavailable`, rather than dropping silently (§2.11). 12. Discard conversation state on `Abandon`, and not reply (§2.14). ### 2.21.2.3 Answering 13. Send exactly one terminal message — `Assertion` or `Refusal` — per conversation (§2.13). 14. Assert only principals within its declared domain (§2.18). 15. Assert group SIDs and identifiers only, never attributes (§2.13). 16. Carry the canonical spelling of the principal's name in `canonical_name` (§2.13), and never assert a name that PGSS §2.15 forbids — validating what it asserts, not only what it creates. 17. Never distinguish an unknown principal from a bad credential — by denial code, by reason, or by timing (§2.13). ### 2.21.2.4 Identifiers 18. Assert **relative** identifiers only, and never apply its own base (§2.20). 19. Allocate from a single counter across every kind of object it holds, so that no number is issued twice (§2.20). 20. Send `0` for any object it does not number, including every group it does not own (§2.20). 21. Never issue an identifier at or past the count it was given (§2.20). ### 2.21.2.5 Queries A source declaring no capabilities (§2.8) is exempt from this section entirely. 22. Declare only capabilities it implements, and answer every message type it declared (§2.8). 23. Answer every key of a `Query`, in order, without reordering, merging or omitting — or refuse the conversation whole (§2.15). 24. Send `Found`, `NotFound` or `Refused` in a result, and never `Unavailable` or `Malformed` (§2.15). 25. Return its own canonical spelling of a name, unqualified (§2.15). 26. Return **relative** identifiers in a result, exactly as in an `Assertion` (§2.15, §2.20). 27. Answer only for principals within its declared domain, on a query as on a logon (§2.18). 28. Answer `Refused`, never an empty `Found`, wherever it is declining rather than reporting an absence — including a membership it will not expose, a key naming an object it does not hold, and a key of the wrong kind (§2.16). 29. Refuse a cursor it can no longer honour, rather than restarting or answering from a changed store (§2.16). 30. Hold no per-cursor state it is unwilling to discard unasked (§2.16). 31. Size a page against the smaller of PSI's message ceiling and PGSS Logon's, since the authority must re-encode it into the latter (§2.16, §2.A). 32. Send `Changed` before the altered answer becomes observable, if it declared `PUSHES_CHANGES` (§2.17). 33. Declare a non-zero `entry_ttl` if it does not push changes and can tolerate its answers being held, and SHOULD declare one even if it does (§2.8, §2.17). ### 2.21.2.6 Credentials 34. Store verifiers that are **not** usable as credential material — nothing a challenge could be recomputed from (PGSS §2.11). 35. Erase credential material, and every buffer it was decoded through, before that memory is released (PGSS §2.12). 36. Never write credential material to a log, audit record, or diagnostic (PGSS §2.12). ## 2.21.3 What a source is not required to do A source is **not** required to store anything, to be local, to be persistent, or to know what a token is. It answers one question: given this identifier and whatever it chose to ask for, who is this? Nor is it required to trust the authority beyond the connection. A source that refuses logons on the strength of `originator`, or declines to answer for principals it holds but does not wish to expose, is conforming. --- # Appendix 2.A Message Reference _Peios / Advanced Peios / PSPU / Principal Source Interface_ > Every PSI message by number and direction, the protocol constants, and the field limits — including the ceiling that actually binds a page. ## 2.A.1 Messages | `msg_type` | Message | Direction | Conversation | Defined in | |---|---|---|---|---| | `0x8001` | `Register` | source → authority | `0` | §2.8 | | `0x0001` | `Registered` | authority → source | `0` | §2.8 | | `0x0002` | `Authenticate` | authority → source | 1+ (opens) | §2.11 | | `0x8002` | `CredentialRequest` | source → authority | 1+ | §2.12 | | `0x0003` | `CredentialResponse` | authority → source | 1+ | §2.12 | | `0x8003` | `Assertion` | source → authority | 1+ (terminal) | §2.13 | | `0x8004` | `Refusal` | source → authority | 1+ (terminal) | §2.13 | | `0x0004` | `Abandon` | authority → source | 1+ (terminal) | §2.14 | | `0x0005` | `Query` | authority → source | 1+ (opens) | §2.15 | | `0x8005` | `QueryResult` | source → authority | 1+ (terminal) | §2.15 | | `0x0006` | `EnumerateSource` | authority → source | 1+ (opens) | §2.16 | | `0x8006` | `EnumerateResult` | source → authority | 1+ (terminal) | §2.16 | | `0x8007` | `Changed` | source → authority | `0` | §2.17 | The high bit marks a message sent by **the source**, which is the authority for its own principals (§2.7). ## 2.A.2 Protocol constants | Constant | Value | Defined in | |---|---|---| | Socket path | the implementation's choice | §2.6 | | Magic | `PPSI` (`50 50 53 49`) | §2.7 | | Version | `1` | §2.7 | | Header size | 20 bytes | §2.7 | | Maximum message size | 81920 bytes | §2.7 | | Reserved conversation | `0` | §2.7 | ## 2.A.3 Field limits | Field | Maximum | Defined in | |---|---|---| | `source_name` | 32 bytes | §2.8 | | `domain` | 68 bytes | §2.8 | | `max_batch` | 64 | §2.8 | | `originator` | 68 bytes | §2.11 | | `user_sid` | 68 bytes | §2.13 | | `canonical_name` | 256 bytes | §2.13 | | `groups` | 128 entries, each SID 68 bytes | §2.13 | | `primary_group` | 68 bytes | §2.13 | | `claims` | 64 entries | §2.13 | | claim `name` | 255 bytes | §2.13 | | claim `values` | 64 per claim | §2.13 | | claim string value | 1024 bytes | §2.13 | | claim octet value | 1024 bytes | §2.13 | | claim SID value | 68 bytes | §2.13 | | `reason` | 512 bytes | §2.13 | | `keys` | 64 entries | §2.15 | | key `name` | 256 bytes | §2.15 | | `results` | 64 entries | §2.15 | | `withheld` | 32 entries | §2.15 | | `values` | 32 entries | §2.15 | | `entries` | 256 entries | §2.16 | | `cursor`, `next` | 256 bytes | §2.16 | 68 bytes is the largest a SID can be: an eight-byte prelude plus fifteen sub-authorities (§2.7). A claim name is bounded at 255 **bytes** of UTF-8 while PCDS §5.9 bounds it at 255 UTF-16 code units. A string's UTF-16 length never exceeds its UTF-8 byte length, so the byte bound is the stricter of the two and satisfies PCDS without transcoding to find out. The claim limits are otherwise tighter than PCDS §5.9 permits — it allows 1024 values per claim. These bound the work an authority does decoding a message it has not yet decided to believe, and nothing needs a thousand-valued claim from a principal source. Fields inside a nested `LogonStart`, `CredentialRequest`, `CredentialResponse` or profile keep PGSS Logon's limits (PGSS §2.A). ## 2.A.4 The ceiling that actually binds a page None of the entry counts above is the constraint on how much a source may return. **An entry that fits a PSI message need not fit the PGSS Logon message the authority must re-encode it into**: this chapter's ceiling is 81920 bytes and PGSS Logon's is 65536, and a `QueryResult` or `EnumerateResult` entry travels outward inside the smaller one. A source MUST therefore bound a reply by the **smaller** of the two ceilings, not by this one, and MUST page rather than fill a PSI message it knows an authority cannot forward. A page that fits here and not there is a page nobody can deliver, and the entry-count bounds do not prevent one — 256 entries of a few hundred bytes each exceeds both. The margin an implementation leaves for the authority's own framing is its own choice; leaving none is a defect. --- # Appendix 2.B Enumerations _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The values PSI defines for itself — key types, result outcomes, change scopes, capabilities, and claim value types and flags. Values PSI defines for itself. Everything else it carries is PGSS Logon's — see PGSS §2.B. Adding a value to any enumeration here is a breaking change requiring a version bump (§2.7). The two exceptions are the capability bitmask below, and PGSS Logon's field bitmask, which PSI carries unchanged and which may gain bits without one. ## 2.B.1 Key types Carried in a `Query` key entry's `key_type` (§2.15) as a `u8`. | Value | Name | Key is in | |---|---|---| | 0 | *none* | The key entry is absent | | 1 | `Name` | `name` | | 2 | `Sid` | `sid` | | 3 | `RelativeId` | `relative_id` | Zero is not a key. It is how an `EnumerateSource` encodes an empty `of` (§2.16), which is the only place it may appear; an encoder MUST NOT send it in a `Query` key and a decoder MUST reject one that arrives there. PGSS Logon's key types (PGSS §2.B) share the first two values and differ in the third, where it carries an **absolute** POSIX identifier. The values are not interchangeable and the tables are deliberately separate: `3` means a rebased number on one side of the authority and a relative one on the other, which is the whole of §2.20 expressed as a number. ## 2.B.2 Result outcomes Carried in a result entry's `outcome` (§2.15) and in `EnumerateResult.outcome` (§2.16) as a `u8`. The values are PGSS Logon's (PGSS §2.B). A source may send only these three: | Value | Name | |---|---| | 1 | `Found` | | 2 | `NotFound` | | 4 | `Refused` | `Unavailable` (`3`) and `Malformed` (`5`) are the authority's to produce and MUST NOT be sent by a source — see §2.15. A decoder MUST reject either arriving from a source, rather than leaving the check to a caller. ## 2.B.3 Change scopes Carried in `Changed.scope` (§2.17) as a `u8`. | Value | Name | |---|---| | 1 | `All` | | 2 | `Object` | ## 2.B.4 Capabilities Carried in `Register.capabilities` (§2.8) as a `u32` bitmask. | Bit | Name | |---|---| | 0 | `QUERIES` | | 1 | `ENUMERATES` | | 2 | `MEMBERS` | | 3 | `PUSHES_CHANGES` | Unlike the enumerations above, a bit MAY be added here without a version bump. A source that does not set a bit has not declared the capability, and an authority MUST NOT send a message the source did not declare it answers (§2.8) — so an authority that predates a bit simply never uses it, and a source that predates one never sets it. Both are the safe reading. `MEMBERS` gates a *field* rather than a message: an authority MUST NOT set the `MEMBERS` field bit of a `Query` (§2.15) against a source that did not declare it. ## 2.B.5 Claim value types and flags A claim's `value_type` and `flags` (§2.13) are PCDS §5.9's, and are not restated here. They are nonetheless **closed** on this interface: a decoder MUST reject a value type or a flag bit it does not recognise, rather than carrying it through to an authority that will put it on a token. Adding one is therefore a breaking change to PSI by the rule above, even though the values themselves belong to PCDS. --- # Appendix 2.C What Is Shared with PGSS Logon _Peios / Advanced Peios / PSPU / Principal Source Interface_ > A checklist of exactly what PSI shares with PGSS Logon unchanged, what it adds, and what it changes. PSI is a superset of PGSS Logon (§2.5). This appendix consolidates exactly what is shared, what is added, and what differs — as a checklist for an implementer building both, and as the list to re-examine whenever either specification changes. ## 2.C.1 Shared unchanged | Element | PGSS | Note | |---|---|---| | `LogonStart` body | §2.7 | Nested whole inside `Authenticate`, never inlined (§2.7) | | `CredentialRequest` body | §2.8 | Byte-for-byte identical | | `CredentialResponse` body | §2.8 | Byte-for-byte identical | | `profile` body | §2.9 | Nested inside `Assertion`, relayed onward unchanged (§2.13) | | Denial codes | §2.B | Reused by `Refusal` (§2.13) | | Lookup result body, `present` onward | §2.16 | Nested inside a `QueryResult` entry (§2.15) | | Field bitmask | §2.B | Carried unchanged by `Query` (§2.15) | | Object kinds | §2.B | Carried unchanged by a key entry (§2.15) | | Lookup outcomes | §2.B | A source may send three of the five (§2.B) | | Header layout, first 12 bytes | §2.6 | Same fields at the same offsets | | Byte order, string encoding, length framing | §2.6 | See §2.7 | | Extensibility rules | §2.6 | Append-only; new enum value is breaking | | Credential-handling obligations | §2.12 | Bind sources too (§2.21) | | Name rules | §2.15 | Bind what a source asserts (§2.13) | An implementation that reimplements any of these rather than sharing one definition has taken on the job of keeping two copies in step. The sharing is the point: a translation layer between two byte-identical formats is a place for them to drift. ## 2.C.2 Added by PSI | Element | Defined in | |---|---| | `conversation` header field | §2.7 | | `Register` / `Registered` | §2.8 | | `Authenticate`, wrapping `LogonStart` plus `originator` | §2.11 | | `Assertion` | §2.13 | | `Abandon` | §2.14 | | Domain claim and its checks | §2.10 | | POSIX identifiers, and the ranges that confine them | §2.13, §2.20 | | Claims carried from a source | §2.13 | | `Query` / `QueryResult`, and batching | §2.15 | | `EnumerateSource` / `EnumerateResult`, and cursors | §2.16 | | `Changed`, and the cache contract | §2.17 | | Source capabilities, TTL and batch limit | §2.8 | | Relative key types, where PGSS Logon's are absolute | §2.15, §2.B | | Identity, membership and numeric scope | §2.18 to §2.20 | ## 2.C.3 Differs | | PGSS Logon | PSI | |---|---|---| | Magic | `PGSL` | `PPSI` | | Header | 12 bytes | 20 bytes | | Maximum message | 65536 | 81920 | | Conversations per connection | one | many | | Connection lifetime | one logon | the source's lifetime | | High bit of `msg_type` | authority → client | source → authority | | Success terminal | `AccessGranted` + token fd | `Assertion` — no session, no token | | Socket path | normative | the implementation's choice | | Key type `3` | absolute POSIX identifier | relative identifier | The success terminal is what makes minting structurally impossible for a source (§2.4); the socket path is normative in PGSS because it is a conformance bar and not here because PSI is not one (§2.1). The message ceiling is the row most likely to catch an implementer out, because the larger number is the one that does *not* bind a reply — see §2.A. ## 2.C.4 When either specification changes A field appended to `LogonStart`, `CredentialRequest`, `CredentialResponse` or `profile` in PGSS appears here automatically, because the bodies are shared. That is the intended behaviour and needs no change to this chapter. The profile is the one shared body that travels in the *opposite* direction to the others: the interrogation bodies pass from the authority outward to the client, while the profile originates at the source and is relayed outward through the authority. It is shared for the same reason regardless — one definition, so the value a source states and the value a client reads cannot drift apart. A new `Denial` value, a new `CredentialType`, or any change to the header's first twelve bytes is a **breaking change to both** and requires a coordinated version bump. An implementer maintaining both MUST NOT bump one alone. --- # Appendix 2.D Prior Art _Peios / Advanced Peios / PSPU / Principal Source Interface_ > The approaches PSI exists to avoid — in-process authentication packages above all — and the ones it adopts. ## 2.D.1 What this exists to avoid The design is shaped more by rejected approaches than adopted ones. **LSA authentication packages.** Windows loads authentication packages as DLLs into the LSA process. A defect in any package is a defect in the most privileged process on the system, and the packages are exactly the components most likely to parse hostile input. PSI puts that boundary at a process, permanently: there is no in-process extension point and no message that could create one. **PAM modules.** The same objection, plus stacking — offering a credential to each module in turn until one accepts, which hands every module the credentials of every other module's users. PSI resolves *which* source answers before any credential is collected (§2.11). **NSS.** Name service switch modules answer "who is this?" as a library call in whatever process asked, with no boundary at all. PSI's answer is a message from a process that was separately identified. ## 2.D.2 What is adopted **RSI's shape**, specified in PSPK. A long-lived connection, sources that dial in and register, multiplexed requests tagged with an identifier, and the authority tearing down a connection it cannot parse. PSI is recognisably the same family, and deliberately so — an implementer who has written a registry source will find little surprising here. The differences are worth naming, because they follow from what is being federated. A registry source is trusted with the correctness of a subtree and the kernel validates its structure; a principal source is trusted with identity, so the checks it faces are about *scope* — which principals, which memberships, which numbers — rather than about well-formedness alone. And the kernel assigns a registry source no numeric range, because there is nothing to project. **PGSS Logon's interrogation, wholesale.** Rather than inventing a parallel vocabulary for prompts and answers, PSI relays PGSS Logon's messages with identical bodies (§2.5). The gain is not brevity but correctness: there is no translation layer to be lossy, and a source's prompt reaches the client exactly as written. ## 2.D.3 Design influences **Sources connect inward.** The authority never dials out. See §2.3 — this is the single most consequential shape decision in the chapter. **Assertion, not minting.** The success terminal deliberately cannot express a session or a token (§2.4). The separation is structural rather than a rule an implementer must remember. **Domains claimed, not assumed.** A source states what it is authoritative for and the authority confines it to that (§2.10, §2.18). The alternative — an authority that trusts whatever a source says about anyone — makes every source as dangerous as the most dangerous one. **Relative numbers.** POSIX identifiers are the one thing a source states that has no namespace of its own to be confined by, so the range supplies one (§2.20). Nothing else in the protocol needed inventing; a SID already carries its domain. --- # 3.1 Scope and Roles _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The three channels by which programs deposit logs and metrics with an observability service, and the roles at each end. This chapter specifies the **observability interfaces**: the three channels by which the programs on a Peios system deposit logs and metrics with an observability service, and by which anything on the system asks that service what it holds. There are three interfaces and they are specified together because they are one contract from the service's side and because two of the three share their encoding, their validation posture and their loss model: - the **Log Ingestion Interface**, on which a producer submits log records (§3.6 to §3.8) - the **Metric Ingestion Interface**, on which a producer submits metric samples (§3.9 to §3.13) - the **Query Interface**, on which a client asks for stored events, logs and metrics and receives records (§3.14 to §3.28) Three roles participate. The **collector** is the process that accepts ingestion on the two datagram channels, serves the query channel, and holds the data in between. There is one collector. It is the party being asked, on all three interfaces — which is why the obligations in this chapter fall mostly on it, and why the producer and client roles are so thin. A **producer** is any process that submits log records or metric samples. The producer role is unrestricted by design: the point of a system log is that everything on the system can write to it. A producer declares what it is (§3.7, §3.11) and the collector does not verify the declaration (§3.4). A **client** is a process that issues a query and reads the result. A client's identity, unlike a producer's, is established by the collector and determines what it may see (§3.28). One program is commonly all three at once. This chapter covers: - the shape of the three channels and why two are datagram and one is a stream (§3.3) - the loss model, which is the load-bearing decision of the whole ingestion design (§3.4) - encoding, timestamps and the timestamp domain (§3.5) - the log record, its fields, and exactly which malformations cost the record and which are merely ignored (§3.6 to §3.8) - the metric data model, the three metric types, the sample record, and what makes two samples the same time series (§3.9 to §3.13) - the query channel, its framing, and the four response messages (§3.14 to §3.17) - the query language: its shape, its lexis, its operators, its ordering and grouping semantics, and the three modes (§3.18 to §3.25) - cross-type filtering and streaming (§3.26, §3.27) - what a client is and is not told about data it may not read (§3.28) - how these interfaces may be extended (§3.29) - the obligations binding on each role (§3.30) This chapter does not cover: - Event *emission*. Events reach a collector through KMES, not through any interface here; the consumer side of that is specified in PSPK, and emission is a kernel interface offered to privileged callers. - Event type vocabulary and payload schemas, which belong to whichever subsystem emits the event. - How a collector stores, indexes, retains or accelerates anything — its own design. The mainline collector's is described in the eventd TRMP. - Which producers a system permits to reach the ingestion channels, and how that is configured. - Administering a collector's contents. The third of those is the point of the whole document. A collector is handed records and asked questions; how it gets from one to the other is exactly what different collectors exist to do differently. ## 3.1.1 These interfaces are not a conformance requirement A system that offers none of these is still Peios. Observability is not in the definition of the platform, and a system that ships a different collector, or none, conforms exactly as well. They are specified because they are *public*. Every service on the system is a log producer, every collection agent is a metric producer, and every dashboard, alerting tool and command-line viewer is a query client. All three of those are third-party positions, and all three need a contract that stays put. > [!NOTE] > The Query Interface reaches *events* as well as logs and metrics, even > though events do not arrive over any interface here. That asymmetry is > deliberate: PSPK's ring-buffer interface is the raw transport, requires > SeSecurityPrivilege, and applies no per-record access control. The > query interface is how everything else reads events, and it is the only > way to read them after the ring buffer has moved on. --- # 3.2 Terminology _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Terms this chapter borrows unchanged from PSPK's KMES event stream and from elsewhere in the corpus. Terms defined in PSPK for the KMES event stream — event, header, payload, stamp, sequence number, origin class — are used here with the same meaning and are not redefined. Terms defined in PCDS — GUID, SID, Security Descriptor, ACL, ACE — likewise. The following terms are specific to this chapter. **Collector**: the process that accepts log and metric ingestion and serves queries. The role, not the program: the mainline collector is eventd, and this chapter never requires that it be. **Producer**: a process that submits log records or metric samples. **Client**: a process that issues a query. **Log record**: one line of output from a program, with the light metadata of §3.7 attached. A log record is text; the collector does not parse it. **Metric sample**: one measurement of one quantity at one moment, belonging to a time series (§3.13). **Time series**: the sequence of samples sharing a name, a label set, and — for histograms — a set of bucket boundaries. Identity is defined in §3.13. **Boot ID**: a GUID identifying one boot of the system. Every record a collector stores carries one, so that records from different boots are never interleaved and per-CPU event sequence numbers, which restart each boot, remain unambiguous. It is assigned outside this interface and is visible to a client only as a queryable field. **Datagram**: one message on an ingestion channel, carrying either one record or a batch of them (§3.7, §3.11). **Effective query range**: the half-open interval a query examines, `[SINCE, UNTIL)`, with the bounds resolved as §3.19 defines. **Concrete identifier**: the event type, log origin or metric name that a stored record actually carries — as distinct from the pattern a query or a Security Descriptor uses to match one. Access control resolves per concrete identifier (§3.28). --- # 3.3 Three Channels _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Two datagram sockets carrying data inward and one stream socket carrying it out — why the split, and how each is protected. A collector listens on three `AF_UNIX` sockets. Two carry data inward, one carries it out. | Channel | Socket type | Direction | Section | |---|---|---|---| | Log ingestion | `SOCK_DGRAM` | producer to collector | §3.6 | | Metric ingestion | `SOCK_DGRAM` | producer to collector | §3.9 | | Query | `SOCK_STREAM` | request and response | §3.14 | The pathnames are configuration and this chapter does not fix them. A collector MUST serve each interface on a distinct socket; it MUST NOT multiplex two of them onto one. ## 3.3.1 Why ingestion is datagram Each submission is an independent message. A datagram either arrives whole or does not arrive, so there is no framing to get wrong, no length prefix to parse, no partial read to reassemble, and no connection state to keep for a producer that submits one line an hour. The record boundary is the datagram boundary. The property that matters more is that a datagram socket **cannot exert backpressure**. When the receive queue is full the kernel discards the datagram and the sender proceeds. That is the behaviour §3.4 requires, and choosing a stream socket would make it unachievable: a full stream buffer blocks the writer, which is precisely the outcome this design forbids. ## 3.3.2 Why the query channel is a stream A query result is arbitrarily large, is delivered in several messages, and must not be silently truncated — the opposite requirements. It also needs a caller identity, and a peer token can be obtained from a connected stream socket. Both push the same way. ## 3.3.3 Separation The three channels are separated for the same two reasons. The first is **admission**. Each listening socket has its own receive queue. Log volume is orders of magnitude above query volume on any normal system, and a burst of either must not delay the other. Three sockets means three populations of caller that cannot starve one another, whatever load any of them is under. The second is **access control**. The set of processes that may write logs is every process on the system; the set that may read them is not. Those want different Security Descriptors, and a descriptor is a property of a socket. The separation is **not** for isolation. One collector serves all three, so a defect or a hang in any of them reaches the others regardless, and this chapter does not pretend otherwise. ## 3.3.4 Protecting the channels A collector MUST protect each socket with a Security Descriptor. This is the whole of the access control on the two ingestion channels: there is no per-record write authorization anywhere in this chapter (§3.4), so the descriptor on the socket is the only thing standing between a process and the ability to write a log line under any name it likes. A collector MUST NOT rely on the socket's POSIX mode bits for this. On Peios an access decision is routed through the object's Security Descriptor, not through mode bits, so a mode set on a socket pathname does not restrict anything; and an inode created without a descriptor is denied to every caller, so a collector that binds a socket into a directory carrying no inheritable ACEs produces a socket nothing can reach. A collector MUST establish the descriptor on each socket before it begins accepting on it. --- # 3.4 Loss and Backpressure _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The decision that shapes both ingestion channels — a producer is never stalled, so data may be lost — and what a collector must not do about it. The single decision that shapes both ingestion interfaces is this: **a producer is never slowed down, and never told that a record was lost.** ## 3.4.1 The obligation A collector MUST NOT exert backpressure on a producer. A producer MUST NOT stall, block, retry or otherwise change its behaviour because a collector is slow, busy, or absent. The consequence is accepted openly. When a collector cannot drain an ingestion socket as fast as producers fill it, the kernel discards datagrams. Neither party is notified. A collector MUST NOT report the loss to the producer, because there is no reply message on a datagram channel and adding one would reintroduce the coupling this rule exists to prevent. ## 3.4.2 Why loss is acceptable here A lost log line is an inconvenience. A lost metric sample is a visible gap in a chart. Neither is a failure of the system, and neither is worth the cost of the alternative — which is either blocking the producer or buffering without bound, and the second is only the first with a delay. Events are the counter-example, and the reason the boundary between events and logs matters. An event may be a security audit record whose absence is itself the finding, so events do not travel on either interface in this chapter: they travel through KMES, where loss is detected, bounded and recorded. A program with data that must not be lost emits an event; a program with output for a human to read writes a log. > [!NOTE] > This is the practical test for producers deciding where output belongs. > If you would want to know that a record went missing, it is not a log > and not a metric. ## 3.4.3 What a collector must not do about it A collector MUST NOT emit an event, write a log entry, or perform any other work proportional to the volume of malformed or unwanted input it receives. Ingestion input is unauthenticated and arrives from arbitrary local processes (§3.3). A collector that reacted to bad input — by logging it, by counting it in a way a client can observe, or by emitting a diagnostic event — would hand every process on the system an amplification primitive: a cheap malformed datagram producing an expensive durable record. Silence is the defence. The rule binds only on responses to *input*. A collector MAY record its own internal conditions, and the mainline collector records several (eventd TRMP §2.6). ## 3.4.4 Ordering and duplication A collector MUST NOT assume that datagrams arrive in the order they were sent, and MUST NOT reorder or deduplicate the records inside one. A producer MUST NOT assume that submitting two datagrams in order causes them to be stored in that order; the timestamp field (§3.7, §3.11) is the only ordering a producer controls. Records are not deduplicated. A producer that submits the same record twice has produced two records. --- # 3.5 Encoding and Time _Peios / Advanced Peios / PSPU / Observability Interfaces_ > MessagePack on every interface, the canonical subset producers must emit, and how timestamps are carried. ## 3.5.1 MessagePack Every structured value on all three interfaces — log records, metric samples, query requests, query responses — is encoded as MessagePack. Strings are UTF-8. The choice is inherited rather than made here: KMES event payloads are a single MessagePack value, so a collector already carries a decoder and a query result can carry an event payload outward without re-encoding it. A decoder MUST accept any valid MessagePack encoding of a value it is given. In particular a producer MAY use any length-prefix width that can represent the value, and a collector MUST NOT require the shortest. ## 3.5.2 Canonical MessagePack Where this chapter requires a **canonical** encoding, the value MUST be encoded as follows: - Nil and booleans use the fixed singleton encodings. - Integers use the shortest encoding that preserves signedness: non-negative values use positive fixint, `uint8`, `uint16`, `uint32` or `uint64`; negative values use negative fixint, `int8`, `int16`, `int32` or `int64`. - Floats are encoded as `float64`. Finite values use their IEEE-754 binary64 representation; the infinities use the normal binary64 encodings; **any** NaN is encoded as the single quiet NaN bit pattern `0x7ff8000000000000`. - Strings, binary values, arrays and maps use the shortest length-prefix form capable of representing the length. - Arrays encode each element recursively, in order. - Maps encode keys and values recursively, with entries sorted by the canonical encoded key bytes; ties are broken by the canonical encoded value bytes. Canonical encoding exists so that two values that are equal are also byte-identical, which is what makes them comparable and orderable without decoding. It is required in exactly two places: histogram sample storage, where it makes a sample map a stable value (§3.11), and array comparison in query ordering and grouping (§3.21). It does **not** constrain what a producer sends. Ingestion accepts any valid encoding. ## 3.5.3 Timestamps A timestamp is wall-clock time in **nanoseconds since the Unix epoch, UTC**, as a signed 64-bit value. The **timestamp domain** is `0` to `9223372036854775807` inclusive. A value outside it is invalid wherever it appears: as a producer-supplied timestamp (§3.8, §3.12), as a query time literal (§3.19), or as a value in a result record. The domain has no negative half. A collector MUST reject a negative timestamp rather than storing a time before 1970, and a query whose time arithmetic lands below zero — `SINCE 100000d ago`, for example — MUST produce an error rather than clamping. > [!NOTE] > The upper bound is `i64::MAX` nanoseconds, which is the year 2262. The > domain is stated as a closed range rather than "whatever fits" because > both parties must agree on where the edge is: a producer that clamps > and a collector that rejects would disagree about the same sample. Wall-clock time is not monotonic. A collector MUST store the timestamp it is given or derives without correcting it, and MUST NOT assume that timestamps within one time series increase (§3.13). A clock step backwards produces records that are out of order with respect to their arrival, and every ordering rule in this chapter is defined to remain total and deterministic when that happens. --- # 3.6 The Log Channel _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The log ingestion socket — its datagram ceiling, the receive queue as the only buffer, and what reachability a producer may assume. A collector MUST expose an `AF_UNIX` `SOCK_DGRAM` socket for log ingestion, protected by a Security Descriptor as §3.3 requires. ## 3.6.1 The datagram ceiling A collector declares a maximum accepted datagram size, the **log datagram ceiling**. A collector MUST receive log datagrams into a buffer of at least that size, and MUST discard a datagram the kernel reports as truncated rather than storing the prefix that fitted. A producer MUST NOT send a log datagram larger than the ceiling. One that does is discarded whole, taking every record in it, and the producer is not told (§3.4). > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. **There is no mechanism by which a producer can learn the ceiling.** A datagram channel has no reply, so a producer either knows the value out of band or assumes the mainline default. A collector that lowers the ceiling below the mainline value MUST expect producers to keep sending at the old one, and silently losing what they send. Raising it is safe; lowering it is a change to the contract with every producer on the system. ## 3.6.2 The receive queue is the buffer A collector MAY enlarge the socket receive queue, to at most four times the datagram ceiling. It MUST NOT buffer beyond it. That queue is the only cushion between a producer and the collector's storage. While a collector is committing a batch it is not draining the socket, and datagrams arriving in that window occupy the queue; when the queue fills, they are discarded. This is the designed degradation (§3.4), not a failure to be tuned away — a larger buffer moves the threshold without changing what happens at it, and an unbounded one converts data loss into memory exhaustion. ## 3.6.3 Reachability Every process that produces output is a log producer, including processes that have not been written with a collector in mind. The mainline arrangement is that the service manager holds each service's standard output and standard error at fork and forwards what it reads (peinit TRM). It is **not** a privileged producer: it uses this socket, this record format and these rules like anything else, and its role is to bridge programs that write to a file descriptor into an interface that expects datagrams. A producer that wants control over its own metadata MAY write to the socket directly instead, with no registration, negotiation or setup of any kind. Direct submission and forwarded submission are the same interface; nothing distinguishes them on the wire, and a collector MUST NOT treat them differently. --- # 3.7 Log Records _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The MessagePack map that carries a log record, one or batched — origin, is_error, timestamp and job_id. A log datagram carries either **one** record, encoded as a MessagePack map, or **several**, encoded as a MessagePack array of maps. A collector MUST accept both forms. A producer MAY use either at any time; there is no mode and no negotiation. ## 3.7.1 Fields | Field | Type | Required | Meaning | |---|---|---|---| | `origin` | string | yes | Non-empty name of the program that produced the line. | | `is_error` | bool | yes | True if the line came from standard error, or the producer marked it an error. False otherwise. | | `message` | string | yes | The log text — one line of output. MAY be empty, which is a blank line. | | `timestamp` | integer | no | When the line was produced, in the timestamp domain (§3.5). Absent means the collector uses its own clock at receipt. | | `job_id` | binary, 16 bytes | no | A GUID in PCDS binary layout correlating this line to one execution of one program. | A collector MUST ignore fields it does not recognise (§3.29). ## 3.7.2 `origin` `origin` is what the producer says it is. A collector MUST NOT verify it, because it has no way to: the channel is a datagram socket and carries no peer identity (§3.4). Two producers MAY use the same origin, and one producer MAY use several. An origin is nonetheless the unit that read access is granted on (§3.28), and a collector matches it against patterns using dot-delimited prefix semantics: the pattern `svc` matches the origin `svc` and any origin beginning `svc.`, and matches neither `svc_daemon` nor `svcfoo`. A producer therefore SHOULD choose an origin that names it stably and distinguishably, and SHOULD use dots for hierarchy, because an administrator writing an access rule has nothing else to write it against. An origin MUST match the identifier grammar of §3.19: ```text [A-Za-z_][A-Za-z0-9_.-]* ``` A collector MUST discard a record whose origin does not. The constraint exists because an origin is not merely a label. It is matched against patterns in which `*` is the wildcard, so an origin containing `*` could not be selected exactly and could match a rule its producer was never meant to satisfy; and it is the name an access rule is stored under, so an origin carrying a path separator or a quoting character could land somewhere other than where the administrator who wrote the rule believes it is. Constraining the producer is the only point at which either can be prevented. Quoted forms remain valid syntax everywhere an origin may be written (§3.24). A conforming origin never needs them, but a *pattern* may, and a collector holding origins stored before this rule applied must still be able to return and select them. ## 3.7.3 `is_error` `is_error` is a boolean and deliberately not a severity level. A forwarding producer can distinguish standard output from standard error and nothing more; inventing five levels out of two file descriptors would be a guess presented as data. A producer with real severity levels either writes them into the message text, where they are text and are searched as text, or emits events, which have types. ## 3.7.4 `timestamp` A producer SHOULD supply the timestamp it captured when the line was produced, not when it submitted it. A producer that batches (§3.8) and omits the field attributes every line in the batch to the moment the collector happened to read it, which discards the timing information the batch was accumulated over. ## 3.7.5 `job_id` `job_id` correlates a line to a single execution rather than to a program. A forwarding producer sets it so that the output of one run of a service can be separated from the run before and the run after; a producer with no such notion omits it. A collector MUST treat it as an opaque 16-byte value. Nothing in this chapter interprets it, and a producer MAY use it for any correlation of its own, provided the value is a GUID. --- # 3.8 Validating a Log Record _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The three scopes of silent failure — the whole datagram, one record, or one field — and what a collector adds on the way in. Every failure on this channel is silent (§3.4). What differs between failures is *how much* is lost: the whole datagram, one record, or only one field. ## 3.8.1 The three scopes of failure **The datagram is discarded** when it cannot be resolved into records at all: - it is not valid MessagePack - it decodes to something that is neither a map nor an array of maps - the kernel reported it truncated (§3.6) **One record is discarded**, and the others in the same datagram are still processed, when the record itself is unusable: - a required field is absent - a required field has the wrong type — `origin` an integer, say - `origin` is the empty string - the map contains a duplicate top-level key **One field is ignored**, and the record is still stored, when an optional field is unusable: - `timestamp` is not an integer, is negative, or is outside the timestamp domain (§3.5) - `job_id` is not binary, or is binary of a length other than 16 A collector MUST implement all three scopes as stated. In particular it MUST NOT discard a record because an optional field was malformed: a producer with a broken clock or a mangled correlation key still has a log line worth keeping, and the field it got wrong is the field of least value in the record. ## 3.8.2 Duplicate keys A record map carrying the same top-level key twice MUST be discarded. A collector MUST NOT resolve the duplicate by taking the first or the last. MessagePack decoders differ on which they keep, and the fields here are entirely producer-controlled, so a rule that depended on decoder behaviour would let a producer choose which of two `origin` values a given collector saw. Discarding is the only answer that is the same everywhere. ## 3.8.3 Batches A batch is validated **per record**. A malformed record in a batch MUST NOT cost the valid records beside it. A producer SHOULD batch under sustained load. Batching amortises the syscall over many records, and the ceiling (§3.6) is per datagram, so a batch is also the only way to use the channel's capacity efficiently. The encoded datagram, batched or not, MUST NOT exceed the ceiling. A producer that batches without bounding the encoded size will eventually build a datagram that is discarded whole — which is the one case where batching loses more than sending singly would have. ## 3.8.4 What a collector adds A collector supplies the boot ID (§3.2) and, when the record omitted `timestamp`, its own clock reading at receipt. It MUST NOT alter any other field, and MUST store `message` byte-for-byte as given. A collector MUST NOT parse `message`. If the text happens to be JSON, or logfmt, or anything else structured, that is the producer's business: this interface carries lines, and a producer with structured data to record emits an event instead (§3.4). --- # 3.9 The Metric Channel _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The metric ingestion socket, separate from the log socket, and why the collector is a sink rather than a scraper. A collector MUST expose an `AF_UNIX` `SOCK_DGRAM` socket for metric ingestion, protected by a Security Descriptor as §3.3 requires, separate from the log socket. The channel works exactly as the log channel does, for the reasons given there: a declared datagram ceiling, truncated datagrams discarded whole, a receive queue of at most four times the ceiling, no backpressure, no notification, and no way for a producer to discover the ceiling (§3.6). > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. ## 3.9.1 A sink, not a collector of its own The collector is **pushed** to. It MUST NOT scrape an endpoint, read a kernel interface, or poll anything to obtain metrics; every sample it holds arrived on this socket because a producer sent it. What gathers the measurements is a separate concern and a separate program. A collection agent that reads system counters and submits them is an ordinary producer here, with no privileged position and no interface of its own. > [!NOTE] > The push direction follows from the loss model rather than from taste. > A pulling collector must reach every producer on a schedule, which > makes it responsible for their availability and makes a slow producer > the collector's problem. Pushing keeps each producer responsible for > its own submissions and keeps a slow or dead producer invisible to > everything except the gap it leaves. ## 3.9.2 Batching Batching matters more here than it does for logs. A collection sweep produces many samples at once — every CPU core, every disk, every interface — and they share a moment, so a producer SHOULD submit a sweep as one batched datagram rather than as one datagram per sample. The rules are the log rules: one map or an array of maps, per-record validation, and the encoded datagram bounded by the ceiling (§3.8, §3.12). --- # 3.10 The Metric Data Model _Peios / Advanced Peios / PSPU / Observability Interfaces_ > What a metric is on this interface — names, labels, and the counter, gauge and histogram types, all valued as binary64. A metric is a quantity that varies and is worth watching over time: a utilisation, a queue depth, a running total, a distribution of latencies. Metrics are dense where events and logs are sparse — many measurements of the same thing rather than a record of a thing that happened — and the model reflects that. A **sample** is one measurement, of one time series, at one moment. It carries: - a **name**, identifying what is measured - a **label set**, identifying which instance of it - a **type**, fixing how the value is to be read - a **timestamp** - a **value**, whose shape depends on the type Name and labels together identify the series (§3.13). The type is a property of the series, not of the sample. ## 3.10.1 Names A name MUST match the identifier grammar of §3.19: ```text [A-Za-z_][A-Za-z0-9_.-]* ``` A collector MUST discard a record whose name does not, for the same two reasons an origin is constrained (§3.7): names are matched against patterns in which `*` is the wildcard, and names are what read access is granted on. Beyond that, naming is convention and a collector MUST NOT enforce any. The conventions in use are a dot-separated hierarchy from general to specific, the unit as the last component, and a cumulative name for a cumulative quantity: ```text system.cpu.usage disk.read.bytes request.duration.seconds http.requests.total ``` ## 3.10.2 Labels Labels are the dimensions of a measurement: which core, which device, which method. `cpu.usage` with `core="0"` and `cpu.usage` with `core="1"` are two series, not two samples of one. Label keys and values MUST be non-empty UTF-8 strings. A key MUST match the identifier grammar above; a value MUST NOT contain `=` (0x3D) or `,` (0x2C), which are reserved as delimiters in the collector's canonical representation of a label set (§3.13). A key MUST NOT be repeated within one sample, and MUST NOT be any of the five fixed field names a metric result carries — `timestamp`, `boot_id`, `name`, `type`, `value` — because labels and fixed fields share one flat namespace in a result record (§3.22) and a collision would make the record ambiguous. A record violating any of these is discarded (§3.12). > [!NOTE] > Label keys are constrained to the identifier grammar for the reason > that matters most in practice: a key that cannot be written in a query > cannot be filtered on, grouped by, or granted access to. A label you > cannot address is storage you cannot reach. **Label cardinality is the producer's responsibility.** Each distinct combination of label values is a distinct series, so labels whose values are unbounded — request identifiers, user-supplied strings, timestamps — produce series without limit, and a collector is required neither to cap them nor to degrade gracefully when they arrive. A producer SHOULD use labels whose value sets are small and known. A dimension that is not bounded belongs in an event payload, where it costs one field, not in a label, where it costs a series. ## 3.10.3 Types The type is fixed when the series is first seen and is **immutable**. A sample that resolves to an existing series but declares a different type is discarded (§3.12), permanently and without notification. A producer that changes the type of a metric it already emits has stopped emitting it, and the only visible symptom is that the series stops advancing. ### 3.10.3.1 Counter A value that only increases, and resets to zero when the producer restarts. Used for cumulative quantities: requests served, bytes transmitted, errors encountered. A counter value MUST be a finite, non-negative binary64 value. The raw value is rarely what a reader wants; the rate of change is (§3.25). A decrease is read as a restart rather than as a negative change, which is why the type must be declared: the same number sequence means something different for a gauge. ### 3.10.3.2 Gauge A value that may move in either direction. Used for current state: a utilisation, an amount in use, a depth, a temperature. A gauge value MUST be a finite binary64 value, and MAY be negative. ### 3.10.3.3 Histogram A distribution of observations across buckets the producer chose. Used where the shape matters more than the mean — latencies above all. A histogram value carries: - **boundaries**: a non-empty array of bucket upper bounds, strictly increasing in the order given - **counts**: one cumulative count per boundary, each being the number of observations less than or equal to that boundary; non-decreasing, and each no greater than the total - **total_count**: the number of observations - **sum**: the finite sum of the observations Boundaries are part of the series identity (§3.13). A collector MUST NOT sort or reinterpret them; a producer that changes them has started a new series, and SHOULD therefore keep them fixed for the life of a metric. The final count MAY be less than the total: observations above the highest boundary are the difference between them, and are not otherwise represented. A total of zero is a valid empty sample, in which case every count and the sum MUST be zero. > [!NOTE] > Observations above the highest boundary are counted but not located. > A reader asking for a high percentile of a distribution whose tail > overflows gets no answer rather than a wrong one (§3.25), so a producer > SHOULD choose a highest boundary above the values it expects to see. ## 3.10.4 Values are floating point Numeric input MAY be a MessagePack integer or a MessagePack float; both are converted to binary64 with round-to-nearest, ties-to-even. Every value a collector stores and every value a query returns is a finite binary64. Non-finite values are refused rather than stored: a record whose value converts to NaN or to either infinity is discarded (§3.12). There is no representation for a missing measurement — a producer with nothing to report sends nothing, and the gap is the answer (§3.13). --- # 3.11 Metric Records _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The MessagePack map that carries one metric sample, the histogram value shape, and why type is per record rather than per series. A metric datagram carries one record, encoded as a MessagePack map, or several, encoded as an array of maps. A collector MUST accept both (§3.9). Each map is one sample of one series. ## 3.11.1 Fields | Field | Type | Required | Meaning | |---|---|---|---| | `name` | string | yes | The metric name (§3.10). | | `labels` | map | no | Key-value string pairs. Absent means the series has no labels, which is not the same as a series whose labels are empty — it is the same series. | | `type` | string | yes | Exactly `"counter"`, `"gauge"` or `"histogram"`, lowercase. | | `timestamp` | integer | no | When the measurement was taken, in the timestamp domain (§3.5). Absent means the collector uses its own clock at receipt. | | `value` | varies | yes | The measurement. A number for counter and gauge; a map for histogram. | A collector MUST ignore fields it does not recognise (§3.29). ## 3.11.2 The histogram value For a histogram, `value` is a map: | Field | Type | Meaning | |---|---|---| | `boundaries` | array of number | Non-empty, finite bucket upper bounds, strictly increasing after conversion to binary64, in the order given. | | `counts` | array of integer | Cumulative count per boundary. Same length as `boundaries`. Non-decreasing, each no greater than `total_count`. | | `total_count` | integer | Number of observations. | | `sum` | number | Finite sum of the observations. | Counts and `total_count` MUST be MessagePack unsigned integers, or non-negative signed integers. `boundaries` and `sum` MAY be integers or floats and are converted as §3.10 requires. ## 3.11.3 `type` is per record, not per series Every record declares its type, including the second and every subsequent sample of a series that already exists. This is redundant on the wire and deliberately so. A producer holds no state about what a collector already knows, has no way to ask, and MUST NOT be required to establish a series before sampling it: the first sample of a series and the millionth are the same message. The redundancy is what makes a producer stateless, and the cost is one short string per sample. The collector uses the declaration only on first sight. Afterwards it is a consistency check, and a record that fails it is discarded (§3.10). ## 3.11.4 Timestamps need not increase A collector MUST store a valid sample whose timestamp is older than samples it already holds for that series. Producers batch, clocks step, and a collection sweep may be submitted out of order or retried. A collector that refused late samples would turn any of those into silent data loss, so it accepts them and defines every ordering it performs over `timestamp` rather than over arrival (§3.21, §3.25). Two samples of one series MAY share a timestamp; the collector orders them deterministically and a client MUST NOT depend on which comes first. --- # 3.12 Validating a Metric Record _Peios / Advanced Peios / PSPU / Observability Interfaces_ > What causes a metric datagram or a single record to be discarded, silently, and why the timestamp rule differs from the log channel's. Failures on this channel are silent, exactly as on the log channel (§3.4, §3.8). The scopes are the same, with one difference that matters: **a metric record has no ignorable field.** Every field a metric record carries participates either in the series identity or in the measurement, so there is nothing whose loss leaves a usable record behind. A malformed `timestamp` costs the log line nothing and costs the sample everything. ## 3.12.1 The datagram is discarded - it is not valid MessagePack - it decodes to something that is neither a map nor an array of maps - the kernel reported it truncated (§3.9) ## 3.12.2 The record is discarded A collector MUST discard a record, leaving the rest of its batch untouched, for any of the following. **Structure** - a required field is absent, or has the wrong type - the map contains a duplicate top-level key - `type` is not exactly `"counter"`, `"gauge"` or `"histogram"` **Name and labels** (§3.10) - `name` is empty or does not match the identifier grammar - a label key or value is not a string, or is empty - a label key does not match the identifier grammar - a label value contains `=` or `,` - a label key is repeated within the record - a label key is one of `timestamp`, `boot_id`, `name`, `type`, `value` **Timestamp** - `timestamp` is present and is not an integer, is negative, or is outside the timestamp domain (§3.5) **Counter and gauge values** - the value is not a number - it converts to a non-finite binary64 - it is negative and the type is counter **Histogram values** - the value is not a map, or its map has a duplicate key - a field is absent or has the wrong type - `boundaries` is empty - a boundary or `sum` converts to a non-finite binary64 - `counts` and `boundaries` differ in length - the converted boundaries are not strictly increasing - a count is negative, the counts are not non-decreasing, or a count exceeds `total_count` - `total_count` is zero and any count or `sum` is non-zero **Series consistency** - the record resolves to an existing series whose type differs (§3.10) ## 3.12.3 Why the timestamp rule differs from logs On the log channel a malformed timestamp is ignored and the record kept; here it discards the record. The asymmetry is not an inconsistency. A log line with the wrong time is still the line, and reading it is still worth doing. A sample is a `(time, value)` pair and nothing else: attaching the collector's receipt time to a measurement taken at an unknown moment does not recover the sample, it fabricates one, and it fabricates one that will be charted next to real ones. Discarding leaves a gap, which is honest (§3.4). Absence is different from malformation. A record that simply omits `timestamp` is asserting "now", and the collector's clock is the right answer to that. ## 3.12.4 Silence, again A collector MUST NOT emit an event, log an error, or increment anything a client can observe in response to any failure in this article — with no exception for the series-consistency failure, which is the one that most looks like it deserves one. A producer that changes a metric's type is misconfigured, and the misconfiguration is permanent and invisible: every sample is discarded for as long as the series exists. The only signal available to an operator is that the series stopped advancing while the producer reported no error, and the only diagnosis is to query the series and read its `type` (§3.25). --- # 3.13 Series Identity _Peios / Advanced Peios / PSPU / Observability Interfaces_ > What makes two samples the same time series — name, labels and histogram boundaries — and what pointedly does not participate. Two samples belong to the same time series when they agree on: - the **name**, compared exactly, byte for byte; and - the **label set**, compared as an unordered set of key-value pairs, each compared exactly; and - for histograms, the **bucket boundaries**, compared as an ordered sequence of binary64 values. Nothing else participates. Type does not: a record whose type disagrees resolves to the series and is then discarded for disagreeing (§3.10). Boot ID does not: a series continues across a reboot, and a client that wants one boot's worth filters for it (§3.25). Time does not. ## 3.13.1 Order does not distinguish a label set `{core: "0", host: "a"}` and `{host: "a", core: "0"}` are the same series. A collector MUST compare label sets as sets. To do so it needs a canonical form, and the form in use is the reason label values may not contain `=` or `,` (§3.10): pairs are sorted by key in unsigned UTF-8 byte order, each written `key=value`, and joined with commas. Because neither delimiter can occur inside a key or a value, no escaping is needed and no two distinct label sets can produce the same string. The byte form itself is the collector's business and this chapter does not require it. What it requires is the property: **a label set has exactly one identity, independent of the order the producer wrote it in.** The delimiter reservation is stated normatively because it binds the *producer*, and a producer cannot see the encoding that motivates it. ## 3.13.2 Absent labels and empty labels A record with no `labels` field, a record with an empty `labels` map, and a record whose labels were all discarded are the same series: the one with no labels. There is no distinction between "unlabelled" and "labelled with nothing". ## 3.13.3 Boundaries are identity, not metadata Two histogram samples with different boundaries are different series even when name and labels agree. This is the consequence that surprises producers, and it is unavoidable: cumulative counts against one set of bucket edges cannot be compared with counts against another, so calling them one series would mean computing percentiles across incommensurable distributions. A producer that re-tunes its buckets each collection cycle creates a series each cycle, each holding a single sample and each surviving until retention removes it. A collector MUST NOT defend against this. It is a producer defect, it is indistinguishable at the interface from legitimately introducing a new metric, and every defence available — capping series, merging near-equal boundary sets, rejecting a second boundary set for a name — would break a correct producer to inconvenience an incorrect one. ## 3.13.4 Series are created, never announced A series comes into existence when its first sample arrives. There is no registration message, no schema, and no way to declare a series in advance or to retire one. A collector MUST NOT require a series to be known before a sample of it is accepted, and MUST NOT limit how many series exist. A series with no remaining samples ceases to exist when retention removes the last of them; nothing else removes one. > [!NOTE] > The absence of a cap is a deliberate and load-bearing decision, and it > is worth being clear about what it costs. Labels are producer-supplied > and unverified (§3.4), so any process that can reach the metric socket > can create series without limit, and each persists until the retention > window elapses. The Security Descriptor on the socket (§3.3) is the > only thing that bounds this. ## 3.13.5 Gaps are preserved A collector MUST NOT interpolate, backfill, or synthesise a sample that a producer did not send. A missing sample is a real fact about the system — the producer was down, the datagram was dropped, the sweep was late — and it is a fact the metric is often being watched for. A series with a hole in it is returned with a hole in it, and a client that wants a value across the hole computes one itself. --- # 3.14 The Query Channel _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The query socket — one query per connection, the identity captured at connect, and why no credentials cross it. A collector MUST expose an `AF_UNIX` `SOCK_STREAM` socket for queries, protected by a Security Descriptor as §3.3 requires. One socket serves all three data types. The mode a query runs in — events, logs or metrics — is determined by parsing the query string (§3.18), never by the transport, so a client needs no connection setup, no mode selection, and no separate endpoint per data type. ## 3.14.1 One query per connection A connection carries exactly one query. A client that wants two concurrent queries opens two connections. A collector MUST close the connection after the terminal message of a non-streaming query (§3.16), and MUST treat a client disconnect as cancellation of a streaming one. > [!NOTE] > This is the opposite of the identity-lookup channel in PGSS §2.14, > which multiplexes tagged requests over one connection. The reason is > the shape of the work rather than a difference in taste: a lookup is > small, uniform and answered in one message, so correlating many on one > connection saves real cost. A query may run for thirty seconds, may > stream indefinitely, and may be cancelled — and every one of those is > expressed by the connection itself, with no correlation identifier and > no cancellation message to specify. ## 3.14.2 Identity A collector MUST establish the client's identity from the connected socket, before executing anything, by obtaining the peer's token. This is possible here and not on the ingestion channels, and it is the whole reason the query channel is a stream (§3.3). The token is captured **once**, at connection time, and is a snapshot. A client whose privileges change while a query runs — and in particular while a streaming query runs, which may be indefinitely — is evaluated throughout against the token it connected with. If a collector cannot obtain the peer token, it MUST refuse the query. It MUST NOT execute a query for an unidentified caller, and MUST NOT fall back to any other means of identifying one. ## 3.14.3 No credentials cross this channel There is no message with which a client offers a credential and none with which a collector asks for one. Identity is established from the connection and from nothing else. ## 3.14.4 Concurrency A collector MUST bound the number of queries it will execute at once, and MUST reject a query beyond the bound with an error rather than queueing it behind the others. The streaming bound is the lower of the two and is enforced separately, because a streaming query holds its resources for as long as its client stays connected while an ordinary one holds them for at most a timeout (§3.16). > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. Both bounds are global. Neither is per-client, because a collector cannot attribute connections to a caller beyond the token it has, and one client MAY therefore occupy every slot. A collector MUST NOT allow that to affect ingestion: queries and ingestion are separate channels precisely so that exhausting one cannot exhaust the other (§3.3). --- # 3.15 Query Framing _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Length-prefixed MessagePack in both directions, the message ceiling, and the requirement that the ceiling admit every record. Every message in either direction is a length-prefixed MessagePack value: | Offset | Size | Field | Value | |---|---|---|---| | 0 | 4 | `length` | Length of the payload in bytes | | 4 | `length` | `payload` | The request or response body | `length` is little-endian, as PSPU §1.2 requires of every integer field in this document. It counts the payload only; the four bytes of the prefix are not included. There is no magic value and no version field. The channel is a Unix socket at a configured path, so there is no possibility of reaching the wrong service by accident in the way a shared header guards against (PSPU §2.7), and versioning is handled as §3.29 describes. ## 3.15.1 The message ceiling A collector declares a maximum payload size, the **query message ceiling**, which bounds messages in both directions. > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. **Inbound.** A collector MUST refuse a request whose `length` exceeds the ceiling. It MUST do so without reading the payload — the point of checking the prefix is to avoid allocating for a request that a malicious or broken client has declared too large — and it MUST send an error response (§3.16) before closing the connection. A collector MUST NOT close on an oversized request silently: a bare close is indistinguishable from a crash, and leaves a client unable to tell that shortening its query is the remedy. **Outbound.** A collector MUST ensure every response payload it sends is within the ceiling, chunking result records across messages as §3.16 describes. ## 3.15.2 The ceiling must admit every record A collector MUST NOT operate with a query message ceiling smaller than the largest record it can store. A single result record is never split across messages (§3.16), so a record larger than the ceiling cannot be returned at all — and it cannot be skipped either, because skipping it would silently misreport what the store holds. It fails the query, and it fails every query whose range covers it, for as long as retention keeps it. One oversized record renders a span of history unreadable. The two are related by configuration and nothing enforces the relation automatically: the ingestion ceilings (§3.6, §3.9) bound the largest record a producer can deposit, and the query message ceiling bounds the largest that can be handed back. An administrator who raises one MUST raise the other. > [!NOTE] > The mainline defaults do not satisfy this. The log and metric datagram > ceilings are 262144 bytes and the query message ceiling is 65536, so a > producer can deposit a log line four times larger than any response > that could carry it. ## 3.15.3 Requests A request is a MessagePack map: | Field | Type | Required | Meaning | |---|---|---|---| | `query` | string | yes | The query string (§3.18). | A collector MUST send an error response and close the connection if the payload is not valid MessagePack, is not a map, omits `query`, gives `query` a non-string value, or contains a duplicate top-level key. It MUST ignore unrecognised fields that are not duplicates (§3.29). Unlike the ingestion channels, nothing here is silent. A query client is identified (§3.14), is one of a bounded number, and is asking a question, so telling it what went wrong is neither an amplification vector nor an information leak — with the one exception §3.28 sets out. --- # 3.16 Responses _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The four response kinds a query can receive, how records are chunked, and how errors and timeouts are reported. A response is a MessagePack map whose `status` field names its kind. There are four. | `status` | Carries | Meaning | |---|---|---| | `"ok"` | `records` | A chunk of result records. | | `"end"` | — | The non-streaming query is complete. | | `"watch"` | — | The streaming query's initial result set is complete. | | `"error"` | `error` | The query failed. | ## 3.16.1 Result messages An `"ok"` message carries `records`, an array of flat maps (§3.22). Records are chunked at record boundaries: a successful query sends one or more `"ok"` messages, each within the message ceiling (§3.15). A collector MUST NOT split one record across two messages. A record too large to fit in a message alone MUST fail the query with an error rather than being truncated, partially sent, or skipped. Each record is self-describing and records in one response MAY carry different sets of keys — event payload fields vary by event type, metric labels vary by series. A client MUST NOT assume a uniform schema across a result set, and MUST NOT infer that a key absent from one record is absent from the data. A successful query with no matching records sends exactly one `"ok"` message with an empty `records` array, then its terminal message. A collector MUST NOT omit it: "no records" and "the query has not yet produced records" are different states and a client must be able to tell them apart. ## 3.16.2 The two terminal messages `"end"` terminates a non-streaming query. `"watch"` marks the point in a streaming query where the stored result set ends and live delivery begins (§3.27). A query sends exactly one of them, never both. Until one has arrived, **the query has not succeeded**. A collector that fails partway through MUST send `"error"`, and a client that receives `"error"` before either terminal message MUST discard every `"ok"` message it received for that query. Partial results are not results: they are an arbitrary prefix of an ordering that was never completed, and a client that kept them would silently under-report. An `"error"` *after* `"watch"` is different. It terminates the stream, and the records already delivered remain valid — they were complete when they were sent, and the ordering they belonged to had already closed. ## 3.16.3 Errors An `"error"` message carries `error`, a human-readable string. There is no error code and no machine-readable classification. This is a deliberate limit on the interface: an error here is a parse failure, a type mismatch, a timeout, a limit, or a refusal, and a client's response to all of them is the same — show it to whoever wrote the query. A client MUST NOT parse the string, and a collector MAY change the wording of any error at any time. A collector MUST NOT include in an error message any value the client was not authorized to read (§3.28). ## 3.16.4 Timeouts A collector MUST bound the time a query may take to reach its terminal message. > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. The clock starts once the request has been decoded and the caller's token obtained, and it covers everything that follows: parsing, access checks, cross-type pre-computation, execution, merging, aggregation, pagination, projection and transmission. A collector MUST send `"end"`, or for a streaming query `"watch"`, before it expires. The timeout bounds the **initial result set only**. Once a streaming query has sent `"watch"` its watch phase is not time-limited; what bounds it instead is the streaming concurrency limit (§3.14), the distinct-value limit (§3.27), and the client's own ability to keep up (§3.27). On expiry a collector MUST cancel the query and send `"error"`. Any `"ok"` messages already sent are discarded by the client under the rule above. --- # 3.17 Value Encoding _Peios / Advanced Peios / PSPU / Observability Interfaces_ > How every value in a result record is encoded — including why a GUID is a string — and why missing and null are the same value. Every value in a result record is encoded as follows. | Value | Encoding | |---|---| | Integer | MessagePack integer | | Float | MessagePack `float64` | | Timestamp | MessagePack integer, nanoseconds since the Unix epoch (§3.5) | | String | MessagePack string | | GUID | MessagePack string, PCDS canonical form | | Binary | MessagePack `bin` | | Boolean | MessagePack boolean | | Array | MessagePack array | | Absent or null | MessagePack nil | A GUID is rendered as a string rather than as sixteen bytes because a result record is read by people as often as by programs, and a raw GUID in a terminal is unreadable. The canonical form is lowercase `8-4-4-4-12` hexadecimal within braces, as PCDS defines it. A query comparing against a GUID accepts either braced or unbraced input, and compares case-insensitively (§3.19); a result always uses the canonical form. ## 3.17.1 Maps do not appear as values An event payload is a MessagePack map, and result records are flat (§3.22). A map in a stored payload is therefore a *container to be flattened*, not a value to be emitted: its entries become top-level keys of the record, joined by dots, and the map itself never appears. Arrays are different. An array is emitted as an array value at its flattened path, and a collector MUST NOT traverse into it. Maps nested inside an array are preserved as that array's contents, unflattened and unqueryable. The asymmetry is deliberate. A map has keys, so its entries have names that can be addressed, granted access to and indexed. An array has positions, and a path like `hops.3.address` would mean something different in every record — so an array is carried across whole and treated as one value. Binary values in a payload stay binary. A collector MUST NOT render `bin` as a string, in either direction. ## 3.17.2 Missing and null are the same value A field absent from an event payload or from a metric's label set encodes as nil, exactly as an explicitly null one does, and the two are indistinguishable in a result record. This is consistent throughout: they compare equal, they sort together, and they group together (§3.20, §3.21). A collector MUST NOT distinguish them anywhere in the query surface, and a client MUST NOT attempt to. --- # 3.18 The Query Language _Peios / Advanced Peios / PSPU / Observability Interfaces_ > One string that names a mode, narrows to some data and says what to do with it — the three modes, the clauses, and their execution order. A query is one string. It names a mode, narrows to some data, and says what to do with it. ```text EVENTS kacs.* SINCE 1h ago WHERE process_guid == "550e8400-e29b-41d4-a716-446655440000" TAKE 100 LOGS FROM loregd ERROR ONLY CONTAINING "connection refused" SINCE 1d ago METRIC cpu.usage[core="0"] SINCE 1h ago AVG_OVER 5m ``` ## 3.18.1 Three modes The first token selects the mode, and a collector MUST reject a query whose first token is not one of them. - **`EVENTS`** searches structured event records, primarily by event type (§3.23). - **`LOGS`** searches log output, primarily by origin (§3.24). - **`METRIC`** evaluates measurements, primarily by name and labels (§3.25). Events and logs are *record-oriented*: collections you search, returning the records that matched. Metrics are *value-oriented*: measurements you evaluate, returning numbers computed from samples. The modes differ because the data differs, and forcing all three through one shape would serve none of them. ## 3.18.2 The primary selector Immediately after the mode comes an optional **primary selector**, specific to the mode: an event type pattern, `FROM` with one or more log origins, or a metric name with an optional label selector. It narrows the data before anything else runs. A primary selector MUST NOT be repeated unless its mode defines a list form — `LOGS FROM a, b` is one selector naming two origins, not two selectors. ## 3.18.3 Clauses Everything after the primary selector is a **clause**, and clauses MAY appear in any order. `EVENTS SINCE 1h ago TAKE 10` and `EVENTS TAKE 10 SINCE 1h ago` are the same query. Order of appearance never affects meaning. Execution follows the fixed sequence below regardless of how the string was written, so a collector MUST NOT derive semantics from clause position. These clauses work identically in all three modes: | Clause | Meaning | |---|---| | `SINCE t` | Lower time bound, inclusive. | | `UNTIL t` | Upper time bound, exclusive. Defaults to the evaluation time. | | `WHERE p` | Filter by a predicate (§3.20). | | `WHERE METRIC …` / `WHERE EVENT …` / `WHERE LOG …` | Filter by a condition on another data type (§3.26). | | `SORT f [ASC\|DESC], …` | Order the results (§3.21). | | `TAKE n` | Return at most `n`. | | `SKIP n` | Discard the first `n` after ordering. | | `STREAM` | Deliver matching records as they arrive (§3.27). | ## 3.18.4 Execution order Whatever order the clauses were written in, a collector MUST evaluate them in this sequence: | | Phase | |---|---| | 1 | Cross-type conditions, producing time ranges (§3.26) | | 2 | The primary selector | | 3 | Access control on the primary and cross-type sources (§3.28) | | 4 | `SINCE` and `UNTIL` | | 5 | `WHERE`, including the ranges from phase 1 | | 6 | `ERROR ONLY` and `CONTAINING`, as `WHERE` predicates (§3.24) | | 7 | Metric transforms (§3.25) | | 8 | `GROUP` | | 9 | `COUNT BY`, `TOP N BY`, `DISTINCT`, and the aggregation functions | | 10 | Metric window aggregations (§3.25) | | 11 | `SORT` (§3.21) | | 12 | `SKIP` and `TAKE` | | 13 | `SELECT` (§3.22) | Two positions in that list are load-bearing. **Access control is third**, before every filter, aggregate, sort and limit. It is part of the query's logical execution and not a filter applied to the output (§3.28). **`SELECT` is last.** It shapes the output and nothing else; a field it omits is still available to every earlier phase (§3.22). ## 3.18.5 Repetition A clause MUST appear at most once, and a collector MUST reject a repeat as a parse error, with two exceptions: - **`WHERE` is repeatable.** Multiple `WHERE` clauses are combined with `AND`, each treated as a parenthesised group: `WHERE a == 1 OR b == 2` followed by `WHERE c == 3` means `(a == 1 OR b == 2) AND c == 3`. - **`SELECT` is repeatable** where it is valid at all, and is additive: `SELECT timestamp SELECT event_type` names both fields. Both exist so that a query can be built up in pieces — by a tool appending a filter, or by a person adding one to a query they already have — without rewriting what is already there. ## 3.18.6 Counts `TAKE`, `SKIP` and the `N` of `TOP N BY` are unsigned decimal integers that MUST fit in 64 bits. A negative, hexadecimal, floating-point or missing count is a parse error. `SKIP` defaults to 0. `TAKE` omitted means no limit. `TAKE 0` and `TOP 0 BY` are valid, and return no records after every earlier phase has run — which is not the same as not running the query, because a `TOP 0 BY` still counts and a `TAKE 0` still enforces access control. > [!NOTE] > A non-aggregating query without `TAKE` has no implicit limit, and a > broad one over a long range may match millions of records. The timeout > (§3.16) is the only backstop. ## 3.18.7 Case Keywords are matched case-insensitively, using ASCII case folding, in grammar positions where a keyword is expected. This document writes them in uppercase by convention only. Identifiers are case-sensitive, except where the language defines a named alias for a value (§3.23). A word spelled like a keyword MAY be used where the grammar expects an identifier or a value: `LOGS FROM stream` selects the origin `stream`, while `LOGS STREAM` enables streaming. A collector MUST resolve the ambiguity by grammar position and MUST NOT reserve keywords globally. --- # 3.19 Lexical Rules and Literals _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The lexical layer — identifiers, strings, binary, numbers, durations and times — and what fixes the evaluation time. A query string is UTF-8. Whitespace separates tokens outside quoted strings and is otherwise insignificant. ## 3.19.1 Identifiers An unquoted identifier is ASCII and matches: ```text [A-Za-z_][A-Za-z0-9_.-]* ``` Identifiers name fields, payload paths, metric names, label keys, event type patterns, log origins and value aliases. `.`, `_` and `-` are permitted inside one; `/`, `:`, whitespace, quotes, brackets, parentheses, commas and the comparison operators are not. This grammar is the same one that constrains a log origin, a metric name and a metric label key at ingestion (§3.7, §3.10), which is what makes every stored identifier writable here without quoting. A value that cannot be written as an identifier MUST be written as a quoted string. Quoted forms are accepted anywhere an identifier is — they are never required for a conforming identifier, but a *pattern* may need one, and a collector holding identifiers stored under an earlier revision must still be able to select them. ## 3.19.2 Strings A string literal is double-quoted UTF-8. The escapes are `\"`, `\\`, `\n`, `\r`, `\t`, and `\uXXXX` for a scalar value in `U+0000` to `U+FFFF` written as four hexadecimal digits. A collector MUST reject any other backslash escape as a parse error, and MUST reject `\uXXXX` naming a surrogate code point in `U+D800` to `U+DFFF`. Surrogate pairs are not decoded: a character outside the basic multilingual plane is written directly as UTF-8, not as two escapes. > [!NOTE] > Refusing surrogates rather than pairing them means there is exactly one > way to write every character, and no way to write a sequence that is > not valid UTF-8. A language that accepts lone surrogates has to decide > what they mean when compared against stored text that cannot contain > them. ## 3.19.3 Binary A binary literal is a lowercase `x`, a double quote, an even number of hexadecimal digits, and a closing quote: ```text WHERE target_sid == x"010500000000000515000000" ``` Hexadecimal digits inside are case-insensitive. `x""` is valid and is the empty byte string. Whitespace inside the payload, an odd digit count, and any non-hexadecimal character are parse errors. A binary literal compares only against MessagePack `bin` values. A collector MUST NOT coerce one to a string or a string to one: `x"6162"` and `"ab"` are different values and never compare equal (§3.20). ## 3.19.4 Integers An integer literal is decimal or hexadecimal. ```text WHERE origin_class == 2 WHERE granted_access == 0x1F01FF ``` A decimal integer MAY carry a leading `-`, in which case it MUST fit in signed 64 bits; without one it MUST fit in unsigned 64 bits. A hexadecimal integer is `0x` followed by one or more digits, is always non-negative, and MUST fit in unsigned 64 bits. A leading `+` is not valid. An out-of-range literal is a parse error. ## 3.19.5 Floats A float literal is a finite decimal number with an optional leading `-` and either a fractional part or an exponent: `42.0`, `0.001`, `1e6`, `-1.25e-3`. A token that looks like an integer, such as `42`, **is** an integer literal and not a float. Float literals are binary64 and MUST be finite. `NaN`, `Infinity`, `-Infinity` and any literal that overflows to infinity are parse errors. A leading `+` is not valid. ## 3.19.6 Booleans and null `true` and `false` are matched case-insensitively with ASCII folding. `NULL`, likewise folded, is valid **only** in `IS NULL` and `IS NOT NULL`. A collector MUST reject `field == NULL` and `field != NULL` as parse errors rather than evaluating them. > [!NOTE] > Equality against null is refused rather than defined because every > definition of it is a trap. Under three-valued logic it is neither > true nor false, which no other operator here does; under two-valued > logic it silently disagrees with SQL. `IS NULL` says what was meant > and cannot be misread. ## 3.19.7 Durations A duration is an unsigned decimal integer followed immediately by `s`, `m`, `h` or `d` — seconds, minutes, hours or days. A zero duration is a parse error. ## 3.19.8 Times | Literal | Meaning | |---|---| | ` ago` | That duration before the evaluation time. | | ` hence` | That duration after it. | | `today` | Midnight of the current day, UTC. | | `yesterday` | Midnight of the previous day, UTC. | | `YYYY-MM-DD` | Midnight of that date, UTC. | | `YYYY-MM-DDTHH:MM:SS` | That instant, UTC. | Absolute literals are a fixed UTC subset. Components MUST be zero-padded exactly as shown, the date MUST be a valid Gregorian date, hours are `00`–`23`, minutes and seconds `00`–`59`. Leap seconds are not accepted. Timezone suffixes and fractional seconds are not part of this revision and MUST produce a parse error. ## 3.19.9 The evaluation time A collector MUST capture the evaluation time **once**, before execution begins, and MUST use that one reading for every `ago`, every `hence`, and for an omitted `UNTIL`, throughout the query — including throughout the watch phase of a streaming query. A query that read the clock more than once could produce a range whose end preceded its start, or a window that grew while it was being scanned. One reading makes the effective query range a fixed interval for the life of the query. `SINCE` is inclusive, `UNTIL` is exclusive: the effective query range is `[SINCE, UNTIL)`. If `SINCE` is greater than or equal to `UNTIL` the query returns no records — which is a successful query with an empty result (§3.16), not an error. A time literal that evaluates outside the timestamp domain MUST produce an error (§3.5). --- # 3.20 Comparison and Logic _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The operators, case-folding on strings, why types never coerce, how absent fields behave, and how predicates combine. ## 3.20.1 Operators | Operator | Meaning | Operand types | |---|---|---| | `==` | Equal | any | | `!=` | Not equal | any | | `>` `>=` `<` `<=` | Ordering | integer, float, timestamp | | `STARTS_WITH` | Prefix | string | | `ENDS_WITH` | Suffix | string | | `CONTAINS` | Substring | string | | `IN` | Member of a set | any | | `NOT_IN` | Not a member | any | | `IS NULL` | Absent or null | any | | `IS NOT NULL` | Present and not null | any | `IN` and `NOT_IN` take a non-empty parenthesised, comma-separated list of literals. An empty list is a parse error. ```text WHERE origin IN ("loregd", "peinit") WHERE origin_class NOT_IN (kacs, lcs) ``` `=` is not a comparison operator and MUST produce a parse error, with one exception: inside a metric label selector, where `=` and `==` are both equality (§3.25). ## 3.20.2 Strings fold case Every string comparison — `==`, `!=`, `STARTS_WITH`, `ENDS_WITH`, `CONTAINS`, `IN`, `NOT_IN` — is **case-insensitive**, using ASCII-only folding: bytes `A`–`Z` compare equal to `a`–`z`, and every non-ASCII byte compares exactly. This applies uniformly: to event header fields, to payload fields, to log messages, to metric label values, and to the pattern matching of primary selectors. Integers, floats, GUIDs, timestamps and binary values are unaffected. > [!NOTE] > Folding is ASCII-only rather than Unicode because full case folding is > locale-dependent, version-dependent and expensive, and because the data > it would be applied to — identifiers, service names, event types — is > ASCII by construction (§3.19). A collector that folded Turkish dotless > `ı` would have to fold it the same way as every other collector, for > every Unicode version, forever. ## 3.20.3 Numbers compare mathematically Integers and floats compare by mathematical value, not by casting both to one storage type. An integer equals a finite float only when the float represents exactly that value. Ordering between an integer and a float MUST be exact, including for integers outside the range binary64 can represent exactly. A collector MUST NOT resolve `9007199254740993 > 9007199254740992.0` by converting the left operand to a float, which would make it false. ## 3.20.4 Types do not coerce Values of different non-numeric types are never equal. The string `"1"` is not the integer `1`, and `!=` between them is true. An ordering operator applied to a field whose runtime value is non-numeric evaluates **false** for that record — not an error, because a payload field's type varies from record to record and a query cannot know in advance. An ordering operator applied to a *known fixed field* whose declared type cannot be ordered is different: a collector MUST reject the query during parsing or planning rather than executing a predicate that can never match. `WHERE message > 5` is a mistake the collector can see, and returning zero records for it would be a wrong answer that looks like a right one. Binary values compare by exact byte equality under `==`, `!=`, `IN` and `NOT_IN`. Ordering is **not defined** for binary values, and a predicate applying an ordering operator to a binary literal MUST produce a parse error. ## 3.20.5 Absent fields A field absent from an event payload or from a metric's label set resolves to null (§3.17). Every comparison against null evaluates false, except `IS NULL`, which is true, and `IS NOT NULL`, which is false. In particular `WHERE field != "x"` does **not** match records lacking the field: a record with no opinion is not a record with a different opinion. ## 3.20.6 Combining predicates Predicates within one `WHERE` combine with `AND` and `OR`. `AND` binds tighter than `OR`. Parentheses override. Multiple `WHERE` clauses combine with `AND`, each parenthesised as a group (§3.18). There is no `NOT`. Negation is written with the negative operators — `!=`, `NOT_IN`, `IS NOT NULL` — and a collector MUST reject `NOT` as a parse error rather than silently treating it as an identifier. --- # 3.21 Ordering, Grouping and Distinct _Peios / Advanced Peios / PSPU / Observability Interfaces_ > SORT, GROUP and DISTINCT — and the two rules that make paging a query safe: one query, one order. Two rules govern this article, and both exist for the same reason. **Ordering MUST be total and deterministic.** For a fixed set of stored records, one query MUST produce one order. Without that, `SKIP` and `TAKE` are meaningless: a client paging through results would see records twice and never see others, and would have no way to tell. **Equality here MUST be the query language's, not the storage engine's.** A collector that grouped by whatever its database considers equal would group differently depending on how it was built. ## 3.21.1 SORT `SORT` orders by one or more fields. Each defaults to ascending; `ASC` may be written, `DESC` reverses that field. ```text SORT timestamp DESC SORT origin ASC, timestamp DESC ``` If the named fields do not uniquely order two records, a collector MUST append internal tiebreakers until the order is total. The tiebreakers are not query-language fields, are never emitted in a result record, and a client MUST NOT depend on their identity — only on their effect, which is that the order is stable. When no `SORT` is present: - **Events and logs** are ordered by timestamp descending, most recent first — the order a person reading a log wants. - **Metrics** are ordered by timestamp ascending, the order a chart wants. ## 3.21.2 Value ordering `SORT` uses the query language's ordering, not the storage engine's. Missing fields and explicit nulls are equivalent. Ascending order sorts by type first, in this order: 1. Null 2. Boolean, `false` before `true` 3. Numeric, integers and floats compared mathematically (§3.20) 4. String and GUID, ASCII-folded 5. Binary, unsigned lexicographic 6. Array `DESC` reverses the whole ordering, type order included. Strings and GUIDs compare with the same ASCII folding as predicates. Two strings equal under folding are ordered by their original UTF-8 bytes, so that folding never costs totality. Binary values compare as unsigned bytes. Arrays compare by their canonical MessagePack encoding (§3.5). Maps do not appear as result values (§3.17) and MUST NOT appear as sort keys. ## 3.21.3 Grouping `COUNT BY`, `TOP N BY`, `GROUP` and `DISTINCT` use query-language equality: - missing and null are one group - integers and floats that are numerically equal are one group - strings and GUIDs group under ASCII folding - binary values group by exact bytes ## 3.21.4 The canonical representative When a group's members are equal under those rules but not byte-identical — `"Loregd"` and `"loregd"`, or `1` and `1.0` — the value emitted for the group MUST be its **canonical representative**: | Group | Representative | |---|---| | Null | nil | | Boolean | the boolean | | Numeric | an integer if every contributing value was an integer; otherwise a `float64` | | String or GUID | the smallest original UTF-8 byte sequence among the members | | Binary | the exact value | | Array | the member with the smallest canonical MessagePack encoding | Choosing the smallest rather than the first makes the representative a property of the *set*, independent of the order records were read in — which matters because a collector may read them from several places at once and merge (eventd TRMP §6.4). ## 3.21.5 Ordering of aggregates `COUNT BY` results are ordered by `count` descending. Ties are broken by the group key under the value ordering above, then by the representative's encoded bytes. `TOP N BY` is exactly `COUNT BY` with `TAKE N` applied after that ordering. `DISTINCT` results are ordered by the distinct value under the value ordering, unless an explicit `SORT` overrides it. --- # 3.22 Fields and Results _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Every result record is a flat map — the event, log and metric field sets, the reserved header names, and how nesting is flattened. Every result record is a **flat** MessagePack map. There is no nesting in a result, in any mode. Flatness is what makes one set of rules — for access control, for ordering, for grouping, for projection — apply uniformly to a header field, a payload field and a metric label alike. A nested result would need a path language, and a path language would need to be reproduced identically by every SD author and every client. ## 3.22.1 Event fields These names resolve to header fields: `timestamp`, `cpu_id`, `sequence`, `origin_class`, `event_type`, `effective_token_guid`, `true_token_guid`, `process_guid`, `boot_id` Every other name resolves to a payload field. ### 3.22.1.1 Header names are reserved Header field names are reserved in the query language and in result maps. If a payload carries a top-level key with a header field's name, **the header wins**. The colliding payload value is stored unchanged, and is retrievable as part of the raw payload by whatever holds it, but it is not exposed through field resolution, `SELECT`, `WHERE`, aggregation, access control or result maps. Suppression is applied *before* descendants are flattened, so a payload key named `timestamp` removes its entire subtree from the query surface, not just itself. An emitter SHOULD avoid payload keys that collide with header names. > [!NOTE] > The header must win because header fields are the ones the kernel > stamped and an emitter could not influence (PSPK §2). If a payload key > could shadow `process_guid`, an emitter could choose what its own > events appeared to come from, and every query filtering on identity > would be answerable by the party being investigated. ### 3.22.1.2 Flattening Payload maps are flattened recursively, path segments joined with `.`: a payload `{source: {name: "x"}}` exposes the field `source.name`. Each map key on a queryable path MUST be a MessagePack string matching: ```text [A-Za-z_][A-Za-z0-9_-]* ``` Note that `.` is **not** permitted in a segment, though it is permitted in an identifier generally (§3.19) — a key containing a dot could not be distinguished from a path through two maps. A key that is not a string, contains `.`, or does not match the grammar is stored unchanged and is **not queryable**: it does not resolve, does not appear in a result map, and has no field identity for access control. An empty map produces no field at all. Maps are containers; every non-map value, arrays included, is emitted at its flattened path (§3.17). If two payload entries flatten to the same path, the **first in MessagePack map order wins** and later duplicates are suppressed. ## 3.22.2 Log fields `timestamp`, `origin`, `is_error`, `message`, `boot_id`, `job_id` The set is closed. There are no payload fields and no flattening, and a collector MUST reject any other log field name as a parse error rather than resolving it to null. A log record has a fixed shape, so a name outside it is a mistake the collector can see — unlike an event payload field, which may legitimately be absent from a given record. `is_error` is a boolean in the query language, and compares against `true`/`false` or against `1`/`0`. ## 3.22.3 Metric fields `timestamp`, `boot_id`, `name`, `type`, `value` Every other name resolves to a label. Ingestion refuses labels colliding with these five (§3.10), so the flat namespace is unambiguous by construction rather than by a precedence rule. `type` is the series type as a string: `"counter"`, `"gauge"` or `"histogram"`. ## 3.22.4 What a record contains **Event records** carry the header fields plus every non-suppressed flattened payload field, as top-level keys. **Log records** carry the log fields. **Raw metric sample records** carry `timestamp`, `boot_id`, `name`, `type`, `value`, and the series' labels as top-level keys. **Aggregated metric results** carry `name`, `type` and `value`, plus labels when the result belongs to one label set. They carry `boot_id` only when the query restricted the samples to exactly one boot by a `boot_id` equality predicate, in which case the value is that boot ID; a result that could span boots omits it rather than picking one. **Aggregation results** in event and log mode carry the group key fields and the aggregate output, with the fixed schemas of §3.23. ## 3.22.5 SELECT `SELECT` narrows a result record to the named fields. It is valid only for non-aggregating event and log queries. A collector MUST reject `SELECT` combined with `COUNT BY`, `TOP N BY`, `DISTINCT` or `GROUP`, and MUST reject it in metric mode: all of those have fixed output schemas, and a clause that reshapes a fixed schema is a contradiction rather than a refinement. `SELECT` is applied **last**, after every other phase (§3.23). It controls the shape of the output and nothing else: a field not selected is still available to `WHERE`, to `SORT`, and to grouping. Narrowing what is displayed MUST NOT narrow what is filtered on. --- # 3.23 Event Queries _Peios / Advanced Peios / PSPU / Observability Interfaces_ > EVENTS mode — the type pattern, origin class aliases, and the aggregations available over events. ```text EVENTS [type_pattern] [clauses…] ``` ## 3.23.1 The type pattern The primary selector is an optional event type pattern, placed immediately after `EVENTS`. ```text EVENTS kacs.access_denied -- exactly that type EVENTS kacs.* -- every type beginning "kacs." EVENTS *.denied -- every type ending ".denied" EVENTS kacs.*.denied -- kacs.access.denied, kacs.token.denied, … EVENTS -- every type ``` `*` is the **only** metacharacter, and matches zero or more of any character, dots included. `?`, `[` and `{` have no special meaning and a collector MUST NOT treat them as any. Matching folds case, like every string comparison (§3.20). A pattern with no `*` is exactly `WHERE event_type == "…"`. A pattern whose only `*` is trailing is exactly `WHERE event_type STARTS_WITH "…"`. Anything else is a glob. ## 3.23.2 Origin class aliases `origin_class` accepts named aliases as well as its integer values: | Alias | Value | |---|---| | `userspace` | 0 | | `kmes` | 1 | | `kacs` | 2 | | `lcs` | 3 | ```text EVENTS WHERE origin_class == kacs SINCE 1h ago ``` These are the only aliased values in the language. A collector MUST accept both forms and MUST treat them as identical. ## 3.23.3 Aggregation Grouping equality, canonical representatives and tie ordering are defined in §3.21. Every aggregation below has a **fixed output schema**, and rejects `SELECT` (§3.22). ### 3.23.3.1 COUNT BY Counts records grouped by one field, ordered by count descending. ```text EVENTS SINCE 24h ago COUNT BY event_type ``` Output: `{: representative, count: }`. ### 3.23.3.2 TOP N BY `COUNT BY` with a limit — the `N` most frequent values. ```text EVENTS SINCE 1h ago TOP 10 BY process_guid ``` Output: the `COUNT BY` schema. ### 3.23.3.3 DISTINCT The distinct values of one field. ```text EVENTS SINCE 24h ago DISTINCT event_type ``` Output: `{: representative}`. ### 3.23.3.4 GROUP Groups by one or more fields, followed by an aggregation function: `COUNT`, or `SUM`, `AVG`, `MIN`, `MAX` with a field argument. ```text EVENTS SINCE 1h ago GROUP origin_class COUNT EVENTS SINCE 1h ago GROUP origin_class, event_type COUNT EVENTS SINCE 1h ago GROUP event_type AVG queue_depth ``` Output, for `GROUP a, b`: | Query | Record | |---|---| | `COUNT` | `{a, b, count}` | | `SUM x` | `{a, b, sum}` | | `AVG x` | `{a, b, avg}` | | `MIN x` | `{a, b, min}` | | `MAX x` | `{a, b, max}` | Group-key fields carry canonical representatives (§3.21). ### 3.23.3.5 What is aggregated For `SUM`, `AVG`, `MIN` and `MAX`, records whose field is null or non-numeric are **excluded** from the aggregate — not treated as zero. `COUNT` counts every record regardless. If no record in a group contributes a numeric value, the group's aggregate is null and the group is still present, because `COUNT` of it is still meaningful. ### 3.23.3.6 Result types - `COUNT` returns an unsigned integer. - `SUM` over integers returns an integer when the exact mathematical sum fits in signed or unsigned 64 bits. If it does not, or if any input was a float, it returns a `float64`. If that would be non-finite, the query MUST fail with an error rather than returning an infinity. - `AVG` returns a `float64` whenever at least one numeric value contributed. - `MIN` and `MAX` return the winning value itself, under exact numeric comparison. When an integer and a float tie, the integer wins. ## 3.23.4 Ordering Without `SORT`, results are ordered by timestamp descending, ties broken as §3.21 requires. ## 3.23.5 INDEX ```text EVENTS INDEX target_sid ``` `INDEX` asks the collector to prioritise a field for query acceleration immediately, rather than waiting for it to be observed often enough to be prioritised automatically. It exists for incident response, where the field that suddenly matters has never been queried before. `INDEX` is an **administrative operation**, not a query. It returns no records. A collector MUST check the caller's token against a Security Descriptor governing administration of the collector — one distinct from the read-path descriptors of §3.28 — and MUST refuse a caller that does not hold it. A collector without such a descriptor MUST refuse `INDEX` outright. A collector MAY treat `INDEX` as advisory and MAY decline the request, shed the acceleration later, or do nothing at all. It is a hint about priority; the accelerations a collector maintains are its own business, and a conforming collector that maintains none accepts `INDEX` and has nothing to do. There is no command to undo it, because there is nothing to undo: a collector reconsiders its own accelerations continuously and the hint decays with disuse. > [!NOTE] > The right to issue `INDEX` MUST NOT be the read right. Accelerating a > field costs write throughput on every record thereafter, so `INDEX` is > a way for a caller to degrade the system for everyone, and a caller > permitted only to read data has not been permitted to do that. --- # 3.24 Log Queries _Peios / Advanced Peios / PSPU / Observability Interfaces_ > LOGS mode — its three optional primary selectors, projection and aggregation, and why there are no payload fields. ```text LOGS [FROM origin[, origin…]] [ERROR ONLY] [CONTAINING "text"] [clauses…] ``` Log mode has three primary selectors rather than one, all optional and all combinable. Each is sugar for a `WHERE` predicate, and each exists because it is the thing a person actually types. ## 3.24.1 FROM Selects by origin. Several may be listed, comma-separated. ```text LOGS FROM loregd LOGS FROM loregd, peinit LOGS ``` `FROM` is exactly `WHERE origin == "…"` for one origin and `WHERE origin IN ("…", "…")` for several. Origins are written as identifiers (§3.19) or as quoted strings. A conforming origin is always an identifier (§3.7). ## 3.24.2 ERROR ONLY Selects lines that came from standard error. ```text LOGS ERROR ONLY LOGS FROM loregd SINCE 1h ago ERROR ONLY ``` It is exactly `WHERE is_error == true`, and like every clause it may appear anywhere after `LOGS` without changing the meaning (§3.18). ## 3.24.3 CONTAINING Selects lines whose message contains the given text — a substring match, folding case like every string comparison (§3.20). ```text LOGS CONTAINING "connection refused" LOGS FROM loregd CONTAINING "failed to open" ``` It is exactly `WHERE message CONTAINS "…"`. `CONTAINING` is a log-specific keyword because searching text is the primary operation on log data, and the primary operation deserves the shortest spelling. It is a substring scan, not an indexed text search: a collector MUST NOT restrict what it matches, and combining it with `SINCE` is what keeps it affordable. ## 3.24.4 Projection and aggregation `SELECT` narrows non-aggregating results to named log fields, and is additive across clauses (§3.22). `COUNT BY`, `TOP N BY`, `DISTINCT` and `GROUP` work exactly as in event mode (§3.23), with the same fixed output schemas, the same result types, and the same prohibition on combining them with `SELECT`. ```text LOGS SINCE 1h ago COUNT BY origin LOGS SINCE 1h ago TOP 5 BY origin ``` ## 3.24.5 Ordering Without `SORT`, results are ordered by timestamp descending, ties broken as §3.21 requires. ## 3.24.6 No payload fields Log mode has a closed field set (§3.22). A collector MUST reject an unknown log field name as a parse error, in a `WHERE`, a `SORT`, a `SELECT` or a grouping clause alike. This differs from event mode, where an unknown name is a payload field that resolves to null. The difference is that a log record's shape is fixed and known: a name outside it cannot be a field that this record happens to lack, so treating it as null would answer a question the client did not ask. --- # 3.25 Metric Queries _Peios / Advanced Peios / PSPU / Observability Interfaces_ > METRIC mode evaluates rather than searches — series selection, homogeneity, transforms, percentiles and scalar aggregations. ```text METRIC name[label_selector] [transform] [aggregation] [clauses…] ``` Metric mode evaluates rather than searches. A collector MUST reject `SELECT` in metric mode: the result schemas are fixed (§3.22). ## 3.25.1 Selecting series The primary selector is a metric name, optionally followed by a label selector in brackets. The name supports `*` with the same glob semantics as an event type pattern (§3.23). ```text METRIC cpu.usage METRIC cpu.* ``` The brackets — present, absent, or present and empty — decide how multiple matching series are handled, and this is the distinction that governs the rest of the mode. **No brackets — aggregate.** Every matching series is combined into one result. ```text METRIC cpu.usage -- average across all cores METRIC cpu.usage MAX -- maximum across all cores ``` **Empty brackets — break out.** Each series is returned separately. ```text METRIC cpu.usage[] -- latest value per core METRIC cpu.usage[] SINCE 1h ago -- a time series per core ``` **Filled brackets — select.** Only series matching the label predicates. ```text METRIC cpu.usage[core="0"] METRIC cpu.usage[core="0", host="srv1"] METRIC disk.usage[device STARTS_WITH "sd"] ``` Label predicates are comma-separated and combined with `AND`. They use the operators of §3.20, and within a label selector `=` is accepted as equality alongside `==`. Label keys are identifiers; values are identifiers or quoted strings. An absent label resolves to null, so `[device IS NULL]` selects the series that carry no `device` label. ## 3.25.2 Homogeneity After the name, the label selector, `WHERE` predicates and access filtering have been applied, the remaining series MUST be **of one type**. A collector MUST reject a selection spanning more than one type at execution time, with an error asking for a narrower name or an explicit `WHERE type == …`. A selection resolving to zero series returns no records — a successful query with an empty result, not an error. The rule exists because every function below is defined on one type. A selection mixing counters and gauges has no meaningful rate, and a selection mixing either with histograms has no meaningful value at all. ## 3.25.3 Function stages Function keywords execute in fixed stages regardless of where they were written: 1. **Transform** — `RATE`, `DELTA`, `P50`, `P95` or `P99`. Operates within each series independently and produces scalars. At most one per query. 2. **Terminal aggregation** — either a **scalar** aggregation (`AVG`, `MIN`, `MAX`, `SUM`) or a **window** aggregation (`AVG_OVER`, `MIN_OVER`, `MAX_OVER`, `SUM_OVER`). At most one per query; specifying both a scalar and a window aggregation is a parse error. The pipeline operates on scalars throughout. Counter and gauge samples are already scalar; a histogram sample is not scalar until a percentile function has been applied. A collector MUST therefore reject, at execution time when the type is known, a query that resolves to a histogram series without a percentile function, or that applies `RATE`, `DELTA`, or any scalar or window aggregation directly to one. Every output is a **finite** binary64. If any computation would produce NaN or an infinity, the query MUST fail with an error rather than returning it. ## 3.25.4 Transforms ### 3.25.4.1 RATE and DELTA `DELTA` is the change between consecutive samples; `RATE` is that change per second. Both apply **only to counter series**, and a collector MUST reject them on a gauge or histogram at execution time. Both use the same pair construction. Samples of one series are taken in ascending timestamp order, with a deterministic tiebreaker among samples sharing a timestamp. Each consecutive pair `(s1, s2)` whose `s2` falls inside the effective query range, and where `s2` is later than `s1`, produces one scalar at `s2`'s timestamp. The **immediately preceding sample before the first in-range one MUST be used as `s1`** for the first pair, when such a sample exists — without it the first point of every range would be missing, and a chart would show a notch at the start of every window. The adjusted delta is `s2 - s1` when the value rose, and `s2` alone when it fell, because a fall means the counter restarted from zero. `RATE` is that adjusted delta divided by the elapsed seconds. A pair with non-positive elapsed time contributes nothing. ```text METRIC http.requests.total SINCE 1h ago RATE METRIC http.requests.total SINCE 1h ago DELTA ``` ### 3.25.4.2 P50, P95, P99 Percentiles of **histogram series only**; a collector MUST reject them on a counter or gauge at execution time. Each histogram sample yields one value. Evaluation is nearest-rank over the sample's cumulative counts: for percentile `q`, compute `rank = ceil(q × total_count)`, and take the first boundary whose cumulative count is at least `rank`. A sample with `total_count == 0` yields no value. A sample whose rank falls **above the final cumulative count** — meaning the percentile lies in the overflow region above the highest boundary — also yields no value, because the distribution does not record where in that region it lies. ```text METRIC request.duration P95 METRIC request.duration[origin="loregd"] SINCE 1h ago P99 ``` > [!NOTE] > The overflow rule is why a producer's highest bucket boundary matters > (§3.10). A `P99` over a histogram where more than one observation in a > hundred exceeds the top boundary returns *no record*, which a client > cannot distinguish from no data. The alternatives are worse — reporting > the top boundary understates the answer, and reporting an infinity > violates the finiteness rule — but the failure is silent, and an > operator seeing an empty high percentile beside a populated low one is > seeing a mis-provisioned histogram. ## 3.25.5 Scalar aggregations `AVG`, `MIN`, `MAX` and `SUM` reduce scalars to one value. They MUST NOT be applied to a histogram series directly. What they aggregate *over* depends on the brackets: - **Bracketed**, so one result per series: over time, within each series. - **Unbracketed without `SINCE`**: over the latest transformed value of each matching series. The result timestamp is the greatest of the contributing timestamps. A series that cannot produce a value — a `RATE` with fewer than two samples, say — contributes nothing. - **Unbracketed with `SINCE`**: valid only when the selector resolves to zero or one series. More than one MUST be rejected with an error asking for a window aggregation. ```text METRIC cpu.usage AVG METRIC http.requests.total RATE SUM METRIC cpu.usage[] SINCE 1d ago AVG METRIC cpu.usage[core="0"] SINCE 1h ago MIN ``` The unbracketed default aggregation, when no `SINCE` and no explicit function is given, is `AVG`. No implicit scalar aggregation is added when a window aggregation is present. If nothing contributes to an aggregation, the query returns no record for that output group. Otherwise the output timestamp is the greatest contributing timestamp — for `RATE` and `DELTA`, the later sample of the contributing pair. ### 3.25.5.1 Why unbracketed plus SINCE needs a window A collector MUST NOT synthesise a merged time series from samples that do not share timestamps. Two series sampled at unrelated moments cannot be averaged point by point without inventing values between the points, and interpolation would make the collector responsible for a number nobody measured. A window aggregation supplies the common time grid explicitly, which is why it is required rather than assumed. ## 3.25.6 Window aggregations `AVG_OVER`, `MIN_OVER`, `MAX_OVER` and `SUM_OVER` take a duration and produce one value per window. They **require `SINCE`**; a collector MUST reject a window aggregation without one as a parse error. Windows are fixed and aligned to Unix-epoch multiples of the duration — not to the query's start — so that the same window boundaries fall in the same places for every query. The output timestamp is the window start. Windows with nothing in them are omitted rather than emitted as null. ```text METRIC cpu.usage SINCE 1d ago AVG_OVER 1h METRIC cpu.usage[] SINCE 1d ago AVG_OVER 5m METRIC http.requests.total SINCE 1h ago RATE SUM_OVER 5m METRIC request.duration P95 SINCE 1h ago AVG_OVER 5m ``` `AVG` and `AVG_OVER` are different keywords and a collector MUST NOT treat them as synonyms: `AVG` produces one value for the range, `AVG_OVER` one per window. For raw and percentile-transformed values, a window contains the scalars whose timestamps fall inside it, and the function is applied to those. No interpolation is performed. For `RATE` and `DELTA` with a window aggregation, each series first produces **at most one scalar per window**: the window `DELTA` is the sum of reset-adjusted deltas for pairs whose later sample is in the window, and the window `RATE` is that divided by the elapsed seconds those pairs covered. The preceding-sample rule applies to the first pair of each window. The terminal aggregation then combines the per-series window values — so `RATE SUM_OVER 5m` sums the series' five-minute rates, and `RATE AVG_OVER 5m` averages them. Where the selector resolves to exactly one series, all four window functions return that series' window value. Bracketed window queries keep labels in the result rows. Unbracketed ones omit them, unless the selector resolved to exactly one series. ## 3.25.7 Without SINCE With no `SINCE`, the query returns the **latest** value. ```text METRIC cpu.usage[core="0"] METRIC cpu.usage[] METRIC cpu.usage ``` "Latest" is per series, by timestamp with the deterministic tiebreaker. For `RATE` and `DELTA`, it is the latest valid consecutive pair with positive elapsed time; a series with no such pair returns nothing. ## 3.25.8 Boot filtering Samples carry `boot_id` but series continue across boots (§3.13). A query MAY restrict to one boot: ```text METRIC cpu.usage[] WHERE boot_id == "550e8400-e29b-41d4-a716-446655440000" ``` A boot-filtered metric query MUST be evaluated from raw samples. A collector MUST NOT serve one from any pre-computed aggregate that is not itself partitioned by boot. ## 3.25.9 Results One record per raw sample; one per valid pair for `RATE` and `DELTA`, timestamped at the later sample; one per histogram sample that yields a percentile; one per window for window aggregations; one for a scalar aggregation. A histogram result carries only the percentile in `value`. The boundaries, counts, total and sum are **not** returned by the query language in this revision, in any mode. ```text {timestamp: 1714000000000000000, boot_id: "{550e8400-…}", name: "cpu.usage", type: "gauge", core: "0", value: 42.7} {timestamp: 1714000300000000000, name: "cpu.usage", type: "gauge", core: "0", value: 39.8} ``` Without `SORT`, metric results are ordered by timestamp **ascending** (§3.21) — the opposite of events and logs, because a metric result is read as a series rather than as a list of occurrences. --- # 3.26 Cross-Type Filtering _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The only correlation mechanism in the language — narrowing one data type by a condition on another, with a lookback limit and no join. A cross-type filter narrows one data type by a condition on another. It is the only correlation mechanism in the language; there is no join. ```text EVENTS kacs.* SINCE 1h ago WHERE METRIC cpu.usage[core="0"] > 80 LOGS FROM loregd SINCE 1h ago WHERE EVENT kacs.access_denied EXISTS METRIC cpu.usage[] SINCE 1h ago WHERE EVENT synthetic.storage_error EXISTS EVENTS kacs.* SINCE 1h ago WHERE LOG loregd CONTAINING "error" EXISTS ``` | Form | Available in | |---|---| | `WHERE METRIC …` | events, logs | | `WHERE EVENT … EXISTS` | logs, metrics | | `WHERE LOG … EXISTS` | events, metrics | ## 3.26.1 How it is evaluated A collector MUST evaluate the cross-type condition **first**, producing the set of time ranges over which it holds, and then apply those ranges as additional timestamp bounds on the primary source. The condition is evaluated against the referenced data's own resolution — the metric's sample interval, or the density of matching events — and **not** once per record of the primary source. It is computed once for the query. ## 3.26.2 Metric conditions `WHERE METRIC` operates on **raw scalar samples of counter and gauge series only**. Transform, scalar aggregation and window aggregation keywords are not valid in one, and a condition resolving to a histogram series MUST be rejected. The selector MUST resolve to **zero or one** series. Zero produces no true ranges. More than one MUST be rejected with an error asking for a bracketed or narrower selector. Within the effective query range, a sample's value is treated as active over `[sample.timestamp, next_sample.timestamp)`, clipped to the range, and the final sample stays active through the upper bound. A collector MUST include the **latest sample before `SINCE`** as the initial state when one exists; without it the condition would be false from the start of every range until the first sample inside it, which for a fifteen-second sampling interval is fifteen seconds of wrongly excluded records. If no earlier sample exists, the condition is false until the first in-range sample. Samples sharing a timestamp are ordered deterministically; the earlier ones create zero-width intervals and the last at that timestamp is the active value. This is interpolation of a kind, and it should be understood as such: it assumes the condition held continuously between two samples. A metric that crossed a threshold and crossed back between samples is invisible. ## 3.26.3 Existence conditions `WHERE EVENT … EXISTS` and `WHERE LOG … EXISTS` are true when at least one matching record lies near the primary record in time. The event type supports `*` globbing (§3.23); the log form names an origin and optionally a `CONTAINING` text. "Near" is a **centred half-open window** of a configured width `W`. With `lower = floor(W / 2)` and `upper = W - lower`, the condition is true for a primary timestamp `t` when a matching record exists with: ```text timestamp >= t - lower timestamp < t + upper ``` Equivalently, a matching record at `e` contributes the true range `[e - lower, e + upper)`. When `W` is odd the extra nanosecond falls on the upper side, so the width is exactly `W` and never `W ± 1`. > [!NOTE] > §3.A gives the mainline width of the existence window and its > adjustable range, alongside every other bound in this chapter. ## 3.26.4 The lookback limit A collector MUST bound how far back a cross-type filter may scan. > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. If the effective query range exceeds the limit, a collector MUST reject the cross-type filter with an error saying the range is too large, and the error SHOULD suggest narrowing it with `SINCE` or `UNTIL`. **A query with a cross-type filter and no `SINCE` MUST be rejected.** An unbounded cross-type scan is never permitted, in any mode, at any configured limit. The reason is that a cross-type filter reads a second store in full before the first query begins. Its cost is set by the *referenced* data's density, which the client did not select and cannot see, so a query that looks cheap can scan a hundred times more than it returns. ## 3.26.5 Cost A cross-type filter is efficient when it is selective — narrow true ranges eliminating most of the primary source — and expensive when it is broadly true, which is the case where it also eliminates nothing. A condition that holds across the whole range costs the full scan of both stores and returns exactly what the query would have returned without it. --- # 3.27 Streaming _Peios / Advanced Peios / PSPU / Observability Interfaces_ > STREAM turns a query into a live watch — what may be streamed, what still applies during the watch phase, and what is rejected. `STREAM` turns a query into a live tail. It is a flag, may appear anywhere in the string, and takes no argument. Streaming is available for **event and log queries only**. A collector MUST reject `STREAM` in metric mode. > [!NOTE] > Metric streaming is absent because metric data is sampled on an > interval — typically fifteen seconds — and the consumer is a dashboard > that polls. Streaming individual samples adds a delivery path with a > latency budget far finer than the data it carries. ## 3.27.1 The shape of a streaming query 1. The collector executes the query normally and sends the initial result set as `"ok"` messages. 2. It sends `"watch"` (§3.16). The query is established at this point and not before. 3. It stays open. As records are committed, it evaluates them against the query and sends those that match. 4. It continues until the client disconnects, an error terminates it, or the collector shuts down. There is no `"end"` message for a streaming query, ever. ## 3.27.2 What may be streamed Raw record queries and `DISTINCT` queries. A collector MUST reject `STREAM` combined with `COUNT BY`, `TOP N BY` or `GROUP` as a parse error — those produce one answer about a set, and a set that is still growing has no answer yet. A collector MUST reject `STREAM` combined with `UNTIL`. An upper time bound and an unbounded live tail are contradictory requests. `SINCE` is permitted and applies to both phases, resolved against the evaluation time captured at query start (§3.19). ## 3.27.3 What still applies during the watch phase Access control, the primary selector, the `SINCE` bound and every `WHERE` predicate — cross-type conditions included — are evaluated against each new record. `SORT`, `TAKE` and `SKIP` apply to the **initial result set only**. Streamed records are delivered in commit order and a collector MUST NOT reorder, limit or skip them: there is no total order over records that have not arrived, and applying `TAKE` to a stream would silently end it. `SELECT` applies to streamed records as it does to initial ones. ## 3.27.4 DISTINCT streaming ```text EVENTS kacs.* DISTINCT process_guid STREAM LOGS DISTINCT origin STREAM ``` A `DISTINCT` stream emits a value the first time it is seen, and never again. The output schema is `DISTINCT`'s fixed one (§3.23) in both phases. The initial result set is the complete distinct set visible at query start, after access control and every filter. The collector then holds a **seen set** initialised from it. Each newly committed record that passes access control and the filters is reduced to its value for the field, and emitted only if that value is not already in the seen set under the grouping equality of §3.21; emitted values are then added. A collector MUST bound the seen set. > [!NOTE] > §3.A gives the mainline value and adjustable range > for this bound and every other in this chapter. If initialising the set or inserting a value would exceed the bound, the collector MUST terminate the query with an error. It MUST NOT evict: "not seen before" is the entire meaning of the output, and a set that forgets would re-emit values it had already reported, which is worse than stopping. A collector MUST reject `DISTINCT … STREAM` combined with `SORT`, `TAKE` or `SKIP`, so that the seen set always corresponds to the complete initial visible set. `SELECT` is already invalid with `DISTINCT` (§3.22). ## 3.27.5 Cross-type conditions during the watch phase The pre-computed time ranges of §3.26 describe the past. A collector MUST NOT reuse them for streamed records. For a **metric** condition, the selector has already been required to resolve to exactly one series (§3.26). For each committed batch, the collector finds that series' active sample at the batch's latest candidate timestamp under §3.26's interval rules and evaluates the condition against it. If no sample is active there, the condition is false. A false condition filters out the whole batch; a true one leaves the batch to be filtered by the remaining predicates as usual. For an **existence** condition, the collector applies §3.26's centred window to each candidate record's own timestamp. These are evaluated **per record**, not per batch, because a matching record may be near some of a batch and not the rest. > [!NOTE] > Evaluating the metric condition once per batch rather than once per > record is an approximation, and an intentional one. A commit batch > spans a fraction of a second while a metric sample spans fifteen, so > every record in a batch normally maps to the same sample. At > sub-second metric resolution it filters more coarsely than per-record > evaluation would, and records near a threshold crossing are included or > excluded as a group. ## 3.27.6 Backpressure If a client cannot keep up, the collector MUST drop the query rather than buffer for it. Backpressure is detected on the socket send buffer: when a result message cannot be sent because the buffer is full, the collector MUST terminate the query immediately and MUST NOT block on the send. It sends an error if the socket will still take one, and closes otherwise. Streaming MUST NOT slow or block ingestion. A streaming client is the lowest-priority consumer of a collector's time, and a slow one is disconnected rather than accommodated — the same principle as §3.4, applied on the way out. ## 3.27.7 Latency Delivery latency is bounded below by the collector's commit interval for the store concerned, because a record is only streamable once it is committed. A client that needs lower latency than that is not served by this interface: the KMES ring buffer is the lower-latency path and is specified in PSPK. > [!NOTE] > Streaming is a convenience for interactive tailing and dashboards. > Where results must be predictable — cross-type conditions evaluated per > batch, a metric threshold sampled coarsely, a disconnect under load — > repeated non-streaming queries with a sliding `SINCE` are more > reliable, and for latency-critical consumption the ring buffer bypasses > a collector entirely. --- # 3.28 What a Client Cannot See _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Read access is enforced on every query against the token captured at connect — the unit of access, and why filtering is silent. A collector MUST enforce read access on every query, against the token captured when the client connected (§3.14). How it does so is its own design, and the mechanism the mainline collector uses is described in the eventd TRMP. What this chapter fixes is the part a client can observe: **which results it gets, and what it is told about the ones it does not.** ## 3.28.1 The unit of access is the concrete identifier Access is resolved per **concrete identifier** — the event type, log origin or metric name a stored record actually carries (§3.2) — and not per query, per store, or per pattern the query happened to write. A collector MUST resolve each identifier that a query's data could touch independently. A broad selector authorizes nothing by itself: `EVENTS` with no pattern, `EVENTS kacs.*`, `LOGS` with no `FROM`, and `METRIC cpu.*` are all resolved identifier by identifier, and a client permitted to read one matching identifier and not another sees only the first. Identifiers are matched to rules by dot-delimited prefix, most specific first, falling back to a wildcard default: for `kacs.access_denied`, a rule for `kacs.access_denied`, then one for `kacs`, then the default. A collector MUST fail closed. If no rule resolves — including because the default is missing — access is denied. ## 3.28.2 Filtering is silent **Records and fields removed by access control are removed without comment.** A collector MUST NOT indicate in a response that anything was withheld, and a client MUST NOT assume a result set is complete. The consequences are precise and a client needs all of them: - A record whose identifier the client may not read is **absent**, not redacted. - A field the client may not read is **absent from the record**, and is indistinguishable from a field the record never carried (§3.17). - `COUNT`, `COUNT BY`, `TOP N BY`, `DISTINCT` and every aggregation reflect **only** authorized records. A count is a count of what the client may see. - A cross-type condition referencing data the client may not read evaluates as though **no matching data exists** (§3.26). It does not fail the query. - `TAKE` and `SKIP` page over the authorized records only. ## 3.28.3 Access control runs before everything A collector MUST remove unauthorized records from the logical row set **before** predicates, transforms, grouping, aggregation, sorting, pagination and projection (§3.18). This is not tidiness. Counting, ordering or paginating over records a client may not read leaks them through the count, through the ordering, and through the gaps in pagination — a client could establish how many records of a type it cannot read exist, and roughly when, without ever seeing one. A collector MAY reach the result however it likes: pushing the authorization down into its storage engine, or reading candidates and discarding them before aggregating. What it MUST NOT do is produce a different answer from the one filtering-first produces. ## 3.28.4 Denied fields do not fail the query When a query references a field in a predicate, a grouping, a sort or an aggregation, and some matching identifier does not grant that field, the records under that identifier contribute nothing — exactly as if their identifier had been denied outright. A collector MUST NOT reject the query. > [!NOTE] > Rejecting would be the more informative behaviour and that is precisely > the objection to it. A rejection tells the client that an identifier > exists, matches its query, and carries a field it may not read — three > facts about data it was not permitted to see, delivered by the > mechanism meant to withhold them. A client could enumerate restricted > event types by watching which queries are refused. Silence costs the > client a result that is narrower than it looks; rejection costs the > system the property the whole model rests on. Authorization for a field is resolved from the field **as written**, against each concrete identifier, and does not depend on whether any record of that identifier actually carries it. Payload fields vary between records of the same type, so a rule that turned on presence would be undecidable before the scan it was meant to authorize. ## 3.28.5 What is not a field Derived aggregate outputs — `count`, `sum`, `avg`, `min`, `max` — are **not** source fields, have no access identity of their own, and are visible whenever the client is authorized for the records and the source fields they were computed from. Values internal to preserving query semantics — row identifiers, series identifiers, ordering tiebreakers, series type checks — are likewise not query-language fields (§3.21). A metric result's `value` **is** a source field, because it is a raw sample or a scalar derived from raw samples. ## 3.28.6 Errors say nothing A collector MUST NOT include a value the client is not authorized to read in any error message (§3.16), including in errors raised by internal consistency checks. ## 3.28.7 Streaming Access decisions made for the initial result set are reused during the watch phase, but a collector MUST resolve any **new** concrete identifier that appears in a streamed batch and check it before using the record or its distinct value — a new event type or a new log origin appearing mid-stream has never been authorized. If a rule changes during a streaming query, a collector MUST re-check subsequent batches against the new rule. The **token** does not change. It was captured at connection (§3.14), so a client whose group memberships change mid-stream continues to be evaluated against what it connected with, and a client whose access is revoked keeps receiving records until it disconnects. ## 3.28.8 The write path is not access-controlled Nothing on either ingestion channel is authorized per record (§3.4). Access control here is a read-path mechanism only, and the Security Descriptor on each ingestion socket is the whole of the write-path control (§3.3). The consequence is that **`origin` and metric `name` are self-asserted** (§3.7, §3.10). Any process that can reach an ingestion socket may write under any origin or metric name it likes, including one belonging to another program — which permits fabricating a plausible operational record, or burying a real one under noise attributed elsewhere. Read-path rules limit who can *see* data written under a given identifier; they do nothing about who wrote it. A collector MUST NOT present a stored `origin` or metric `name` as evidence of provenance, and a client MUST NOT treat one as authenticated. > [!NOTE] > Closing this needs something the interface cannot supply on its own: a > way to obtain the peer's token for a datagram, as a collector obtains > one for a stream connection (§3.14). Until that exists, a producer's > claim about itself is unverifiable, and confining the set of processes > that can reach the socket at all is the only available control. --- # 3.29 Extension _Peios / Advanced Peios / PSPU / Observability Interfaces_ > There is no version number on any of the three interfaces — so unknown fields are ignored, unknown values are refused, and only some things may change. There is no version number on any of the three interfaces. No datagram carries one, no query message carries one, and there is no exchange in which either party could state or discover what the other speaks. That is a deliberate consequence of the shapes chosen, and it is worth being explicit about, because it means every rule below is the *only* mechanism available. Ingestion is one-way over a datagram socket: there is no reply in which a collector could announce a version and no state in which a producer could remember one. The query channel could carry a version — it is a stream, and it has a request message — and does not, because a version field is only useful if a party may then behave differently, and a client cannot usefully vary: it either asks a question the collector understands or does not. What replaces negotiation is a set of rules under which both sides may change without either being told. ## 3.29.1 Unknown fields are ignored A collector MUST ignore fields it does not recognise in a log record (§3.7), in a metric record (§3.11), and in a query request (§3.15). This is what allows a field to be added. A producer built against a later revision may send a field this collector has never heard of, and the record is still stored; a producer built against an earlier one omits a field that has since been added, and the record is still stored because everything added is optional. A field added to any of these three maps MUST therefore be optional, and a collector MUST NOT require one to be present. ## 3.29.2 Unknown values are refused, not ignored The rule does **not** extend to values. An unrecognised `type` in a metric record discards the record (§3.12); a first token that is not a mode fails the query (§3.18); an unrecognised keyword is a parse error. A collector MUST NOT guess at an unrecognised value, and MUST NOT skip a field it recognised but could not interpret. The asymmetry is the point. An unknown field is something the sender knows about and this collector does not, and ignoring it loses only what was never understood. An unknown *value* in a known field is the sender saying something specific about this record, and proceeding without understanding it stores something other than what was sent. ## 3.29.3 Response statuses A client MUST treat a response whose `status` it does not recognise as an error terminating the query, and MUST discard the `"ok"` messages it has received for that query unless `"end"` or `"watch"` had already arrived (§3.16). A status is the control flow of the response stream, so there is no ignoring one: a client that skipped an unknown status would be waiting for a terminal message that had already been sent, or treating an incomplete result as complete. Failing is the only safe reading. A collector MUST NOT introduce a new status for a condition that the four existing ones can express. ## 3.29.4 What may change without notice - **New optional fields** in a log record, a metric record or a query request. - **New fields in result records.** A client MUST tolerate a key it does not recognise, and MUST NOT reject a record for carrying one. - **New query keywords, clauses and functions.** A client sending one the collector does not know receives a parse error, which is the correct answer. - **Wording of any error string** (§3.16). - **New event types, log origins and metric names.** These are data, not interface; nothing enumerates the valid set of any of them. ## 3.29.5 What may not change - **The meaning of an existing field**, in either direction. A field is added or it is left alone. - **The type of an existing field.** - **The four response statuses**, or the rule that exactly one terminal message ends a query. - **The framing** of §3.15, which has no version field and therefore no way to change compatibly. - **A required field becoming optional, or an optional one becoming required.** ## 3.29.6 Limits are not the interface The declared bounds — the datagram ceilings (§3.6, §3.9), the query message ceiling (§3.15), the concurrency and timeout bounds (§3.14, §3.16), the existence window and lookback limit (§3.26) — are configuration, and an administrator may change any of them. A collector MUST behave identically at any value in its supported range. A producer or client MUST NOT infer a bound from having exceeded one, or from not having exceeded one, and MUST NOT depend on the mainline defaults quoted in this chapter. The one place this bites is the log and metric datagram ceilings, which a producer cannot discover and which silently discard what exceeds them (§3.6). Lowering either is a change to the contract with every producer on the system, and there is no mechanism by which any of them will find out. --- # 3.30 Conformance _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Every requirement of this chapter collected by role — collector, producer and client — plus what is deliberately not required. A conforming implementation of any role MUST satisfy every requirement in this chapter. This section collects the obligations that are not tied to one message. ## 3.30.1 A collector **Serve three separate channels.** Two `SOCK_DGRAM` for ingestion, one `SOCK_STREAM` for queries, each on its own socket, each protected by a Security Descriptor established before it accepts anything (§3.3). **Never exert backpressure.** No producer stalls because of a collector, under any load, in any failure state (§3.4). **Never react to input.** No event, no log entry, no client-observable counter, in response to a malformed, unwanted or excessive submission (§3.4). **Validate at the stated scope.** Datagram, record, or field — as §3.8 and §3.12 set out, and no more broadly. In particular a malformed record MUST NOT cost the valid records batched with it, and a malformed optional field MUST NOT cost a log record. **Store what you were given.** A log message byte-for-byte, an event payload unmodified, a timestamp uncorrected, a histogram's boundaries in the order sent (§3.8, §3.10, §3.5). **Preserve gaps.** No interpolation, no backfill, no synthesised sample (§3.13). **Identify every query client** from the connection, before executing anything, and refuse the query if you cannot (§3.14). **Order totally and deterministically.** Every result, for a fixed set of stored records, in one order — so that `SKIP` and `TAKE` mean something (§3.21). **Use query-language semantics, not your storage engine's**, for every comparison, ordering, grouping and equality test the language defines (§3.20, §3.21). **Enforce access before you compute**, per concrete identifier, failing closed, and silently (§3.28). **Bound everything a client can consume**: concurrent queries, streaming queries, message size, query time, distinct-stream values, cross-type lookback (§3.14, §3.15, §3.16, §3.26, §3.27). **Behave identically across your configured ranges** (§3.29). ## 3.30.2 A producer **Send well-formed records** and accept that malformed ones vanish without notice (§3.8, §3.12). **Stay within the datagram ceiling**, batched or not — and know that you cannot discover it (§3.6). **Choose a stable, conforming identifier.** An origin or metric name matching the identifier grammar, naming you distinguishably, and using dots for hierarchy — because it is what access rules are written against and what queries select on (§3.7, §3.10). **Timestamp at production**, not at submission (§3.7). **Bound your label cardinality**, and keep histogram boundaries fixed for the life of a metric (§3.10, §3.13). **Never assume delivery.** No acknowledgement exists, none is coming, and a record that mattered should have been an event (§3.4). **Never change a metric's type.** Doing so ends the series silently and permanently (§3.10). ## 3.30.3 A client **Tolerate unknown keys** in result records, and unknown statuses as errors (§3.29). **Discard partial results.** An error before `"end"` or `"watch"` means every `"ok"` message for that query is void (§3.16). **Assume nothing about completeness.** Results are silently filtered by access, counts count only what you may see, and an absent field is indistinguishable from a denied one (§3.28). **Assume nothing about provenance.** An `origin` and a metric `name` are what the producer claimed (§3.28). **Do not parse error strings** (§3.16). **Open one connection per query** (§3.14). ## 3.30.4 What this chapter does not require of a collector A conforming collector need not accelerate anything, pre-compute anything, shard anything, or retain anything for any particular period. It need not honour `INDEX` beyond accepting it (§3.23). Its storage, indexing, retention and query planning are entirely its own, and every requirement above is stated about the answer rather than about how the answer is reached. --- # Appendix 3.A Limits _Peios / Advanced Peios / PSPU / Observability Interfaces_ > Every bound a collector must enforce, with the mainline collector's value and adjustable range, and the one relation that ties two of them together. Every bound this chapter requires a collector to enforce, with the value and adjustable range of the mainline collector. The mainline values are **informative**: a conforming collector chooses its own, and a producer or client MUST NOT depend on any of them (§3.29). The mainline configuration key names are those of eventd, whose configuration is catalogued in the eventd TRMP §A. ## 3.A.1 Ingestion | Bound | Mainline value | Mainline range | Key | Section | |---|---|---|---|---| | Log datagram ceiling | 262144 B | 4096 – 1048576 | `MaxLogDatagramBytes` | §3.6 | | Metric datagram ceiling | 262144 B | 4096 – 1048576 | `MaxMetricDatagramBytes` | §3.9 | | Receive queue, either socket | ≤ 4 × the ceiling | — | — | §3.6 | ## 3.A.2 Queries | Bound | Mainline value | Mainline range | Key | Section | |---|---|---|---|---| | Query message ceiling | 65536 B | 1024 – 16777216 | `MaxQueryMessageBytes` | §3.15 | | Query timeout | 30000 ms | 1000 – 300000 | `QueryTimeoutMs` | §3.16 | | Concurrent queries | 128 | 1 – 4096 | `MaxConcurrentQueries` | §3.14 | | Concurrent streaming queries | 64 | 1 – 1024 | `MaxStreamingQueries` | §3.14 | | Values per DISTINCT stream | 100000 | 1000 – 10000000 | `MaxDistinctStreamValues` | §3.27 | ## 3.A.3 Cross-type filtering | Bound | Mainline value | Mainline range | Key | Section | |---|---|---|---|---| | Existence window `W` | 15000 ms | 1000 – 300000 | `CrossTypeWindowMs` | §3.26 | | Maximum lookback | 604800 s | 3600 – 2592000 | `CrossTypeMaxLookbackSeconds` | §3.26 | ## 3.A.4 Fixed by this chapter These are not configuration and a collector MUST NOT vary them. | Quantity | Value | Section | |---|---|---| | Timestamp domain | `0` – `9223372036854775807` ns | §3.5 | | GUID field width | 16 bytes | §3.7, §3.11 | | Message length prefix | 4 bytes, little-endian | §3.15 | | Transforms per query | at most 1 | §3.25 | | Terminal aggregations per query | at most 1 | §3.25 | | Queries per connection | exactly 1 | §3.14 | ## 3.A.5 The relation between two of them The query message ceiling MUST NOT be smaller than the largest record a collector can store, because a record that will not fit in a response fails every query that reaches it (§3.15). The ingestion ceilings bound what a producer can deposit; the query message ceiling bounds what can be handed back. Nothing enforces the relation automatically, and the mainline defaults do not satisfy it. --- # Appendix 3.B Query Language Reference _Peios / Advanced Peios / PSPU / Observability Interfaces_ > An index of the query language and where each construct is valid — shape, clause validity, rejected combinations, functions, operators and fields. An index of the language, and of where each construct is valid. The normative definitions are in §3.18 to §3.27; nothing here adds a rule. ## 3.B.1 Shape ```text EVENTS [type_pattern] [clauses…] LOGS [FROM o[, o…]] [ERROR ONLY] [CONTAINING s] [clauses…] METRIC name[label_selector] [transform] [aggregation] [clauses…] ``` ## 3.B.2 Clause validity | Clause | EVENTS | LOGS | METRIC | Section | |---|---|---|---|---| | `SINCE` / `UNTIL` | yes | yes | yes | §3.19 | | `WHERE` | yes | yes | yes | §3.20 | | `WHERE METRIC` | yes | yes | no | §3.26 | | `WHERE EVENT … EXISTS` | no | yes | yes | §3.26 | | `WHERE LOG … EXISTS` | yes | no | yes | §3.26 | | `SORT` | yes | yes | yes | §3.21 | | `TAKE` / `SKIP` | yes | yes | yes | §3.18 | | `SELECT` | non-aggregating only | non-aggregating only | no | §3.22 | | `COUNT BY` / `TOP N BY` | yes | yes | no | §3.23 | | `DISTINCT` | yes | yes | no | §3.23 | | `GROUP` + function | yes | yes | no | §3.23 | | `STREAM` | yes | yes | no | §3.27 | | `ERROR ONLY` / `CONTAINING` | no | yes | no | §3.24 | | `INDEX` | yes | no | no | §3.23 | `WHERE` and `SELECT` are the only repeatable clauses (§3.18). ## 3.B.3 Combinations that are rejected | Combination | Rejected at | Section | |---|---|---| | `SELECT` with `COUNT BY`, `TOP N BY`, `DISTINCT` or `GROUP` | parse | §3.22 | | `SELECT` in metric mode | parse | §3.22 | | `STREAM` with `COUNT BY`, `TOP N BY` or `GROUP` | parse | §3.27 | | `STREAM` with `UNTIL` | parse | §3.27 | | `DISTINCT … STREAM` with `SORT`, `TAKE` or `SKIP` | parse | §3.27 | | Window aggregation without `SINCE` | parse | §3.25 | | Scalar and window aggregation together | parse | §3.25 | | Two transforms | parse | §3.25 | | Cross-type filter without `SINCE` | parse | §3.26 | | `=` outside a label selector | parse | §3.20 | | `== NULL` or `!= NULL` | parse | §3.19 | | Ordering operator on a binary literal | parse | §3.20 | | Ordering operator on a fixed field that cannot be ordered | parse or planning | §3.20 | | Unknown log field name | parse | §3.24 | | Effective range beyond the lookback limit | planning | §3.26 | | Selected metric series spanning more than one type | execution | §3.25 | | `RATE` or `DELTA` on a gauge or histogram | execution | §3.25 | | Percentile on a counter or gauge | execution | §3.25 | | Histogram series with no percentile function | execution | §3.25 | | Unbracketed metric query with `SINCE` resolving to several series | execution | §3.25 | | Cross-type metric selector resolving to several series | execution | §3.26 | | Result record larger than the message ceiling | execution | §3.16 | | Aggregation producing a non-finite value | execution | §3.23, §3.25 | "Parse" failures need no data. "Execution" failures depend on what the store holds, so the same query string may succeed on one system and fail on another. ## 3.B.4 Metric functions | Keyword | Stage | Valid on | Produces | |---|---|---|---| | `RATE` | transform | counter | per-second change | | `DELTA` | transform | counter | absolute change | | `P50` `P95` `P99` | transform | histogram | one value per sample | | `AVG` `MIN` `MAX` `SUM` | scalar aggregation | counter, gauge | one value | | `AVG_OVER` `MIN_OVER` `MAX_OVER` `SUM_OVER` | window aggregation | counter, gauge | one value per window | Transforms feed aggregations; a query may have at most one of each (§3.25). ## 3.B.5 Operators `==` `!=` `>` `>=` `<` `<=` `STARTS_WITH` `ENDS_WITH` `CONTAINS` `IN` `NOT_IN` `IS NULL` `IS NOT NULL`, combined with `AND` and `OR` (§3.20). There is no `NOT` and no `=`. ## 3.B.6 Literals | Kind | Form | Section | |---|---|---| | Identifier | `[A-Za-z_][A-Za-z0-9_.-]*` | §3.19 | | String | `"…"` with `\"` `\\` `\n` `\r` `\t` `\uXXXX` | §3.19 | | Binary | `x"0a1b…"`, even digit count | §3.19 | | Integer | decimal or `0x…` | §3.19 | | Float | finite, with a fraction or exponent | §3.19 | | Boolean | `true`, `false` | §3.19 | | Null | `NULL`, in `IS NULL` only | §3.19 | | Duration | `s` `m` `h` `d`, non-zero | §3.19 | | Time | ` ago`, ` hence`, `today`, `yesterday`, `YYYY-MM-DD`, `YYYY-MM-DDTHH:MM:SS` | §3.19 | | GUID | `8-4-4-4-12`, braced or not | §3.19 | ## 3.B.7 Fields | Mode | Fixed fields | Everything else | |---|---|---| | EVENTS | `timestamp` `cpu_id` `sequence` `origin_class` `event_type` `effective_token_guid` `true_token_guid` `process_guid` `boot_id` | a flattened payload path, or null | | LOGS | `timestamp` `origin` `is_error` `message` `boot_id` `job_id` | a parse error | | METRIC | `timestamp` `boot_id` `name` `type` `value` | a label, or null | ## 3.B.8 Aliases `origin_class` accepts `userspace` (0), `kmes` (1), `kacs` (2), `lcs` (3). These are the only aliased values in the language (§3.23). ## 3.B.9 Default ordering | Mode | Without `SORT` | |---|---| | EVENTS, LOGS | timestamp descending | | METRIC | timestamp ascending | | `COUNT BY`, `TOP N BY` | count descending | | `DISTINCT` | by the distinct value | All ties are broken to a total order (§3.21). --- # Appendix 3.C Prior Art _Peios / Advanced Peios / PSPU / Observability Interfaces_ > The well-known counterparts these three interfaces were shaped against, and where they sit relative to them. The three interfaces here are not novel, and each has a well-known counterpart whose shape informed it. What follows compares the *contracts* — this appendix is about wire shapes and the obligations they place on either side. The eventd TRMP §1.4 compares the systems. ## 3.C.1 Log ingestion The closest relative is journald's native socket: a Unix datagram socket, world-writable, accepting a self-describing record from any local process, with no acknowledgement and no notification of loss. The agreements are substantive — datagram rather than stream, self-asserted identity, silent drop under pressure, a forwarder bridging programs that only know standard output. The differences are three. The record here is MessagePack rather than a line-oriented key-value text format, because the collector already carries a MessagePack decoder for event payloads and a second parser would be a second thing to get wrong. Severity is a boolean rather than a syslog priority, because a forwarder can distinguish two file descriptors and inventing eight levels from two would be a guess presented as data (§3.7). And a batch is a first-class datagram shape rather than a stream of records, which is what lets a forwarder amortise the syscall without giving up the datagram's all-or-nothing property. Classic syslog over `/dev/log` is the older relative, and the departure from it is the same one journald made: a record with named fields rather than a formatted line that every consumer re-parses with a regular expression. ## 3.C.2 Metric ingestion The shape is StatsD's: push, datagram, fire-and-forget, no registration, sender-named series. It is the opposite of Prometheus's, where the collector pulls from endpoints it has been configured to know about. The choice follows from the loss model rather than from taste (§3.9). A pulling collector must reach every producer on a schedule, which makes it responsible for their availability; pushing keeps a slow or dead producer invisible except for the gap it leaves. What is taken from the Prometheus data model rather than from StatsD is the *identity* of a series: a name plus a set of labels, with each distinct label combination a distinct series, and the cardinality warning that comes with it (§3.10). The histogram is Prometheus's cumulative-bucket form, including the property that the top bucket is an overflow whose contents are counted but not located. Two things are deliberately absent. There is no text exposition format, because nothing scrapes. And there is no summary type — a producer that has already computed its own quantiles cannot submit them, because quantiles do not aggregate and a stored one could not be combined with another (§3.25). ## 3.C.3 The query interface The unusual choice here is having a query *language* at all. journald exposes a cursor and a set of field matchers, and computation belongs to the client. The Windows Event Log exposes XPath over an XML representation. Prometheus exposes PromQL, a genuine language, but only for metrics. This interface puts one language over all three data types, with a shared clause vocabulary and per-type modes (§3.18). The reason is access control. Filtering, grouping and aggregation must happen on the side that knows what the caller may see, because a count computed by a client is a count of what the client was given and a count computed by the collector can be a count of what the client is entitled to (§3.28). A cursor interface pushes the computation across the trust boundary and takes the enforcement point with it. The framing — a length-prefixed MessagePack request, a sequence of chunked result messages, one terminal message — is unremarkable and deliberately so. What it does not have is more interesting: no version field (§3.29), no error codes (§3.16), no multiplexing (§3.14), and no cursor. A query is one connection, and paging is `SKIP` and `TAKE` over a total order (§3.21) rather than an opaque token the collector must keep state for. ## 3.C.4 Where these interfaces sit | Concern | Where it is specified | |---|---| | Event emission and the ring-buffer transport | PSPK | | Event types and payload schemas | the emitting subsystem's own documentation | | Tokens, SIDs and Security Descriptors | PCDS, and the Peios Kernel TRM | | Forwarding a service's output | the peinit TRM | | Storage, indexing, retention, query planning | the collector's own design; for the mainline one, the eventd TRMP | --- # 4.1 Scope and Roles _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The two interfaces a Peios service manager offers — the control channel and the notification channel — and what this chapter leaves out. This chapter defines the two interfaces a Peios service manager offers: the **control channel**, by which a program manages services, and the **notification channel**, by which a supervised service reports on itself. Both are Unix-domain sockets between userspace parties, and both have a publicly implementable side. A monitoring tool, an orchestration agent, a shell utility, or a privileged action broker implements the client side of the control channel. Every supervised service that reports readiness, sends keepalives, or preserves file descriptors across a restart implements the producer side of the notification channel. ## 4.1.1 The roles **The manager** is the process that supervises services. It listens on both channels. On Peios this is peinit, running as PID 1, but nothing here depends on that beyond the manager being a single process holding both sockets. **A client** connects to the control channel to issue commands and read answers. A client is any process; it holds no special relationship with the manager beyond the one its token establishes. **A service** is a process the manager started, and speaks the notification channel about itself. A service does not connect to the control channel in that capacity — a program that does both is acting in two roles. Requirements are stated against the role, not the program. ## 4.1.2 What this chapter covers - the two channels, their addressing, and how each is reached - message framing and encoding on both - how a client's identity is established, and how a command is authorised - the command set, the response shapes, and the error vocabulary - what a command does to a service in each of its states - how a service's notification is authenticated, and what a service may say - the file-descriptor store - the rules under which either channel may be extended - the conformance requirements for each role ## 4.1.3 What this chapter does not cover - **How the manager supervises anything.** Dependency resolution, restart policy, timers, cgroups, the boot sequence and shutdown are the manager's own design. This chapter defines what a client can ask for and what it is told, not how the answer comes about. - **How service definitions are expressed.** On Peios they are registry keys, administered like any other registry data. That is the service manager's own design. - **What a service state means.** The vocabulary is fixed here (§4.B) because it appears on the wire; what causes a service to be in one of those states is not. - **Kernel interfaces.** Establishing a peer's identity and evaluating an access decision are kernel operations, specified in PSPK and in the kernel's own reference manual. --- # 4.2 Terminology _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Terms this chapter defines for itself — service, job, operation and advisory — and the ones it borrows unchanged. **Service.** A named unit of execution the manager supervises. Service names are opaque to this chapter except for the character restriction in §4.8. **Job.** One process execution. A service that has been restarted has had more than one job. **Operation.** A requested state machine action on a service, with an identity and a lifecycle of its own. Lifecycle commands do not act directly; they create operations, and an operation is what a client observes and waits on. **Activation generation.** A counter the manager increments each time a service begins starting. It distinguishes one incarnation of a service from the next. **Right.** A named permission on a service or on the manager itself, represented as a bit in an access mask and evaluated against a Security Descriptor. §4.7. **Dependent-satisfying state.** A service state in which the services that depend on the service may proceed. Which states these are is the manager's design; that a state is or is not one of them is observable through the state vocabulary. **Terminal state.** For an operation, one of `completed`, `failed`, `cancelled`, `merged` or `aborted`. An operation in a terminal state does not change again. **Frame.** One newline-terminated line on the control channel, carrying exactly one JSON object. **Datagram.** One message on the notification channel, carrying zero or more `KEY=VALUE` lines and optionally file descriptors. --- # 4.3 The Two Channels _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Why the notification channel is a datagram socket and the control channel a stream, and how a caller reaches either. The two channels differ in almost every respect, and the differences are deliberate. | | Control | Notification | |---|---|---| | Socket type | `SOCK_STREAM` | `SOCK_DGRAM` | | Who connects | The client | Nobody; a service sends | | Addressing | A fixed path | A path given to each service | | Direction | Request and response | One-way | | Framing | Newline-delimited JSON | `KEY=VALUE` lines | | Identity | The peer's token, at connect | The sender's kernel-attested PID | | Authorisation | An access check per command | Membership: is the sender this service? | | Loss | None. A stream, or an error | Possible. A datagram may be dropped | | Ordering | Guaranteed within a connection | Not guaranteed | ## 4.3.1 Why the notification channel is a datagram socket A service reporting on itself must not be able to block the manager, and must not block itself. A stream socket gives both parties a queue that fills, and a service writing into a full queue either blocks — hanging a service on the manager's scheduling — or gets an error it has to handle in the middle of doing something else. A datagram socket has neither problem. A send either goes or is dropped, and the manager can drain at whatever rate it manages. The cost is that a notification can be lost, which is why nothing in §4.19 is a transaction: every field is either idempotent or a statement of current condition, and a service that needs a lost keepalive to have arrived sends another one. ## 4.3.2 Why the control channel is a stream socket A command has an answer, and a client waiting for one needs to know it did not arrive rather than assuming. It also needs framing: a request can be large, and a response certainly can. ## 4.3.3 Reaching either socket Both sockets are protected by the Security Descriptor on the socket's own inode, and a party that may not reach the socket is refused when it connects or sends, before any content is exchanged. The manager MUST NOT rely on POSIX mode bits for this. On a Peios system access to a filesystem object is routed through its Security Descriptor, mode bits are not consulted, and a `chmod` on either socket has no effect whatever. The manager MUST ensure that each socket, and each directory containing one, carries a Security Descriptor that admits the parties intended to use it. A socket created where nothing inheritable applies acquires no descriptor, and an object with no descriptor is denied to every caller — so a manager that leaves this to chance produces a socket nobody can reach, including principals its own default policy grants access to. > [!NOTE] > The failure is quiet in both directions and neither direction > announces itself. A socket in a permissive place is reachable by > anything, and a socket in a bare one is reachable by nothing, and in > both cases the manager binds successfully, reports itself ready, and > serves no one it meant to. --- # 4.4 The Control Channel _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The manager's stream socket at a well-known path, what a connection is, and the limits placed on one. The manager MUST listen on a Unix `SOCK_STREAM` socket at a well-known path. On Peios that path is: ``` /run/services/peinit/control.sock ``` The socket MUST exist for as long as the manager is serving, and the manager MUST unlink it when it stops. The manager MUST create the listening socket and every accepted connection with close-on-exec set, so that no connection descriptor is inherited by a process the manager starts. ## 4.4.1 A connection A client connects, issues one or more commands, and closes. The manager MUST NOT require a client to issue any command before another, and MUST NOT hold state across connections: a connection carries an identity (§4.6) and nothing else. Requests on one connection MUST be answered in the order they were received. The manager MAY read no further frames from a connection while a response on it is outstanding. ## 4.4.2 Limits The manager MUST enforce three limits, and MUST make their values discoverable to an administrator through the same configuration surface that sets them. The values a Peios service manager uses by default are in §4.A. **Concurrent connections.** A connection accepted while the manager is already at its limit MUST be closed at the socket level, without a response. There is no error code for this condition: the manager has declined to enter the protocol at all, and a client MUST treat an immediate close with no response as a refusal rather than as a protocol error. **Request size.** A request frame whose content exceeds the limit MUST be answered with `REQUEST_TOO_LARGE` and the connection MUST then be closed. The limit applies to the frame's content and MUST NOT count the terminating newline, so a request of exactly the limit plus its newline is within bounds. **Idle timeout.** A connection with no request outstanding MAY be closed once it has been idle for the configured period. The manager MUST NOT treat a connection as idle while a request on it is outstanding — in particular a connection blocked on a `wait=true` operation (§4.13) is not idle, however long the operation runs, and MUST be held open until the operation resolves. Such a connection is bounded by the operation's own timeout, not by the idle timeout. A connection closed for idleness MUST be closed without a response. --- # 4.5 Framing and Encoding _Peios / Advanced Peios / PSPU / Service Control and Notification_ > One compact JSON object per newline-terminated frame, in both directions — what counts as malformed, and when the manager closes. ## 4.5.1 Frames Every message in both directions is one frame: a single JSON object, serialised compactly, followed by one `0x0A` byte. This applies to requests and to responses alike, and the manager MUST terminate every response with a newline. Framing is byte-oriented and is performed before any JSON is parsed. A `0x0A` byte ends the frame wherever it appears, so a raw newline inside what a sender intended as a JSON string does not produce one frame with an embedded newline — it produces two malformed ones. (A raw `0x0A` inside a JSON string is not valid JSON in any case; the `\n` escape sequence is unaffected and is the way to carry a newline in a value.) The manager MUST NOT emit pretty-printed JSON, and MUST NOT emit more than one object per frame. ## 4.5.2 Encoding Frames are UTF-8. The manager MUST reject a frame that is not well-formed UTF-8 with `MALFORMED_REQUEST`. ## 4.5.3 What is malformed The manager MUST answer with `MALFORMED_REQUEST` when a frame: - is empty — a bare newline with no content; - is not well-formed UTF-8; - is not valid JSON; - is valid JSON but not an **object**. An array, a string, a number, `true`, `false` and `null` are all malformed requests. ## 4.5.4 Closing after an error The manager MUST distinguish two classes of failure, because they say different things about the connection. A **frame-level** failure means the manager cannot trust the stream's framing any more: it does not know where the next frame begins. `MALFORMED_REQUEST` for an empty frame and `REQUEST_TOO_LARGE` are both frame-level. The manager MUST send the error response, discard any buffered input, and close the connection. A **command-level** failure means the frame was well-formed and the command in it could not be carried out: unparseable JSON content, an unknown command, missing arguments, a denied access check, an unknown service. The manager MUST send the error response and MUST keep the connection open. A client MUST NOT assume a connection survives an error response, and MUST be prepared for either. ## 4.5.5 Timestamps Every timestamp field the manager emits MUST be a UTC RFC 3339 string with exactly **nine** fractional-second digits and the literal offset marker `Z`: ``` "2026-06-01T12:34:56.123456789Z" ``` The manager MUST NOT emit a numeric offset in place of `Z`, and MUST NOT vary the number of fractional digits. These are wall-clock instants, presented for a reader. The manager MUST NOT derive elapsed-time decisions — timeouts, retries, ordering — from wall-clock differences, and a client MUST NOT assume that two timestamps in the same response were taken from a clock that did not move between them. --- # 4.6 Peer Identity _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The manager establishes every client's identity from the kernel, at connect, and a client can never assert who it is. The manager MUST establish the identity of every client from the kernel. There is no credential exchange in this protocol, and a client MUST NOT be able to assert who it is. ## 4.6.1 Obtaining the identity On accepting a connection, the manager MUST obtain the peer's token from the kernel. On Peios this is `kacs_open_peer_token`, which returns a token descriptor for the peer. The token obtained is the peer **thread's effective token at the moment of the call**. A client that is impersonating another principal is therefore captured as the principal it is impersonating, not as its own service identity — which is the intended behaviour: access decisions reflect the identity a client is actually acting under. ## 4.6.2 When it is captured The manager MUST capture the identity once, when the connection is accepted, and MUST use that identity for every command on the connection. A client MUST NOT expect a change of identity mid-connection to affect authorisation. A client that needs to act under a different identity MUST open a new connection. > [!NOTE] > Capturing once is what makes the identity meaningful. A per-command > capture would evaluate each command against whatever the peer happened > to be at the moment the manager got round to reading it, which is a > race a client could steer. ## 4.6.3 Failure If the manager cannot obtain the peer's identity, it MUST close the connection without a response. There is no error code, because the manager has no basis on which to decide whether this caller may be told anything at all. A client MUST treat an immediate close with no response as a refusal. This is the same observable outcome as exceeding the connection limit (§4.4), and a client cannot distinguish the two — deliberately, since distinguishing them would tell an unauthenticated caller about the manager's state. ## 4.6.4 The identity is not a UID The manager MUST NOT use the peer's UID or GID as an authorisation input. Identity on a Peios system is a token, and the token is what the kernel attests. --- # 4.7 Authorising a Command _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Every command is checked against a Security Descriptor using the peer's token — the rights, the per-command mapping, and why some results are filtered rather than denied. Every command is authorised against a Security Descriptor, using the peer's token. There is no command the manager performs without a check, and no principal exempt from one. ## 4.7.1 Rights Commands acting on a service are checked against that service's descriptor: | Right | Bit | Grants | |---|---|---| | `SERVICE_QUERY_STATUS` | 0x0001 | Query the service's state and detail. | | `SERVICE_START` | 0x0002 | Start the service. | | `SERVICE_STOP` | 0x0004 | Stop the service. | | `SERVICE_INTERROGATE` | 0x0008 | Reload the service. | | `SERVICE_ALL_ACCESS` | 0x000F | All four. | Commands acting on the system are checked against the manager's own descriptor: | Right | Bit | Grants | |---|---|---| | `SYSTEM_SHUTDOWN` | 0x0001 | Initiate a shutdown. | | `SYSTEM_RELOAD_CONFIG` | 0x0002 | Re-read the configuration. | ## 4.7.2 Generic mappings The manager MUST use these generic mappings when evaluating a descriptor, so that a descriptor written in generic terms means the same thing to every implementation. For a service descriptor: | Generic right | Maps to | |---|---| | `GENERIC_READ` | `SERVICE_QUERY_STATUS` | | `GENERIC_WRITE` | `SERVICE_START` \| `SERVICE_STOP` \| `SERVICE_INTERROGATE` | | `GENERIC_EXECUTE` | `SERVICE_START` \| `SERVICE_STOP` \| `SERVICE_INTERROGATE` | | `GENERIC_ALL` | `SERVICE_ALL_ACCESS` | For the manager's descriptor: | Generic right | Maps to | |---|---| | `GENERIC_READ` | 0 | | `GENERIC_WRITE` | `SYSTEM_RELOAD_CONFIG` | | `GENERIC_EXECUTE` | `SYSTEM_SHUTDOWN` | | `GENERIC_ALL` | `SYSTEM_SHUTDOWN` \| `SYSTEM_RELOAD_CONFIG` | `GENERIC_READ` maps to nothing on the manager's descriptor because it governs two actions and no queries. ## 4.7.3 Per command | Command | Right required | |---|---| | `start` | `SERVICE_START` | | `stop` | `SERVICE_STOP` | | `restart` | `SERVICE_START` and `SERVICE_STOP` | | `reload` | `SERVICE_INTERROGATE` | | `reset` | `SERVICE_STOP` | | `status` | `SERVICE_QUERY_STATUS` | | `list` | Evaluated per service; see below | | `operation-status` | `SERVICE_QUERY_STATUS` on the operation's target | | `shutdown` | `SYSTEM_SHUTDOWN` | | `reload-config` | `SYSTEM_RELOAD_CONFIG` | `reset` requires `SERVICE_STOP` because clearing a terminal state is the tail of stopping something rather than the head of starting it. ## 4.7.4 The sequence 1. If the manager is shutting down, apply §4.15's restriction. The shutdown restriction is evaluated **before** the access check, so a caller who would have been denied is told the command is invalid for the current state. A client MUST NOT infer anything about its own rights from an `INVALID_STATE` received during shutdown. 2. Resolve the target. A command naming no service the manager knows of MUST be answered `UNKNOWN_SERVICE`. The manager MUST NOT synthesise a descriptor for a service that does not exist. 3. Evaluate the access check with the peer's token, the target's descriptor, the appropriate generic mapping, and the required right. 4. On denial, answer `ACCESS_DENIED`, and record the attempt with at least the caller's SID, the target, and the right requested. The manager MUST NOT deny silently. 5. On grant, proceed. ## 4.7.5 Filtering rather than denying `list` MUST return only the services the caller may query, and MUST **omit** the rest rather than denying the command. A caller with no query rights on anything receives an empty list and a successful response. The manager MUST NOT reveal, through the response, that services were omitted. Reporting the omissions would answer the question the filtering exists to leave unanswered. ## 4.7.6 Not revealing what a caller may not see Where a command names an object the caller may not query, the manager MUST NOT let the answer distinguish "this does not exist" from "you may not see this". For `operation-status` this means the authorisation check MUST be evaluated before the operation's existence is reported: a caller lacking `SERVICE_QUERY_STATUS` on an operation's target MUST receive `ACCESS_DENIED` whether or not the identifier names a real operation, and MUST NOT receive `UNKNOWN_OPERATION` for one that exists. Where the caller's rights cannot be established because the target cannot be resolved, `UNKNOWN_OPERATION` is the correct answer. --- # 4.8 Requests _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The fields a request JSON object carries, which apply to which commands, and the rules service names must satisfy. A request is one JSON object. ```json {"command": "start", "service": "jellyfin", "wait": true} ``` ## 4.8.1 Fields | Field | Type | Required | Meaning | |---|---|---|---| | `command` | string | always | The command to run. §4.11, §4.14, §4.15 | | `service` | string | for service commands | The target service's name. | | `wait` | bool | no | Whether to block until the operation resolves. §4.13 | | `type` | string | for `shutdown` | `poweroff`, `reboot` or `halt`. | | `operation_id` | string | for `operation-status` | The operation to report on. | `command` MUST be present and MUST be a string naming a command the manager implements. A request whose `command` is absent, is not a string, or names no known command MUST be answered `INVALID_COMMAND`. `service` MUST be present and a string for `start`, `stop`, `restart`, `reload`, `reset` and `status`. Its absence, or a non-string value, MUST be answered `INVALID_ARGUMENTS`. `wait` MUST be a boolean when present. A non-boolean MUST be answered `INVALID_ARGUMENTS`. Its default is per command (§4.13). `type` MUST be present and MUST be exactly one of the three values for `shutdown`. Anything else MUST be answered `INVALID_ARGUMENTS`. `operation_id` MUST be present and a string for `operation-status`. A value that is not a well-formed identifier MUST be answered `INVALID_ARGUMENTS`. ## 4.8.2 Fields that do not apply A field the command does not use MUST be ignored, not rejected. A `service` on a `list`, or a `wait` on a `status`, is accepted and has no effect. This is what makes the request shape extensible: a client written against a later revision may send a field an earlier manager does not know, and the earlier manager ignores it. §4.21. ## 4.8.3 Service names A service name is 1 to 128 bytes drawn from `[A-Za-z0-9._-]`. The manager MUST NOT accept a name outside that set, and a client MUST NOT send one. The restriction exists because service names are used as path components and as configuration key names by managers that store their definitions in a hierarchy. `/` and `:` are excluded specifically: the first because it is a separator wherever the name is used as a path component, and the second because it is conventionally reserved for a manager's own synthetic naming. --- # 4.9 Responses _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The four response shapes — acknowledgement, status, system and error — and what may be null in each. Every response carries a `status` field, which MUST be exactly `"ok"` or `"error"`. What else it carries depends on which of four shapes it is. ## 4.9.1 The acknowledgement shape Returned by a lifecycle command that created, merged into, queued, cancelled, cleared or executed an operation. ```json {"status": "ok", "operation_id": "a1b2c3d4-…", "service": "jellyfin", "state": "active", "cause": "explicit_start", "warnings": []} ``` | Field | Type | Meaning | |---|---|---| | `operation_id` | string | The operation to observe. | | `service` | string | The target. | | `state` | string | The service's state when the response was formed. §4.B | | `cause` | string or null | Why the service last transitioned. §4.B | | `warnings` | array of strings | Human-readable warnings. Often empty. | | `mode` | string | For a `reload` only. §4.13 | `warnings` here is an array of **strings**. The `status` response uses the same field name for an array of objects (§4.14); a client MUST distinguish them by which command it sent, not by inspecting the array. ## 4.9.2 The status shape Returned by `status`, and also by a lifecycle command that had no effect — see §4.12. §4.14 gives it in full. ## 4.9.3 The system shape Returned by `shutdown`: ```json {"status": "ok"} ``` Nothing else. A shutdown has no operation to observe and no service to report on. `reload-config` has its own shape (§4.15). ## 4.9.4 The error shape ```json {"status": "error", "code": "ACCESS_DENIED", "message": "caller lacks SERVICE_START on jellyfin"} ``` `code` MUST be one of the values in §4.10. `message` is human-readable and is not normative: a client MUST NOT parse it, match on it, or branch on its content. Two managers answering the same request with the same code MAY word the message differently. ## 4.9.5 Nullability A field that does not apply to the current state MUST be present and `null` rather than omitted, except where this chapter says otherwise. A client MUST accept `null` for any field this chapter marks nullable, and MUST NOT treat a `null` as an error. The two exceptions are `mode`, which appears only on a reload response, and `job_id` in the notification event payloads, which is omitted when there is no job. --- # 4.10 Errors _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The closed set of error codes a manager may emit, the distinctions a client can rely on, and why the set cannot grow without a version bump. The `code` field of an error response MUST be one of these values. The manager MUST NOT emit any other code, and a client MUST treat a code it does not recognise as an unrecoverable error for that request (§4.21). | Code | Meaning | Closes? | |---|---|---| | `MALFORMED_REQUEST` | The frame is not a single well-formed JSON object. §4.5 | On an empty frame | | `REQUEST_TOO_LARGE` | The request exceeds the configured maximum. §4.4 | Yes | | `INVALID_COMMAND` | `command` is absent, not a string, or names no known command. | No | | `INVALID_ARGUMENTS` | A field the command requires is absent or malformed. | No | | `UNKNOWN_SERVICE` | The named service has no definition the manager can act on. | No | | `UNKNOWN_OPERATION` | The operation identifier names nothing the manager holds — it never existed, or its retention has elapsed. §4.14 | No | | `ACCESS_DENIED` | The access check denied the requested right. §4.7 | No | | `INVALID_STATE` | The command is not valid for the service's current state (§4.12), or the manager is shutting down (§4.15). | No | | `OPERATION_TIMEOUT` | A `wait=true` request's operation did not reach a terminal state in time. §4.13 | No | | `INTERNAL_ERROR` | The manager failed while executing the command. | No | ## 4.10.1 Distinctions a client can rely on **`UNKNOWN_SERVICE` versus `ACCESS_DENIED`.** A caller that may not query a service still receives `UNKNOWN_SERVICE` for a name that does not exist and `ACCESS_DENIED` for one that does but which it may not touch. This chapter does not attempt to hide the existence of services from a caller that can name them: the `list` filtering (§4.7) hides them from a caller that cannot. **`INVALID_STATE` versus `ACCESS_DENIED` during shutdown.** During shutdown the state restriction is evaluated first, so a caller who would have been denied receives `INVALID_STATE` instead. A client MUST NOT infer that it holds a right from receiving `INVALID_STATE`. **`OPERATION_TIMEOUT` does not cancel anything.** It reports that the client's wait ended, not that the operation did. The operation continues and can still be observed with `operation-status`. ## 4.10.2 Codes are not extensible without a version The manager MUST NOT introduce a new code without the version negotiation in §4.21. A client written against this revision will not recognise one, and the only safe thing it can do with an unrecognised code is fail the request — so a new code silently converts a handled condition into an unhandled one. --- # 4.11 Lifecycle Commands _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The five commands that move a service through its state machine — none acting directly, each creating, merging into, queuing or cancelling an advisory. Five commands move a service through its state machine. None of them acts directly: each creates, merges into, queues or cancels an **operation**, and the operation is what actually happens. | Command | Effect | Default `wait` | |---|---|---| | `start` | Start the service. | true | | `stop` | Stop the service, escalating if it does not exit. | true | | `restart` | Stop then start, under one operation. | true | | `reload` | Tell the service to re-read its configuration. | **false** | | `reset` | Clear a terminal state, returning the service to inactive. | false | `reload` defaults to not waiting because a reload's outcome is often advisory, and a client usually wants the identifier rather than the block. `reset` is synchronous and completes before the response is sent, so waiting on it would mean nothing. ## 4.11.1 Operations The manager MUST return an operation identifier from any lifecycle command that created, merged into, queued, cancelled, cleared or executed an operation. The client uses it to poll (§4.14) or to correlate. The manager MUST NOT invent an operation solely so that it has an identifier to return. Where a command had no effect, or the service was already in the state asked for, the manager MUST return the status shape instead of an acknowledgement (§4.12). ## 4.11.2 Merging Where an operation of the same type is already in flight for the same service, the manager MUST merge the new request into it and MUST return the **existing** operation's identifier. A merged caller therefore receives an identifier that may be older than its own request, whose `requested_at` precedes the moment it sent the command. This is correct — that is when the work being waited on began — and a client MUST NOT treat an identifier older than its request as an error. The manager MUST NOT tell the caller that a merge occurred. A merge is not a distinguishable outcome, and a client cannot do anything with the knowledge. ## 4.11.3 What completion means | Command | The operation completes when | |---|---| | `start` | The service reaches a dependent-satisfying state, or a state indicating its start-time conditions did not apply. | | `stop` | The service is no longer running. | | `restart` | The service reaches its normal successful start target after the restart. | | `reload` | The reload resolves, whatever its mode. | | `reset` | Immediately. | ## 4.11.4 Timeouts Every operation has a maximum lifetime, derived from the target service's own configured timeouts. **The lifetime is measured from the operation's creation, including any time it spent queued.** From the caller's point of view they have been waiting since they sent the command, and an operation that sat behind another for longer than its lifetime MUST fail rather than begin. An operation whose lifetime expires while it is still queued MUST fail, and MUST fail its waiters. Expiry of the operation object MUST NOT by itself authorise the manager to act on the service — a stop operation that timed out while waiting its turn does not license signalling the service ahead of that turn. --- # 4.12 Command Outcomes by State _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The defined answer for every command sent to a service in an unexpected state — the manager never silently does nothing. A command sent to a service in an unexpected state MUST receive a defined answer. The manager MUST NOT silently do nothing. | | inactive | starting | active | reloading | stopping | completed | backoff | failed | abandoned | skipped | |---|---|---|---|---|---|---|---|---|---|---| | `start` | act | merge | already | already | queue | act | defer | act | invalid | act | | `stop` | noop | cancel + act | act | act | merge | clear | cancel | noop | invalid | noop | | `restart` | act | queue | act | act | queue | act | act | act | invalid | act | | `reload` | invalid | invalid | act | merge | invalid | invalid | invalid | invalid | invalid | invalid | | `reset` | noop | invalid | invalid | invalid | invalid | invalid | invalid | clear | clear | clear | | `status` | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ## 4.12.1 The outcomes **act** — create an operation and execute it. The manager returns the acknowledgement shape. **merge** — an operation of this type is in flight. The command merges into it (§4.11) and the caller receives that operation's identifier. **queue** — the operation is created and left pending; it executes once the operation ahead of it completes. The caller receives the new operation's identifier. **defer** — an automatic restart is already pending for this service. The manager MUST create a pending start operation, or merge into a deferred one that already exists, and MUST NOT execute it until the existing delay has elapsed. A `start` MUST NOT shorten a pending restart's delay. **already** — the service is in the state the command would take it to and no operation of this type is in flight. The manager MUST return the **status** shape, not an error and not an acknowledgement. **noop** — the command has no effect. The manager MUST return the status shape. **clear** — the service returns to inactive. This is a synchronous outcome; the manager returns an acknowledgement. **cancel** — abort or cancel the operation in flight, then proceed. **invalid** — the command is not valid for this state. The manager MUST answer `INVALID_STATE`. **ok** — `status` is answered from any state. ## 4.12.2 The `backoff` column A service in `backoff` is down with an automatic restart pending, and the four lifecycle commands mean different things there: - `start` defers, as above, and honours the remaining delay. - `stop` cancels both the pending restart and any deferred start, and the service becomes inactive. - `restart` cancels the automatic restart and performs a caller-initiated one. - `reload` and `reset` are invalid: there is no process to reload and no terminal state to clear. ## 4.12.3 The `abandoned` column Every lifecycle command except `reset` is invalid on an abandoned service. `reset` clears it. Nothing else is meaningful while processes the manager could not terminate are still present. ## 4.12.4 A service being withdrawn A manager MAY keep supervising a service whose definition has been removed while an instance of it is still running. In that condition the manager MUST answer `start`, `restart` and `reload` with `UNKNOWN_SERVICE`, MUST accept `stop`, and MUST report the condition in the status shape (§4.14). This holds whatever the service's state. --- # 4.13 Wait Semantics _Peios / Advanced Peios / PSPU / Service Control and Notification_ > What ending a wait means, and how the wait flag changes when a lifecycle command responds. `wait` decides whether a lifecycle command's response is sent immediately or held until the operation resolves. With `wait: false`, the manager MUST respond as soon as it has accepted the operation, with the acknowledgement shape and the service's state at that moment. With `wait: true`, the manager MUST hold the connection open and respond when the operation reaches a terminal state, with the same shape and the service's state, cause and warnings observed at that time. A connection blocked on a wait is not idle (§4.4). The manager MUST NOT close it for idleness however long the operation runs. ## 4.13.1 When a wait ends | Ending | Response | |---|---| | The operation reaches a terminal state | The acknowledgement shape. | | The operation's lifetime expires | `OPERATION_TIMEOUT`. | | The operation is no longer held by the manager | `UNKNOWN_OPERATION`. | `OPERATION_TIMEOUT` ends the client's wait, not the operation. The operation continues, and the client MAY still observe it with `operation-status` using the identifier it never received — which it does not have. A client that needs to survive a timeout SHOULD issue the command with `wait: false`, keep the identifier, and poll. ## 4.13.2 Reload mode A response to a `reload` command MUST carry a `mode` field saying how the reload resolved: | Value | Meaning | |---|---| | `confirmed` | The service acknowledged the reload by signalling readiness. The reload demonstrably happened. | | `advisory` | The manager issued the reload and the service did not acknowledge it. The reload probably happened; nothing confirms it. | | `failed` | The reload did not happen. An external reload command exited non-zero or timed out. | `mode` MUST be present on every response to a `reload`, including one sent with `wait: false` — in which case it MUST be `advisory`, since nothing has been observed yet. A client MUST treat these three as an exhaustive set and MUST NOT expect a fourth. A manager MUST NOT introduce one without §4.21. The distinction between `confirmed` and `advisory` is the whole value of the field: a service that implements the reload handshake (§4.19) can be *known* to have reloaded, and one that does not cannot. `failed` does not mean the service stopped — a failed reload leaves a running service running. --- # 4.14 Query Commands _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The three commands that read state and change nothing — status, list and operation-status — and how long results are retained. Three commands read state and change nothing. ## 4.14.1 status Returns everything the manager knows about one service. ```json { "status": "ok", "service": "jellyfin", "state": "active", "cause": "explicit_start", "status_text": "Listening on port 8096", "current_job": { "id": "a1b2c3d4-…", "type": "service_main", "pid": 1234, "started_at": "2026-06-01T12:34:56.123456789Z", "identity": "jellyfin-svc" }, "current_operation": { "id": "e5f6g7h8-…", "type": "start", "source": "admin" }, "health": "healthy", "uptime_seconds": 86400, "definition_removed": false, "warnings": [] } ``` | Field | Type | Meaning | |---|---|---| | `state` | string | §4.B. | | `cause` | string or null | Why the service last transitioned. §4.B. | | `status_text` | string or null | The most recent status string the service sent (§4.19). | | `current_job` | object or null | The current main job, or null if none. | | `current_operation` | object or null | The current operation, or null if none. | | `health` | string or null | `healthy`, `unhealthy`, `unknown`, or null when the service has no health check configured. | | `uptime_seconds` | integer or null | Whole seconds since the current job started. Null when nothing is running. | | `definition_removed` | bool | True while the service's definition has been withdrawn and an instance is still draining (§4.12). | | `warnings` | array of objects | Conditions worth an operator's attention. | `current_job` carries `id`, `type` (§4.B), `pid`, `started_at` and `identity`. `pid` and `started_at` are independently nullable. `identity` is the identity string the manager resolved for the execution, which is not necessarily what the resulting token contains. `current_operation` carries `id`, `type` and `source` (§4.B). The manager MUST clear `status_text` to null at the start of every activation generation. A status string from a previous incarnation MUST NOT survive a restart and be reported as though it described the current process. ### 4.14.1.1 Status warnings `warnings` in the status shape is an array of **objects**, not strings: ```json {"path": "/sys/fs/cgroup/peinit/jellyfin/health", "type": "health", "detected_at": "2026-06-01T12:34:56.123456789Z"} ``` | Field | Type | Meaning | |---|---|---| | `path` | string | What the warning is about. | | `type` | string | The kind of warning. §4.B. | | `detected_at` | string | When the manager noticed. §4.5. | A client MUST accept a `type` it does not recognise and MUST NOT discard the warning, since a warning it cannot classify is still one an operator should see. ## 4.14.2 list Returns every service the caller may query, with a compact summary. ```json { "status": "ok", "services": [ {"service": "jellyfin", "state": "active", "cause": "explicit_start", "health": "healthy"}, {"service": "registryd", "state": "active", "cause": "dependency_start", "health": null} ] } ``` Exactly four fields per entry. Services the caller may not query are omitted (§4.7). A service whose definition has been withdrawn is listed, and the list entry does not say so. A client that needs to know MUST issue a `status`. ## 4.14.3 operation-status Returns one operation by identifier. ```json { "status": "ok", "operation": { "id": "e5f6g7h8-…", "type": "start", "service": "jellyfin", "source": "admin", "state": "completed", "result": "active", "merged_into": null, "error": null, "requested_at": "2026-06-01T12:34:56.123456789Z", "started_at": "2026-06-01T12:34:56.223456789Z", "completed_at": "2026-06-01T12:34:58.923456789Z" } } ``` | Field | Meaning | Present when | |---|---|---| | `id` | The operation's identifier. | Always. | | `type` | §4.B. | Always. | | `service` | The target. | Always. | | `source` | Why the manager created it. §4.B. | Always. | | `state` | §4.B. | Always. | | `result` | The resulting service state. | `completed`. | | `error` | Why it did not complete. | `failed`, `cancelled`, `aborted`. | | `merged_into` | The surviving operation's identifier. | `merged`. | | `requested_at` | When it was created. | Always. | | `started_at` | When it began executing. | Once running. | | `completed_at` | When it reached a terminal state. | Once terminal. | Fields that do not apply to the current state MUST be null. `error` carries a reason for all three non-success terminal states, not only for `failed`. A client MUST NOT read a non-null `error` as meaning the operation failed — it MUST read `state` for that. Cancellation and abortion have reasons worth reporting, and a separate field for each would give a client three places to look for one fact. ## 4.14.4 Retention The manager MUST hold an operation record for at least a grace period after it reaches a terminal state, so that a client polling for the result can retrieve it. The value a Peios service manager uses is in §4.A. An identifier that never existed, and one whose record has been dropped after its grace period, MUST both be answered `UNKNOWN_OPERATION`. A client MUST NOT distinguish them, and MUST treat `UNKNOWN_OPERATION` after a successful acknowledgement as meaning the result is no longer available rather than that the operation never ran. --- # 4.15 System Commands _Peios / Advanced Peios / PSPU / Service Control and Notification_ > shutdown and reload-config — the two commands that act on the manager itself, and why reload is atomic but not a live update. Two commands act on the manager rather than on a service. ## 4.15.1 shutdown ```json {"command": "shutdown", "type": "reboot"} ``` `type` MUST be one of: | Value | Meaning | |---|---| | `poweroff` | Stop everything and remove power. | | `reboot` | Stop everything and restart the machine. | | `halt` | Stop everything and halt, leaving the machine powered. | The response is `{"status": "ok"}` and nothing else. There is no operation to observe: a shutdown is a mode the manager enters, not an action on a service, and by the time it has finished there is nobody left to tell. A client MUST NOT expect the connection to survive. The manager MAY close it at any point after the response. ## 4.15.2 reload-config Re-reads the configuration and rebuilds whatever the manager derives from it. ```json { "status": "ok", "summary": { "added": ["jellyfin"], "updated": ["sshd"], "restored": [], "marked_removed": ["old-migration"], "discarded": ["obsolete-timer"] }, "warnings": [] } ``` | Field | Type | Meaning | |---|---|---| | `added` | array of strings | Services that did not exist before. | | `updated` | array of strings | Services whose definition changed. | | `restored` | array of strings | Services whose withdrawal was reversed. | | `marked_removed` | array of strings | Services whose definition is gone but which are still running. | | `discarded` | array of strings | Services removed outright. | | `warnings` | array of strings | Human-readable warnings about the new configuration. | Every member of `summary` MUST be present, even when empty. A client MUST accept a member of `summary` it does not recognise, and MUST ignore it (§4.21). ### 4.15.2.1 It is atomic The manager MUST validate the new configuration in full before adopting any of it, and MUST adopt it only if validation succeeds. If validation fails, the manager MUST leave the previous configuration in force and MUST answer `INVALID_STATE`, reporting what was wrong. A partially applied configuration is worse than the one already running: the running one at least booted. ### 4.15.2.2 It does not live-update The manager MUST NOT reconfigure a running service. A changed definition takes effect the next time that service starts. ## 4.15.3 During shutdown Once the manager is shutting down, it MUST reject every command except `status`, `list` and `operation-status` with `INVALID_STATE`. Those three are permitted because they change nothing and because a client watching a shutdown proceed has a legitimate reason to keep looking. Everything else — including a second `shutdown` — is refused: the manager has committed to a course of action and a command that would alter it arrives too late to be honoured consistently. As §4.7 says, this restriction is evaluated before the access check, so a caller who would have been denied receives `INVALID_STATE` instead. --- # 4.16 The Notification Channel _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The datagram socket a service reports on itself over — how it is addressed, how delivery works, and the bounds on it. A service reports on itself over a Unix `SOCK_DGRAM` socket the manager binds and holds for the lifetime of the system. ## 4.16.1 Addressing The manager MUST make the socket's path available to each service it starts, in the `NOTIFY_SOCKET` environment variable, set in the service process's environment before exec. The path is not part of this contract, and a service MUST NOT hardcode one. The manager MAY bind one socket for all services or one per service; a service cannot tell and MUST NOT depend on either. The manager MUST set `NOTIFY_SOCKET` unconditionally, for every service it starts, whatever readiness protocol that service uses. A service uses this channel for keepalives, status, timeout extension and the descriptor store as well as for readiness, and a manager that set the variable only for services expected to signal readiness would make the rest unreachable. The manager MUST NOT allow `NOTIFY_SOCKET` to be overridden by any configurable environment layer. A service that could override it would silently disable its own supervision. ## 4.16.2 Direction and delivery The channel is one-way. The manager does not reply, and a service MUST NOT wait for one. Delivery is not guaranteed. A datagram MAY be dropped, by the kernel under load or by the manager. Every field in §4.19 is therefore either idempotent or a statement of a current condition, and a service that needs an effect to have taken hold sends the field again rather than waiting for an acknowledgement that does not exist. The manager MUST NOT let this channel exert backpressure on a service. A service MUST NOT be able to block by sending, and the manager MUST NOT require a service to slow down. ## 4.16.3 Bounds The manager MUST accept a datagram of at least the size in §4.A, and MUST accept at least the number of file descriptors in §4.A in one datagram's control message. The manager MUST detect a datagram that exceeded either bound and MUST reject the whole datagram (§4.17). It MUST NOT process a truncated datagram: a truncation can leave a tail that parses as a complete, valid line, which would apply a field the sender did not send. A service MUST NOT send a datagram exceeding either bound. --- # 4.17 Datagram Framing _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Newline-separated KEY=VALUE lines — what makes a datagram malformed, the three ways one line can fail, and why rejection is recorded rather than answered. A datagram carries zero or more lines, separated by `0x0A`. Each line is `KEY=VALUE`. ``` READY=1 STATUS=Listening on port 8096 ``` A trailing newline on the last line is permitted and is not a line of its own. A trailing `0x0D` on any line MUST be stripped before the line is interpreted, so a sender that emits CRLF is understood. The datagram MUST be well-formed UTF-8. ## 4.17.1 Applying a datagram The manager MUST parse every line before applying any of them, and MUST apply every line of a datagram it accepts, in order. **If any line is malformed, the manager MUST reject the entire datagram and apply nothing from it.** Any file descriptors it carried MUST be closed. Partial application is the failure this rule exists to prevent. A datagram saying `RELOADING=1` and something unintelligible has an ambiguous meaning, and applying the half that parsed picks one reading of it silently. ## 4.17.2 What is malformed A **line** is malformed when it is non-empty and: - it contains no `=`; or - its key is empty. An **empty line** is not malformed. It is skipped. A datagram is malformed when it is not well-formed UTF-8, or when it exceeded a bound in §4.16. ## 4.17.3 Three ways a line can fail to take effect These are distinct and a service author needs the distinction: | Situation | Effect on the datagram | Effect on the line | |---|---|---| | A malformed line | Rejected entirely | — | | An **unrecognised key** | Applied normally | Ignored | | A recognised key with an **unexpected value** | Applied normally | Ignored | The second is what makes the field set extensible (§4.21): a service built against a later revision may send a field an older manager does not know, and the older manager applies the rest. The third is the one that surprises. `READY=0` is not a malformed line and does not reject the datagram; `READY` expects the value `1` and anything else is silently ignored. A service MUST NOT send a recognised key with a value the field does not define, and MUST NOT expect to be told when it does. §4.19 gives each field's accepted values. ## 4.17.4 Rejection is recorded, not answered The manager MUST record a rejected datagram, with at least the sender's identity and the reason, and MUST attribute it to a service where the sender could be identified. It MUST NOT reply. There is nothing to reply on. --- # 4.18 Sender Authentication _Peios / Advanced Peios / PSPU / Service Control and Notification_ > A datagram claims to be a service talking about itself; the steps by which a manager establishes that it is, and what it must never use. A datagram on this channel claims to be a service talking about itself. The manager MUST establish that it is. ## 4.18.1 The requirements The manager MUST enable `SO_PASSCRED` on the socket, and MUST reject any datagram arriving without a kernel-attested credentials control message. It MUST then establish all of the following, and MUST drop the datagram if any fails: 1. **The sender is a service's current main job.** The manager MUST match the attested PID against the main jobs it is supervising. A hook process, a health check, or a child a service forked MUST NOT be able to notify on the service's behalf. 2. **That job has exec'd and is running.** A job still in setup has not become the service yet. 3. **That job has a kernel handle on the process** — a pidfd, or an equivalent that refers to one specific process rather than to a number. 4. **The handle still refers to the attested PID.** The manager MUST verify the PID against the handle rather than trusting the PID alone. 5. **The job's activation generation is the service's current one.** ## 4.18.2 Why steps 3 and 4 exist A PID identifies a process only until that process exits. Between a service writing a datagram and the manager reading it, the service can die and its PID be recycled onto something else — and PID matching alone would then attribute the unrelated process's message to the service, or attribute the service's message to whatever now holds the number. A handle obtained atomically at fork does not have that property. Verifying the attested PID against the handle is what turns a probable match into a certain one. ## 4.18.3 Why step 5 exists A datagram sent by an incarnation of a service that has since been restarted MUST NOT be applied to its replacement. Without the generation check, a `READY=1` written by a process moments before it crashed could mark the process that replaced it ready — declaring a service healthy on the strength of a message from the one that just failed. Readiness is per activation generation, and so is everything else on this channel. ## 4.18.4 What the manager MUST NOT use The manager MUST NOT use the sender's UID or GID as an authorisation input, and MUST NOT accept any identity a service asserts in the datagram's content. Identity on this channel is *which supervised process this is*, and only the kernel can attest that. A service does not have a name here that it gets to state. --- # 4.19 Notification Fields _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Every field a service may send — lifecycle, health, reporting and descriptor-store — and the ones deliberately not supported. Every field a service may send. A manager MUST implement all of them. A service MUST NOT send a recognised key with a value the field does not define (§4.17). ## 4.19.1 Lifecycle | Field | Value | Meaning | |---|---|---| | `READY` | `1` | Startup is complete and the service is serving. | | `RELOADING` | `1` | Configuration reload has begun. | | `STOPPING` | `1` | Graceful shutdown has begun. | **`READY=1`** is what a service using notification readiness sends when it is genuinely able to serve, not when its process exists. Anything depending on the service starts on the strength of it, so a service that signals early declares its dependents' assumptions true before they are. **`RELOADING=1`** opens a reload. The manager waits a bounded period after issuing a reload for this field; a service that sends it MUST follow with `READY=1` when the reload is complete, and the pair is what lets the manager report the reload `confirmed` rather than `advisory` (§4.13). A service that never sends either still reloads — it just cannot be observed to have done so. **`STOPPING=1`** tells the manager the service is already shutting down. A manager that receives it MUST NOT send a further termination signal to that service. It MUST NOT extend or reset the stop timeout: the service still has to exit within it, and a service needing longer sends `EXTEND_TIMEOUT_USEC`. ## 4.19.2 Health | Field | Value | Meaning | |---|---|---| | `WATCHDOG` | `1` | A keepalive. | | `WATCHDOG_USEC` | unsigned integer | Change the expected keepalive interval, in microseconds. | | `EXTEND_TIMEOUT_USEC` | unsigned integer | Extend the current transition's deadline, in microseconds. | **`WATCHDOG_USEC`** with a value above zero sets the interval and MUST re-arm the timer from the moment the message arrives, rather than letting the new interval apply only from the next keepalive. A value of zero MUST disable the watchdog. The value MUST NOT persist across a restart. A restarted service gets the interval its definition specifies. **`EXTEND_TIMEOUT_USEC`** sets the current transition's deadline to expire that many microseconds from the message's arrival. It **replaces** the deadline rather than adding to it, and MAY be sent repeatedly. Because it replaces, a value smaller than the time remaining shortens the deadline, and zero expires it immediately. A service MUST NOT send a value expecting it to be treated as a floor. The manager MUST cap the extended deadline at four times the base timeout of the phase being extended, and MUST **clamp** rather than reject a value beyond the cap. During a system shutdown the manager MUST additionally cap it at the time remaining in the shutdown, and where both apply the stricter MUST win. A message arriving while the service is not in a transition MUST be ignored. There is no deadline to extend. ## 4.19.3 Reporting | Field | Value | Meaning | |---|---|---| | `STATUS` | free text | A human-readable statement of what the service is doing. | | `ERRNO` | free text | An errno-style error number. | | `EXIT_STATUS` | free text | An exit status, informationally. | All three MUST be authenticated like any other field and MUST be recorded by the manager as structured events. They MUST NOT be forwarded to a log sink as though they were the service's output — they are the service speaking to the manager. `STATUS` MUST additionally be retained and exposed as `status_text` in the status shape (§4.14). `ERRNO` and `EXIT_STATUS` MUST NOT be retained. A service MUST NOT include a newline or carriage return in a `STATUS` value: it would frame as two lines, the second of which is almost certainly malformed. ## 4.19.4 The descriptor store | Field | Value | Meaning | |---|---|---| | `FDSTORE` | `1` | Store the descriptors attached to this datagram. | | `FDNAME` | free text | The name to store or remove them under. | | `FDSTOREREMOVE` | `1` | Remove the descriptors stored under `FDNAME`. | | `FDPOLL` | `0` | Do not monitor the stored descriptors for error conditions. | §4.20. ## 4.19.5 Fields that are not supported | Field | Why | |---|---| | `MAINPID` | A manager supervises the process it forked, through a kernel handle obtained at fork. There is no mechanism for redirecting supervision to another process, and there is deliberately none: a service that could nominate its own supervision target could nominate anything. | | `BUSERROR` | Peios has no D-Bus. | Neither is rejected distinctly. Both are simply unrecognised keys and are ignored like any other (§4.17). A service MUST NOT rely on being told that it sent one. --- # 4.20 The Descriptor Store _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Handing file descriptors to the manager and getting them back after an unchosen restart — storing, removing, returning, and when the store is emptied. A service may hand file descriptors to the manager and get them back after a restart it did not choose. This is what lets a stateful daemon — one holding a listening socket, say — restart without dropping what it already had. The manager MUST support a per-service maximum, which MAY be zero. Zero disables the store for that service, and a service MUST NOT assume a store exists. ## 4.20.1 Storing On an authenticated datagram carrying `FDSTORE=1` with descriptors attached, the manager MUST: 1. If the store is disabled for this service, **close** the descriptors and record the rejection. 2. If the store already holds its maximum, **close** the descriptors and record the rejection. It MUST NOT evict an existing entry — a full store is full, and silently discarding something the service is relying on to survive a restart would be worse than refusing the new one. 3. Store them under the value of `FDNAME` if present and non-empty, and under the name `stored` otherwise. 4. Note `FDPOLL=0` if present. A datagram MAY carry several descriptors. Each becomes its own entry under the one name, and each is independently subject to the maximum — so a datagram carrying more than will fit has some stored and the rest closed. Several entries MAY share a name. `FDPOLL=0` asks the manager not to monitor the descriptors for error conditions. A manager MAY monitor stored descriptors and remove ones that have become invalid; a manager that does not MUST still accept the field. ## 4.20.2 Removing `FDSTOREREMOVE=1` with `FDNAME` MUST remove every entry of that name and close its descriptors. A name matching nothing is a no-op and MUST NOT be an error. `FDSTOREREMOVE=1` **without** `FDNAME` MUST be treated as a malformed line, rejecting the whole datagram (§4.17). A remove with no name has no defined meaning, and the alternative readings — remove everything, remove the default name, do nothing — are far enough apart that guessing between them silently is worse than refusing. ## 4.20.3 Returning them When the service starts again, the manager MUST pass the stored descriptors to the new process: 1. Placed consecutively, starting at descriptor **3**, with close-on-exec cleared. 2. `LISTEN_FDS` set to the number of descriptors passed. 3. `LISTEN_FDNAMES` set to the names, **colon-separated**, in the same order as the descriptor numbers. 4. `LISTEN_PID` set to the new process's own PID. 5. The store cleared. `LISTEN_PID` is what lets a service verify that the variables are addressed to it rather than inherited from an ancestor. A conforming client checks it against its own PID before trusting `LISTEN_FDS`, and treats a mismatch as meaning no descriptors were passed — so a manager that omits it hands descriptors to a service that will not take them. All four variables MUST be absent when no descriptors are passed, and the manager MUST NOT allow any of them to be set by a configurable environment layer. A `LISTEN_FDS` reaching a service that was passed nothing points its descriptor-adopting code at whatever happens to be at descriptor 3. Descriptors are returned to the service's main process only. A hook or a probe MUST NOT receive them. The store MUST be cleared once the descriptors have been passed. The manager MUST NOT clear it when a start attempt fails before that point — the descriptors are still the service's, and the next attempt should get them. ## 4.20.4 When the store is emptied The manager MUST clear the store, closing its descriptors, when: - the service is stopped deliberately — by a client, or as part of a system shutdown; or - the service's definition is withdrawn and its entry is finally discarded. The manager MUST NOT clear it on a restart the service did not ask for — a crash, or a restart policy acting on one. That case is the entire purpose of the mechanism: the descriptors survive exactly the restart the service could not prepare for. > [!NOTE] > `LISTEN_FDS`, `LISTEN_FDNAMES` and `LISTEN_PID` are the convention > established by systemd's `sd_listen_fds`, and are specified here in > the same form deliberately. Software already written to adopt > descriptors that way works against a Peios service manager without > modification. --- # 4.21 Extension _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Neither channel carries a version number — what may be added freely, what may not, and what a client must do with something it does not recognise. Neither channel carries a version number. Both are extended by the rules below, which are what allow a client and a manager built against different revisions to interoperate. ## 4.21.1 What may be added **A request field.** A manager MUST ignore a request field it does not recognise (§4.8). A client MAY therefore send a field a manager may not know, and MUST NOT depend on the field having had an effect. **A response field.** A client MUST ignore a response field it does not recognise, and MUST NOT treat its presence as an error. This includes an unrecognised member of `summary` in a `reload-config` response, and an unrecognised key in `current_job` or `current_operation`. **A notification field.** A manager MUST ignore an unrecognised key (§4.17). A service MAY therefore send a field a manager may not know. **A `type` value in a status warning.** A client MUST accept a warning whose `type` it does not recognise and MUST NOT discard it. An unclassifiable warning is still a warning. ## 4.21.2 What may not be added without a version Anything a client must *recognise* in order to behave correctly cannot be added compatibly, because an older client's only options are to fail or to misbehave. A manager MUST NOT, without a negotiated version: - introduce an **error code** outside §4.10; - introduce a **service state**, **transition cause**, **operation state**, **operation type**, **operation source** or **job type** outside §4.B; - introduce a **reload mode** outside the three in §4.13; - introduce a **command**, or change what an existing command does; - change the **shape** of an existing response, including changing a field's type or making a non-nullable field nullable. A client encountering one of these has no correct behaviour available. Faced with an unknown `state` it cannot decide whether the service is running; faced with an unknown error code it cannot decide whether to retry. ## 4.21.3 What a client must do with the unknown A client MUST treat an unrecognised **enumerated value** in a field it depends on as an error for that request, and MUST NOT map it onto the nearest value it does know. Guessing that an unfamiliar state is probably like `active` is how a monitoring tool reports a broken system as healthy. A client MUST treat an unrecognised **error code** as unrecoverable for that request. It MUST NOT retry, since it cannot know whether the condition is transient. ## 4.21.4 Versioning, when it comes A future revision introducing an incompatible change MUST do so through an explicit negotiation, in which a client states what it understands and the manager answers within that. Until such a mechanism exists, this chapter's contract is fixed and the rules above are the whole of the supported way for it to grow. --- # 4.22 Conformance _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Every requirement of this chapter collected by role — manager, client and service — and what conformance deliberately is not. ## 4.22.1 A conforming manager **The channels.** Listens on a Unix stream socket at a well-known path and on a Unix datagram socket whose path it gives each service in `NOTIFY_SOCKET`. Ensures both sockets, and the directories containing them, carry a Security Descriptor admitting the parties intended to reach them, and relies on no POSIX mode bits (§4.3, §4.4, §4.16). **Framing.** Emits exactly one compact JSON object per newline-terminated frame. Answers a malformed frame with `MALFORMED_REQUEST` and an oversized one with `REQUEST_TOO_LARGE`, closing the connection after a frame-level failure and holding it open after a command-level one (§4.5). **Identity.** Obtains every client's identity from the kernel once, at accept, and uses no UID, GID or asserted identity (§4.6). **Authorisation.** Checks every command against the appropriate Security Descriptor with the mappings in §4.7, records every denial, filters `list` rather than denying it, and does not let `operation-status` distinguish an operation the caller may not see from one that does not exist. **Commands.** Implements all ten, with the outcomes in §4.12 for every command-and-state pair, the response shapes in §4.9, §4.14 and §4.15, and only the error codes in §4.10. **Operations.** Returns an identifier from every lifecycle command that produced one and none where it did not; merges same-type requests and returns the surviving identifier; measures every operation's lifetime from its creation including queue time; and holds a terminal record for at least the grace period (§4.11, §4.14). **Waiting.** Honours the per-command `wait` default, holds a waiting connection open past the idle timeout, and carries a `mode` on every reload response (§4.13). **Notification.** Authenticates every datagram through all five steps of §4.18, including verifying the attested PID against a kernel handle and checking the activation generation. Applies all lines of an accepted datagram and none of a rejected one. Rejects a truncated datagram rather than processing it. Implements every field in §4.19. **The descriptor store.** Closes rather than keeps what it refuses; returns descriptors from 3 upward with `LISTEN_FDS`, `LISTEN_FDNAMES` and `LISTEN_PID` set; clears the store on a deliberate stop and keeps it across a restart the service did not ask for (§4.20). **Extension.** Ignores unrecognised request and notification fields, and introduces nothing from §4.21's closed list without a negotiated version. ## 4.22.2 A conforming client Sends one compact JSON object per newline-terminated frame. Treats an immediate close with no response as a refusal. Does not parse `message`. Accepts `null` for every nullable field, and unrecognised fields everywhere it is told to. Treats an unrecognised enumerated value or error code as an error for that request rather than guessing. Reads `state` rather than the presence of `error` to decide whether an operation succeeded. Does not infer its own rights from an `INVALID_STATE` received during shutdown. Opens a new connection to act under a different identity. ## 4.22.3 A conforming service Reads `NOTIFY_SOCKET` from its environment and hardcodes no path. Sends `READY=1` when it can genuinely serve, not when its process exists. Sends no recognised key with an undefined value, and no newline inside a `STATUS` value. Sends no datagram exceeding the bounds in §4.A. Expects no reply, and no acknowledgement that a field was applied. Treats `EXTEND_TIMEOUT_USEC` as replacing a deadline rather than adding to one. Checks `LISTEN_PID` against its own PID before adopting any descriptor. ## 4.22.4 What conformance is not A system that offers neither channel is still Peios (PSPU §1.2). These are contracts for the components that do offer them, not a bar the platform requires anything to clear. --- # Appendix 4.A Limits and Defaults _Peios / Advanced Peios / PSPU / Service Control and Notification_ > The limits and defaults a Peios service manager uses on both channels, and which of them are configurable. The values a Peios service manager uses. A manager MAY use different ones; where a value is configurable, it MUST be discoverable to an administrator through the same surface that sets it. ## 4.A.1 Control channel | Bound | Value | Configurable | Defined in | |---|---|---|---| | Socket path | `/run/services/peinit/control.sock` | No | §4.4 | | Concurrent connections | 32 | Yes | §4.4 | | Request size | 65536 bytes, excluding the terminating newline | Yes | §4.4 | | Idle timeout | 30 seconds | Yes | §4.4 | | Listen backlog | 32 | No | §4.4 | ## 4.A.2 Operations | Bound | Value | Defined in | |---|---|---| | Terminal record retention | 60 seconds | §4.14 | | Operation lifetime | The target service's own start or stop timeout | §4.11 | ## 4.A.3 Notification channel | Bound | Value | Defined in | |---|---|---| | Maximum datagram | 65536 bytes | §4.16 | | Descriptors per datagram | 64 | §4.16 | | First returned descriptor | 3 | §4.20 | | Descriptor store maximum | Per service; 0 disables | §4.20 | | Timeout extension cap | 4 × the phase's base timeout | §4.19 | ## 4.A.4 Composing the two channels A `STATUS` value a service sends on the notification channel is returned as `status_text` on the control channel. The notification datagram bound is 65536 bytes and the control response is not bounded by the request limit, so a status string that fits in a datagram is always returnable. The bounds are stated at their values here rather than left to each implementation because a producer has no other way to learn them. Lowering either without telling anyone breaks every service that was sizing to the old one, and the failure — a truncated datagram, or a connection closed mid-request — does not name its cause. --- # Appendix 4.B Wire Vocabulary _Peios / Advanced Peios / PSPU / Service Control and Notification_ > Every enumerated value on the control channel — response status, service state, transition cause, health, job and operation types. Every enumerated value that appears on the control channel. All are lower snake case. A manager MUST NOT emit a value outside these sets without the version negotiation in §4.21, and a client MUST treat one it does not recognise as an error for that request rather than mapping it onto a value it knows. ## 4.B.1 Response status `ok`, `error` ## 4.B.2 Service state | Value | Process? | Satisfies dependents? | |---|---|---| | `inactive` | No | No | | `starting` | Maybe | No | | `active` | Yes | Yes | | `reloading` | Yes | Yes | | `stopping` | Briefly | No | | `completed` | No | Yes | | `backoff` | No | No | | `failed` | No | No | | `abandoned` | Yes, unkillably | No | | `skipped` | No | Yes | Exactly three states satisfy dependents: `active`, `completed` and `skipped`. A client deciding whether something depending on this service could be running MUST use that set and no other. ## 4.B.3 Transition cause `explicit_start`, `dependency_start`, `restart_policy`, `binds_to_recovery`, `timer`, `explicit_stop`, `explicit_reload`, `explicit_reset`, `conflict_eviction`, `binds_to_propagation`, `shutdown_wave`, `process_crash`, `clean_exit`, `clean_exit_restart`, `readiness_timeout`, `watchdog_timeout`, `health_check_failure`, `pre_hook_failure`, `parent_setup_failure`, `pre_exec_failure`, `dependency_failure`, `restart_budget_exhausted`, `cycle_detected`, `validation_error`, `assertion_error`, `condition_skipped`, `process_unkillable` A `cause` may also be `null`, for a service that has not transitioned. ## 4.B.4 Service health `healthy`, `unhealthy`, `unknown` `health` is `null` when the service has no health check configured, which is distinct from `unknown` — the latter means one is configured and has not produced a result yet. ## 4.B.5 Job type `service_main`, `pre_exec_hook`, `post_exec_hook`, `reload_hook`, `health_check`, `ad_hoc` Only `service_main` appears in `current_job`. ## 4.B.6 Operation type `start`, `stop`, `restart`, `reload`, `reset` ## 4.B.7 Operation state | Value | Terminal? | Meaning | |---|---|---| | `pending` | No | Queued, not yet executing. | | `running` | No | Executing. | | `completed` | Yes | Reached its goal. | | `failed` | Yes | Did not reach its goal, or expired while queued. | | `merged` | Yes | Merged into another operation. | | `cancelled` | Yes | Terminated while pending. Never executed. | | `aborted` | Yes | Terminated while running. | ## 4.B.8 Operation source `admin`, `boot`, `shutdown`, `dependency_propagation`, `restart_policy`, `timer`, `binds_to_recovery`, `binds_to_propagation`, `conflict_resolution`, `on_failure` `admin` is the only source a client's own command produces. The rest describe operations the manager created for its own reasons, and a client observing one has learned something about what the manager is doing rather than about anything it asked for. ## 4.B.9 Reload mode `confirmed`, `advisory`, `failed` ## 4.B.10 Status warning type `service_tree`, `health`, `hooks` These name what part of a service's process containment could not be reclaimed, `service_tree` being the whole of it and therefore the most serious. A client MUST accept a value outside this set and MUST NOT discard the warning (§4.21). ## 4.B.11 Shutdown type `poweroff`, `reboot`, `halt` Request-only; the manager does not echo it. --- # 5.1 Scope and Roles _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What this chapter specifies — the .peipkg artifact and the repository protocol that serves it — its roles, and what it leaves out. This chapter specifies the **peipkg package format** and the **peipkg repository protocol**: the artifact by which compiled software is distributed to a Peios system, and the static-HTTP protocol by which a system discovers, trusts, and fetches those artifacts. A package is the binary distribution primitive of Peios — the unit of build, distribution, and trust. It is deliberately narrow: it defines how binaries reach a system, not how they are integrated into services, roles, or features. Higher-level artifacts reference packages; a package knows nothing of them. ## 5.1.1 Roles Three roles speak this specification. A requirement is stated against the role, not the program; one program may serve more than one. | Role | Obligation | |---|---| | **Producer** | Builds package files. Everything a `.peipkg` contains is a producer obligation. | | **Repository** | Publishes a descriptor, two indexes, and package files over static HTTP, and signs the metadata. | | **Consumer** | Fetches, verifies, and installs packages. Every validation and rejection rule binds the consumer. | A repository operator is usually also a producer, but need not be: a repository may publish packages built elsewhere, and the format's signatures survive the journey. ## 5.1.2 In scope - The on-wire package file: container, internal layout, manifest schema, payload layout, per-file integrity - Package identity: names, versions, version comparison, architectures - How a package expresses its relationships to other packages, and what it means for one to satisfy another - Package signing: algorithm, envelope, verification - The repository protocol: descriptor, active and archive indexes, URL conventions, freshness and rollback protection - Establishing and maintaining trust in a repository - The rules under which the format may be extended ## 5.1.3 Out of scope - **How a consumer decides what to install.** Given several candidates that all satisfy a dependency, which one it picks, in what order it applies a plan, and how it recovers from an interrupted one are the consumer's own design. - **How a consumer stores its state.** The installed-package database, its transaction journal, and its cache format are private. - **How a producer builds a package.** Recipes, build farms, and source trees are producer mechanics; only their output is specified here. - **Roles, role features, core features, and applets.** These are separate subsystems that reference packages. - **Integration metadata attached to packages** — service definitions, registry seeds, reconciller manifests. These belong to the artifacts that compose packages, not to packages. - **Security descriptor semantics.** A package carries security descriptor bytes; what they mean is specified with the kernel's access-control subsystem. ## 5.1.4 Relationship to other chapters Nothing in this chapter is a conformance requirement on a Peios system: a system that ships software some other way is still Peios (§1.1). What this chapter guarantees is that the format and the protocol are written down and will not move. --- # 5.2 Terminology _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The terms this chapter defines — package, manifest, root, repository — used with these meanings throughout. - **Package** — a binary distribution artifact: one or more files, metadata describing its identity and relationships, and, when signed, a signature. The unit of build, distribution, and installation. - **Manifest** — the JSON document at `.peipkg/manifest.json` inside a package that declares its identity, relationships, side-effect requirements, and build provenance. The manifest is authoritative for a package's metadata (§5.18). - **Files manifest** — the JSON document at `.peipkg/files.json` carrying one content hash per regular payload file (§5.25). - **Payload** — the tar entries of a package that are not metadata: the files, directories, and symlinks it installs. - **Repository** — a collection of packages addressable as a unit, identified by its base URL. - **Repository descriptor** — the small JSON document at a well-known path within a repository declaring its identity, signing keys, and the locations of its indexes (§5.31). - **Index** — a signed JSON document listing packages available from a repository. Every repository publishes two: an **active index** (§5.33) listing the current version of each package, and an **archive index** (§5.35) listing every version ever shipped. - **Virtual name** — a capability name, rather than a package name, that a package may require or provide (§5.4). - **Role** — a virtual name that several installed packages may contend to own on the filesystem, with at most one *holding* it (§5.23). - **Claim** — the binding of a contended filesystem name (a *claim path*) to a file supplied by the package that holds a role (a *target*). - **Holder** — the single installed package that currently owns a role. A role with no holder is *unheld*. - **Side-effect declaration** — a manifest flag naming a standard maintenance operation to be invoked after install, drawn from a closed set (§5.24). - **Installation root** — a self-contained filesystem tree into which packages are installed. The default root is the system root; a system may define others (§5.19). - **Epoch**, **upstream version**, **peios revision** — the three components of a version string (§5.5). - **Trust anchor** — a key fingerprint supplied to a consumer out-of-band, against which a repository's descriptor signature is first verified (§5.37). --- # 5.3 Package Names _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What a package name may contain, how it is structured and cased, the filename convention, and the sub-package conventions. A package's name identifies it within a repository and across every repository that may serve it. ## 5.3.1 Character set A package name MUST consist of ASCII characters drawn from: - lowercase letters `a`–`z` - digits `0`–`9` - hyphen `-` - period `.` - plus sign `+` A package name MUST NOT contain uppercase letters, whitespace, underscores, or any character outside that set. ## 5.3.2 Structure A package name MUST start with a lowercase letter or a digit, and MUST end with a lowercase letter, a digit, or a plus sign. The hyphen and the period are **separator** characters. The plus sign is not a separator but an ordinary name character: it is intrinsic to names such as `libstdc++` and `g++`, so it MAY repeat and MAY end a name. A package name MUST NOT contain two consecutive separators — `--`, `..`, `-.`, or `.-`. A package name MUST be at least 2 and at most 64 characters long. > [!NOTE] > The character set admits the common upstream patterns: library > suffixes (`libstdc++`), architecture prefixes (`lib32-foo`), and > dotted module names (`python3.example`). Underscores are excluded so > that the filename separator below stays unambiguous. ## 5.3.3 Case Package names are case-sensitive. Because uppercase letters are forbidden, this is equivalent to byte-for-byte equality. ## 5.3.4 Filename convention A package file's name, on disk and in URLs, MUST be: ``` __.peipkg ``` The separator between fields is the underscore, and the extension is `.peipkg`. A filename is parsed by splitting at the **first** underscore and then at the **second**: what precedes the first is the name, what lies between them is the version, and what follows the second — up to the `.peipkg` extension — is the architecture. The underscore MUST NOT appear in the name (§5.3) or in the version (§5.5). It MAY appear in the architecture (§5.8), and does in `x86_64`, which is why the architecture field is defined as the remainder rather than as the text after the last underscore. Examples: ``` nginx_1.26.2-3_x86_64.peipkg jq_1.7.1-2_x86_64.peipkg peios-docs_0.22-1_noarch.peipkg libstdc++_13.2.1-4_x86_64.peipkg ``` A consumer MUST NOT derive a package's identity from its filename. The manifest is authoritative (§5.18); the filename is a convenience for humans and for static hosting. ## 5.3.5 Sub-package conventions Packages shipping related but separable content SHOULD use a hyphen-suffix convention: | Suffix | Content | |---|---| | `-doc` | Documentation, man pages, examples | | `-debug` | Debug symbols | | `-dev` | Headers, static libraries, build-time dependencies | | `-source` | Corresponding source (§5.14) | These are advisory. The format does not enforce them, and other suffixes MAY be used for other purposes. --- # 5.4 Virtual Names _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > A capability expressed as a name rather than a package — the grammar, and why virtual and real names share one namespace. The `name` of a `dependencies`, `optional_dependencies`, or `provides` entry (§5.21) MAY be a **virtual name** rather than a real package name. A virtual name expresses a capability that is required or provided but is not itself a package — most importantly a machine-derived capability such as an ELF soname or a pkg-config module (§5.22). `conflicts` and `replaces` entries target real packages, and so MUST use the package-name grammar of §5.3, not the grammar below. ## 5.4.1 Grammar The virtual-name grammar is a strict superset of the package-name grammar, in two respects. **Uppercase letters are permitted.** A virtual name often mirrors an exact machine identifier — `libGL.so.1`, `libICE.so.6`, a foreign module name — which is case-sensitive. Case MUST be preserved: folding it would be unsound, because a case-sensitive dynamic loader treats `libGL.so.1` and `libgl.so.1` as distinct. **A namespaced form `namespace(argument)` is permitted**, for capabilities drawn from a foreign namespace. The `namespace` is lowercase letters and digits, beginning with a letter. The `argument` is bracketed by parentheses, is non-empty, and may contain letters, digits, the separators `-`, `.`, `+`, and additionally `_`, `:`, and `/` — so that `pkgconfig(gtk+-3.0)`, `perl(Foo::Bar)`, and `python3dist(ruamel.yaml)` are all well-formed. Outside the namespaced form, a virtual name uses the package-name character set extended with the underscore `_`, which is common in real sonames (`libgcc_s.so.1`, `libnss_files.so.2`). It MUST start with a letter or a digit and MUST end with a letter, a digit, or `+`. Unlike a package name, a virtual name MAY contain consecutive separators, so that `libstdc++.so.6` is well-formed. A virtual name MUST be at least 2 and at most 128 characters long. ## 5.4.2 One namespace Virtual names share a namespace with real package names. A dependency on `libssl` is satisfied by a package literally named `libssl`, or by any package whose `provides` includes `libssl`. The namespaced form exists to keep machine-derived capabilities from colliding with package names: `pkgconfig(zlib)` is unambiguously the pkg-config module, never a package. --- # 5.5 Versions _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The structure of a package version string — epoch, upstream version and Peios revision — and how it is parsed. Every package carries a version string that identifies one build of that package. Version strings have a defined structure and a defined comparison order (§5.6), so that "newer" and "older" are unambiguous across every implementation. ## 5.5.1 Structure ``` [:]- ``` - **Epoch** — an OPTIONAL non-negative integer, separated from the rest by a colon. Absent means zero. - **Upstream** — the version the upstream project assigned, or, for Peios-native software, the version Peios assigned as vendor. - **Peios revision** — a REQUIRED positive integer identifying the build of this upstream version produced by the distributor. ``` 1.26.2-3 upstream 1.26.2, revision 3 1.26.2-rc.1-1 upstream 1.26.2-rc.1, revision 1 2:0.5.0-1 epoch 2, upstream 0.5.0, revision 1 0.22-1 upstream 0.22, revision 1 (Peios-native) ``` ## 5.5.2 Epoch The epoch MUST be encoded as ASCII decimal digits with no leading zeros, except that zero is encoded as the single digit `0`. The separator is a single colon. Epoch exists solely to override the natural ordering of upstream version strings when an upstream regression makes a later release compare as older than an earlier one. Bumping it SHOULD be a deliberate, documented decision; a routine version update MUST NOT bump it. > [!NOTE] > An upstream project releases v2.0, abandons that line, and releases > v0.5 as its new stable branch. Without an epoch, v0.5 compares as > older than v2.0 and nobody on v2.0 can upgrade. Bumping the epoch to 1 > says "v0.5 in this epoch is newer than anything in epoch 0". ## 5.5.3 Upstream version The upstream version is everything between the optional epoch separator and the final hyphen preceding the revision. It MUST consist of ASCII characters drawn from: letters `a`–`z` and `A`–`Z`, digits `0`–`9`, period `.`, plus sign `+`, hyphen `-`, and tilde `~`. It MUST start with a digit or a letter, and MUST NOT contain whitespace or any character outside that set. > [!NOTE] > The set is permissive because upstream projects format versions in > every way imaginable: numeric (`1.26.2`), hyphenated pre-release > (`1.0.0-rc.1`), concatenated pre-release (`16beta1`), build metadata > (`1.0+build.42`), and tilde-separated pre-release (`1.0~rc.1`). ## 5.5.4 Peios revision The peios revision MUST be a positive integer encoded as ASCII decimal digits with no leading zeros. It is incremented when the distributor produces a new build of the same upstream version — a backported security patch, a build-configuration change, a dependency bump, a packaging fix. The first revision of any upstream version MUST be `1`. Revision `0` is reserved and MUST NOT appear in a published package. ## 5.5.5 Parsing A version string is parsed as follows: 1. If the string contains a colon, split at the **first** colon: what precedes it is the epoch, what follows is the remainder. Otherwise the epoch is 0 and the remainder is the whole string. 2. Split the remainder at the **last** hyphen: what follows is the peios revision, what precedes is the upstream version. 3. The peios revision MUST parse as a positive integer. 4. The upstream version MUST satisfy the constraints above. A version string that does not parse is invalid, and an implementation MUST reject it. ## 5.5.6 Stability The comparison algorithm of §5.6 is frozen. Any two conforming implementations MUST produce identical comparison results for every pair of valid version strings. An implementation that disagrees with another on any such pair is non-conformant, whichever of the two is at fault. --- # 5.6 Comparing Versions _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The three-stage comparison that decides which of two versions is newer, including pre-release segments and their rank. Two version strings are compared in three stages: 1. Compare epochs as integers. If they differ, the higher epoch is greater. 2. If equal, compare upstream versions by the algorithm below. 3. If equal, compare peios revisions as integers. The higher revision is greater. 4. If all three are equal, the versions are equal. ## 5.6.1 Tokenising the upstream version A tokeniser walks the upstream string left to right and emits segments: 1. The non-alphanumeric characters `.`, `+`, `-`, and `~` are separators and belong to no segment. 2. A maximal run of digits forms a **numeric** segment. 3. A maximal run of letters forms an **alphabetic** segment. 4. A transition between a digit and a letter ends the current segment and begins a new one. ## 5.6.2 Pre-release segments A segment is a **pre-release segment** if it falls at or after the earlier of: - the first `~` separator — the tilde and every segment following it; or - the first **recognised pre-release token**: a segment whose token carries a rank of 0 to 4 in the table below, that segment and every segment following it. Once the pre-release tail begins it extends to the end of the upstream version: every later segment is a pre-release segment, whatever the separators between them. A `-` separator is an ordinary separator; it is not itself a pre-release marker. > [!NOTE] > In `1.0.0-rc.1`, `rc` is recognised at rank 4, so `rc` and the > following `1` are pre-release. In `16beta1`, `beta` is recognised at > rank 2, so `beta` and `1` are pre-release with no separator involved. > In `1.0-foo`, `foo` is alphabetic but not recognised: it is an > ordinary rank-5 segment, and the `-` before it begins nothing. | Upstream | Segments | |---|---| | `1.26.2` | `1`, `26`, `2` | | `1.0.0-rc.1` | `1`, `0`, `0`, `rc` (pre), `1` (pre) | | `1.0~rc1` | `1`, `0`, `rc` (pre), `1` (pre) | | `16beta1` | `16`, `beta` (pre), `1` (pre) | ## 5.6.3 Pre-release rank | Token | Rank | |---|---| | `dev` | 0 | | `alpha` | 1 | | `a` | 1 | | `beta` | 2 | | `b` | 2 | | `pre` | 3 | | `rc` | 4 | | any other alphabetic token | 5 | Rank 0 sorts lowest. Rank lookup MUST be case-insensitive: `Alpha`, `ALPHA`, and `alpha` all carry rank 1. ## 5.6.4 Comparing two segments The pre-release flag is compared first, before the kinds. If exactly one of the two segments is a pre-release segment, **that segment is the lesser**, whatever either segment contains. A pre-release segment sits at or after the point where the version was marked as preceding a release, and that is a property of position rather than of content. When both segments carry the same flag — both pre-release, or neither — their kinds decide: 1. **Both numeric** — compare as integers. Leading zeros are insignificant. 2. **Both alphabetic** — compare by pre-release rank. When the ranks are equal: - at a rank of 0 to 4, the segments are **equivalent**. The table assigns several tokens to one rank as aliases, so two segments at the same recognised rank sort equal whichever alias appears. - at rank 5, the segments tiebreak by ASCII byte order against other rank-5 tokens. 3. **One numeric, one alphabetic** — the alphabetic segment is the lesser if the pair is a pre-release pair, and the greater if it is not. (Where only one of them is a pre-release segment, the rule above has already decided.) > [!NOTE] > ``` > 1.0~alpha-1 == 1.0~a-1 # both rank 1 > 1.0~beta-1 == 1.0~b-1 # both rank 2 > 1.0~Alpha-1 == 1.0~ALPHA-1 # case-insensitive recognition > 1.0-foo-1 < 1.0-zzz-1 # rank-5 lexical tiebreak > ``` > Two upstream versions that differ only in which alias they use are > equal. A repository publishing both forms produces archive entries at > the same logical version; the active index may carry either. A > producer SHOULD pick one canonical form — conventionally the long > token, `alpha` and `beta` — and keep to it. ## 5.6.5 Unequal lengths When the segments of one version run out and every common segment compared equal, the next segment of the longer sequence decides. Its pre-release flag decides it, and its kind is irrelevant: | Next segment in the longer | Result | |---|---| | a pre-release segment | the shorter is **greater** | | anything else | the shorter is less | | Example tail | | Result | |---|---|---| | `~1` | numeric, pre-release | the shorter is **greater** | | `~rc` | alphabetic, pre-release | the shorter is **greater** | | `.1` | numeric | the shorter is less | | `-foo` | alphabetic, rank 5 | the shorter is less | ## 5.6.6 Worked examples | A | B | Result | Why | |---|---|---|---| | `1.0~2` | `1.0-2` | A < B | the pre-release flag decides before the kinds | | `1.0~foo` | `1.0-foo` | A < B | the same, for two rank-5 tokens | | `1.0` | `1.0` | A = B | identical | | `1.0` | `2.0` | A < B | numeric segment differs | | `1.10` | `1.9` | A > B | numeric, not lexical | | `1.0` | `1.0.1` | A < B | longer continues numerically | | `1.0` | `1.0-rc.1` | A > B | longer continues with a pre-release | | `1.0-rc.1` | `1.0-rc.2` | A < B | numeric segment within the tail | | `1.0-alpha` | `1.0-beta` | A < B | rank 1 < rank 2 | | `1.0-rc` | `1.0-pre` | A > B | rank 4 > rank 3 | | `1.0a1` | `1.0a2` | A < B | numeric within a concatenated tail | | `1.0a1` | `1.0b1` | A < B | rank 1 < rank 2 | | `1.0~rc1` | `1.0` | A < B | the tilde forces a pre-release | | `1.0~1` | `1.0` | A < B | the tilde forces a pre-release, numeric or not | | `5.2~20240101` | `5.2` | A < B | a dated snapshot precedes its release | | `0:1.0` | `1:0.5` | A < B | epoch dominates | | `1.0-1` | `1.0-2` | A < B | peios revision differs | | `1.0-foo-1` | `1.0-1` | A > B | `foo` is rank 5, sorting after a number | --- # 5.7 Version Constraints _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > How a relationship restricts which versions satisfy it — the operators, how they combine, and revision-relaxed operands. A **version constraint** restricts which versions of a package satisfy a relationship (§5.21). ## 5.7.1 Operators | Operator | Meaning | |---|---| | `=` | exactly equal | | `>` | strictly greater than | | `>=` | greater than or equal | | `<` | strictly less than | | `<=` | less than or equal | | `!=` | not equal | Comparison is by §5.6 in every case. A bare version string with no operator is equivalent to `=`. ## 5.7.2 Combining Multiple expressions within one constraint string are separated by commas and combined with logical AND. A version satisfies the constraint if and only if it satisfies every expression. ``` libssl >= 3.0 libssl >= 3.0, < 4.0 nginx = 1.26.2-3 ``` Whitespace around operators and commas is optional and MUST be ignored. A constraint string MUST parse as one or more operator-and-version expressions separated by commas. One that does not parse is invalid, and an implementation MUST reject it. ## 5.7.3 Revision-relaxed operands A constraint's version operand MAY omit the `-` that a complete version string otherwise requires. An operand written without a revision — `>= 3.0` — constrains the epoch and the upstream version only. A candidate satisfies it whenever its epoch and upstream version satisfy the operator, whatever its revision. An operand written in full constrains the revision as well. > [!NOTE] > This is what lets a dependency track a capability level rather than a > packaging iteration. `libssl >= 3.0` is satisfied by `3.0-1` and by > `3.0-7` alike; nothing about a repackaging of the same upstream > release should change whether a dependency is met. --- # 5.8 Architectures _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The architecture identifier, the defined set, the triplets derived from them, and what noarch means for installability. A package's architecture identifies the instruction-set architecture its binaries were built for. It is a separate identifier from the name and the version. ## 5.8.1 Identifier format An architecture identifier MUST consist of lowercase letters `a`–`z`, digits `0`–`9`, and the underscore `_`. It MUST start with a lowercase letter and MUST NOT exceed 16 characters. ## 5.8.2 Defined architectures | Identifier | Meaning | |---|---| | `x86_64` | 64-bit x86 (AMD64, Intel 64) | | `aarch64` | 64-bit ARM (ARMv8-A or later) | | `noarch` | architecture-independent | An implementation MUST recognise all three. `x86_64` is the primary target; every other architecture is secondary in this version of the specification. Additional identifiers MAY be defined in a future version. A new identifier MUST satisfy the format above and SHOULD be the canonical Linux machine name — the value `uname -m` reports — where one exists. ## 5.8.3 Triplets Each architecture identifier that is not `noarch` has a corresponding **triplet**, used in the install paths where arch-specific content is namespaced (§5.15): ``` -linux-peios ``` | Identifier | Triplet | |---|---| | `x86_64` | `x86_64-linux-peios` | | `aarch64` | `aarch64-linux-peios` | `noarch` has no triplet form, and architecture-independent payload MUST NOT be installed under an arch-namespaced path. > [!NOTE] > The `peios` suffix distinguishes Peios binaries from foreign-arch > binaries originating elsewhere — Debian uses `gnu`, Alpine uses > `musl`. It leaves room for a future multi-architecture system to host > foreign-distribution binaries without filesystem-path collisions. ## 5.8.4 Architecture-independent packages The `noarch` identifier denotes a package whose payload contains no architecture-dependent content: documentation, configuration templates, scripts in interpreted languages, or metadata only. A package MUST NOT declare `noarch` if its payload contains compiled binaries, shared libraries, or any other content whose semantics depend on the target architecture. ## 5.8.5 Installability Each Peios system has a single **primary architecture**, fixed at install time. - A package whose architecture equals the system's primary architecture MAY be installed. - A package whose architecture is `noarch` MAY be installed on any system. - A package whose architecture is neither MUST NOT be installed. > [!NOTE] > This version of the specification does not define multi-architecture > systems — systems installing foreign-architecture packages alongside > native ones. Running foreign-architecture binaries for emulation, > cross-compilation, or legacy compatibility is addressed by mechanisms > outside the package format. The triplet convention above applies > regardless, so that a package conforming to this version stays > forward-compatible with such an extension. --- # 5.9 Document Conventions _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The rules every JSON artifact in this chapter obeys — parser hardening, hashes, signatures, strings, URLs and compression. Every artifact this chapter defines — the manifest, the files manifest, the signature envelope, the repository descriptor, and both indexes — is a JSON document. The rules below apply to all of them. ## 5.9.1 JSON Documents conform to RFC 8259 and are UTF-8 encoded (RFC 3629). Field names are lowercase with underscores between words: `schema_version`, never `schemaVersion`. Field order is not significant. Unknown fields MUST be ignored on parse, so that the format can be extended compatibly (§5.38). The signature envelope (§5.28) is the one exception, and mandates strict parsing. ## 5.9.2 Parser hardening A consumer's JSON parser processes attacker-supplied input. It MUST therefore enforce the following, on every document defined in this chapter: - **Duplicate keys in any object MUST cause the document to be rejected.** A parser that silently takes first-wins or last-wins is not conformant. - Integer fields MUST fit in the unsigned 64-bit range and MUST NOT use exponent notation. - Nesting depth MUST be capped at 64; a document exceeding that depth MUST be rejected. - A string value MUST NOT exceed the document size limit applicable to its containing artifact (§5.A). - A Unicode escape within a string MUST resolve to a valid code point per RFC 8259 §7. > [!NOTE] > Duplicate-key rejection is the load-bearing rule of the list. A parser > that takes last-wins and a parser that takes first-wins disagree about > what a document says, which means the bytes a producer signed and the > bytes a consumer acts on can differ while every signature still > verifies. Several widely used JSON libraries take last-wins silently; > conformance requires the rejection to be added deliberately. ## 5.9.3 Hashes Hash values are encoded in lowercase hexadecimal unless stated otherwise. Hash algorithms are identified by their IANA-registered names (`sha256`, `blake3`). ## 5.9.4 Signatures Signatures use Ed25519 as defined in RFC 8032 unless stated otherwise. Signature values are encoded in base64 (RFC 4648 §4) **without padding**. A base64 value carrying padding MUST be rejected. ## 5.9.5 Strings String comparison uses byte-for-byte equality unless stated otherwise. ## 5.9.6 URLs URLs follow RFC 3986. Relative URLs in a repository index are resolved against the repository descriptor's URL (§5.36). ## 5.9.7 Compression Compression uses the Zstandard format (RFC 8478). This specification does not constrain the compression level. --- # 5.10 The Container _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > A package is one Zstandard-compressed tar archive — the extension, tar format, compression settings, and what makes it streamable. A package is a single file: a tar archive compressed with Zstandard. ## 5.10.1 Extension A package file's extension MUST be `.peipkg`. There is no intermediate `.tar` form; a producer emits the compressed form directly, and the compressed file is the whole artifact. ## 5.10.2 Tar format The tar archive MUST conform to the POSIX pax interchange format (IEEE Std 1003.1-2017, Chapter 14). > [!NOTE] > pax supports arbitrary path lengths through extended headers. The > older ustar format limits paths to 100 characters, or 255 with a > prefix, which is not enough for some real packages. GNU tar, > libarchive, and BSD tar all read and write pax by default. ## 5.10.3 Compression The archive MUST be compressed with Zstandard (RFC 8478). The compression level is at the producer's discretion. zstd is deterministic at every level, so a producer MAY choose any level to trade build time against on-wire size. Levels 19 and above, including `--ultra`, increase build time substantially for a smaller result; level 3 is a common default. ## 5.10.4 Reproducibility A package MUST be reproducible: given identical source inputs, an identical build environment, identical metadata — including the build timestamp recorded in the manifest — and identical compression parameters, two independent producers MUST produce byte-identical package files. The determinism rules of §5.11 are what make this achievable at the format level. They are necessary rather than sufficient: they constrain what the archive looks like, not how the producer arrived at its contents. Byte-identity is a property of the **uncompressed** tar stream and of the compression applied to it. This specification fixes the former completely and the latter not at all: the compression level, the Zstandard implementation, its version, and its frame parameters all affect the resulting bytes and are none of them constrained here. Two producers seeking byte-identical output MUST therefore agree on their compression parameters out of band. What the format guarantees unconditionally is that the *signed* bytes — the uncompressed tar prefix of §5.28 — are identical, so a signature survives recompression at any level. ## 5.10.5 Streaming A consumer MAY process the archive as a stream. The internal layout (§5.12) places metadata before payload precisely so that a consumer can read a package's identity and reject a mismatched package without buffering the payload. ## 5.10.6 No outer wrapping The compressed archive contains tar entries and nothing else: no enclosing directory, no concatenated archives, no container metadata outside the tar entries themselves. --- # 5.11 Determinism _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The rules a tar archive must obey to be byte-reproducible, every one of which a consumer rejects a package for violating. To make a package byte-reproducible (§5.10), the tar archive MUST obey every rule below. A consumer MUST reject a package that violates any of them. 1. Tar entries MUST be ordered lexicographically by **the entry name as written into the tar header**, compared byte-for-byte over its UTF-8 bytes. A directory's entry name carries a trailing `/`, and that slash participates in the comparison. 2. Every entry's modification time MUST equal the value of `build.timestamp` in the manifest (§5.18), which MUST NOT carry sub-second precision. 3. Every entry's owner numeric ID and group numeric ID MUST be 0. 4. Every entry's owner name and group name MUST be the string `root`. 5. Entries MUST NOT carry extended attributes. Security descriptors are applied at file-creation time (§5.20), never through tar attributes; other install-time attributes are applied through side-effect declarations (§5.24) or by higher-level mechanisms outside this specification. 6. Entry permission bits MUST be `0777` for every entry, with the setuid and setgid bits cleared (§5.16). 7. PAX extended header records, when present, MUST appear in a fixed canonical order: `path` first, then `linkpath` if present, then any other record sorted by record name lexicographically. 8. The tar header magic MUST be `ustar\0` and the version MUST be `00`. 9. The `devmajor` and `devminor` header fields MUST be 0 for every entry type this specification permits — none of which is a device entry. 10. Header padding bytes MUST be NUL (`0x00`). 11. PAX global header records (typeflag `g`) MUST NOT appear. 12. PAX extended header records (typeflag `x`) MUST appear only when an entry's `path` exceeds the ustar 100-byte limit, in which case a `path` record is emitted, or its `linkname` exceeds that limit, in which case a `linkpath` record is emitted. A record with any other key MUST NOT be emitted. 13. A path exceeding the ustar 100-byte limit MUST be carried by a `path` record. The ustar `prefix` field MUST NOT be used to split such a path across `prefix` and `name`. 14. An extended header entry's own name MUST be the containing directory's path, then `PaxHeaders.0/`, then the base name of the entry it describes. For an entry at the archive root the directory part is absent. Rules 13 and 14 exist because a tar library given a long path may legitimately choose either encoding, and either choice produces a different byte stream from the same input. Determinism requires the choice be made here rather than by whichever library a producer reached for. > [!NOTE] > Rule 2's sub-second prohibition is not fussiness. A timestamp carrying > a fractional part forces an `mtime` extended header onto every entry, > violating rule 12 across the whole archive and changing the block > count of the signature entry's own header — which is exactly the > quantity a verifier uses to find the end of the signed range (§5.28). > The symptom is a signature that does not verify, reported against the > key rather than against the timestamp. > [!NOTE] > Rules 3, 4, and 6 together make a payload identity-free and > permission-free: bytes in transit, carrying no claim about who may > read them. What access control an installed file gets is decided at > install time (§5.20), not declared by the package. §5.16 states the > consequence for anyone extracting a package with a generic tar tool. --- # 5.12 Internal Layout _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Metadata entries under the reserved prefix and payload entries beneath — which are required, in what order, and which entry types are permitted. A package's tar entries divide into **metadata** entries under a reserved prefix and **payload** entries — the files that will be installed. ## 5.12.1 The reserved prefix Every metadata entry MUST appear under the path prefix `.peipkg/` at the archive root. That prefix is reserved: a payload entry MUST NOT use any path beginning with `.peipkg/`, and no payload entry may be named literally `.peipkg`. ## 5.12.2 Required entries | Entry path | Purpose | Section | |---|---|---| | `.peipkg/manifest.json` | Authoritative package metadata | §5.18 | | `.peipkg/files.json` | Per-file integrity manifest | §5.25 | | `.peipkg/signature` | Inline package signature | §5.28 | `.peipkg/manifest.json` and `.peipkg/files.json` MUST be present in every package. `.peipkg/signature` MUST be present in every signed package; a package without it is *unsigned* (§5.28). ## 5.12.3 Entry order Tar entries MUST appear in exactly this order: 1. `.peipkg/manifest.json` 2. `.peipkg/files.json` 3. Any optional metadata entries, sorted lexicographically by path 4. All payload entries, sorted lexicographically by path (§5.11 rule 1) 5. `.peipkg/signature` The manifest comes first so that a streaming consumer can read a package's identity and reject a mismatched one — wrong name, wrong version, wrong architecture — before reading any payload. The signature comes last because it signs everything preceding it (§5.28). A consumer MUST reject a package in which any named entry follows `.peipkg/signature`. ## 5.12.4 Optional metadata entries A package MAY carry additional entries under `.peipkg/`. The set this specification recognises is fixed at the three above; an unrecognised entry under `.peipkg/` MUST be ignored on parse and MUST NOT prevent installation. When present, an optional metadata entry MUST appear between `.peipkg/files.json` and the first payload entry. > [!NOTE] > A future version may introduce build attestations, reproducibility > manifests, or a software bill of materials as further metadata > entries. A producer targeting that future version MAY emit them into a > package conforming to this one; a consumer conforming to this version > ignores them. ## 5.12.5 Permitted entry types A payload entry MUST be one of: - a regular file (typeflag `0` or `\0`) - a directory (typeflag `5`) - a symbolic link (typeflag `2`) Any other entry type MUST cause the package to be rejected. This excludes hardlinks (typeflag `1`), character devices (`3`), block devices (`4`), FIFOs (`6`), contiguous files (`7`), and every vendor-specific type. > [!NOTE] > Hardlinks are excluded because they share an inode with their target, > which would let a package install a payload entry aliasing an existing > system file and so gain shared access to it. Kernel hardlink-creation > permissions provide some defence, but excluding the entry type at the > format level is simpler and removes the class entirely. Device, FIFO, > and contiguous entries have no use in a package payload. --- # 5.13 Payload Paths _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > A payload entry's tar path is its install location — the constraints on it, why nothing is canonicalised, and what an empty payload means. A payload entry's tar path is its install location, resolved against the installation root (§5.19). A tar entry at `usr/bin/nginx` installs to `/usr/bin/nginx`. ## 5.13.1 Constraints A payload path MUST: - be relative — it MUST NOT begin with `/` - contain no segment equal to `.` or `..`, or any encoding thereof - be valid UTF-8 (RFC 3629) - contain no NUL byte (`0x00`) and no ASCII control character (`0x01`–`0x1F`, `0x7F`) - contain no backslash (`\`) - be in Unicode Normalization Form C, per Unicode 16.0 - have every component at most 255 bytes when encoded as UTF-8 - be at most 4096 bytes in total when encoded as UTF-8 - have at most 256 components - not begin with `.peipkg/`, and not be literally `.peipkg` (§5.12) A consumer MUST validate every payload path against these constraints **before any further processing of the entry**. A package containing a non-conforming payload path MUST be rejected. ## 5.13.2 No canonicalisation Path resolution MUST NOT canonicalise away `..` or `.` segments by interpretation. Such segments are forbidden above; any appearance is a format error, not a question of path canonicalisation. > [!NOTE] > The distinction matters. A consumer that normalises `a/../b` to `b` > and proceeds has accepted a path this specification forbids, and has > done so by a rule the producer never agreed to. Rejecting is the only > behaviour both sides can predict. ## 5.13.3 Empty payloads A package MAY have zero payload entries. Such a package carries only metadata; installing it records the package and runs any declared side effects (§5.24). > [!NOTE] > An empty-payload package is useful as a virtual aggregator: it ships > no binaries of its own and exists to declare dependencies on a curated > set of other packages. --- # 5.14 Install Destinations _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The permitted top-level destinations, why package-owned storage under /usr is separate from the root-level runtime views, and the drop-in directories. Peios separates package-owned vendor storage under `/usr` from the root-level runtime views such as `/bin`, `/lib`, and `/sbin`. Those views are filesystem topology assembled by the boot and base-filesystem layers; they are not package storage. A package installs its files under `/usr`, and the runtime topology projects them at their canonical paths. Within `/usr`, executables split by kind. `/usr/sbin/` holds **system binaries** — daemons, init and boot binaries, and service executables not normally invoked directly by a person. `/usr/bin/` holds everything else, including administrative tools a person does invoke directly, even those requiring administrator privileges. ## 5.14.1 Permitted top-level destinations | Path | Purpose | |---|---| | `/usr/bin/` | Executables that are not system binaries — user-facing tools, and admin tools invoked directly | | `/usr/sbin/` | System binaries — daemons, init and boot binaries, service executables | | `/usr/lib//` | Architecture-specific libraries and arch-dependent data (§5.15) | | `/usr/lib/debug/` | Separated debug information, mirroring the install paths of the files it describes | | `/usr/lib/modules//` | Kernel content for one kernel release: its modules, and the kernel image, `System.map`, and build config alongside them | | `/usr/lib/firmware/` | Device firmware blobs, addressed by device rather than by host triplet | | `/usr/lib/os-release` | The freedesktop OS-identity file, at a fixed external contract path | | `/usr/libexec/` | Architecture-independent helper executables run by another program rather than by a person | | `/usr/share/` | Architecture-independent data | | `/usr/include/` | Header files | | `/usr/src/debug/` | Debugger source files, mirroring the build's source tree | | `/usr/src/dist/` | Corresponding source shipped by `-source` packages | | `/usr/etc/` | Vendor-shipped default configuration for legacy applications — the bottom layer of the `/etc` merge | | `/usr/conf/` | Vendor-shipped defaults for the supplementary configuration of native applications — the bottom layer of the `/conf` merge | | `/var/` | Runtime variable state directories, empty at install time | | `/boot/` | `/boot/initramfs/`, a complete independent root filesystem, and `/boot/efi/`, the EFI System Partition | | `/hooks/` | Initramfs boot hooks, discovered and ordered when the initramfs cpio is packed | | `/++/` | Initramfs early-cpio segments, prepended uncompressed ahead of the main archive | A payload entry MUST NOT install under any other top-level path, unless the package declares itself a special system package (below). A consumer MUST enforce this at install time. Producer-side validation proves nothing about a package file that arrives from elsewhere. ## 5.14.2 Notes on individual destinations Only the `debug/` and `dist/` subtrees of `/usr/src/` are permitted; the rest of `/usr/src/` is administrator territory. `/usr/etc/` is where package configuration goes. A package never writes `/etc` directly, because `/etc` is a merged view resolving `/usr/etc` < `/system/retc` < `/lcl/etc`, not storage. `/usr/conf/` is the equivalent bottom layer of the `/conf` merge (`/usr/conf` < `/lcl/conf`); native software reads the registry directly, so there is no reconciled layer between them. `/var/` accepts **empty directories only**, establishing locations a runtime will write to — `/var/log//`, `/var/state//`. Populated content under `/var/` is invalid: variable state is owned by the runtime, not by the package. `/hooks/` is meaningful only in an initramfs root, where the cpio packer scans it. In an ordinary system root it is an unused permitted destination. An entry under `/boot/` SHOULD be a symlink whose target resolves to a regular file under one of the other permitted destinations — typically `/usr/lib//` for a kernel image, initramfs, or device tree. `/boot/` is a discovery directory a bootloader reads, not storage where real package content lives. This is a SHOULD rather than a MUST because recovery images and embedded bootloader integrations that cannot follow symlinks exist; a format-level validator does not enforce it. A package reaching `/boot/initramfs/` is cross-targeting a different root, not installing into this one (§5.19). > [!NOTE] > What is absent from the list, and why: > > - `/bin`, `/sbin`, `/lib`, `/lib64`, `/libexec`, `/share`, `/include`, > `/etc`, `/conf` — **merged views**, computed from the trees beneath > them so that software sees one path while authority stays separated. > A package writes the `/usr` layer and the view does the rest. > - `/lcl` — the **operator's** tree, the peer to `/usr` that is backed > up and survives reinstall. `/lcl/policy` in particular holds inputs > that grant authority; writing there yields arbitrary code at boot, > which is why packages are excluded structurally rather than by rule. > - `/system` — **derived** from the image, registry, or platform, and > always reconstructible. > - `/opt` — **operator territory**, deliberately off the list. Software > there brings its own layout, is invisible to package verification, > and participates in none of these guarantees. > - `/dev`, `/proc`, `/sys`, `/run`, `/tmp` — kernel interfaces and > runtime-only directories; not storage a package can populate, not > even as empty directories. > - `/srv`, `/data`, `/home`, `/media`, `/mnt` — operator and user > namespaces. > - `/usr/local` — does not exist. `/usr` is meant to be untouchable, > and a writable subdirectory of it makes that a lie; `/lcl` is the > honest version. > - `/root` — does not exist. There is no root. ## 5.14.3 Special system packages A few packages exist precisely to lay down the structure these rules protect — the base-filesystem package that mints the runtime mountpoint tree is the archetype. For those, the allowlist is not a guardrail but the thing being installed. Such a package MAY set `special_system_package` in its manifest (§5.18). The declaration waives the layout checks **at production time only**. It grants nothing at install time: a consumer MUST refuse an out-of-layout payload unless the operator has *also* explicitly opted in, through a distinct and deliberate act naming that intent. This is two keys held by two parties. A package may propose its own exemption; only whoever installs it can grant one. When a consumer meets the declaration without having been given the opt-in, it MUST refuse the package with an error **naming the refused request**, so that an operator can tell "this package asked for an exemption I did not grant" from "this package is malformed". `/lcl/policy` MUST NOT be reachable by this route under any circumstance. It is the tree whose contents grant authority, and an exemption that could reach it would convert a structural guarantee into a policy one. ## 5.14.4 Drop-in directories Several subdirectories of the `/etc` merge are **drop-in directories**: their contents are interpreted as code or configuration by other tools, notably the side-effect tools of §5.24 and system daemons that read configuration drop-ins. A package writing into one has indirect influence on the behaviour of components that read it. A package from a repository other than the system's official repository MUST NOT install a file at the top level of the `/usr/etc` layer of any of these: - `ld.so.conf.d/` - `profile.d/` - `sudoers.d/` - `cron.d/`, `cron.daily/`, `cron.hourly/`, `cron.weekly/`, `cron.monthly/` - `sysctl.d/` - `modules-load.d/` - `modprobe.d/` - `binfmt.d/` - any directory the system declares as a drop-in directory through its configured list The consumer's drop-in directory list MUST be stored under a security descriptor granting write access only to a recovery-class operator principal, never to the principal performing installs. Operator configuration MAY add entries to the list but MUST NOT remove an entry this specification requires: the list is purely additive. A non-official-repository package whose payload installs to one of those paths MUST be rejected at install time. A non-official-repository package MAY install drop-in files under its own subdirectory of a drop-in path — for example `/usr/etc/ld.so.conf.d//.conf` — provided the subdirectory is namespaced by both the repository's name and the package's name, so that two such packages cannot collide. > [!NOTE] > The restriction closes a concrete chain. A low-trust repository > installs a configuration file at the top level of `ld.so.conf.d/` that > extends the loader's library search path. At the next transaction that > declares `ldconfig` — from any repository, not necessarily the same > one — the drop-in is honoured and the loader's behaviour changes. > Restricting top-level drop-in writes to the official repository, with > a namespaced escape hatch for legitimate non-official content, breaks > the chain without forbidding the use case. --- # 5.15 The Architecture Triplet _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What an architecture-specific package must install under /usr/lib//, the exemptions, and where architecture-independent data goes. A package whose architecture is not `noarch` MUST install all of the following under `/usr/lib//`, where `` is the architecture triplet of §5.8: - shared libraries (`.so`, `.so.*`) - static libraries (`.a`) - loadable modules — plugin shared objects, and kernel modules outside `/usr/lib/modules/` - architecture-*dependent* helper binaries not on the user's search path - any other arch-dependent file that is not a user-facing binary Architecture-*independent* helper executables — a shell script run by another program, say — go under `/usr/libexec/` instead, which carries no triplet rule because the rule is scoped to `/usr/lib/`. A package whose architecture is `noarch` MUST NOT install any file under `/usr/lib//`. A `noarch` package containing any of the categories above is invalid. ## 5.15.1 Exemptions Three arch-dependent payload categories are exempt from the triplet path, because each is addressed by something other than the host triplet: | Category | Path | Addressed by | |---|---|---| | Kernel content | `/usr/lib/modules//` | kernel release | | Device firmware | `/usr/lib/firmware/` | device | | Separated debug information | `/usr/lib/debug/` | the install path of the file it describes | A `noarch` package MUST NOT install under `/usr/lib/modules/` or under `/usr/lib/debug/`: kernel content and debug information are both arch-dependent. `/usr/lib/firmware/` carries no such restriction, firmware being opaque data rather than host-architecture content. Debug files mirror the full install path of what they describe. The debug information for `/usr/bin/foo` is `/usr/lib/debug/usr/bin/foo.debug`; for `/usr/lib//libfoo.so.1` it is `/usr/lib/debug/usr/lib//libfoo.so.1.debug`. Debug files MAY additionally be indexed by build ID under `/usr/lib/debug/.build-id/`. The freedesktop `os-release` file is a fourth exemption of a different kind: it installs at exactly `/usr/lib/os-release`, a fixed external contract path the ecosystem hard-codes. Unlike debug information it is arch-*independent*, so a `noarch` package — the OS-identity package — MAY ship it. It is conventionally paired with a `/usr/etc/os-release` symlink, which the `/etc` merge projects to `/etc/os-release`. ## 5.15.2 Source The debugger *source* files that debug information references install under `/usr/src/debug/`, not under `/usr/lib/`. Source is architecture-independent, so `/usr/src/debug/` carries neither a triplet rule nor the `noarch` restriction: it is a plain permitted destination that both arch-specific and `noarch` packages MAY use. The same applies to `/usr/src/dist/`, the home of corresponding-source packages. > [!NOTE] > A corresponding-source package conventionally lays out > `/usr/src/dist/-/` with `upstream/` — the pristine > source artifact, byte-identical to the producer's pinned input — > `patches/` for the applied patch series, and `recipe/` for the > build-controlling recipe files including the source lock that makes > the shipped upstream artifact verifiable. This is a producer > convention, not a format rule. ## 5.15.3 Architecture-independent data `/usr/share/` holds architecture-independent files shared across every architecture of a system: documentation, man pages, locales, configuration templates, and static data such as icons, images, and fonts. Both `noarch` and arch-specific packages MAY install under `/usr/share/`. A file installed there by an arch-specific package MUST be byte-identical across every architecture build of the same upstream version. --- # 5.16 Payload Entries _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Tar permission bits are distribution metadata and establish no access control — plus empty directories, path ownership, and forward compatibility. ## 5.16.1 Permissions Tar entry permission bits in a package are distribution-format metadata only. They establish no access control on the installed file. Access control is the consumer's responsibility: on Peios, through a security descriptor applied at file-creation time (§5.20); on any other system extracting a package for inspection or migration, through that system's native mechanism applied after extraction. Every payload entry's permission bits MUST be `0777`. Any other value MUST cause the package to be rejected. The setuid and setgid bits MUST NOT be set on a payload entry. Privilege escalation on Peios is mediated by the kernel's access-control subsystem, not by filesystem-level setuid; a setuid bit is meaningless to the access-check path and MUST NOT appear in installed content. > [!NOTE] > The `0777` rule is honest signalling. A `0644` or `0755` mode would > imply a permission contract the format does not enforce and the kernel > does not consult. Fixing every entry to `0777`, combined with the > uid/gid and owner/group rules of §5.11, treats a payload as > identity-free, permission-free transport bytes. > > One consequence: extracting a package with a generic tar tool on a > non-Peios host produces world-writable files. This is intended. > Tooling that needs sensible host-native permissions on extracted files > is responsible for applying them after extraction. ## 5.16.2 Empty directories A package MAY install an empty directory: a tar entry of type directory with no content. Empty directories establish paths a runtime will need, and are the only content permitted under `/var/` (§5.14). ## 5.16.3 One package per path Two packages MUST NOT install a file at the same install path. A consumer MUST detect the collision and reject the second install. A package MAY install content into a directory another package created; directory creation is idempotent. The rule applies to non-directory entries only. The one exception is a claim link (§5.23), which belongs to the consumer rather than to any package and is materialised only at a path no installed package owns. ## 5.16.4 Forward compatibility The triplet path convention of §5.15 is designed so that a future multi-architecture system MAY install foreign-architecture packages alongside native ones without filesystem-level collisions. In this version only one architecture's packages may be installed on a given system at a time (§5.8). The triplet convention applies regardless, so that a package conforming to this version stays forward-compatible with such an extension. --- # 5.17 Symlinks _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Symlinks as first-class payload entries — target constraints, cross-package targets, how they are covered by integrity, and their descriptors. Symlinks are first-class payload entries. The tar entry's linkname is the symlink target. ## 5.17.1 Target constraints A symlink target MUST be a relative path. A symlink target MUST resolve, when joined with the symlink's parent directory, to a path that is either within the package's own payload tree or under one of the permitted top-level install destinations of §5.14. An absolute target is forbidden, as is a target whose resolution escapes those destinations entirely. A symlink target is subject to the same path-validity constraints as a payload path (§5.13): valid UTF-8, no NUL bytes, no ASCII control characters, no backslashes, NFC normalisation, and the length limits. A consumer MUST validate every symlink target against these constraints before extracting the entry. A package containing a non-conforming symlink target MUST be rejected. > [!NOTE] > Format-level validation cannot distinguish a symlink whose resolved > target is an installable path that no package actually installs. A > target reached by `..` traversal into a permitted destination passes > validation and produces a dangling symlink; one that would overwrite > an existing owned file is caught by the one-package-per-path rule > (§5.16). Both are consumer-side outcomes, not format errors. ## 5.17.2 Cross-package targets A producer MAY emit a symlink whose target resolves into a different package's payload tree, provided the resolved path is under a permitted destination. The canonical case is the conventional library split, where a `-dev` package ships a developer link (`libfoo.so`) whose target (`libfoo.so.1`) lives in the corresponding runtime package. A producer SHOULD declare the target's owning package as a dependency, so that the target is present at extraction time. The format does not record this relationship at the symlink level; it is captured at the package level through `dependencies` (§5.21). > [!NOTE] > Prohibiting cross-package symlinks was considered — it would have > forced restructured `-dev` splits with developer links living in > runtime packages — and rejected. Forbidding them deviates from > universal Linux convention with no corresponding security gain: the > defence below operates on resolved-target validity, not on whether the > resolution crosses a package boundary. ## 5.17.3 Integrity A symlink has no content body, and so is not hashed in the files manifest (§5.25). Its target is integrity-checked directly: the linkname stored in the tar header is what the consumer compares, and that header is inside the signed bytes (§5.28). ## 5.17.4 Security descriptors A symlink does not carry an independent security descriptor. Access to a symlink is governed by access to its target. A security descriptor override (§5.20) MUST NOT target a symlink entry. > [!NOTE] > The classic symlink TOCTOU attack — install `/usr/share/foo` as a > symlink to a system file, then have a later write at > `/usr/share/foo/bar` traverse it — is prevented in two layers. The > format forbids symlink targets resolving outside the managed tree, and > extraction resolves every path component without following a symlink, > so even an in-tree symlink cannot redirect a later write. --- # 5.18 The Manifest _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The authoritative JSON metadata for a package — identity, relationships, side-effect requirements and build provenance — with its full schema. The manifest is the authoritative metadata for a package: its identity, its relationships, its side-effect requirements, and its build provenance. It is a JSON document at `.peipkg/manifest.json`. ## 5.18.1 Schema ```json { "schema_version": 1, "name": "", "version": "", "architecture": "", "description": "", "license": "", "homepage": "", "default_root": "", "special_system_package": , "dependencies": [...], "optional_dependencies": [...], "conflicts": [...], "provides": [...], "replaces": [...], "side_effects": [...], "size_installed": , "sd_overrides": [...], "build": {} } ``` ## 5.18.2 Required fields | Field | Type | Description | |---|---|---| | `schema_version` | integer | MUST be 1 in this version. | | `name` | string | Package name conforming to §5.3. | | `version` | string | Version conforming to §5.5. | | `architecture` | string | Architecture identifier conforming to §5.8. | | `dependencies` | array | Required dependencies. MAY be empty; MUST be present. | | `conflicts` | array | Conflicting packages. MAY be empty; MUST be present. | | `size_installed` | integer | Total size in bytes of the installed payload. | | `build` | object | Build provenance. | A manifest missing any required field MUST be rejected. ## 5.18.3 Optional fields | Field | Type | Description | Absent means | |---|---|---|---| | `description` | string | One-line human-readable description. | empty string | | `license` | string | SPDX identifier or expression. | empty string | | `homepage` | string | URL of the upstream project. | empty string | | `default_root` | string | The root a *top-level* install of this package lands in when the operator names none (§5.19). | the operator's current root | | `special_system_package` | boolean | Declares the package exempt from the §5.14 layout rules at production time (§5.14). | `false` | | `optional_dependencies` | array | Dependencies that enhance but are not required. | empty array | | `provides` | array | Virtual names this package satisfies. | empty array | | `replaces` | array | Packages this one supersedes. | empty array | | `side_effects` | array | Maintenance operations to invoke (§5.24). | empty array | | `sd_overrides` | array | Per-entry security descriptor overrides (§5.20). | empty array | A manifest carrying an unknown field MUST NOT be rejected; the unknown field MUST be ignored (§5.9). ## 5.18.4 The build object ```json { "timestamp": "", "farm_id": "", "source_ref": "" } ``` | Field | Required | Description | |---|---|---| | `timestamp` | yes | RFC 3339 timestamp of the build. MUST be UTC, MUST end with `Z`, and MUST NOT carry sub-second precision. | | `farm_id` | yes | Identifier of the build farm that produced this package. | | `source_ref` | yes | Reference to the build inputs, sufficient to reproduce the build. | | `source_package` | no | Name of the corresponding-source package produced from the same recipe and inputs (§5.15). | | `recipe_ref` | no | VCS identity of the recipe tree the build ran from — for example `git:`, suffixed `+dirty` when the work tree held uncommitted changes. | | `builder` | no | Identity and revision of the producing tool, for example `pekit/`. | A consumer MUST treat an absent optional field as the empty value. `timestamp` is also the modification time of every tar entry (§5.11 rule 2). A producer MUST set both identically. `source_ref` is producer-defined but SHOULD be a machine-resolvable reference. The conventional form is a version-control URL with an explicit ref: ``` git+https://git.peios.org/sources/nginx#refs/tags/v1.26.2-3 ``` > [!NOTE] > The build object exists to make reproducibility verifiable: a third > party in possession of the build inputs and the recorded timestamp can > re-run the build and compare the output bytes. The format does not > mandate that verification; it supplies the inputs for it. ## 5.18.5 Field constraints `description`, when present, MUST consist only of printable ASCII in the range `0x20`–`0x7E`. ASCII control characters and non-ASCII bytes MUST NOT appear. It SHOULD be a single line under 80 characters; longer descriptions belong in upstream documentation. > [!NOTE] > The byte-range rule is deliberately cruder than a Unicode-category > test. `description` is displayed to an operator deciding whether to > install, so it is both a terminal-escape-injection surface and a > homoglyph surface. A hard ASCII whitelist closes both without anyone > having to reason about which Unicode categories are safe to render. `license`, when present, SHOULD be a valid SPDX expression. A producer MAY use another form; this specification does not validate license strings. `homepage`, when present, MUST be a syntactically valid URL per RFC 3986 and MUST use the `https` or `http` scheme. Any other scheme MUST cause the package to be rejected. `size_installed` MUST be a non-negative integer, and MUST equal the sum of the `size` fields of every entry in the files manifest (§5.25). A consumer MUST verify that equality and MUST reject a package where it does not hold. > [!NOTE] > `size_installed` is not merely advisory. It is the input to the > decompression bound of §5.27, so a package whose declared size is > smaller than what it actually unpacks to is a resource-exhaustion > attempt, and one that is larger inflates the bound. Tying it to a > quantity a consumer can independently compute is what makes it > trustworthy. ## 5.18.6 Authoritative status The manifest is authoritative for a package's metadata. Where it disagrees with any other source — the repository index, the filename, secondary documentation — the manifest MUST be treated as correct, and the disagreement MUST be reported (§5.32). ## 5.18.7 Encoding The manifest MUST be UTF-8 encoded JSON and MUST end with a single newline. A producer that intends its packages to be byte-reproducible MUST serialise the manifest canonically: compact, with no insignificant whitespace, with HTML-escaping of `<`, `>`, and `&` disabled, with fields in the declaration order of the schema above, and with every optional field either always emitted or never emitted for a given producer. > [!NOTE] > The manifest's bytes are inside the archive that gets hashed and > signed, so two semantically identical manifests with different > whitespace produce different packages. Pinning the serialisation is > what makes independent reproduction possible at all; leaving it to the > producer's discretion means only that producer can ever reproduce its > own output. --- # 5.19 Installation Roots _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > A self-contained tree packages install into — the default root, named roots, and why dependency satisfaction is per-root. An **installation root** is a self-contained filesystem tree into which packages are installed. The default root is the system root; a system MAY define additional **named roots** — an initramfs image built and maintained alongside the main system is the motivating case. How roots are registered, and how a name resolves to a filesystem location, is consumer mechanics and is not part of the package format. ## 5.19.1 Root references A **root reference** is the string form by which a manifest names a root. Within a manifest a root reference MUST be a **named reference**: one or more segments separated by `.`, where each segment matches `[a-z0-9][a-z0-9_-]*`. Nesting is expressed by further segments, so `initramfs.subroot` names the root `subroot` registered within the root `initramfs`. A root reference in a manifest MUST NOT be an absolute or relative filesystem path. A package names roots and never dictates a filesystem location: placement is the installing system's prerogative. A manifest whose root reference is not syntactically valid is invalid and MUST cause the package to be rejected. Whether the named root *exists* is a consumer-side resolution concern, not a format-validity one. ## 5.19.2 `default_root` The manifest's `default_root` field (§5.18) governs **only** the placement of a *top-level* install of the package — an operator request naming this package directly, with no explicit root. It has no effect when the package is pulled in as a dependency; dependency placement is governed by the depending package and by the dependency's own `root` field (§5.21). An explicit operator-supplied root always overrides `default_root`. ## 5.19.3 Satisfaction is per-root The identity of a satisfier is the pair **(name, root)**. The same package name installed in two different roots is two independent satisfactions, possibly at different versions, and a dependency is satisfied only by an installation in the named — or defaulted — root. A `constraint` or architecture qualifier is evaluated against that installation. > [!NOTE] > Cross-root dependencies let a root be composed through the dependency > graph. An initramfs package may depend on ordinary packages — a shell, > core utilities — and have them placed into the initramfs root, either > implicitly by living in that root itself or explicitly through the > dependency's `root` field. The depended-on package declares no root > affinity of its own; where it lands is the depender's and the > operator's choice. This generalises the fixed build-host/target split > other systems draw to an open set of named roots. --- # 5.20 Security Descriptor Overrides _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Installed files inherit their descriptor by default — how a package declares an override, and the consumer's obligation when it does. Every installed file and directory carries a security descriptor. A consumer applies it at file-creation time, through the kernel's file-creation interface — never through a tar attribute, which §5.11 rule 5 forbids outright. ## 5.20.1 The default is inheritance When a payload entry has no override, a consumer MUST create the entry without supplying an explicit security descriptor, so that the kernel computes one by inheritance from the parent directory's inheritable entries at creation time. Inheritance is the default for the overwhelming majority of installed entries, and most packages declare no overrides at all. An override is appropriate when a file needs more restrictive access than its parent would give it, when it needs explicit access for a principal absent from the parent's inheritable entries, or when a directory needs to begin a new inheritance scope. ## 5.20.2 Declaring an override An entry in the manifest's `sd_overrides` array has the form: ```json { "path": "", "sd": "" } ``` | Field | Description | |---|---| | `path` | Payload-relative path, matching a tar entry exactly. | | `sd` | Base64-encoded binary self-relative security descriptor, per RFC 4648 §4 without padding. | The `sd_overrides` array MUST be sorted lexicographically by `path`, and MUST NOT contain two entries with the same `path`. `path` MUST refer to a regular-file entry or a directory entry. An override MUST NOT target a symlink entry, which carries no independent descriptor (§5.17). An override referring to a non-existent payload entry, or to a symlink entry, is invalid and MUST cause the package to be rejected. `sd` MUST decode to a syntactically valid binary self-relative security descriptor. One whose decoded bytes do not parse is invalid and MUST cause the package to be rejected. A consumer MUST perform all three of those checks — entry existence, entry type, and descriptor parseability — before installing anything from the package. Deferring them to the moment the descriptor is applied turns a malformed package into a partially completed install. ## 5.20.3 The consumer's policy obligation The kernel validates that a declared descriptor is well-formed. It does **not** validate that the producer of a package had any authority to declare that descriptor on behalf of the principals it grants access to. A package can therefore declare a descriptor granting access to any principal the system knows about. The format treats the bytes as opaque; whether a given package may declare a given descriptor is policy, and that policy is the consumer's to enforce. A consumer MUST enforce a per-repository override policy: 1. Before applying any override, the consumer MUST surface it to the operator in human-readable form, including the payload path, the principals and rights granted, and a diff against what inheritance would have produced. 2. For a package from the system's official repository, overrides MAY be applied without per-operation confirmation, but the operator-visible install report MUST list every override applied. 3. For a package from any other repository, the consumer MUST require explicit operator confirmation before applying an override that grants rights to a principal outside a configured allowlist. The default allowlist contains the well-known system principals, plus any principal the operator has added to that repository's allowlist. It MUST NOT contain any principal derived from the package itself — from its manifest fields, its build metadata, or its payload. A package cannot elect its own principals into the allowlist. 4. A package whose overrides the policy rejects MUST be refused. A consumer MUST NOT silently drop the overrides and proceed with inheritance defaults. ## 5.20.4 Inherited descriptors are covered too The policy applies both to explicitly declared descriptors and to descriptors that *result from* inheritance from a directory whose own descriptor was declared by any package's overrides. Specifically: when installing a file under a directory whose descriptor was overridden by any package — from any repository — the resulting inherited descriptor MUST pass the policy as if the installing package had declared it. > [!NOTE] > Without this, package A declares an override on a directory that > grants A's author rights, and package B's files installed under that > directory silently inherit it with the operator never prompted. The > check fires on *what descriptor ends up on the file*, regardless of > how it got there — which also closes the case where a > carefully-mimicked "looks like the default" override would have > evaded a test for non-default descriptors. ## 5.20.5 Failure If file creation fails because the kernel rejects the descriptor — most often because it references a principal the system does not know — the install MUST be treated as failed and any partial state rolled back. A package MUST NOT be installed into a parent directory whose descriptor denies the caller the access required to create the entry. A consumer detects this at install time and treats it as an install failure. --- # 5.21 Relationships _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The five manifest fields expressing what a package needs, conflicts with, provides and replaces — with their constraints. A package expresses its relationships to other packages in five manifest fields: - `dependencies` — packages that must be installed for this one to function - `optional_dependencies` — packages that enhance functionality but are not required - `conflicts` — packages that must not be installed alongside this one - `provides` — virtual names this package satisfies on behalf of dependencies declared elsewhere - `replaces` — packages this one supersedes ## 5.21.1 Dependency entries An entry in `dependencies` or `optional_dependencies`: ```json { "name": "", "constraint": "", "arch": "", "root": "", "claims": { "": { "path": "" } } } ``` | Field | Required | Description | |---|---|---| | `name` | yes | The depended-on package name (§5.3) or virtual name (§5.4). | | `constraint` | no | A version constraint per §5.7. Absent means any version satisfies. | | `arch` | no | An architecture qualifier. Default `any`. | | `root` | no | A root reference (§5.19) naming the root this dependency is placed and satisfied in. Absent means the same root as the depending package. | | `claims` | no | Claim paths this dependency expects a holder to materialise (§5.23). | `root`, when present, MUST be a syntactically valid named root reference — never a filesystem path. An entry whose `root` is not one is invalid. ## 5.21.2 Conflict entries An entry in `conflicts` has the same shape as a dependency entry, minus `root` and `claims`, and expresses incompatibility rather than requirement: a package MUST NOT be installed simultaneously with any package matching the entry. A conflict whose `constraint` is absent expresses incompatibility with any version of the named package. ## 5.21.3 The architecture qualifier `arch` restricts the qualified package's architecture. In this version the only valid value is `any`, which is the default. Any other value MUST be rejected. `any` means: the qualified package's architecture MUST equal the depending package's **effective architecture**, or be `noarch`. A depending package's effective architecture is its own architecture when arch-specific, and the system's primary architecture (§5.8) when the depending package is `noarch`. A `noarch` label describes an architecture-independent payload, not an architecture-independent resolution context: a `noarch` package's dependencies on arch-specific packages — a script on its interpreter, a meta-package on native tools — resolve against the concrete system being assembled, exactly as a native package's do. > [!NOTE] > A future version may permit explicit architecture identifiers here, to > support multi-architecture systems. Reserving the field now is what > makes such an extension parse compatibly. ## 5.21.4 Provides entries ```json { "name": "", "version": "", "claims": { "": { "target": "", "path": "" } } } ``` | Field | Required | Description | |---|---|---| | `name` | yes | The virtual name provided, conforming to §5.4. | | `version` | no | The version of the capability provided. Parsed revision-relaxed (§5.7), because a provides version is a capability level rather than a packaging iteration. Absent means any version of the name is provided. | | `claims` | no | Filesystem targets this package materialises when it holds the named role (§5.23). | A virtual name that collides with a real package name MAY be provided; both are then valid satisfiers of a dependency on that name. `provides.version` SHOULD reflect the providing package's actual functional compatibility level. A `provides.version` greater than the providing package's own `version` MUST generate an operator warning at install time, because an inflated provides-version defeats constraint-based resolution. > [!NOTE] > The attack the warning exists for is concrete: a package at version > `1.0-1` declaring `provides: [{name: libfoo, version: "5.0"}]` > satisfies a dependency on `libfoo >= 4.0` and gets installed in place > of a real `libfoo` — most easily when the real one is absent from the > candidate set, which is precisely the shadowing case. The provides relation does not flow transitively: providing `smtp-server` does not provide whatever `smtp-server` itself provides. ## 5.21.5 Replaces entries ```json { "name": "", "constraint": "" } ``` `name` is required and MUST conform to the package-name grammar (§5.3). `constraint` is optional; absent means this package replaces any version of the named one. A replaces entry expresses supersession. During upgrade the replaced package is removed and this one installed in its place: files owned by the replaced package that no longer exist in this one are removed, and files existing in both are updated. A replaces entry does not imply a conflict. A package MAY both replace and conflict with the same target, but a replaces entry is typically sufficient on its own. > [!NOTE] > Replaces is the rename mechanism. When `nginx-core` becomes `nginx`, > the new `nginx` declares `{ "name": "nginx-core" }` in `replaces`, and > existing systems transition on their next upgrade. ## 5.21.6 Field constraints Each of the five fields is an array of objects matching the appropriate schema. `dependencies` and `conflicts` MUST be present, and MAY be empty. The other three MAY be omitted, which is equivalent to an empty array. Within a single field, entries MUST be sorted lexicographically by `name`, and two entries MUST NOT carry identical `name` values. A package with several constraints on one target MUST combine them into that entry's single `constraint` string. ## 5.21.7 What satisfies a dependency A dependency is **satisfied** by a candidate package when all of the following hold: 1. The candidate's name equals the dependency's `name`, **or** the candidate has a `provides` entry whose name equals it. 2. If the dependency carries a `constraint`, the version satisfies it — the candidate's own version when matched by name, and the matching `provides` entry's version when matched through `provides`. 3. The candidate's architecture satisfies the `arch` qualifier. 4. The candidate is installed, or is being installed, in the dependency's root (§5.19). A conflict is **triggered** by a candidate when the same conditions hold with respect to a `conflicts` entry. A `claims` field has no effect on satisfaction. A dependency on a role is satisfied by any installed eligible provider regardless of which one currently holds the role; claims govern which installed file owns a contended filesystem name, nothing more (§5.23). --- # 5.22 Derived Capabilities _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Capabilities derived mechanically from built contents — shared libraries and pkg-config modules — and why derivation is the producer's job. Some capabilities are derived mechanically from a package's built contents rather than declared by hand. So that a producer and a consumer agree on the name whichever way it arrived, the conventions below are normative for the capability `name`. | Capability | Virtual name | Version | |---|---|---| | Shared library | The ELF soname, verbatim — `libssl.so.3`. | None by default. | | pkg-config module | `pkgconfig()`, where `` is the `.pc` file's base name — `pkgconfig(glib-2.0)`. | The `.pc` file's `Version:` field, matched as an ordered constraint per §5.7. | ## 5.22.1 Shared libraries A shared-library dependency is the soname listed in a binary's `DT_NEEDED`; the corresponding provide is the soname in the providing library's `DT_SONAME`. The soname's ABI-version field is part of the name and is matched by exact equality: `libssl.so.3` is never satisfied by `libssl.so.4`. A version MAY be carried on a soname provide when the library's symbol versions are commensurable with the providing package's own version, as they are for a C library shipping versioned symbols. ## 5.22.2 pkg-config modules A pkg-config dependency is a module named in a `.pc` file's `Requires:` or `Requires.private:`; the corresponding provide is the `.pc` file itself. ## 5.22.3 Derivation is a producer concern Whether a producer derives these automatically is its own business. This section fixes only the names, so that a hand-written entry and a derived entry for the same capability are byte-identical. > [!NOTE] > A producer that derives from ELF metadata will encounter cases this > section does not name: a symlink that carries no `DT_SONAME` of its > own, a shared-library-shaped file with no soname at all, symbol > version tokens from `DT_VERNEED` that could be turned into a version > constraint. Any additional constraint a producer synthesises is a > constraint like any other and is evaluated by §5.7; what this section > forbids is inventing a different *name* for a capability that has one. --- # 5.23 Claim Declarations _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > How several installed packages contend for one shared filesystem name with exactly one owning it — the vocabulary, eligibility, and the consumer's part. `provides` (§5.21) lets several installed packages satisfy one virtual name. A **claim** extends that to the filesystem: it lets several installed packages contend for a single shared filesystem name, with exactly one owning it at a time. The canonical case is a role daemon — two registry sources may both be installed, but only one may own `/usr/bin/registryd`. This section specifies what a package declares. Which provider holds a role, and when the consumer re-evaluates that, is consumer mechanics. ## 5.23.1 Vocabulary - **Role** — a virtual name (§5.4) that one or more packages contend to own. A role is identified by the `name` of a `provides` or dependency entry carrying a `claims` field. - **Slot** — a named channel within a role. Each slot materialises one filesystem name. A role has one or more slots. - **Claim path** — the absolute path a slot materialises at. A slot MAY have more than one. - **Target** — the file a claim path points at while a given provider holds the slot. The target is a payload file of the holding package. - **Holder** — the single installed package that currently owns a role. A role with no holder is *unheld*. > [!NOTE] > The two halves of a claim are declared by two different parties. The > *consumer* — whatever hard-codes `/usr/bin/registryd` and expects to > find a registry daemon there — declares the path. The *provider* > declares the target, the file of its own that should answer it. The > consumer joins the two by (role, slot). Splitting the declaration this > way puts the path where the dependency on it lives and the target > where the implementation lives. ## 5.23.2 The `claims` field A claim is declared by adding a `claims` field to a dependency entry, an optional-dependency entry, or a `provides` entry. It maps a slot name to a slot descriptor: ```json "claims": { "": { "path": "", "target": "" } } ``` A slot name MUST conform to the package-name grammar (§5.3). Which of the two descriptor fields is permitted depends on where the `claims` field appears: - On a **dependency** or **optional-dependency** entry — the consumer side — each slot descriptor MUST contain `path` and MUST NOT contain `target`. A consumer declares only where it expects the name; it supplies no implementation. - On a **`provides`** entry — the provider side — each slot descriptor MUST contain `target` and MAY contain `path`. A provider declares the file that answers the slot, and MAY additionally declare a default claim path. A `claims` field MUST NOT appear on a `conflicts` or `replaces` entry. Slot keys within a `claims` object are an unordered JSON object and carry no ordering requirement. The enclosing arrays remain sorted and unique by `name` (§5.21). Example — one package consumes the role, another provides it: ```json // the consumer's manifest "dependencies": [ { "name": "registryd", "claims": { "binary": { "path": "/usr/bin/registryd" } } } ] // the provider's manifest "provides": [ { "name": "registryd", "claims": { "binary": { "target": "/usr/sbin/loregd" } } } ] ``` ## 5.23.3 Where a target may point A `target` MUST name a path the declaring package itself installs as a payload entry, and MUST therefore lie within the permitted install destinations of §5.14. A `target` that does not correspond to one of the declaring package's own payload paths is invalid and MUST cause the package to be rejected. A consumer MUST verify this itself, against the payload it actually received. Producer-side validation says nothing about a package built elsewhere. ## 5.23.4 Where a claim path may lie A claim path is not a payload entry — it is the location of a consumer-managed link — and is governed by its own rule. It MUST satisfy the payload path-syntax and safety constraints of §5.13, and it MUST lie in one of: - the permitted install destinations of §5.14; - under `/run/`; or - the well-known root-level name `/init`. Any other location MUST cause the package to be rejected. > [!NOTE] > A link location and a target are constrained differently because they > are different kinds of object. A target is a real file the provider > ships, so it obeys the ordinary payload rules. A claim path is a name > the consumer owns and points at that file; permitting `/run/` for it — > without permitting packages to ship payload there — lets a role expose > a runtime socket while keeping `/run` off-limits to package payloads. > `/init` is admitted for the same reason in the other direction: it is > a well-known file a provider must own directly, and it is not a > payload destination. `/lcl/policy` MUST NOT be reachable as a claim path under any circumstance, by the same rule and for the same reason as §5.14. ## 5.23.5 Eligibility A package is an **eligible provider** of a role when it has a `provides` entry whose `name` is the role and whose `claims` field declares a `target` for at least one of the role's slots. Only an eligible provider may hold a role. A package that depends on a role and declares a claim path for it, but does not provide the role, is a **consumer only**: it contributes claim paths and can never hold. ## 5.23.6 What a consumer guarantees The materialised links for a role MUST at all times equal the cross-product of the role's computed claim paths with the holder's targets, where the computed claim path set for a slot is the union of: - every `path` declared for that slot by an installed consumer, and - the `path` declared for that slot by the holder's own `provides` entry, if present. A consumer MUST re-evaluate that set within any transaction that changes its inputs — a change of holder, or the installation or removal of any package declaring a claim path or a target for the role. A claim path declared for an already-held role MUST be materialised retroactively against the current holder; the holder is not re-decided. A role MAY be held with no materialised links at all, when its computed path set is empty. Holder state is therefore recorded independently of whether any link exists. A claim link is owned by the consumer, not by any package. It MUST NOT appear in any package's payload and MUST NOT be recorded as a package-owned path. This is what lets two eligible providers coexist: neither ships the contended path, so the one-package-per-path rule (§5.16) is never engaged by the providers themselves. A claim path MUST NOT collide with a path owned by any installed package, evaluated against the state the containing transaction will produce rather than the state it started from. On collision, materialisation MUST fail and the transaction MUST be rolled back. A holder swap MUST repoint every one of the role's links within a single transaction, and each repoint MUST be atomic, so that no consumer of a claim path ever observes the path absent. > [!NOTE] > Atomic repoint matters because a claim path is typically on the > critical path of a running system: a contended daemon binary may be > executed at any moment. Tearing the old link down and building the new > one as two steps exposes a window in which the name does not exist. ## 5.23.7 What claims are not - Not a general symlink mechanism. A package needing a fixed symlink among its own files ships a payload symlink entry (§5.17). Claims exist for names contended by several packages. - Not a service-registration or activation mechanism. A materialised claim link is a symlink and nothing more. - Not an input to dependency resolution (§5.21). - Not a way to escape the one-package-per-path rule for ordinary payload files. Only consumer-owned claim links are exempt, and only at paths no package owns. --- # 5.24 Side-Effect Declarations _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The closed set of maintenance operations a package may request after install or removal — depmod and man-db — and how they are hardened, plus why Peios needs no shared-library cache. Some standard maintenance operations must run after files are installed or removed for a system to function: rebuilding the kernel module dependency cache when modules change, rebuilding the man page index when man pages are added. These are not install scripts. **The format does not permit a package to specify its own install script.** A package instead declares which of a closed, enumerated set of maintenance operations it requires, and the consumer invokes them. ## 5.24.1 Schema `side_effects` is an array of strings: ```json "side_effects": ["depmod", "man-db"] ``` Each string MUST be drawn from the set below. An unknown value is invalid and MUST cause the package to be rejected. The array MUST NOT contain duplicates. It MAY be empty, or omitted entirely, for a package requiring no maintenance operation. ## 5.24.2 The recognised set ### 5.24.2.1 `depmod` Rebuilds the kernel module dependency cache for a kernel release. A package MUST declare `depmod` if its payload contains kernel module files (`.ko`, `.ko.*`) under `/usr/lib/modules/`. A package MUST NOT declare it if its payload contains no kernel modules. The consumer MUST invoke it once **per affected kernel release**, naming that release. A package shipping modules for two releases causes two invocations. > [!NOTE] > Naming the release matters more than it appears to. An invocation with > no release operand acts on the *running* kernel, which during a kernel > update or an image build is precisely not the kernel whose modules > were installed — leaving that release's dependency cache unbuilt and > its modules unloadable. ### 5.24.2.2 `man-db` Rebuilds the man page index, so that lookups by keyword are fast. A package SHOULD declare `man-db` if its payload contains man pages under `/usr/share/man/`. > [!NOTE] > `man-db` is SHOULD rather than MUST because man page lookup degrades > gracefully without it — queries fall back to a filesystem scan. A > package omitting it is suboptimal, not broken. ## 5.24.3 Semantics A side effect MUST be idempotent: running it several times in succession MUST leave the system in the same state as running it once. The recognised set has this property by construction. A side effect MUST be safe to invoke non-interactively. A consumer MUST invoke each declared side effect **once per transaction**, after every file operation in that transaction is in place and after the transaction has committed. Side effects MUST be deduplicated across the packages in a transaction: several packages each declaring `man-db` cause one invocation, not several. A consumer MUST also invoke a side effect when a transaction *removes* files whose absence affects that effect's target — removing a kernel module requires `depmod`, removing a man page requires `man-db` — whether or not any package in the transaction declared it. ## 5.24.4 Why there is no shared-library cache Other systems carry an `ldconfig` side effect to rebuild `/etc/ld.so.cache`. Peios has no such cache and no such side effect, and this is a property of the layout rather than an omission. A cache exists to do two things: make lookup fast when the loader must search many directories, and let it find libraries in directories it would not otherwise search. Peios has neither problem. The C library is configured with its library directory, its system library directory and its runtime-loader directory all set to `/usr/lib/`, and the loader carries that path compiled in as its default. **There is exactly one shared-library directory, and it is the one the loader already searches.** So the rule that replaces the declaration is a layout rule, and it is normative: a package shipping a shared library MUST install it into `/usr/lib/`. A library installed anywhere else will not be found, and no maintenance operation exists to make it findable. Reintroducing a cache would mean reintroducing everything a cache brings with it — a file to keep coherent with the filesystem, a tool in the base to regenerate it, and a failure mode where the two disagree. That trade is only worth making if Peios ever needs more than one library directory. ## 5.24.5 Ordering Side effects are invoked in an implementation-defined order. The recognised set is chosen so that order between distinct effects is not significant, and a consumer MAY invoke them concurrently. ## 5.24.6 Invocation hardening A consumer MUST invoke a side-effect tool with: - a **fixed absolute path** to the tool. The set is closed, so the consumer knows each tool's location; it MUST NOT search a path variable. - a **cleared environment** containing only well-defined variables. Environment inherited from the invoking context MUST NOT be passed through. - **standard input closed**. > [!NOTE] > If a consumer located the tool by searching a path variable, a package > could shadow the intended tool through an inherited search path. > Because the set is closed the consumer needs no configurable > allowlist — it invokes a known path. A cleared environment closes the > matching injection route. A consumer MUST invoke the tool against the **installation root the transaction acted on**, not against the root the consumer itself is running from. ## 5.24.7 Failure Side effects run after the transaction commits, so a side-effect failure does not — and cannot — roll the transaction back. A consumer MUST report the failure to the operator; the transaction stands. Because side effects are idempotent, a failed one is self-correcting: re-invoking it, explicitly or as part of the next transaction that declares it, reaches the correct state. A consumer SHOULD make re-invocation straightforward. > [!NOTE] > Rolling a committed transaction back because a cache rebuild exited > non-zero would be disproportionate: the packages installed correctly > and only a cache lagged, recoverably. ## 5.24.8 Extension A future version MAY recognise further identifiers — likely candidates include `update-mime-database`, `update-desktop-database`, and `udev-reload`, all excluded here as irrelevant to the scope Peios is built for. A conforming implementation of this version MUST reject a manifest declaring any identifier outside the set above. A future version introducing a new identifier MUST state whether it is order-independent with respect to the existing set. An order-dependent side effect, if one is ever added, MUST be specified with a normative ordering relative to every other recognised effect. ## 5.24.9 What side effects are not - Not a general install-script mechanism. The closed enumeration is what prevents arbitrary code execution at install time. - Not a way to register a service with the init system. Service integration belongs to the higher-level artifacts that compose packages. - Not a way to seed registry state. - Not a way to apply security descriptors, which are applied at file-creation time (§5.20). A package whose required behaviour cannot be expressed through the manifest is incomplete and cannot be installed through the package format alone. That behaviour MUST be supplied by the higher-level artifact that composes the package. --- # 5.25 The Files Manifest _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Per-file integrity beneath the package-level hash — the schema, what it must cover, and the two threats the two levels answer. A package's integrity is verified at two levels. At the **package level**, the whole file has a hash and a signature proving it has not been altered since signing. At the **per-file level**, each payload file has an individual hash proving its content has not been altered between archive creation and installation. The per-file level lives in the files manifest at `.peipkg/files.json`. ## 5.25.1 Schema ```json { "schema_version": 1, "algorithm": "sha256", "entries": [ { "path": "", "size": , "hash": "" } ] } ``` | Field | Description | |---|---| | `schema_version` | MUST be 1 in this version. | | `algorithm` | MUST be `sha256` in this version. | | `entries` | One entry per regular-file payload entry. | | Entry field | Description | |---|---| | `path` | Payload-relative path, identical to the corresponding tar entry path. | | `size` | Size in bytes of the file's content. | | `hash` | Lowercase hexadecimal hash of the file's content under the declared algorithm. | The `entries` array MUST be sorted lexicographically by `path` and MUST NOT contain duplicates. ## 5.25.2 Coverage The files manifest MUST contain **exactly one entry per regular-file payload entry**, and MUST NOT contain an entry for a metadata entry under `.peipkg/`, a directory entry, or a symlink entry. A regular-file payload entry with no corresponding files-manifest entry is invalid. A files-manifest entry with no corresponding tar entry is invalid. Either MUST cause the package to be rejected on parse. > [!NOTE] > The correspondence is checked in both directions because each > direction catches a different failure. A file with no entry is an > unverifiable file smuggled into the payload; an entry with no file is a > hash for something that was never shipped, which makes the manifest's > own count untrustworthy. Symlinks are integrity-checked through the tar entry's linkname directly, and directories have no content. The files manifest covers only what is verifiable by content hash. ## 5.25.3 The package hash The package hash is the hash of the entire `.peipkg` file in its compressed on-wire form, computed with the algorithm declared in the repository index (§5.33). The required algorithm in this version is SHA-256. It is recorded in the repository index, to verify that a downloaded file matches what the repository advertises, and in the signature payload (§5.28), to bind a signature to that exact file. The package hash is **not** recorded inside the package: a package cannot contain its own hash. ## 5.25.4 Algorithm agility This version supports SHA-256 only. The `algorithm` field here and the hash identifier in the index reserve syntactic space for future algorithms. A conforming implementation of this version MUST reject any algorithm value other than `sha256`. > [!NOTE] > BLAKE3 is a likely future addition for performance on large packages. > The reservation means such a migration can be additive: producers > continue emitting SHA-256 for compatibility with this version while a > future version permits BLAKE3 as well. ## 5.25.5 Two levels, two threats The two levels defend against different things, and a consumer MUST verify both. The package hash plus the signature defends against substitution of the package as a whole. The files manifest defends against corruption or tampering during extraction, after the signature has been verified. Verifying only the signature leaves extraction errors and on-disk corruption undetectable. Verifying only the per-file hashes leaves the files manifest itself untrusted. --- # 5.26 Verifying a Package _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The ordered steps a consumer performs before installing anything, why nothing is observable before step three, and how a transaction commits. A consumer MUST perform the following steps, in this order, before installing anything from a package: 1. Compute the SHA-256 of the downloaded `.peipkg` file. 2. Compare it against the hash recorded in the repository index (§5.33). If they differ, the package is corrupted or substituted; abort. 3. Verify the inline signature (§5.30). If verification fails, the package's authenticity is unproven; abort. 4. Decompress and parse the tar archive, enforcing the layout rules of §5.12 and the determinism rules of §5.11. 5. Read `.peipkg/manifest.json` and verify it against §5.18. 6. Read `.peipkg/files.json` and verify it against §5.25, including the two-way coverage check and the `size_installed` equality of §5.18. 7. For each payload entry, compute its content hash and compare it against the files manifest. If any file's hash does not match, abort. 8. Compare the manifest against the index entry that led here (§5.32). If any field disagrees, abort. A consumer MUST NOT install any payload before all eight steps complete successfully. Partial installation after a verification failure leaves the system indeterminate and is forbidden. ## 5.26.1 Ordering is logical, not temporal A consumer MAY compute the hashes for steps 1, 3, and 7 in a single streaming pass: feeding the compressed bytes simultaneously through a hasher and a decompressor, piping the decompressed bytes through a second hasher up to the signature entry, and hashing each file's content as the tar walk reaches it. What the ordering requires is that no payload is committed to its final install path, and no decompressed byte is made visible outside the consumer's own private state, until every step has completed. ## 5.26.2 Nothing observable before step 3 A consumer MUST NOT make any decompressed payload byte visible to another process — including through a staging directory reachable from outside the consumer's own process tree — before signature verification has succeeded. Streaming decompression and hashing are permitted. Observable filesystem effects are not. ## 5.26.3 Verifying the whole transaction first When a consumer installs several packages together, it MUST complete steps 1 through 8 for **every** package before extracting **any** package's payload, and it MUST do so across every installation root the operation touches. > [!NOTE] > This closes a class of multi-package attack: package A is verified, > extracted, and its contents then influence the verification or > extraction of package B — A installs a tool B's extraction invokes, or > A creates a directory whose descriptor decides where B's files land. > Verifying everything first means extraction operates on a known-good > set of payloads. The rule binds across roots as well as within one, > because a package extracted into one root is just as present on the > filesystem as one extracted into another. ## 5.26.4 Committing a payload A consumer MUST resolve every path component of an install location relative to a verified parent-directory file descriptor, without traversing any symbolic link — including one the consumer itself created earlier in the same operation. A resolution that would traverse a symlink MUST abort the operation. A pre-existing symlink at an install path MUST be removed atomically before the write, and MUST NOT be followed. A well-formed package never contains a payload entry whose ancestor component is a symlink. A resolution failure therefore indicates a malformed or hostile package, or hostile filesystem state. On Linux this is achieved with `openat2(..., RESOLVE_NO_SYMLINKS)` anchored at the relevant permitted top-level destination (§5.14), or with equivalent semantics using `O_NOFOLLOW` on every component against a carried directory descriptor. A consumer SHOULD additionally apply `RESOLVE_BENEATH`, `RESOLVE_NO_XDEV`, and `RESOLVE_NO_MAGICLINKS` as defence in depth, and SHOULD commit a staged file with `renameat2` against the same pinned parent descriptor rather than a re-walked path string. > [!NOTE] > The format's symlink-target rules (§5.17) are the first layer of the > defence and this is the second, and neither is sufficient alone. The > format forbids a symlink whose target resolves outside the managed > tree; component-wise resolution ensures that even an in-tree symlink — > or one already on the filesystem before this package arrived — cannot > redirect a write. Without the second layer, a package shipping a > symlink in one transaction and a file *under* that symlink in the next > silently writes outside where it claims to, with no race required. --- # 5.27 Decompression Bounds _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The two bounds every consumer enforces against extreme compression ratios, checked continuously, and what happens when one is exceeded. A consumer MUST bound package decompression, to prevent resource-exhaustion attacks by packages with extreme compression ratios. Two bounds apply, and both MUST be enforced. ## 5.27.1 The index-declared bound The repository index entry's `size_compressed` and `size_installed` fields (§5.33) bound the legitimate sizes of the compressed and uncompressed forms. During streaming decompression a consumer MUST verify that: - the cumulative compressed bytes consumed do not exceed `size_compressed` by more than **the lesser of 1% or 16 MiB**; and - the cumulative decompressed bytes produced do not exceed `size_installed` plus a fixed overhead allowance of **320 MiB**. Both figures MUST be taken from the **index entry**, not from the package's own manifest. The manifest lives inside the compressed stream and is therefore under the control of whoever produced the bytes being bounded. > [!NOTE] > This is the whole point of the bound, and it is easy to get backwards. > A consumer that reads `size_installed` out of the manifest it is in > the middle of decompressing has derived its cap from the input it is > defending against: a hostile package simply declares a large enough > figure. The index entry is signed by the repository, independently of > the package, which is what makes it usable as a bound. The 320 MiB decompressed allowance bounds the structural overhead a conforming package may legitimately carry above its installed payload: tar headers and block padding for up to the §5.A limit of 100,000 entries, plus the metadata files at their maximum sizes. A typical package's overhead is a tiny fraction of it; the allowance is sized so that a consumer never rejects a package conforming to §5.A. ## 5.27.2 The absolute cap Independently of any declared size, a consumer MUST abort decompression when the cumulative decompressed output exceeds an absolute cap. The default cap is **4 GiB**. A consumer MAY raise it through operator configuration but MUST NOT raise it silently. ## 5.27.3 Checked continuously Both bounds MUST be checked on **every chunk** of output, not at end-of-stream. > [!NOTE] > Constructed Zstandard payloads can achieve compression ratios beyond > 10,000:1, so a 10 MB package can decompress to terabytes. A check > deferred to end-of-stream never runs. ## 5.27.4 On exceeding a bound Exceeding either bound MUST cause the package to be rejected with no further processing and nothing committed to disk. ## 5.27.5 Cross-checking the declared size The manifest's `size_installed` and the index entry's `size_installed` MUST be equal, and a consumer MUST verify that equality (§5.32). Together with the files-manifest sum required by §5.18, this makes the figure a quantity all three of the producer, the repository, and the consumer can compute independently and agree on. --- # 5.28 Package Signatures _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What a signature binds, which bytes are signed, the envelope carrying it, and how an unsigned package is treated. A package signature binds a package's bytes to a signing key. Verifying it establishes that the package has not been altered since signing, and that the signer held the trusted private key. ## 5.28.1 The signature entry The signature is the final entry in the tar archive, at `.peipkg/signature` (§5.12). Its content is a UTF-8 JSON document. Every tar attribute of the entry — mode, owner, mtime, magic — follows the determinism rules of §5.11 unmodified, so the entry is mode `0777` like every other, and §5.16's rationale applies to it identically. ## 5.28.2 The signed bytes The signature is over a SHA-256 hash computed across the concatenation of every complete tar entry block — header, content, and content-block padding to the next 512-byte boundary — for every entry **preceding** `.peipkg/signature`, in archive order. The signed bytes do **not** include: - the tar entry header or content of `.peipkg/signature` itself; - the two trailing zero blocks that terminate a tar archive; - any compression artifact — signing operates on the uncompressed tar bytes. > [!NOTE] > Compressing the signed tar produces the on-wire file. Compression is > independent of signing: the same signed tar compressed at different > levels verifies identically once decompressed. This is what makes > §5.10's inability to fix compression parameters harmless for > authenticity. ## 5.28.3 The envelope ```json { "schema_version": 1, "algorithm": "ed25519", "key_fingerprint": "", "signature": "" } ``` | Field | Description | |---|---| | `schema_version` | MUST be 1 in this version. | | `algorithm` | MUST be `ed25519` in this version. | | `key_fingerprint` | Fingerprint of the public key (§5.29). Lowercase hex, 64 characters. | | `signature` | The signature value, base64 per RFC 4648 §4 without padding. For Ed25519 the decoded value is 64 bytes. | An envelope MUST contain all four fields. A missing field, or an unrecognised `algorithm` or `schema_version`, MUST cause the package to be rejected. ## 5.28.4 Strict parsing The envelope MUST NOT contain any field beyond the four above, and MUST NOT contain a duplicate key. An implementation of this version parsing an envelope from a future version MUST reject the package with an error **naming the schema version mismatch**, rather than silently ignoring the unknown fields. This is a deliberate exception to §5.9's forward-compatibility rule. For security-critical signing data, strict parsing is preferred to permissive ignoring. > [!NOTE] > The error must name the version, not the field. An implementation that > reports "unknown field `x`" when the real condition is "this envelope > is from a newer specification version" sends the operator looking for > a malformed package instead of an outdated tool. ## 5.28.5 Signing procedure To sign a package, a producer: 1. Constructs every tar entry except `.peipkg/signature`. 2. Serialises them as an uncompressed tar byte stream in archive order. 3. Computes the SHA-256 of that stream. 4. Signs the resulting 32-byte hash with its Ed25519 private key, per RFC 8032. 5. Constructs the envelope with the signature value and key fingerprint. 6. Appends the `.peipkg/signature` entry — header, JSON content, and padding — to the tar byte stream. 7. Compresses the complete stream to produce the `.peipkg` file. Note that the Ed25519 message is the 32-byte SHA-256 digest, not the tar stream itself. A verifier MUST do the same. > [!NOTE] > The digest indirection is deliberate and applies to every signature in > this chapter, package and detached alike: one signature construction > covers a small JSON index and a multi-gigabyte package, and the large > case is verifiable in a single streaming pass without buffering. ## 5.28.6 Determinism Given identical signed bytes and an identical key, the Ed25519 signature is deterministic per RFC 8032 §5.1.6. A producer that builds the same tar archive and signs it with the same key MUST produce a byte-identical signature entry. ## 5.28.7 Unsigned packages A package without a `.peipkg/signature` entry is **unsigned**. The format permits unsigned packages. A consumer MAY install one if the originating repository's trust policy permits it (§5.37). An unsigned package MUST conform to every other requirement of this chapter. The manifest, the files manifest, the payload rules, and the integrity rules apply identically to signed and unsigned packages. > [!NOTE] > The official repository requires signatures. Other repositories may > permit unsigned packages for development, homelab, or air-gapped use. > The format is permissive; the policy is enforced by the consumer, > per repository. --- # 5.29 Keys and Fingerprints _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Ed25519 keys — their encoding, how a fingerprint is computed, the key roles, and what makes up a repository's trust set. ## 5.29.1 Algorithm Signatures use Ed25519 as defined in RFC 8032. A conforming implementation MUST support Ed25519 signing and verification. Other algorithms are reserved for future versions. > [!NOTE] > Ed25519 was chosen for a small public key (32 bytes) and signature (64 > bytes), deterministic output with no nonce-reuse risk, fast > verification, resistance to most side channels, and broad library > support. ## 5.29.2 Public key encoding A public key is the raw 32-byte Ed25519 public key value of RFC 8032 §5.1.5. When published as a file, a public key MUST be encoded either as the raw 32 bytes, or as a PEM `PUBLIC KEY` block per RFC 7468 in the SubjectPublicKeyInfo form. Tooling MUST accept both. A published public key file MUST contain only the public key, in one of those two encodings. ## 5.29.3 Fingerprint A public key's fingerprint is the lowercase hexadecimal SHA-256 of the **raw 32-byte public key**: ``` fingerprint = lowercase_hex(sha256(public_key_bytes)) ``` The fingerprint is 64 hexadecimal characters. It is computed over the raw key bytes and never over a PEM or SubjectPublicKeyInfo encoding of them. The fingerprint is the canonical identifier of a public key throughout this chapter: the signature envelope's `key_fingerprint` (§5.28) and the repository descriptor's signing key declarations (§5.31) both use this form. A consumer that fetches a public key MUST verify the key's fingerprint against the fingerprint that identified it before admitting it to a trust set. > [!NOTE] > Fingerprints SHOULD be displayed to people in a form that makes a > mismatch visually obvious — conventionally four-character groups > separated by spaces or colons. The normative on-wire form remains the > unbroken 64-character lowercase hex string. ## 5.29.4 Key roles Two roles are distinguished by usage, not by structure: - **Signing keys** are used by a producer to sign packages, descriptors, and indexes. - **Trusted keys** are configured into a consumer as keys whose signatures it accepts. A single key MAY play both roles. ## 5.29.5 The trust set A consumer maintains a **trust set**: the public keys whose signatures it accepts. The trust set MUST be partitioned per repository. Each configured repository contributes its declared signing keys to the trust set, scoped to that repository's content. A signature MUST be accepted only if its `key_fingerprint` matches a key in the trust set **scoped to the repository the content was fetched from**. > [!NOTE] > Cross-repository acceptance is forbidden. A package fetched from > repository R and signed by a key trusted only for repository S MUST be > rejected, even though both keys are in the consumer's overall > configuration. Without this, a compromised low-trust repository could > serve packages whose signatures verify against a high-trust key, which > is an escalation from one context to the other. ## 5.29.6 Private keys Private key material is not the concern of this specification. Its generation, storage, custody, and rotation are operational matters for the key holder. Hardware security modules, threshold signing schemes, and air-gapped signing are all compatible with this format, so long as the resulting signature conforms to the envelope of §5.28. This specification cares about the bytes, not how they were produced. --- # 5.30 Verifying a Signature _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The streaming verification procedure, how the end of the signed range is located, and every condition that fails it. To verify a package's signature, a consumer MUST: 1. Decompress the `.peipkg` file to the uncompressed tar bytes. 2. Walk the tar archive in order, accumulating each entry's complete blocks — header, content, and content-block padding — until reaching the entry at path `.peipkg/signature`. 3. Stop at that entry. What has been accumulated is the signed byte range of §5.28. 4. Parse the content of `.peipkg/signature` as the signature envelope. 5. Validate the envelope's `schema_version` and `algorithm`. Reject if either is unrecognised, naming the version mismatch where that is the cause. 6. Look up the public key by `key_fingerprint` in the trust set scoped to the originating repository (§5.29). If no matching key is in that trust set, reject. 7. Determine whether the key is usable for verification given its status (§5.32). Reject a revoked key, and a transitioning key past its validity, **before** performing any cryptographic operation. 8. Compute the SHA-256 of the signed bytes. 9. Verify the signature against that hash with the looked-up key, per RFC 8032. If step 9 succeeds the signature is valid. If it fails, reject the package. > [!NOTE] > Step 7 before step 9 is deliberate. A revoked key's signatures are > rejected *regardless of cryptographic validity*, so checking status > first means a revoked key can never produce a "signature verified" > result anywhere in the implementation, even transiently. ## 5.30.1 Streaming The accumulation in step 2 is conceptual. An implementation MAY hash the signed bytes incrementally as it walks, without retaining the stream; steps 8 and 9 then operate on the running hash state. Streaming MUST NOT be conflated with early commitment. A consumer hashing incrementally MUST still defer every externally observable filesystem effect until step 9 has succeeded (§5.26). ## 5.30.2 Locating the end of the signed range A verifier computing the signed range by subtracting a fixed header size from a stream offset MUST account for an extended header block preceding the signature entry. §5.11 makes such a header unnecessary for a short-named entry, but a verifier that assumes it away will mis-locate the range for any archive that carries one, and report a signature failure for what is really a framing difference. ## 5.30.3 Failure conditions A consumer MUST reject a package as unverified when any of these holds: 1. The package contains no `.peipkg/signature` entry **and** the trust policy for its originating repository requires signed packages. 2. The `.peipkg/signature` entry is not the last named entry in the archive. 3. The envelope does not parse, or carries an unknown or duplicate field. 4. The envelope's `schema_version` is not 1. 5. The envelope's `algorithm` is not recognised. 6. The envelope's `key_fingerprint` matches no key in the trust set scoped to the originating repository. 7. The matching key's status does not permit verification. 8. The cryptographic verification fails. A rejected package MUST NOT be installed, and the consumer MUST report which condition triggered the rejection. > [!NOTE] > Condition 2 is not redundant with §5.12's ordering rule; it is the > same rule stated where its consequence is visible. Everything after > the signature entry is unsigned, so a consumer that tolerates a > trailing entry has accepted attacker-chosen bytes inside a package > whose signature verifies. ## 5.30.4 A package with no originating repository A consumer MAY accept a package supplied directly rather than fetched from a configured repository — a file handed to it on the command line. Such a package has no originating repository, and therefore no trust set to verify against. A consumer that accepts one MUST treat it as unverified: it MUST NOT report the package as signature-verified, and it MUST surface to the operator that the package's authenticity was not established. ## 5.30.5 What verification proves A verified signature establishes **integrity** — the archive bytes preceding the signature entry have not been altered since signing — and **authenticity** — the signer held the private key corresponding to a trusted public key at the time of signing. It does not establish that the signed bytes encode meaningful content: a consumer MUST still validate the manifest, the files manifest, and the per-file integrity (§5.26). It does not establish that the signer intended the package for any particular system. And it does not establish that the content is free of bugs or malice. Signing certifies provenance, not safety. ## 5.30.6 Replay and substitution Signature verification alone does not prevent replay — an attacker substituting an older, validly signed package for a newer one. Defence against substitution comes from the repository index (§5.33), which is itself signed, declares the current authoritative version of each package, and records each package's hash. A consumer MUST consult the index and verify the package's hash against it before accepting the package, even when the signature verifies. > [!NOTE] > An old signed package whose signature still validates against a > still-trusted key is otherwise indistinguishable from the current one. > The index's per-package hash binds "what is current" to "this exact > file", which is what closes the gap. --- # 5.31 The Repository Descriptor _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The small JSON document at a well-known URL that is a repository's entry point — its schema, signing keys, index pointers and canonical form. A repository descriptor is a small JSON document at a well-known URL describing a repository's identity, its signing keys, and where its indexes live. It is the entry point a consumer fetches when adding or refreshing a repository. ## 5.31.1 Location A repository's descriptor MUST be reachable at `/repo.json`, where `` is the base URL the repository was added under (§5.36). It MUST be served as static content. ## 5.31.2 Schema ```json { "schema_version": 1, "repo": { "name": "", "description": "", "signing": { "algorithm": "", "keys": [...] } }, "indexes": { "active": { "url": "", "signature_url": "" }, "archive": { "url": "", "signature_url": "" } } } ``` | Field | Description | |---|---| | `schema_version` | MUST be 1 in this version. | | `repo.name` | A short identifier for the repository. MUST be non-empty. SHOULD be kebab-case. | | `repo.description` | OPTIONAL. A human-readable one-line description. | | `repo.signing` | Signing key information. | | `indexes.active` | Pointer to the active index (§5.33). | | `indexes.archive` | Pointer to the archive index (§5.35). REQUIRED. | The archive pointer is required even when the archive is empty, as it is for a newly established repository. A repository without an archive index is non-conformant. ## 5.31.3 The signing object ```json { "algorithm": "ed25519", "keys": [ { "fingerprint": "", "url": "", "status": "active" } ] } ``` | Field | Description | |---|---| | `algorithm` | MUST be `ed25519` in this version. | | `keys` | One or more keys. MUST contain at least one with status `active`. | | Key field | Description | |---|---| | `fingerprint` | The key's fingerprint (§5.29): lowercase hex, 64 characters. | | `url` | Where the public key file is published. MAY be relative to ``. | | `status` | One of `active`, `transitioning`, `revoked` (§5.32). | | `valid_until` | RFC 3339 UTC timestamp after which a `transitioning` key MUST NOT be accepted. REQUIRED for `transitioning`; ignored otherwise. | The `keys` array MUST be sorted lexicographically by `fingerprint`. Two entries with the same fingerprint in one descriptor are invalid. ## 5.31.4 Index pointers | Field | Description | |---|---| | `url` | Where the index is published. MAY be relative to ``. | | `signature_url` | Where the index's detached signature is published. MAY be relative to ``. | The conventional URLs are: ``` /index/active.json /index/active.json.sig /index/archive.json /index/archive.json.sig ``` A repository MAY use other URLs by declaring them. The descriptor's URLs are authoritative; the conventional paths are defaults for tooling that has nothing else to go on. ## 5.31.5 Descriptor signing The descriptor MUST be accompanied by a detached signature published at `/repo.json.sig`. The detached signature is a signature envelope (§5.28) over the SHA-256 digest of the descriptor file's exact bytes — the same construction as a package signature. The signing key MUST be one of the keys listed in the descriptor's own `repo.signing.keys`, with status `active` or `transitioning`. > [!NOTE] > The descriptor signature defends against an attacker who can serve > content from `` substituting alternate signing keys. > Without it, whoever can substitute the descriptor can substitute the > keys, and from there re-sign the indexes and the packages. The > chicken-and-egg at first add is broken by the operator supplying an > expected fingerprint out of band (§5.37). A repository configured to permit unsigned content MAY publish an unsigned descriptor and unsigned indexes. This is a security weakening opted into per repository, and a consumer MUST NOT treat the absence of a signature as a fetch failure for such a repository. ## 5.31.6 Canonical form The descriptor SHOULD be canonically formatted so that signing is reproducible: fields in the schema's order, key arrays sorted as specified, no trailing whitespace, a single trailing newline. ## 5.31.7 Naming A consumer MAY refer to a repository by a local handle of its own choosing. When it does, it MUST NOT require that handle to equal `repo.name`, and MUST NOT compare an index's `repo` field against the local handle. An index's `repo` field is compared against the descriptor's `repo.name`. --- # 5.32 Signing Key Status and Rotation _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What each key status means for verification, how rotation works, the offline emergency key, and what happens on compromise. ## 5.32.1 Statuses A key's `status` describes its role in the repository's current operation. | Status | Used for new signatures | Accepted for verification | |---|---|---| | `active` | yes | yes | | `transitioning` | no | yes, until `valid_until` | | `revoked` | no | **never**, whatever the cryptography says | - **`active`** — the key currently signs new packages and indexes. A consumer MUST accept its signatures. - **`transitioning`** — the key was active and remains acceptable for verification until its `valid_until` timestamp, but no longer produces new signatures. A consumer MUST accept its signatures while the current time is at or before `valid_until`, and MUST reject them afterwards. A `transitioning` key entry MUST carry a `valid_until`. - **`revoked`** — the key is no longer trusted under any circumstance. A consumer MUST reject its signatures regardless of when they were produced and regardless of whether they verify cryptographically. A status other than these three is invalid. A repository MAY have several `active` keys, permitting parallel signing; any number of `transitioning` keys, each with its own `valid_until`; and any number of `revoked` keys. `revoked` is the explicit signal of a compromise event. `transitioning` is for routine rotation only, and the two MUST NOT be conflated. ## 5.32.2 Retention of revoked entries A revoked entry MUST be retained in the descriptor for at least one year after the revocation. Removing it prematurely would hide the public acknowledgement of compromise from consumers with stale caches. A repository MUST continue to serve the public key file of a revoked key for as long as its entry is retained, so that a consumer fetching the descriptor can resolve every key it declares. > [!NOTE] > Keeping a revoked key in the descriptor is useful even though its > signatures are rejected regardless: it is a public acknowledgement a > consumer can audit, and it means a consumer encountering a signature > from that key gets "this key was revoked" rather than a generic "key > not in trust set". ## 5.32.3 Rotation A repository rotates a signing key by: 1. Generating a new key pair. 2. Adding the new public key to the descriptor's key list alongside the existing one. 3. Beginning to sign new content with the new key. 4. After a transition period during which both are advertised, marking the old key `transitioning` with a `valid_until`, and eventually removing it. During the transition, content signed with either key is acceptable. After the old key's validity lapses, only the new key's signatures remain acceptable. The length of the transition period is operational policy and is not specified here. Its purpose is to give consumers time to fetch the updated descriptor and learn the new key before old signatures stop being honoured. ## 5.32.4 The offline emergency key A repository SHOULD maintain at least one **offline** active signing key in addition to its routine signing keys. The offline key's private material is stored separately from build infrastructure and is used only for descriptor updates and emergency rotations. The offline key exists to break a chicken-and-egg in compromise response. Revoking a compromised signing key requires publishing a new descriptor, which must itself be signed. If the only trusted key is the compromised one, the operator must sign the revocation with the compromised key — giving an attacker who holds that same key the ability to substitute their own revocation that adds a key of their choosing. With an offline key, the operator signs the descriptor update revoking the compromised key without relying on the compromised key at all. Consumers holding the offline key in their trust set accept the update; consumers who do not must perform an out-of-band trust-anchor refresh (§5.37). > [!NOTE] > This is SHOULD rather than MUST because the implementation question — > hardware module, air-gapped machine, threshold custody — is the > operator's. A future version may make it mandatory. ## 5.32.5 Compromise A key SHOULD be considered compromised if its private material may have been obtained by an unauthorised party. A compromised key MUST be marked `revoked` in the descriptor immediately on discovery. Packages signed with it SHOULD be re-signed with a fresh key and re-published at new revisions. The `revoked` status is the in-band revocation channel this specification defines. It defends at descriptor-update granularity: a consumer that successfully refreshes learns of the revocation at once, and a consumer caching an older descriptor retains trust in the revoked key only until it re-syncs — a window bounded by the maximum trusted age of §5.37. > [!NOTE] > A richer out-of-band mechanism — signed revocation lists, a key > transparency log — is reserved for a future version. The combination > of `revoked` status, a bounded maximum trusted age, and signed > descriptor updates closes the practical compromise-response window > without requiring separate revocation infrastructure. > > The corollary a consumer must not miss: **revocation only protects the > paths that consult the trust set.** Any code path that installs a > package without resolving its key against the repository's trust set — > a build or image-composition path, say — is a path on which revocation > has no effect at all. --- # 5.33 The Active Index _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > The index listing the current version of every advertised package — its schema, entries, derivation rule, and deliberate omissions. The active index lists the current version of every package a repository advertises. It is the index a consumer fetches on a routine sync. ## 5.33.1 Location and signing The active index's URL and its detached signature's URL are declared by the descriptor (§5.31). The index MUST be accompanied by a detached signature: a signature envelope (§5.28) over the SHA-256 digest of the index file's exact bytes. The signing key MUST be one of the descriptor's keys with status `active` or `transitioning`. A repository configured to permit unsigned content MAY publish the active index unsigned. > [!NOTE] > Detached metadata signatures and package signatures share a single > construction — an envelope over a SHA-256 digest — so a verifier > implements it once. The digest indirection is free for a small JSON > index and lets a multi-gigabyte package be verified in one streaming > pass. ## 5.33.2 Schema ```json { "schema_version": 1, "repo": "", "kind": "active", "index_version": , "generated_at": "", "packages": [...] } ``` | Field | Description | |---|---| | `schema_version` | MUST be 1 in this version. | | `repo` | The repository's name, matching `repo.name` in the descriptor. | | `kind` | MUST be `active`. | | `index_version` | A monotonically increasing positive integer identifying this index revision (§5.34). | | `generated_at` | RFC 3339 UTC timestamp of generation. | | `packages` | One entry per package currently advertised. | A consumer MUST verify that `repo` matches the descriptor's `repo.name` and that `kind` matches the index it requested. An archive index served in place of an active one MUST be rejected. ## 5.33.3 Package entries ```json { "name": "", "version": "", "architecture": "", "description": "", "license": "", "homepage": "", "default_root": "", "dependencies": [...], "optional_dependencies": [...], "conflicts": [...], "provides": [...], "replaces": [...], "side_effects": [...], "size_compressed": , "size_installed": , "hash": { "algorithm": "", "value": "" }, "url": "", "build": { "timestamp": "", "farm_id": "" } } ``` An entry MUST contain `name`, `version`, `architecture`, `dependencies`, `conflicts`, `provides`, `replaces`, `side_effects`, `size_compressed`, `size_installed`, `hash`, and `url`. The array fields MUST be present even when empty, emitted as `[]`. The remaining fields are RECOMMENDED and MAY be omitted. `name`, `version`, and `architecture` MUST each be validated against §5.3, §5.5, and §5.8 respectively on parse — with the same strictness a manifest receives. An index is fetched from the network and its values flow into URL construction and into the consumer's own records. `size_compressed` and `size_installed` are required because they are the input to the decompression bound of §5.27. `hash` carries `algorithm`, which MUST be `sha256` in this version, and `value`, the lowercase hexadecimal SHA-256 of the `.peipkg` file in its compressed on-wire form. ## 5.33.4 The derivation rule The active index is a **derived view** of the packages it advertises. Every field of an entry MUST exactly match the corresponding field of that package's manifest where one exists, and MUST exactly match the properties of the actual package file for `hash`, `size_compressed`, and `url`. Tooling generating an index MUST extract values directly from package manifests. Editing an index by hand is forbidden. Where a manifest contradicts an index entry, the manifest is authoritative (§5.18) — and the contradiction is a defect in the repository, not a difference to accommodate. A consumer MUST compare the downloaded package's manifest against the index entry that led to it, across **every** field the index carries, and MUST reject the package on any mismatch (§5.26 step 8). > [!NOTE] > Comparing only name, version, and architecture is not enough, and the > gap is not theoretical. A consumer builds its entire dependency and > conflict graph from index claims. A repository that publishes an entry > declaring no `conflicts` for a package whose manifest declares one > against a critical installed package gets a plan computed on the lie, > approved by the operator on that basis, and applied — with the real > relations discovered by nobody. The index is a convenience for > planning without downloading; it is not a second source of truth. ## 5.33.5 Deliberate omissions The index omits three manifest fields: - `sd_overrides` — not relevant to planning, and potentially large. - `build.source_ref` — long and low in information density; consult the package when it is wanted. - the manifest's own `schema_version` — the index carries its own. These remain in the manifest and are available to a consumer that fetches the package. Because they are omitted rather than mismatched, they are outside the comparison above. ## 5.33.6 URLs `url` declares where the package file is fetched from, and MAY be relative or absolute (§5.36). The conventional form is relative: ``` "url": "/p/nginx/1.26.2-3/nginx_1.26.2-3_x86_64.peipkg" ``` This keeps an index portable: the same file is valid at any `` hosting the same package files. ## 5.33.7 Ordering The `packages` array MUST be sorted lexicographically by `name`. Two entries with the same `name` in an active index are invalid: each name appears exactly once. > [!NOTE] > Per-name uniqueness is what makes the index "active" — one current > version of each package. The archive index (§5.35) relaxes exactly > this constraint and nothing else. ## 5.33.8 Unknown fields A consumer MUST ignore unknown fields, at the top level and per package, per §5.9. A producer MAY emit additional fields in a future schema version. The exception is a field whose meaning is critical to correctness, such as a hash algorithm identifier. Such changes are expected to arrive through a `schema_version` bump, not as a silent addition. ## 5.33.9 Size and caching For a repository of a few hundred packages the active index is on the order of 100 KB compressed. A consumer SHOULD fetch with HTTP-level compression where it is offered, and SHOULD cache the parsed index between invocations: the index changes only when the repository publishes, which is far less often than a consumer reads. A cached index MUST be stored under a security descriptor granting write access only to the principal permitted to install packages. A consumer MUST re-verify a cached index's signature on **every** operation that relies on it, rather than trusting its cached state across operations. Caching avoids re-parsing; it does not avoid re-verifying. A consumer SHOULD additionally cross-check a cached index against its own recorded freshness state (§5.34), and reject a cached index whose `index_version` or `generated_at` disagrees with what it recorded. > [!NOTE] > Without re-verification, whoever can write to the cache can substitute > metadata between the cache write and the next read. Re-verifying on > every use closes that race, and the cost is negligible: verifying a > signature over a few hundred kilobytes is sub-millisecond. The > cross-check against recorded state closes the matching hole, where an > attacker substitutes an older *validly signed* index directly into the > cache, bypassing the refresh path where §5.34's floor is enforced. --- # 5.34 Freshness and Rollback Protection _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > An index that verifies is not necessarily current — monotonic versions, staleness limits, and the defences against rollback and freeze. An index that verifies is not necessarily current. This section defends against **rollback** — replaying an older signed index to hide newer packages — and **freeze** — holding a consumer at a current-but-stale index while its clock runs on. Every requirement here applies to **both** the active index and the archive index. An attacker who can replay one can replay the other, and the archive index is the candidate source for every downgrade and pin. ## 5.34.1 Monotonic index versions Each publication of an index MUST set `index_version` to a value strictly greater than any previously published value for the same repository. A consumer MUST record, per repository, the highest `index_version` it has ever observed. On each fetch it MUST reject an index whose `index_version` is less than that recorded value, **even when the index is correctly signed by a still-trusted key**. A consumer MUST also record the `generated_at` of the last index it trusted, and MUST reject an index whose `generated_at` is older than the recorded value. ## 5.34.2 No progress is a failed fetch A fetch returning an index whose `index_version` **and** `generated_at` both equal the recorded values is a **failed** refresh, not a successful one. A consumer MUST NOT advance its "last successful refresh" timestamp on such a fetch. > [!NOTE] > This is the anti-freeze rule, and it is the one most often skipped. > Without it, an attacker who can serve the same signed index > indefinitely keeps every consumer's refresh timestamp advancing while > the content never changes, so the maximum-trusted-age check of §5.37 > never fires and the consumer never notices it has been pinned. ## 5.34.3 The initial floor Adding a repository bootstraps the consumer's recorded floor. To defend against an attacker serving a stale-but-signed index at that moment, a repository SHOULD distribute a **minimum acceptable `index_version`** alongside its trust anchors, through the same out-of-band channel. A consumer SHOULD use that minimum as its initial floor, and MUST refuse the add when the first index fetched falls below it. A consumer MUST NOT reset a recorded floor as a side effect of any operation other than removing the repository. In particular, re-adding an already-configured repository MUST NOT lower the floor: the operation either applies the recorded floor as a refresh would, or is refused. > [!NOTE] > Repository-add reads as idempotent, and configuration-management > convergence loops treat it that way. If adding an already-known > repository rewrites the floor unconditionally, a rollback that the > refresh path correctly refuses becomes permanent and invisible the > next time that loop runs. Removing the repository first is the > sanctioned reset (§5.37), because it discards the trust state too. ## 5.34.4 Maximum index staleness A consumer MUST enforce a maximum staleness window on the index itself, measured from its `generated_at`. An index older than **90 days** MUST trigger a refresh attempt before any install operation proceeds. The 90-day default MAY be tuned by operator configuration; a value greater than 365 days SHOULD generate a warning each time it is exercised. > [!NOTE] > This is a different measurement from the maximum trusted age of §5.37, > and both are needed. Trusted age asks "how long since I successfully > refreshed"; index staleness asks "how old is the metadata I am acting > on". A repository that bumps `index_version` on every publication > while stamping an ancient `generated_at` satisfies the first check > forever and fails the second immediately. ## 5.34.5 What these checks buy Per-package signing and index signing both still verify under a rollback: the attacker is replaying genuine, correctly signed content. What changes is the *set* of packages the consumer believes is current. The monotonic version check is what makes that set unable to move backwards, and the no-progress rule is what stops it from being frozen in place. --- # 5.35 The Archive Index _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Every version a repository has ever advertised, its retention and schema, and how it relates to the active index. The archive index lists every version of every package a repository has ever advertised, including versions superseded by newer releases. It is the source of historical data for downgrade, version pinning, and forensic queries. ## 5.35.1 Retention A repository MUST retain every package version it has ever advertised. Once a package has been published at version V, the repository MUST continue to make V fetchable indefinitely, and the archive index MUST continue to list it. > [!NOTE] > This is a deliberate departure from rolling-only models. Retention is > what supports rollback, reproducible deployment, security forensics, > and long-running systems held at an older version. A repository MAY retire pre-release or development versions under a stated retention policy. Retirement MUST NOT silently remove a package a consumer might be using, and SHOULD be coordinated with consumer notice. A pruned package MUST also be removed from the repository's package storage: the archive index MUST NOT reference a package file that is no longer fetchable. > [!NOTE] > Reasonable policies include retaining all stable releases > indefinitely, retaining pre-releases for a year after each successor, > or never pruning anything. This specification neither mandates nor > forbids pruning. ## 5.35.2 Location and signing The archive index's URL and its detached signature's URL are declared by the descriptor (§5.31). It MUST be signed under the same rules as the active index (§5.33). ## 5.35.3 Schema The top-level schema is identical to the active index (§5.33), except: - `kind` MUST be `archive`; - the `packages` array MAY contain several entries with the same `name`, at different versions. The per-package entry schema is identical. Each historical version contributes one entry. `index_version` semantics are identical, and every freshness and rollback requirement of §5.34 applies to the archive index exactly as it does to the active one. ## 5.35.4 Ordering The `packages` array MUST be sorted lexicographically first by `name`, then within a name by `version` **descending** per §5.6. The first entry for any name is its highest version; subsequent entries for that name are progressively older. Where two entries of one name share a version — differing only in architecture — the ordering between them MUST be total and MUST be stated by the producer's tooling, so that the file is reproducible. > [!NOTE] > Highest-first ordering puts the most recently shipped version of each > package at the top of its name group, so a consumer scanning for the > latest version satisfying a constraint can stop as soon as the > constraint is satisfied or exceeded. ## 5.35.5 Relationship to the active index For every entry in the active index there MUST be at least one entry in the archive index with the same `name`, `version`, `architecture`, and `hash`. The archive index is a superset of the active index. Equivalently: the active index is the per-name maximum projection of the archive index, where "maximum" is the highest version per name under §5.6. A repository publishing both indexes SHOULD publish them at the same `index_version` and `generated_at`, so that a consumer holding one has a usable floor for the other. ## 5.35.6 Fetch frequency The archive index is large compared to the active index — potentially many megabytes for a long-running repository. A consumer SHOULD fetch it only when it is needed: for a historical query, for a pin or a downgrade, or when its cached copy expires. A routine sync SHOULD fetch only the active index. A consumer SHOULD cache the archive index aggressively, since it changes only when a version is published or pruned. --- # 5.36 URL Conventions _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Every URL maps to a static file — the repository base, the conventional paths, sibling artifacts, hosting and network failure. Every URL in this chapter maps to a static file. The protocol requires no server-side computation, no dynamic response, and no content negotiation beyond optional HTTP-level compression. ## 5.36.1 The repository base A repository is identified by a base URL, ``. The base URL MUST be a syntactically valid HTTP or HTTPS URL per RFC 3986, and MUST NOT have a trailing slash: the well-known relative paths below are appended directly. HTTPS MUST be used, unless the consumer has been configured with an explicit **per-repository** insecure-transport allowance. There is no global form of that allowance, and its use MUST generate a per-operation warning. Enabling the allowance on a repository that has already been added MUST require explicit operator authorisation and MUST emit an audit event. Setting it as part of the initial add is covered by the operator's trust decision at that moment and requires no separate event beyond the add's own. > [!NOTE] > Insecure transport is intended for a trusted local network during > development. Relying on package signing alone for transport integrity > leaves the consumer exposed to traffic analysis and to > metadata-substitution attacks even when content verification succeeds. A consumer MAY additionally support a `file://` base URL for local development. A `file://` repository MUST be subject to the same per-repository allowance as an HTTP one: it is not HTTPS, and admitting it silently makes removable or network-mounted media a trusted source without the operator ever acknowledging it. ## 5.36.2 Conventional paths | Path | Content | |---|---| | `/repo.json` | Repository descriptor (§5.31) | | `/repo.json.sig` | Detached signature on the descriptor | | `/index/active.json` | Active index (§5.33) | | `/index/active.json.sig` | Detached signature on the active index | | `/index/archive.json` | Archive index (§5.35) | | `/index/archive.json.sig` | Detached signature on the archive index | | `/keys/.pub` | Public key file, named by full fingerprint | | `/p///` | Package file | A repository SHOULD use these paths unless it has a reason not to; when it does not, the descriptor declares the ones it uses. A consumer that knows only `` MUST be able to locate `repo.json` at the conventional path. The descriptor carries the URLs for everything else. ## 5.36.3 Package URLs ``` /p/// ``` where `` conforms to §5.3, `` is the full version string of §5.5, and `` is `__.peipkg`. ``` https://pkgs.peios.org/p/nginx/1.26.2-3/nginx_1.26.2-3_x86_64.peipkg ``` ## 5.36.4 Sibling artifacts The directory containing a package file MAY hold additional siblings for that version. These are reserved for future use and are not normative here: ``` /p///.debug.peipkg /p///.sbom.json /p///.attestation.json ``` A consumer conforming to this version MUST NOT attempt to fetch a sibling artifact. A producer MAY publish them; their meaning is defined by a future version. ## 5.36.5 Relative URLs A URL field in a descriptor or an index MAY be absolute or relative. - An absolute URL, carrying a scheme, is used as-is. - A URL beginning with `/` is resolved against `` by prepending the base. - A URL with neither a scheme nor a leading `/` is resolved against the URL of the document containing the reference, per RFC 3986 §5. > [!NOTE] > Relative URLs are RECOMMENDED: they keep an index portable, valid > under any `` hosting the same layout. An absolute URL pins > the index to a host and requires regeneration when the host changes. ## 5.36.6 Hosting A conformant repository may be hosted on a plain HTTP server, an object store with an HTTP frontend, a static site host, a CDN in front of any of those, or a combination — descriptor and indexes on a static host, package files on object storage behind redirects. ## 5.36.7 Network failure A consumer that fails to fetch a URL MUST NOT silently fall back to outdated cached data. Using a stale cache without explicit operator consent can mask substituted content or a revoked-key update. A consumer SHOULD offer a way to configure cache-staleness tolerance per repository. A consumer whose cached index for a configured repository fails to load or verify MUST treat that as a failure of the operation rather than proceeding without that repository. > [!NOTE] > Silently continuing is worse than it looks. Dropping a repository from > consideration mid-operation does not merely lose candidates: it can > silently promote a lower-priority repository's package into a role the > dropped one was filling, which is an escalation dressed as a warning. --- # 5.37 Establishing Trust in a Repository _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Trust is configured per repository and never globally — adding one, guarding a mistyped fingerprint, signature policy and refresh. Trust is configured **per repository**, never globally. Each repository a consumer is configured with has its own trusted signing keys (§5.29), its own signature policy, and its own priority. ## 5.37.1 Adding a repository To add a repository, the operator supplies its `` URL, one or more expected key fingerprints — the **trust anchors** — and a signature policy. The consumer then: 1. Fetches `/repo.json` and `/repo.json.sig`. 2. Fetches the public key for each supplied anchor fingerprint, from the conventional URL or from the URL the descriptor declares, and verifies each fetched key against the fingerprint that named it (§5.29). 3. Verifies the descriptor's signature against those anchor keys **and only those**. 4. On success, records the descriptor's contents — including every signing key and status — as the repository's initial trust state. 5. On failure, rejects the add and reports why. A consumer MUST NOT add a repository whose signing key it learned from the repository itself without prior verification against an anchor. Trust anchors are obtained out of band: through a project's website, its documentation, or the operating system image. > [!NOTE] > Step 2 necessarily issues requests derived from an unverified > document, since the descriptor is what names the key URLs. A consumer > SHOULD limit those requests to the keys matching the supplied anchors, > and MUST NOT follow a key URL for a fingerprint the operator did not > name — otherwise a substituted descriptor can direct the consumer to > fetch from arbitrary hosts before anything has been verified. ## 5.37.2 Guarding against a mistyped fingerprint When presenting a fetched key for confirmation, a consumer MUST: - display the 64-character fingerprint in groups separated by spaces or colons — conventionally four characters per group, as `1a2b 3c4d 5e6f ...`; - display the fetched key's fingerprint **alongside** the one the operator supplied, for visual comparison, before recording any trust state; - require explicit confirmation before recording. Automatic confirmation on the basis of a bit-for-bit match is permitted only in a non-interactive context where the operator pre-supplied the fingerprint through a configured channel. When a repository declares several `active` keys, the operator is RECOMMENDED to supply anchors for at least two of them, as defence-in-depth against a single mistyped anchor. A consumer MUST report an anchor mismatch by naming both the anchor the operator supplied and the fingerprints the descriptor actually declares. A mismatch is most often a transcription error, and that is precisely the diagnostic needed to find one. ## 5.37.3 Signature policy | Policy | Meaning | |---|---| | `required` | Every package and index MUST be signed and verify. Unsigned content from this repository is rejected. | | `optional` | Signed content is verified. Unsigned content is accepted with a per-operation warning. | These are the only two policies. There is no silently-accept-unsigned policy: a consumer intentionally permitting unsigned content does so through `optional`, which always warns. The warning MUST surface on **every** install, upgrade, and refresh that accepts unsigned content — not once per session — so that a misconfigured trust state stays continuously visible. A consumer's default policy for a newly added repository SHOULD be `required` unless the operator explicitly chooses otherwise, and the official repository SHOULD be configured `required`. `optional` means signed content **is** verified. A consumer MUST NOT treat the absence of trust anchors as licence to stop verifying: a repository under `optional` that publishes signatures MUST have them verified, and one that publishes none MUST produce the warning rather than a fetch error. > [!NOTE] > The two failure modes here are mirror images and both are real. A > consumer that demands a signature file under `optional` cannot add a > repository that §5.31 explicitly permits to publish none. A consumer > that stops verifying entirely because no anchors were configured > leaves a repository fully substitutable by anyone on the network path, > permanently, even after it starts publishing good signatures. ## 5.37.4 Refresh A consumer SHOULD refresh its cached repository state periodically. A refresh MUST: 1. Fetch the current descriptor and its signature. 2. Verify the signature against any key whose status was `active` or `transitioning` in the **previously trusted** descriptor. 3. On success, record the new descriptor as the current trust state, replacing the previous key set with the new one. 4. Fetch the active index and verify it against the new descriptor's keys, applying §5.34. 5. Optionally fetch and verify the archive index, applying §5.34 to it as well. A failed refresh MUST leave the previous trust state in place and be reported. A consumer MUST NOT fall back to unverified state. > [!NOTE] > A failed refresh may mean the repository is unavailable, the network > is interrupted, or the signing key was rotated to one the previously > trusted set does not contain. These are distinct operational concerns; > the consumer surfaces the failure and lets the operator distinguish > them. ## 5.37.5 Maximum trusted age A consumer MUST track the time of the last successful refresh per repository. When that exceeds the **maximum trusted age**, the consumer MUST attempt a refresh before any install, upgrade, or downgrade against that repository. If the attempt fails, the consumer MUST report the failure and refuse the operation, unless the operator explicitly authorises proceeding on stale trust state. The default maximum trusted age is **30 days**. It MAY be tuned by operator configuration; a value above 180 days SHOULD produce a per-operation warning, so that a configuration effectively disabling the check stays visible. > [!NOTE] > The maximum trusted age bounds the window in which a > compromised-but-not-yet-revoked key can be used against a consumer > that has not refreshed. Without it, a long-offline consumer trusts a > rotated key indefinitely. ## 5.37.6 Priority A consumer MAY configure several repositories. Each has a numeric priority: a positive integer, where a **lower number is a higher priority**. A consumer's default assignment SHOULD give the official repository the lowest number. Other repositories receive priorities at the operator's discretion. ## 5.37.7 Removal A consumer MAY remove a configured repository at any time. Removal deletes the cached state and the trust set scoped to that repository. It does **not** uninstall packages already installed from it; those remain installed, and their origin is retained. Re-adding a removed repository performs the full trust ceremony afresh; previous state is not implicitly restored. ## 5.37.8 Orphaned packages A package whose originating repository has been removed or revoked is **orphaned**: its trust chain is no longer verifiable by the current trust state. A consumer MUST: - display an orphaned package with a clear indicator in query output; - surface the orphan state on any operation involving it, and recommend an audit before proceeding; - refuse an upgrade to an orphaned package unless a currently trusted repository now claims it by name. A consumer MUST NOT treat an unknown origin as an *absent* origin. Wherever this chapter gates an operation on the relative priority of two repositories, an orphaned package's origin MUST be treated as at least as trusted as any configured repository, so that the gate still fires. > [!NOTE] > The failure this prevents is subtle and severe: a repository removed > *because its keys were stolen* leaves packages behind whose origin no > longer resolves. If an unresolvable origin is quietly treated as > lowest-priority, every cross-repository guard below stops firing for > exactly those packages — so revoking a repository would *lower* the > protection on what it left behind, and any newly added low-trust > repository could take an orphaned ex-official package over without > confirmation. Operators meeting an orphaned package SHOULD audit it: verify the installed files' hashes against trustworthy out-of-band records, and consider reinstalling or removing it through a trusted repository. ## 5.37.9 Between repositories When two configured repositories publish a package of the same name, no conflict exists at the format level; the consumer resolves which to install by priority. The same applies to overlapping `provides` or `replaces` relations: the higher-priority repository's claim wins. An operator publishing a `provides` that shadows a package of the official repository SHOULD document it clearly, and a consumer SHOULD warn when a lower-priority repository's `provides` shadows a higher-priority package. Two guards require explicit operator confirmation, and neither may be satisfied by a general "proceed" affirmation: - Applying a `replaces` declared by a lower-priority repository against a package originally installed from a higher-priority one. A repository silently replacing a more-trusted package is a real escalation path, and confirmation stops it happening as a side effect of a routine upgrade. - Applying a `conflicts` declared by a lower-priority repository that would cause the cascade-removal of a package from a higher-priority one. That is a denial-of-availability vector, and confirmation stops a low-trust install from silently uninstalling a high-trust package. A consumer that resolves a conflict by rejecting the plan outright, rather than by cascading removals, satisfies the second guard vacuously. ## 5.37.10 Compromise response If a repository's signing key is suspected of compromise, a consumer SHOULD disable the repository immediately, audit the packages installed from it for tampering, and, once the operator has published a new descriptor with the compromised key removed, perform a fresh trust-add with new anchors. This version defines no automated revocation mechanism beyond the `revoked` key status (§5.32). Compromise response is operational. --- # 5.38 Extension _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > What may be added to these documents without a version bump, what may not, the reserved space, and how deprecation works. Every document in this chapter carries a `schema_version`, currently 1. A change that a conforming implementation of this version cannot process correctly requires a version bump; a change it can safely ignore does not. ## 5.38.1 Additive changes The following are additive and do **not** require a version bump: - A new optional field in the manifest, an index entry, or the repository descriptor. A consumer ignores it (§5.9). - A new optional metadata entry under `.peipkg/` (§5.12). A consumer ignores it, and its presence MUST NOT prevent installation. - A new sibling artifact alongside a package file (§5.36). A producer emitting an additive extension MUST ensure that a consumer ignoring it still behaves correctly. An extension whose omission changes what gets installed is not additive. ## 5.38.2 Changes requiring a version bump - Adding, removing, or changing the meaning of a **required** field. - Adding a value to a closed enumeration: the side-effect identifiers of §5.24, the architecture identifiers of §5.8, the hash algorithms of §5.25, the signature algorithms of §5.28, the key statuses of §5.32, the index kinds, the constraint operators of §5.7, or the signature policies of §5.37. A conforming implementation of this version MUST reject a value outside each of those sets, so a new value is not ignorable. - Any change to the version comparison algorithm of §5.6, which is frozen. - Any change to the determinism rules of §5.11, which decide the bytes. - Any change to the signature envelope of §5.28, which is strictly parsed by construction. ## 5.38.3 Reserved space This version reserves syntactic room in three places, so that a future extension can be additive where it would otherwise not be: - The `algorithm` fields of the files manifest and the index hash object reserve room for a further hash algorithm. - The `arch` qualifier on a dependency reserves room for explicit architecture identifiers, for a multi-architecture system. - The sibling-artifact paths of §5.36 reserve room for build attestations and bills of material. An implementation of this version MUST reject a value in a reserved space rather than guess at it. ## 5.38.4 Deprecation A field this specification requires MUST NOT be removed within a `schema_version`. When a field becomes unnecessary, a producer continues emitting it and a future version removes it under a new `schema_version`. --- # 5.39 Conformance _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Every requirement collected by role — producer, repository and consumer — and what conformance deliberately does not require. ## 5.39.1 Producer A conforming producer: - emits packages satisfying §5.10 through §5.17: the container, every determinism rule, the internal layout, the payload path constraints, the install destinations, the triplet rule, the entry rules, and the symlink rules; - emits a manifest satisfying §5.18, with names, versions, and architectures satisfying §5.3 through §5.8; - emits a files manifest satisfying §5.25, covering exactly the regular-file payload entries, with `size_installed` equal to the sum of its sizes; - declares relationships satisfying §5.21, sorted and unique within each field, using the derived-capability names of §5.22 where a capability is machine-derived; - declares claims satisfying §5.23, with every target a payload path of its own; - declares side effects satisfying §5.24, declaring each that its payload requires and none that it does not; - signs packages satisfying §5.28, or emits them unsigned knowing they will be accepted only under a permissive policy. ## 5.39.2 Repository A conforming repository: - publishes a descriptor satisfying §5.31 with a valid detached signature, and serves the public key file of every key the descriptor declares, including revoked ones (§5.32); - publishes an active index satisfying §5.33 and an archive index satisfying §5.35, each with a valid detached signature, each derived directly from package manifests; - increases `index_version` strictly on every publication (§5.34); - retains every version it has ever advertised, and removes a pruned package from both its archive index and its storage (§5.35); - serves everything over HTTPS at the URLs its descriptor declares (§5.36). ## 5.39.3 Consumer A conforming consumer: - verifies a package by §5.26 in full, including the transaction-wide rule and the path-resolution rule, before installing anything; - enforces the decompression bounds of §5.27 continuously, from the index-declared sizes; - verifies signatures by §5.30, against a trust set scoped to the originating repository, honouring key status before any cryptography; - rejects a package violating any rule of §5.10 through §5.25 — the determinism rules and the payload rules included, on the way in, not only on the way out; - enforces the freshness and rollback rules of §5.34 on both indexes, and never lowers a recorded floor except by removing the repository; - establishes and maintains trust by §5.37, including the fingerprint comparison, the per-operation warnings, the maximum trusted age, the orphan rules, and the two cross-repository guards; - enforces the security descriptor policy of §5.20; - invokes side effects by §5.24, once per transaction, by fixed absolute path, with a cleared environment, against the root the transaction acted on; - materialises claims by §5.23, and never at a path an installed package owns. ## 5.39.4 What conformance does not require A conforming consumer is not required to resolve dependencies by any particular algorithm, to store its state in any particular form, to recover from an interrupted operation by any particular mechanism, or to offer any particular command surface. Those are its own design, and §5.1 places them outside this chapter deliberately. What it *is* required to do is reach the same answer as any other conforming consumer about whether a given package satisfies a given dependency (§5.21), and about which of two versions is newer (§5.6). Those two questions are the ones a producer's declarations depend on, and they are frozen. --- # Appendix 5.A Limits and Defaults _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Every limit as a minimum conformance figure — package structure, manifest arrays, identity, documents, decompression and repository defaults. Every limit below is a **minimum conformance** figure: a consumer MUST process a package or document whose characteristics fall within it, and MUST reject one that exceeds it. A consumer MAY raise a limit through operator configuration, but MUST NOT raise one silently: an operator-tuned value SHOULD be logged and surfaced in diagnostic output. A producer SHOULD stay well below these figures. They exist to bound a consumer's resource use when processing a maliciously crafted package, not to describe the scale of a well-formed one. ## 5.A.1 Package structure | Limit | Maximum | |---|---| | Payload entries | 100,000 | | `.peipkg/manifest.json` size | 16 MiB | | `.peipkg/files.json` size | 64 MiB | | `.peipkg/signature` size | 64 KiB | | Single payload path component (UTF-8 bytes) | 255 | | Complete payload path (UTF-8 bytes) | 4096 | | Path nesting depth (components) | 256 | | Single claim path (UTF-8 bytes) | 4096 | ## 5.A.2 Manifest arrays | Limit | Maximum | |---|---| | `dependencies` | 10,000 | | `optional_dependencies` | 10,000 | | `conflicts` | 10,000 | | `provides` | 10,000 | | `replaces` | 1,000 | | `sd_overrides` | 100,000 | | Single `sd_override` decoded `sd` length | 64 KiB | | Slots per `claims` field | 64 | | Claim paths materialised per role | 256 | The claim-path figure is a **materialisation** limit, not a manifest limit: it bounds the union computed across every installed package declaring a path for that role, which is the quantity an adversary controls by installing many consumer-only packages. ## 5.A.3 Identity | Limit | Value | |---|---| | Package name length | 2 to 64 characters | | Virtual name length | 2 to 128 characters | | Architecture identifier length | at most 16 characters | ## 5.A.4 Documents | Limit | Value | |---|---| | JSON nesting depth | 64 | | Integer field range | unsigned 64-bit | ## 5.A.5 Decompression | Bound | Value | |---|---| | Compressed overrun allowance over `size_compressed` | the lesser of 1% or 16 MiB | | Decompressed overhead allowance over `size_installed` | 320 MiB | | Absolute decompressed cap | 4 GiB (default; operator-tunable) | ## 5.A.6 Repository defaults | Default | Value | |---|---| | Maximum trusted age | 30 days | | Maximum trusted age producing a warning | above 180 days | | Maximum index staleness | 90 days | | Maximum index staleness producing a warning | above 365 days | | Revoked key retention | at least 1 year | | Repository priority | positive integer; lower is higher priority | | Default signature policy for a new repository | `required` | --- # Appendix 5.B Enumerated Values _Peios / Advanced Peios / PSPU / Package Format and Repository Protocol_ > Every closed set in this version — architectures, pre-release ranks, constraint operators, side-effect identifiers and hash algorithms. Every set below is **closed** in this version. A conforming implementation MUST reject a value outside it, and a new value requires a `schema_version` bump (§5.38). ## 5.B.1 Architecture identifiers | Identifier | Triplet | Notes | |---|---|---| | `x86_64` | `x86_64-linux-peios` | primary target | | `aarch64` | `aarch64-linux-peios` | secondary target | | `noarch` | none | architecture-independent | Defined in §5.8. ## 5.B.2 Pre-release rank tokens | Token | Rank | |---|---| | `dev` | 0 | | `alpha` | 1 | | `a` | 1 | | `beta` | 2 | | `b` | 2 | | `pre` | 3 | | `rc` | 4 | | any other alphabetic segment | 5 | Rank 0 sorts lowest. Rank-5 tokens compare lexically against each other. Recognition is case-insensitive. Defined in §5.6. ## 5.B.3 Constraint operators | Operator | Meaning | |---|---| | `=` | exactly equal | | `>` | strictly greater than | | `>=` | greater than or equal | | `<` | strictly less than | | `<=` | less than or equal | | `!=` | not equal | A bare version with no operator means `=`. Comma is the AND separator. Defined in §5.7. ## 5.B.4 Side-effect identifiers | Identifier | Declared when | Invoked as | |---|---|---| | `depmod` | the payload contains kernel modules (MUST) | once per affected kernel release, naming it | | `man-db` | the payload contains man pages (SHOULD) | the tool, in quiet mode | Defined in §5.24. ## 5.B.5 Hash algorithms | Algorithm | Identifier | Status | |---|---|---| | SHA-256 | `sha256` | REQUIRED; the only valid value | | BLAKE3 | `blake3` | RESERVED for a future version | Defined in §5.25. ## 5.B.6 Signature algorithms | Algorithm | Identifier | Status | |---|---|---| | Ed25519 | `ed25519` | REQUIRED; the only valid value | Defined in §5.29. ## 5.B.7 Signing key statuses | Status | Signs new content | Accepted for verification | |---|---|---| | `active` | yes | yes | | `transitioning` | no | until `valid_until` | | `revoked` | no | never, regardless of cryptographic validity | Defined in §5.32. ## 5.B.8 Index kinds | Kind | Content | |---|---| | `active` | the current version of each package | | `archive` | every version ever shipped | Defined in §5.33 and §5.35. ## 5.B.9 Signature policies | Policy | Unsigned content | |---|---| | `required` | rejected | | `optional` | accepted with a per-operation warning | There is no silently-accept-unsigned policy. Defined in §5.37. ## 5.B.10 Reserved metadata paths | Path | Required | |---|---| | `.peipkg/manifest.json` | yes | | `.peipkg/files.json` | yes | | `.peipkg/signature` | in every signed package | The `.peipkg/` prefix is reserved; a payload entry MUST NOT use it. Defined in §5.12. ## 5.B.11 Permitted entry types | Type | Typeflag | |---|---| | Regular file | `0` or `\0` | | Directory | `5` | | Symbolic link | `2` | Every other type MUST cause the package to be rejected. Defined in §5.12. ## 5.B.12 Permitted top-level install destinations `/usr/bin/`, `/usr/sbin/`, `/usr/lib//`, `/usr/lib/debug/`, `/usr/lib/modules//`, `/usr/lib/firmware/`, `/usr/lib/os-release`, `/usr/libexec/`, `/usr/share/`, `/usr/include/`, `/usr/src/debug/`, `/usr/src/dist/`, `/usr/etc/`, `/usr/conf/`, `/var/`, `/boot/`, `/hooks/`, `/++/`. A payload entry MUST NOT install under any other top-level path, unless the package declares itself a special system package **and** the operator has separately opted in. `/lcl/policy` is unreachable under every circumstance. Defined in §5.14. ## 5.B.13 Permitted claim path locations The destinations above, plus `/run/` and the well-known root-level name `/init`. Defined in §5.23.