# eventd

> Technical Reference Manual Proposal — the observability daemon as designed: event, log and metric ingestion, three storage engines, adaptive acceleration, and query execution.

---

# 1.1 Overview

_Peios / Advanced Peios / eventd / Introduction_

> eventd is the single persistent sink for everything a Peios system records about itself — the shape of the daemon, and what it is not.

eventd is the observability daemon: the single persistent sink for
everything a Peios system records about itself. Events, logs and metrics
all end in eventd, and every query for any of them is answered by it.

It is one of the platform daemons the service manager starts at boot,
signed at TCB level, and it is Critical — a system that loses it loses
its audit trail.

The three data types are genuinely different and eventd treats them
differently at every layer.

**Events** are structured, typed records carrying identity stamps the
kernel applied and an emitter could not influence. They arrive through
KMES, in per-CPU shared-memory ring buffers, and eventd is their primary
consumer. They are the audit and security telemetry, and losing one is a
real failure — so the event path is the one with sequence numbers, gap
detection, per-transaction durability, and a synthetic record written
whenever anything is lost.

**Logs** are text: a line a program wrote, with light metadata attached.
They arrive on a datagram socket, mostly from the service manager
forwarding what it read from a service's standard output and standard
error. Losing one is an inconvenience, and the ingestion path is
designed around that tolerance rather than against it.

**Metrics** are numeric measurements over time — dense series rather
than discrete occurrences. They arrive on a second datagram socket,
pushed by whatever is doing the measuring. eventd is a sink, not a
collector: it scrapes nothing and polls nothing.

## 1.1.1 The shape of the daemon

Three ingestion paths, three storage engines, one query surface.

The event path runs one drain thread per CPU, each attached to one ring
buffer, handing events over a bounded channel to a writer thread that
batches them into a SQLite shard. Shards are independent — separate
files, separate write-ahead logs, separate writer threads, no shared
write-path state — so write throughput scales with the shard count
(§2.3).

The log and metric paths each run a single thread that reads datagrams
and writes them, to one database each. Neither contends with the event
path.

Underneath all three sits a decision that shapes the event pipeline
entirely: **the KMES ring buffers are the only buffer.** eventd holds no
large intermediate queue. Events move from the ring buffer through a
small bounded handoff straight into a transaction, and when the writer
falls behind, backpressure propagates backwards until the ring buffer
absorbs it — and when the ring buffer cannot, the loss is detected and
recorded rather than hidden (§2.5).

Two subsystems adapt to the workload rather than being tuned for it.
Adaptive indexing watches which fields queries filter on and maintains
indexes for them, shedding those indexes under write pressure because
throughput outranks query latency (§3.4). Adaptive rollups pre-compute
the metric aggregations that are asked for often (§5.6).

Everything eventd holds is readable only through access checks that KACS
performs, per event type, per log origin, per metric name, and per field
within a record (§7).

## 1.1.2 What eventd is not

**It is not a log framework.** eventd stores lines; it does not parse
them, does not understand severity beyond a single error flag, and does
not care whether the text happens to be JSON.

**It is not a metric collector.** Nothing in eventd reads `/proc`,
scrapes an endpoint, or polls a service. Something else measures and
pushes.

**It is not a tracing system.** Distributed tracing is out of scope
entirely.

**It is not the low-latency path to events.** A consumer that needs
events in microseconds attaches to the KMES ring buffers directly, as
`revstrm` does. eventd sits above that transport, adds persistence and
access control, and costs a batch commit interval in latency.

**It is not the only KMES consumer**, and holds no privileged position
among them.

---

# 1.2 What This Manual Is

_Peios / Advanced Peios / eventd / Introduction_

> A TRM proposal for software that has not been written — what in it is a contract, and how versions and constants are treated.

This is a **Technical Reference Manual Proposal**. eventd has not been
written.

Everything in this manual is therefore a description of a design rather
than of an artifact: the schemas, the thread structure, the algorithms,
the constants and the failure behaviour are what eventd is specified to
do, not observations of what it does. Nothing here has been checked
against an implementation, because there is none to check against.

It is written in the indicative mood, exactly as a reference manual for
existing software would be, and it makes no conformance demands. When
the code lands, this document becomes eventd's Technical Reference
Manual — corrected wherever the implementation and the design turn out
to disagree, and with no change in kind. That is the whole reason for
the proposal form: a design document that will be read as a manual is
better written as one from the start.

The distinction a reader needs to keep is that **a TRM's authority comes
from the software and this one has none of that yet**. Where a statement
here is surprising, it is a design decision that has not survived
contact with an implementation.

## 1.2.1 What is a contract and what is not

eventd's three external interfaces are specified separately, in PSPU §3:
the log ingestion channel, the metric ingestion channel, and the query
channel with its query language. Those are contracts, binding on
anything that speaks them, and they are normative where this manual is
not.

This manual covers the other side of that boundary — how eventd fulfils
them:

- how it consumes KMES and what it does when it falls behind (§2)
- what it stores, in what schema, and how it accelerates and expires it
  (§3, §4, §5)
- how a query in the language of PSPU §3 becomes an answer (§6)
- how access decisions are reached (§7)
- how it starts, stops, and behaves when something breaks (§8, §9)

Where a chapter touches the contract, it references PSPU §3 rather than
restating it, and describes only what eventd adds.

The access control **mechanism** is here rather than in PSPU because it
is behind the abstraction: a client sees only which records and fields
it received (PSPU §3.28). The field GUID derivation in §7.3 is the one
part of it that a third party may need to reproduce — an administrator
writing a Security Descriptor computes the same GUIDs — and it is a
candidate for promotion into a specification if that turns out to be a
common thing to do.

## 1.2.2 Versions and constants

Constants, configuration keys and catalogues are collected in the
appendices at the end of the manual, so that a chapter's reference
material does not interrupt the prose that explains it. Where an
appendix defines a value, the body references it rather than repeating
it.

---

# 1.3 Terminology

_Peios / Advanced Peios / eventd / Introduction_

> Terms this manual borrows unchanged from elsewhere in the corpus, and the ones it introduces.

Terms defined elsewhere are used with the same meaning and are not
redefined here: event, header, payload, stamp, ring buffer, consumer,
origin class and sequence number from the KMES chapters of the Peios
Kernel TRM and PSPK; token, GUID, SID, Security Descriptor, ACL, ACE and
privilege from the Peios Kernel TRM and PCDS; registry, hive, key, value
and layer from the Peios Kernel TRM; producer, client, log record,
metric sample, time series and concrete identifier from PSPU §3.2.

Syscall numbers and signatures for the kernel interfaces eventd calls —
`kmes_attach`, `kacs_open_peer_token`, `kacs_access_check` and
`kacs_access_check_list` — are in the Peios Kernel TRM's generated ABI
appendices, §2.A and §3.A. This manual names them and does not repeat
their numbers.

The following are specific to eventd.

**Drain thread**: one of the threads that reads from a per-CPU KMES ring
buffer. There is exactly one per CPU (§2.2).

**Writer thread**: the sole writer to one event shard. There is exactly
one per shard, and no other thread writes to that database (§2.3).

**Shard**: one of the independent SQLite databases the event store is
split across, each with its own file, write-ahead log and writer thread.
A shard is a write-path construct only; the query path treats the whole
directory as one store (§2.3).

**Active shard**: a shard in the current configuration's numbering.
**Historical shard**: a shard database left behind by a previous
configuration, opened read-only and still queried (§3.3).

**Handoff channel**: the bounded queue between drain threads and a
writer thread. It is a staging area for the current batch, not a buffer
(§2.3).

**Synthetic event**: a record eventd generates itself and writes
directly to a shard, bypassing KMES. Synthetic events carry no identity
stamps and no sequence numbers, and are distinguished by a
`synthetic.`-prefixed type (§2.6).

**Gap record**: the synthetic event recording that events were lost on
one CPU, and which sequence numbers went missing (§2.5).

**Event store directory**: the directory holding every shard database
and the metadata database. **Metadata database**: `eventd-meta.db`, the
one database that is not a shard and survives shard reconfiguration
(§3.5).

**Desired index set**: the global, priority-ordered list of fields eventd
aims to have indexed across all shards. **Material indexes**: the indexes a
given shard actually has, which converge toward the desired set when the
shard is quiet and diverge from it under pressure (§3.4).

**Shedding**: dropping secondary indexes to protect write throughput
(§3.4).

**Rollup**: a pre-computed metric aggregate for one series, one
function and one time window (§5.6). **Rollup registry**: the global set
of (function, window) pairs being pre-computed.

**Series cache**: the bounded in-memory map from series identity to
series row, which keeps metric ingestion off SQLite in the common case
(§5.3).

**Logical live size**: `(page_count - freelist_count) × page_size` for a
SQLite database — the space actually holding data, excluding pages freed
by deletion and available for reuse. Retention is enforced against this
rather than against the file size (§3.6).

**Quarantine**: renaming a database SQLite has reported corrupt, aside
from the path eventd uses, and creating an empty one in its place
(§3.3).

---

# 1.4 Prior Art

_Peios / Advanced Peios / eventd / Introduction_

> Where eventd sits against Windows Event Log, ETW, journald, Prometheus and OpenTelemetry, and what it deliberately leaves to others.

eventd is not a port of anything. It occupies a position several
existing systems also occupy, and differs from each of them in ways
worth being explicit about. PSPU §3.C compares the wire contracts; this
compares the systems.

## 1.4.1 Windows Event Log and ETW

The closest structural match. Windows splits the job in two: Event
Tracing for Windows delivers events from kernel and application
providers, and the Event Log service persists and serves them. Peios
splits it the same way, with KMES in the ETW position and eventd in the
Event Log service's.

Held in common: kernel-mediated delivery with metadata the emitter
cannot forge, a userspace service owning persistence, and access control
by Security Descriptor on a named channel — which in eventd becomes a
descriptor per event type pattern (§7.2).

Different: eventd unifies three data types where Windows separates them
across the Event Log, ETL trace files and Performance Counters. eventd
stores in SQLite rather than a proprietary binary format. And ETW's
buffering is a kernel-managed trace session, where KMES exposes
shared-memory ring buffers with a lock-free consumer protocol that
eventd drains directly (§2.2).

## 1.4.2 journald

journald is the systemd system journal: it captures service output and
structured messages and stores them in an indexed binary journal.

Held in common: a single daemon for system-wide log ingestion, capturing
standard output and standard error as records with metadata attached,
stored in a binary format with indexes.

Different: journald is log-only, and a systemd system needs a separate
stack for metrics. journald's access control is Unix file permissions
and polkit, where eventd's is KACS Security Descriptors evaluated per
record and per field (§7). And journald reads kernel messages from
`/dev/kmsg`, a text interface with no identity, where eventd receives
kernel events through KMES with the kernel's own identity stamps intact.

The similarity worth noting is that journald's storage is also its
index, and its query surface is a matcher over fields rather than a
language. eventd goes further in that direction — an actual query
language, over all three data types — for the reason PSPU §3.C gives:
computing on the collector's side is what lets access control constrain
the computation.

## 1.4.3 Prometheus and OpenTelemetry

Prometheus is a pull-based metrics system with a local time-series
database; OpenTelemetry is a vendor-neutral collection framework
spanning traces, metrics and logs.

eventd's metric store fills roughly the role of a local Prometheus TSDB,
and takes the Prometheus data model — a name plus labels identifying a
series, cumulative-bucket histograms — more or less wholesale (§5.2).

Different: eventd is pushed to rather than scraping. It implements no
distributed tracing at all. And where OpenTelemetry's answer to the
three-signal problem is a common collection framework in front of three
backends, eventd's is one backend with one access control model and one
query surface — which is the whole design bet.

## 1.4.4 Deliberately elsewhere

| Concern | Where it lives |
|---|---|
| Event emission, buffering and delivery | KMES, in the Peios Kernel TRM; the consumer protocol in PSPK |
| Event type vocabulary and payload schemas | the emitting subsystem's own documentation |
| Access control primitives | KACS, in the Peios Kernel TRM |
| Configuration storage | LCS and loregd |
| Daemon lifecycle, and forwarding service output | peinit |
| The three interfaces eventd exposes | PSPU §3 |

---

# 2.1 The Pipeline

_Peios / Advanced Peios / eventd / Event Ingestion_

> The four stages from shared memory to a committed row, why the ring buffers are the only buffer, and how sharding scales writes.

eventd is the primary consumer of the KMES ring buffers. Events travel
from shared memory to a committed database row in four stages.

1. **Drain.** One thread per CPU reads events from that CPU's ring
   buffer, following the lock-free read protocol PSPK specifies (§2.2).
2. **Detect.** The drain thread compares each event's sequence number
   against the last it saw for that CPU. A jump means events were lost,
   and the loss becomes a gap record (§2.5).
3. **Hand off.** The drain thread copies the event out of the mapped
   region and passes it to the writer thread that owns the shard it
   routes to (§2.3).
4. **Write.** The writer thread accumulates events into a transaction
   and commits, sizing the batch to whatever throughput allows (§2.4).

Two principles govern the whole pipeline, and most of its behaviour
follows from them rather than from anything specific to a stage.

## 2.1.1 The ring buffers are the only buffer

eventd holds no large intermediate queue between KMES and SQLite. The
handoff channel is bounded by the maximum batch size and nothing else
accumulates.

When the writer falls behind, the channel fills; when the channel is
full, the drain thread stops reading; when the drain thread stops
reading, events accumulate in the ring buffer, which is exactly what a
ring buffer is for. Backpressure propagates all the way back to the
kernel, and the absorption capacity is the ring buffer's, which an
administrator already sizes.

If the ring buffer also fills, KMES overwrites its oldest events, the
drain thread notices the sequence jump when it resumes, and the loss is
recorded (§2.5). That is the designed worst case: **eventd loses events
visibly rather than buffering without bound and dying**.

The alternative — a large in-process queue — would move the same
capacity into a place where losing it is invisible, where it competes
with the page cache for memory, and where an out-of-memory kill takes
the whole queue with no record that it existed.

## 2.1.2 Sharding scales writes linearly

Each shard is a self-contained SQLite database with its own file, its
own write-ahead log and its own writer thread. Shards share no
write-path state, so the write path has no cross-shard lock, no shared
counter and no coordination point (§2.3).

The consequence for the query path is that a shard means nothing to it.
A shard database holds whatever CPUs happened to route to it in whatever
eventd lifetime wrote it, so a query filtering by CPU reads every shard,
and the query path never assumes a relationship between a shard and a
CPU (§6.4).

---

# 2.2 KMES Consumption

_Peios / Advanced Peios / eventd / Event Ingestion_

> Discovering CPUs and attaching to their rings, the drain threads, copying, generation changes and sequence tracking.

## 2.2.1 Attachment

At startup eventd discovers the CPU count by calling `kmes_attach` with
incrementing CPU identifiers from 0 until the call returns `EINVAL`.
Each successful call returns one file descriptor for that CPU's ring
buffer, and eventd maps each one. The mapping size is derived from the
`capacity` value the call reports, as PSPK defines it.

Attachment requires SeSecurityPrivilege in the effective token, which is
the privilege that grants an unfiltered view of every event on the
system. eventd holds it because it is the party that then applies
per-event access control on everything it stores (§7).

Discovering zero CPUs is a startup failure (§8.2). There is no
configuration for the CPU count and no way to attach to a subset.

## 2.2.2 Drain threads

There is one drain thread per CPU, and each reads exactly one ring
buffer. A drain thread never reads another CPU's buffer, which is what
makes per-CPU sequence tracking a thread-local variable rather than
shared state.

Each thread follows the read protocol PSPK specifies:

- `read_pos` starts at `tail_pos` on first attachment, so eventd begins
  at the oldest surviving event rather than at the newest.
- The drain loop loads `write_pos` with acquire ordering, checks
  `tail_pos` for lapping, validates the event's structural integrity,
  and advances `read_pos` by `event_size`.
- After reading an event it re-reads `tail_pos` — the torn-read check —
  to detect that KMES overwrote the event while it was being copied.
- With nothing to read it uses the notification protocol, setting
  `need_wake` and waiting on the futex, rather than spinning.

## 2.2.3 Copying

A drain thread copies the event — header and payload — into
process-local memory before it advances `read_pos`.

Nothing derived from the mapped region ever reaches a writer thread. The
region is producer-owned and KMES may overwrite any part of it the
moment `read_pos` moves past, so a pointer handed across the channel
would be a pointer into memory that another CPU is entitled to rewrite
before the writer gets to it.

The copy is bounded by the event's own `event_size`, and the drain
thread never reads beyond it.

## 2.2.4 Generation changes

An administrator changing the ring buffer capacity causes KMES to
replace the buffers, which it signals by changing the `generation`
field. A drain thread checks it after each drain cycle, and on a change:

1. Records the sequence number of the last event it processed.
2. Calls `kmes_attach` again for its CPU, obtaining a descriptor for the
   resized buffer.
3. Maps the new buffer.
4. Unmaps the old one and closes the old descriptor.
5. Scans the new buffer for the first event whose sequence number
   exceeds the recorded one.
6. Resumes draining from there.

Each drain thread handles this independently. There is no barrier, no
coordination and no shared state, because each attaches only to its own
CPU's buffer — so a resize is a per-CPU event that happens to occur on
every CPU at roughly the same time.

The scan in step 5 is what makes the transition lossless in both
directions: no event is skipped, and none is written twice.

## 2.2.5 Sequence tracking

Each drain thread holds the last sequence number it saw for its CPU. It
serves gap detection (§2.5) and resumption after a generation change or
a restart.

At startup, if committed rows already exist for the current boot, the
resume point for each CPU is derived from those rows:

```sql
MAX(sequence)
WHERE boot_id = current_boot_id
  AND cpu_id = cpu
  AND sequence IS NOT NULL
```

across **every readable event shard database**, historical shards
included. Historical shards can hold current-boot rows: an eventd
restarted within one boot under a smaller shard count leaves its
higher-numbered shards behind, and the events it wrote to them before
the restart are still that boot's events.

The `sequence IS NOT NULL` clause excludes synthetic events, which have
no sequence numbers (§2.6).

Committed rows are the authority. The metadata database holds sequence
checkpoints and the shutdown event records the same numbers (§3.5,
§8.4), but both are diagnostic: a checkpoint can be stale in exactly the
case that matters, where eventd died between its last commit and its
last checkpoint, and trusting it would mean re-reading events already
stored or skipping a gap.

With no prior rows for the current boot, the tracker starts at 0. The
first event on each CPU carries sequence number 1, so a tracker at 0
expects 1 and detects a gap correctly if the first event it sees is
later.

---

# 2.3 Sharding

_Peios / Advanced Peios / eventd / Event Ingestion_

> Event writes distributed across up to 256 independent databases — the count, the assignment, the writer threads and the handoff.

## 2.3.1 The shard count

Event writes are distributed across one to 256 independent SQLite
databases. The count comes from `StorageShards` (§A); zero means "as
many shards as there are CPUs", and is the default.

Two properties make a count perform well, and neither is enforced. A
power of two lets routing use a bitwise AND rather than a modulo. A
multiple of the CPU count distributes shards evenly across CPUs. The
default satisfies the second by construction.

## 2.3.2 Assignment

Shard-to-CPU assignment is computed once at startup and is fixed for the
process lifetime.

For each CPU `c`, eventd assigns every shard `j` where
`j % cpu_count == c`. If that produces nothing — which happens when
there are fewer shards than CPUs — CPU `c` is instead assigned
`c % shard_count`. Every CPU ends with at least one write path.

The three cases behave differently:

| Relation | Result |
|---|---|
| shards == CPUs | one shard per CPU, the 1:1 case |
| shards < CPUs | several CPUs share a shard |
| shards > CPUs | a CPU owns several shards |

A drain thread owning several shards distributes its events round-robin,
sending each successive event to the next shard it owns.

When the counts do not divide evenly, some CPUs carry one shard more
than others, or some shards receive from one CPU more than others. The
resulting imbalance is one shard's worth of throughput, which is
negligible against the whole.

## 2.3.3 Shards are not a query-path concept

Assignment is not persisted. A shard database accumulates events from
whatever CPUs routed to it during whatever eventd lifetimes wrote it, so
a single shard file may hold events from a different set of CPUs in
different regions of its history.

The query path therefore assumes nothing: it reads every database in the
directory, and a query filtering on `cpu_id` scans all of them (§6.4).
Sharding is a write-path optimisation that the read path pays a fan-out
for.

## 2.3.4 Writer threads

Each shard has exactly one writer thread, and that thread is the only
writer to that database. No other thread and no other connection writes
to it, which is what makes the single-writer assumptions in §5.3 and
§3.4 safe.

Drain threads never write to SQLite. When several drain threads share a
shard they hand off concurrently, so the handoff channel is
multi-producer and single-consumer.

## 2.3.5 The handoff channel

Each writer thread has one bounded channel through which drain threads
submit events. Its capacity does not exceed the maximum batch size
(§2.4).

When the channel is full the drain thread **stops reading from the ring
buffer** and waits. It does not drop events to relieve the pressure and
it does not grow the channel. Events accumulate in the ring buffer
instead, which is the designed path (§2.1):

```text
writer slow → channel fills → drain pauses → ring buffer absorbs
            → KMES overwrites oldest if full → gap detected on resume
```

When the writer commits and the channel has room, the drain thread
resumes immediately.

The channel is a staging area for one batch, not a second buffer. Its
bound is the batch size precisely so that it cannot become one.

## 2.3.6 Lifecycle and reconfiguration

Shard databases are created in the event store directory on first use.
eventd never deletes or overwrites one left by a previous configuration:
starting with fewer shards than exist leaves the excess in place, and
the query path continues to read them (§3.3).

Changing `StorageShards` takes effect at the next restart. The
configuration watch notices the change and eventd defers it rather than
reassigning CPUs or creating shards while running (§8.3).

Shard count changes are expected to be rare — set once from the hardware
profile, one for a small board and a multiple of the CPU count for a
server, and then left alone. Live migration would mean rebalancing
writer threads and channels while events are in flight, for a
configuration change that happens once in a machine's life.

---

# 2.4 The Batch Writer

_Peios / Advanced Peios / eventd / Event Ingestion_

> How each writer thread commits — explicit transactions, adaptive batch sizing, WAL checkpointing and prepared statements.

## 2.4.1 Transactions

Each writer thread writes to its shard with explicit transactions: a
`BEGIN`, one `INSERT` per event, a `COMMIT`. The commit is the
durability boundary.

The database runs in WAL mode with `synchronous = FULL`, so every commit
fsyncs the write-ahead log. This is the strictest of the three stores'
settings, and the only one where per-transaction durability is bought at
per-transaction cost — because an event may be an audit record and
losing the last second of them to a power cut is a real loss.

## 2.4.2 Adaptive batch sizing

The writer sizes each batch to balance throughput against how much sits
uncommitted at any moment.

**Throughput is always the priority.** If eventd falls behind the
emission rate, ring buffers fill and events are overwritten, which is
irrecoverable; a shorter power-loss window is not worth that trade. The
algorithm maximises resilience *within* the constraint that throughput
is maintained, never against it.

1. When the first event is available, the writer opens a transaction and
   records the start time.
2. It reads available events from its drain threads and inserts them.
3. After each group of inserts it commits if any of these holds:
   - no assigned drain thread currently has an event available
   - the batch holds `MaxBatchSize` events
   - `MaxBatchLatencyMs` has elapsed since the first event entered it
4. Otherwise it keeps reading and inserting.
5. With nothing available and an empty batch, it sleeps until a producer
   wakes it.

The first condition is what makes the algorithm adaptive. Under light
load the input drains immediately, so a batch of three events commits at
once and the exposure window is microseconds. Under sustained load
batches grow until they hit the size cap or the latency cap, whichever
comes first, and the per-commit fsync is amortised across thousands of
rows.

The writer chooses its own insert-group size, subject to a group never
letting a batch exceed the size cap or stay open past the latency cap.

Both bounds are configuration (§A). The defaults are 10000 events and
100 milliseconds — the tightest latency of the three stores, for the
same reason the durability setting is the strictest.

## 2.4.3 WAL checkpointing

WAL mode accumulates log data until a checkpoint copies it back into the
main database file. Under sustained writes the log grows.

Each writer triggers a checkpoint when its write-ahead log reaches
`WalCheckpointPages` (§A), in `SQLITE_CHECKPOINT_PASSIVE` mode —
checkpointing as much as it can without blocking readers. If a passive
checkpoint cannot make progress because readers hold pages, the writer
does not block: it keeps writing and retries after a later commit.

Checkpointing runs on the writer thread and briefly serialises with
insert work, which is inherent to SQLite rather than a choice — a
database cannot be checkpointed and written concurrently. Passive mode
is the lightest option available, yielding immediately when readers hold
pages, and the per-checkpoint cost is bounded by the threshold.

## 2.4.4 Prepared statements

Each writer prepares its `INSERT` once at startup and reuses it for
every row, which keeps SQL parsing and planning off the hot path
entirely.

---

# 2.5 Gap Detection

_Peios / Advanced Peios / eventd / Event Ingestion_

> Per-CPU sequence numbers are the whole mechanism — what causes a gap, what a gap record holds, and how lapping is detected.

Every event carries a per-CPU, per-boot sequence number, and a drain
thread knows what it last saw. That is the whole mechanism: an event
whose sequence number exceeds the expected next one means the numbers in
between belonged to events eventd never received.

## 2.5.1 Causes

- **Ring buffer overrun.** KMES overwrote events before eventd read
  them. The most serious case: it means audit events were lost
  irrecoverably.
- **Structural drops.** KMES declined to write an event that exceeded
  its size limits.
- **Downtime.** Events emitted while eventd was not running.

All three appear identically at the consumer, which is why the record
says what was lost rather than why.

## 2.5.2 Gap records

On detecting a jump, the drain thread generates a gap record carrying:

- the CPU identifier
- the first missing sequence number, the last seen plus one
- the last missing sequence number, the revealing event's minus one
- the count of missing events
- the timestamp of the last event successfully processed on this CPU,
  where one is known
- the timestamp of the event that revealed the gap

The record is written into the shard database through the normal write
path — handed to the same writer thread, batched with ordinary events,
committed in the same transaction. It is not emitted through KMES, which
would be circular: the mechanism for recording that the event transport
lost something cannot depend on that transport.

The gap details are stored as a MessagePack map in the `payload` column
(§3.2), and gap records are queryable exactly like any other event.

## 2.5.3 The CPU column

A gap record populates `cpu_id` and leaves the other KMES header columns
— `sequence`, `origin_class`, the identity GUIDs — null (§3.1).

`cpu_id` is populated deliberately, and it is the one place a synthetic
event carries a header field. Without it, `EVENTS WHERE cpu_id == 3`
would return every event from CPU 3 *except* the record saying that
events from CPU 3 went missing — which is the one record such a query
most needs to return.

`sequence` stays null because a gap record has no place in the sequence:
it describes numbers that were skipped, and giving it one of them would
make it indistinguishable from the event that was lost. It is also what
keeps gap records out of the resume-point derivation in §2.2.

## 2.5.4 Lapping

If `read_pos` falls behind `tail_pos`, the consumer has been lapped and
the PSPK read protocol advances it to `tail_pos`.

Lapping needs no special handling here. The next event read is the
oldest survivor, its sequence number is far beyond what the thread
expected, and ordinary gap detection records the difference. The
lapping case and the restart case produce the same record by the same
path.

---

# 2.6 Synthetic Events

_Peios / Advanced Peios / eventd / Event Ingestion_

> Records eventd generates about itself, written straight to a shard and never through KMES — when, where and in what order.

Synthetic events are records eventd generates about itself. They are
written straight to a shard database and never touch KMES.

They carry no KMES header: no identity stamps, no sequence number, no
origin class. What they have is a wall-clock timestamp taken when eventd
generated the record, and an event type string prefixed `synthetic.`
which is what distinguishes them in the `events` table — no separate
record-type column exists (§3.1).

## 2.6.1 When they are generated

| Condition | Type |
|---|---|
| Lost events detected on a CPU | `synthetic.gap` (§2.5) |
| eventd started and attached to KMES | `synthetic.startup` |
| Graceful shutdown beginning | `synthetic.shutdown` |
| A write to any store failed | `synthetic.storage_error` |
| A configuration value changed at runtime | `synthetic.config_change` |

Payload schemas for all five are in §3.2.

Note what is absent: there is no synthetic event for malformed
ingestion input. Log and metric datagrams arrive unauthenticated from
arbitrary local processes, and emitting a durable record per bad
datagram would hand every process an amplification primitive
(PSPU §3.4). These five are conditions eventd observed about itself, not
reactions to what it was sent.

## 2.6.2 Which shard

**CPU-specific** synthetic events — gap records — go to the shard
assigned to the CPU that generated them, handed to that writer thread
alongside that CPU's ordinary events. A gap record travels with the
events it describes.

**Daemon-wide** ones — startup, shutdown, configuration changes, storage
errors — go to shard 0 when shard 0 is writable, and otherwise to the
lowest-numbered writable active shard. If no shard is writable at all
the event is skipped, with the failure logged to standard error.

A storage error is the case that needs the fallback. It describes a
failure on one particular shard but is itself a daemon-wide
notification, so it is not written to the failing shard unless that
shard has since been replaced and is writable again (§9.2) — writing the
record of a shard's failure into that shard would lose it exactly when
it matters.

These events are infrequent enough that concentrating them on shard 0
costs nothing measurable in balance.

## 2.6.3 Storage and ordering

Synthetic events live in the same shard databases as KMES events and
participate in the same batching, the same retention and the same
queries. Access control treats their types like any other (§7.2), so
`Machine\System\eventd\Security\Events\synthetic` governs them.

They are ordered by their eventd-assigned timestamp and take no part in
per-CPU sequence numbering.

That timestamp is when eventd **noticed**, not when the condition
occurred. A gap record is stamped at detection, which may be long after
the events it describes were overwritten — and after a restart, may be
the first thing written in a new boot about events lost in the previous
one.

---

# 3.1 The Events Table

_Peios / Advanced Peios / eventd / Event Storage_

> One table per shard with every KMES header field as its own column, the untouched payload blob, and the write-time indexes.

Every shard database holds one `events` table.

| Column | Type | Contents |
|---|---|---|
| `id` | INTEGER PRIMARY KEY | SQLite rowid, monotonic within the shard. |
| `boot_id` | BLOB NOT NULL | 16-byte boot ID GUID in PCDS binary layout. |
| `timestamp` | INTEGER NOT NULL | Nanoseconds since the Unix epoch. From the KMES header for a real event; eventd's clock at generation for a synthetic one. |
| `cpu_id` | INTEGER | From the KMES header. Null for daemon-wide synthetic events; populated for gap records (§2.5). |
| `sequence` | INTEGER | Per-CPU, per-boot sequence from the KMES header. Null for every synthetic event. |
| `origin_class` | INTEGER | 0 userspace, 1 KMES, 2 KACS, 3 LCS. From the header. Null for synthetic events. |
| `event_type` | TEXT NOT NULL | From the header; or a `synthetic.`-prefixed string. |
| `effective_token_guid` | BLOB | 16-byte GUID for the effective token at emission. Null for synthetic events; the null GUID when identity was unavailable at emission. |
| `true_token_guid` | BLOB | 16-byte GUID for the process's primary token. Null for synthetic events. |
| `process_guid` | BLOB | 16-byte GUID for the emitting process. Null for synthetic events. |
| `payload` | BLOB | MessagePack. For a KMES event, the raw payload bytes exactly as received. For a synthetic event, a map (§3.2). Null when the event carries none. |

## 3.1.1 Header fields are columns

Every KMES header field is extracted into its own column rather than
left inside the payload blob. That is what lets a predicate on
`process_guid` or `event_type` become a SQL comparison rather than a
decode of every candidate row, and it is what makes those fields
indexable by ordinary column indexes (§3.4).

`event_type` is the sole discriminator between real and synthetic
records. No record-type column exists, because the `synthetic.` prefix
already partitions the type namespace and a second column would be a
second thing to keep consistent.

## 3.1.2 The payload is not touched

For a KMES event the payload column holds the bytes KMES delivered,
unmodified. eventd does not decode them on the write path, does not
re-encode them, and does not validate them beyond what the ring-buffer
protocol already checked.

The payload is a MessagePack value whose schema belongs to the emitting
subsystem, and eventd has no catalogue of those schemas. It decodes on
the *read* path when a query needs a payload field (§6.1), which is also
the only point at which the flattening rules of PSPU §3.22 apply.

Storing the bytes verbatim is also what keeps a payload field that
collides with a header name recoverable: the value is suppressed from
the query surface but remains in the blob.

## 3.1.3 Identity may be absent two ways

`effective_token_guid` distinguishes two cases that would otherwise
look alike. **Null** means the record is synthetic and never had an
identity. The **null GUID** — sixteen zero bytes — means the record is a
real KMES event whose identity was not available at emission time,
because it was emitted before or outside a context that had one.

The distinction matters for audit: "eventd wrote this" and "the kernel
emitted this and could not attribute it" are different facts.

## 3.1.4 Write-time indexes

One index is created with the table:

- `idx_events_timestamp` on `events(timestamp)`

Time-range filtering is the foundational access pattern — nearly every
query carries a `SINCE` — and it is the one index eventd never sheds,
whatever the write pressure (§3.4). Every other index is the adaptive
system's business.

## 3.1.5 Schema version

Each shard holds a `metadata` table:

| Column | Type | Contents |
|---|---|---|
| `key` | TEXT PRIMARY KEY | Metadata key. |
| `value` | TEXT NOT NULL | Metadata value. |

with two required entries: `schema_version`, and `created_at` as a UTC
timestamp formatted `YYYY-MM-DDTHH:MM:SSZ`. The current version is in
§B.

eventd checks the version at startup and applies the lifecycle rules of
§3.3. It does not migrate: an unrecognised version is a startup failure
for an active shard and an exclusion for a historical one. Migration is
an administrative operation, deliberately not an automatic one — a
daemon that silently rewrote an audit store's schema on first start
after an upgrade would be doing the one thing an audit store must not do
unattended.

---

# 3.2 Synthetic Event Payloads

_Peios / Advanced Peios / eventd / Event Storage_

> The MessagePack schema of each of the five synthetic event types, whose field names are stable query-language surface.

Each of the five synthetic event types (§2.6) carries a MessagePack map
in `payload`, with the schema below. These field names are stable
query-language payload field names after flattening (PSPU §3.22), except
where a value is a nested array or map, which flattening does not
traverse.

## 3.2.1 `synthetic.startup`

| Field | Type | Contents |
|---|---|---|
| `boot_id` | string | The current boot ID, PCDS canonical GUID form. |
| `restart` | bool | True when committed rows for this boot already existed at startup; false on the boot's first eventd start. |
| `shard_count` | unsigned integer | Active shard count after resolving `StorageShards`. |
| `resume_points` | array of map | One entry per CPU, ordered by `cpu_id` ascending. Each has `cpu_id` and `sequence`, both unsigned integers. |

`restart` is the boot-boundary decision of §3.7 recorded as data, which
makes "did eventd crash during this boot, and how often" answerable by
query rather than by inference from gaps.

## 3.2.2 `synthetic.shutdown`

| Field | Type | Contents |
|---|---|---|
| `last_sequences` | array of map | One entry per CPU, ordered by `cpu_id` ascending. Each has `cpu_id` and `sequence` — the last committed sequence for that CPU this boot, or 0 if none was. |

Diagnostic only. Startup derives its resume points from committed rows,
never from this payload (§2.2).

## 3.2.3 `synthetic.gap`

| Field | Type | Contents |
|---|---|---|
| `cpu_id` | unsigned integer | Where the gap was detected. |
| `first_sequence` | unsigned integer | First missing sequence number. |
| `last_sequence` | unsigned integer | Last missing sequence number. |
| `count` | unsigned integer | How many are missing. |
| `last_seen_timestamp` | timestamp or nil | The last event successfully processed before the gap, when known. |
| `revealing_timestamp` | timestamp | The event or ring position that revealed the gap. |

`cpu_id` appears both here and in the `cpu_id` column (§2.5). The column
is what a `WHERE cpu_id == N` predicate matches; the payload field is
what a reader of the record sees without joining anything.

## 3.2.4 `synthetic.config_change`

| Field | Type | Contents |
|---|---|---|
| `key` | string | The key name, relative to `Machine\System\eventd\`. |
| `old_value_type` | string | `absent`, `REG_SZ`, `REG_DWORD`, `REG_QWORD` or `REG_BINARY`. |
| `old_value` | string or nil | The previous value rendered as below; nil when the type is `absent`. |
| `new_value_type` | string | The same five. |
| `new_value` | string or nil | The new value; nil when `absent`. |

Values are rendered deterministically so that two eventd instances
observing the same change record the same bytes: `REG_SZ` as the string
in UTF-8, `REG_DWORD` and `REG_QWORD` as unsigned decimal without
leading zeroes, `REG_BINARY` as lowercase hexadecimal, two digits per
byte.

Everything is a string, including numbers, because the field is the same
field for all five types and a query filtering `WHERE key == "…"` should
not have to know which.

## 3.2.5 `synthetic.storage_error`

| Field | Type | Contents |
|---|---|---|
| `store` | string | `event`, `log`, `metric` or `metadata`. |
| `shard_index` | unsigned integer or nil | The shard for event-store errors; nil for the other three. |
| `error` | string | Human-readable description. |

`error` is diagnostic text and its wording is not stable. `store` and
`shard_index` are the fields worth alerting on.

---

# 3.3 Database Lifecycle

_Peios / Advanced Peios / eventd / Event Storage_

> The event store directory — naming, creation, opening an active shard, quarantine and historical shards.

## 3.3.1 The event store directory

Every shard database and the metadata database live in one directory,
named by `EventStorePath` (§A). There is no compiled-in default: a
missing or invalid value is a startup failure, and eventd writes event
databases nowhere else.

eventd creates the directory if it is absent.

> [!NOTE]
> Nothing in eventd's design constrains the protection on that
> directory. The three sockets get Security Descriptors (PSPU §3.3), but
> the store paths are ordinary configuration, and the metadata database
> inside this directory holds the descriptor governing administrative
> operations (§3.5). A process that can write the directory can rewrite
> that descriptor.

## 3.3.2 Naming

Active shards are `shard-NNNN.db`, with the shard index zero-padded to
four digits — the index assigned at startup, which is not a CPU number
(§2.3).

Starting with more shards than exist creates the new ones. Starting with
fewer leaves the excess in place: they become historical shards, are
never deleted, and remain available to the query path.

## 3.3.3 Creation

A shard database that does not exist is created with:

1. WAL mode, `PRAGMA journal_mode=WAL`
2. `PRAGMA synchronous=FULL`
3. the `events` and `metadata` tables (§3.1)
4. the `idx_events_timestamp` index
5. the `schema_version` and `created_at` entries

## 3.3.4 Opening an active shard

1. Open in WAL mode.
2. Set synchronous to FULL.
3. Read and verify `schema_version`. Missing or unrecognised is a
   **startup failure**. No migration is attempted.
4. Verify structural integrity — the required tables and write-time
   indexes exist. Failing this, with SQLite reporting no corruption, is
   a **startup failure**.
5. If SQLite reports corruption while opening or verifying, quarantine
   and replace (below).

Steps 3 and 4 fail rather than repair because an active shard is
required: eventd has no degraded mode that runs without one (§8.2).

## 3.3.5 Quarantine

When SQLite reports corruption in a required store, eventd renames the
database aside and starts a fresh one at the original path. The main
database file and any matching `-wal` and `-shm` files are renamed with
the suffix `.corrupt.<timestamp_ns>`, all three using the same suffix
from one operation, and a new empty `shard-NNNN.db` is created.

If a target name is taken, eventd appends `.N` with the lowest positive
integer that makes it unique — which happens when two quarantines land
in the same nanosecond, and when a previous quarantine already used the
name.

Quarantining rather than deleting is the point: the corrupt file is the
only copy of whatever it held, recovering data from it is an
administrative operation, and eventd attempts no automatic repair.

The corruption is logged and a `synthetic.storage_error` event is
emitted once a shard is available to write it to (§9.2).

## 3.3.6 Historical shards

A historical shard is never required for startup. If one has a missing
or unrecognised schema version, fails structural verification, or cannot
be opened read-only, eventd logs the error and **excludes it from the
query path for this run** — it does not fail startup and does not
quarantine it.

The asymmetry is deliberate. An active shard that will not open means
eventd cannot do its job; a historical one that will not open means some
old data is unreadable, which is a smaller problem than refusing to boot
the audit daemon over it.

## 3.3.7 Query path discovery

The query path opens every file in the directory matching
`shard-NNNN.db` that has a recognised schema and passes structural
verification — active and historical alike. It assumes no particular
number of them.

It explicitly does **not** treat every `.db` file in the directory as a
shard: `eventd-meta.db` is excluded by the naming pattern, along with
anything else that happens to be there.

Each is opened with a read-only connection. Read-only connections in WAL
mode do not contend with the writer's connection.

## 3.3.8 Concurrency

Each shard has exactly one read-write connection, owned by its writer
thread, and any number of read-only connections owned by query handlers.
WAL mode permits concurrent readers alongside one writer without
blocking either.

Writer threads never share connections. Each creates and owns its own
for the process lifetime, which is what makes the prepared statement in
§2.4 a per-thread object with no locking around it.

---

# 3.4 Adaptive Indexing

_Peios / Advanced Peios / eventd / Event Storage_

> Which secondary indexes are worth their write cost depends on what a deployment queries — how eventd decides, converges and sheds.

Secondary indexes make queries fast and writes slow. Which indexes are
worth that trade depends on what a particular deployment actually
queries, which varies between systems and over time, and which nobody
wants to tune by hand.

eventd observes the queries and maintains the indexes they imply —
subject throughout to the rule that **throughput outranks query
latency**.

## 3.4.1 Three decoupled parts

**Query frequency counters.** Query handlers increment a per-field
counter when a field appears in a `WHERE` predicate. This is the
write-heavy path — once per query, per predicate. Counters live in
memory and are flushed periodically to the metadata database (§3.5),
never to a shard, because they are global state that must survive shard
reconfiguration.

**Index policy.** A periodic process reads the counters, applies the
creation and removal thresholds, and computes the **desired index set**
— an ordered list of fields, highest priority first. It runs every
`AdaptiveIndexPolicyIntervalMinutes` (§A), and it is the only writer to
the desired set.

**Shard convergence.** Writer threads read the desired set and move
their material indexes toward it. They never read the counters and never
write the desired set.

The separation exists so that the high-frequency counter updates never
contend with the writer threads. The policy is the bridge between them
and runs on the order of once an hour, so it is never a contention
point.

The desired set is **global** — one list applying to every shard.
Individual shards do not make independent decisions; they differ only in
how far they have got.

## 3.4.2 Convergence

A shard converges when it is quiet. When a writer thread has no pending
events and its material indexes do not match the desired set, it takes
**one** convergence action — creating the highest-priority missing
index, or dropping the lowest-priority material index no longer wanted —
then rechecks write pressure before considering another.

Creation uses `CREATE INDEX IF NOT EXISTS`; removal uses
`DROP INDEX IF EXISTS`. Both run on the shard's writer thread, which is
the only thread permitted to write that database (§2.3).

**Index creation is cancellable.** If drain threads detect rising write
pressure during a build, they signal the writer to abort; the writer
cancels the `CREATE INDEX`, SQLite rolls back the partial index cleanly,
and the writer returns to event batches immediately. The abandoned build
is retried at the next quiet period.

Cancellation responsiveness matters more than it looks. `sqlite3_interrupt`
sets a flag checked at SQL VM opcode boundaries, and during B-tree
construction for a large index the gap between checks can be tens of
milliseconds — long enough to overrun a ring buffer at a high event
rate. `sqlite3_progress_handler()`, registering a callback invoked every
thousand opcodes that checks a cancellation flag and returns non-zero to
abort, gives cancellation that tracks the pressure signal rather than
lagging it.

Shards converge at their own pace. One under sustained pressure may lag
the desired set indefinitely, and that is the correct outcome: it is
prioritising throughput.

## 3.4.3 Shedding

Under sustained pressure a shard drops indexes to cut per-insert cost.

**Graduated shedding.** If more than `SheddingBatchPercent` of a shard's
batches within a `SheddingWindowSeconds` sliding window exceeded 75% of
`MaxBatchSize`, the shard drops its lowest-priority secondary index —
the one whose column has the lowest query frequency in the desired set.
If pressure persists, the next-lowest goes, and so on. The check runs
once per batch commit.

**Emergency shedding.** If a shard is at maximum batch size and its
drain thread signals rising ring buffer pressure, it drops **all**
secondary indexes at once. `DROP INDEX` is a metadata operation
measured in milliseconds, so it is safe to do under pressure in a way
that creation is not.

The pressure signal comes from the drain thread watching the gap between
`write_pos` and its own `read_pos`. Exceeding
`EmergencySheddingBufferPercent` of ring buffer capacity raises it. It
is a distinct signal from the index-build cancellation one: this
triggers shedding whether or not a build is in progress.

`idx_events_timestamp` is **exempt**. It is never shed at any pressure,
because time-range queries are the access pattern everything else is
built on and the store is unusable without it.

When pressure subsides, shedding reverses: the shard rebuilds toward the
desired set under the same quiet-period scheduling and the same
cancellability, highest priority first.

## 3.4.4 Candidates

Any field that can appear in a `WHERE` predicate is a candidate.

**Header columns.** `event_type`, `origin_class`, `cpu_id`,
`effective_token_guid`, `true_token_guid`, `process_guid`, `boot_id`.
`timestamp` is always indexed and is not adaptively managed.

**Payload fields.** Any queryable flattened path that appears in a
predicate is a candidate for an expression index. A path suppressed by
the flattening rules of PSPU §3.22 — a top-level key colliding with a
header field, a key that is not a valid segment, a duplicate path —
never receives one, because it is not a query-language field at all.

The raw `payload` column never receives a plain column index. Indexing
an opaque blob accelerates nothing.

## 3.4.5 Payload indexes are an optimisation, not an authority

An expression index extracts a field from the payload on every insert
and indexes a deterministic private key for it. The exact key bytes are
internal to eventd and are not part of the storage contract.

eventd may implement these as SQLite expression indexes with
deterministic extraction functions, as generated columns, or by any
equivalent SQLite-backed means. What every mechanism has in common is
that it implements the same field resolution and flattening as
PSPU §3.22 — and that where the index cannot reproduce the query
language's comparison semantics exactly, it is used only to **narrow
candidate rows**, with the real predicate applied after the row is
loaded (§6.3).

SQLite's native dynamic-type equality and ordering never substitute for
the query language's string case folding, numeric comparison, binary
comparison, array comparison, or null and missing-field handling.
Getting a smaller answer faster is worthless if it is a different
answer.

Rows where the field is absent, unqueryable or suppressed index as null,
or are otherwise excluded in a way that preserves those semantics.

Payload indexes are otherwise ordinary members of the desired set, with
the same priority ordering, shedding and convergence.

## 3.4.6 Naming

Header column indexes are `idx_events_<column>` —
`idx_events_event_type`, `idx_events_process_guid`.

Payload expression indexes are named from the field GUID (§7.3), to
avoid both collisions and characters SQLite will not accept in an
identifier:

```text
idx_events_payload_<field_guid_hex>
```

where `field_guid_hex` is the UUID v5 field GUID for the flattened path,
as 32 lowercase hexadecimal digits with braces and hyphens stripped. The
path `source.name` yields
`idx_events_payload_` followed by the 32 hex digits of
`uuid_v5(EVENTD_FIELD_NAMESPACE, "source.name")`.

Deriving the name from the GUID rather than from the path means the same
path always produces the same index name, and no path — however it is
spelled — can produce a name that collides with another's or that
SQLite rejects.

---

# 3.5 The Metadata Database

_Peios / Advanced Peios / eventd / Event Storage_

> The one database in the store that is not a shard — its tables, its concurrency, and why recovering it is cheap.

One database in the event store directory is not a shard:
`eventd-meta.db`. It holds the state that is global to eventd rather
than to any shard — the adaptive index and rollup state, diagnostic
sequence checkpoints, and the administrative Security Descriptor — and
it is the one database that survives shard reconfiguration untouched.

It is created on first startup if absent, opened in WAL mode with
`synchronous=NORMAL`. It is written once per policy interval and read at
startup, so per-transaction durability buys nothing: losing the last
interval's counters costs some adaptation, not any data.

The query path excludes it explicitly, since it is in the same directory
as the shards (§3.3).

## 3.5.1 Tables

**`index_counters`** — query frequency per field (§3.4).

| Column | Type | Contents |
|---|---|---|
| `field_path` | TEXT PRIMARY KEY | Field name or payload path: `event_type`, `granted_access`, `source.name`. |
| `query_count` | INTEGER NOT NULL | Queries filtering on it within the current window. |
| `window_start` | INTEGER NOT NULL | When the window started, nanoseconds since the epoch. |

**`desired_indexes`** — the computed desired index set.

| Column | Type | Contents |
|---|---|---|
| `field_path` | TEXT PRIMARY KEY | Field name or payload path. |
| `priority` | INTEGER NOT NULL | Rank; lower is higher priority. |
| `is_expression` | INTEGER NOT NULL | 1 for a payload expression index, 0 for a column index. |

**`rollup_counters`** and **`desired_rollups`** — the same pair for
metric rollups (§5.6), keyed by `function_window`, a composite of
function name and window size such as `avg_3600`.

| Column | Type | Contents |
|---|---|---|
| `function_window` | TEXT PRIMARY KEY | Function and window, e.g. `avg_3600`. |
| `query_count` | INTEGER NOT NULL | Queries using the pair within the window. |
| `window_start` | INTEGER NOT NULL | When the window started. |

`desired_rollups` carries `function_window` and `priority`.

**`sequence_checkpoints`** — diagnostic only.

| Column | Type | Contents |
|---|---|---|
| `boot_id` | BLOB NOT NULL | The boot the checkpoint applies to. |
| `cpu_id` | INTEGER NOT NULL | CPU identifier. |
| `sequence` | INTEGER NOT NULL | Last committed sequence for that pair when written. |
| `updated_at` | INTEGER NOT NULL | When it was written. |

Primary key `(boot_id, cpu_id)`. Startup resumption derives its points
from committed event rows and never from this table (§2.2). The table
exists so that an operator can see what eventd believed at shutdown and
compare it with what the rows say — the two disagreeing is itself
diagnostic.

**`meta`** — key-value.

| Column | Type | Contents |
|---|---|---|
| `key` | TEXT PRIMARY KEY | Metadata key. |
| `value` | BLOB NOT NULL | Strings as UTF-8 bytes, binary as raw bytes. |

with three required entries: `schema_version` (§B), `created_at` as a
UTC `YYYY-MM-DDTHH:MM:SSZ` string, and **`admin_sd`**, a self-relative
Security Descriptor governing administrative operations — the `INDEX`
command above all (§7.2).

The default `admin_sd` grants SYSTEM and Administrators.

> [!NOTE]
> `admin_sd` is the only access control state eventd keeps outside the
> registry. Everything on the read path lives under
> `Machine\System\eventd\Security\` where the registry's own access
> control protects it; this one sits in a file in the event store
> directory, whose protection is not otherwise specified (§3.3).

## 3.5.2 Concurrency

One writer connection, owned by the index and rollup policy thread. No
other thread opens the database read-write.

Query handlers write only to the in-memory counters; the policy thread
flushes them at each interval. Writer threads and query handlers read
the desired sets from memory, never from the database. Graceful shutdown
writes `sequence_checkpoints` through the same connection, after policy
activity has stopped (§8.4).

With a single writer and no other database-level access, SQLite's WAL
mode is the whole of the concurrency control needed.

The policy thread checkpoints the write-ahead log at
`WalCheckpointPages` in passive mode, and does not block when readers
hold pages — the same rule as every other store (§2.4).

## 3.5.3 Recovery is cheap

If the schema version is missing or unrecognised, or any required table
or `meta` entry is missing or malformed, eventd logs an error and
**recreates the database from defaults**.

This is the opposite of the rule for a shard, which fails startup
(§3.3), and the difference is what is at stake. A shard holds the only
copy of audit data. This database holds an optimisation policy, some
diagnostics, and a descriptor with a known default — all of it
reconstructible, none of it irreplaceable. Losing it costs the
adaptation eventd had accumulated, and the counters begin refilling
immediately.

The one thing recreation does lose is a customised `admin_sd`, which
reverts to the default.

## 3.5.4 Startup

1. Open `eventd-meta.db`, creating it if absent.
2. Verify the schema version and required `meta` entries; recreate from
   defaults on failure.
3. Load `index_counters` and `rollup_counters` into memory.
4. Load `desired_indexes` and `desired_rollups` into memory.
5. Load `sequence_checkpoints`, for diagnostics only.
6. Discover the material indexes in each shard from its schema and
   compare against the desired set.

eventd resumes convergence from wherever each shard happens to be. It
neither drops nor rebuilds indexes at startup: a shard's material set is
a fact to be observed, not a state to be restored.

---

# 3.6 Retention

_Peios / Advanced Peios / eventd / Event Storage_

> Bounding disk growth on two axes, age and size, with both enforced — how the pass runs and how space is reclaimed.

Retention bounds disk growth. eventd deletes on two axes, age and size,
and enforces both — an event goes when it exceeds either threshold.

The v0.23 model is deliberately minimal, and a later one is expected to
support rules resembling queries: retain KACS events for ninety days,
synthetic events for seven, userspace-origin events for fourteen; and to
prune during ingestion rather than only in arrears. What is here is the
least that prevents unbounded growth.

## 3.6.1 Age

Rows are deleted from `events` where `timestamp` is older than
`EventRetentionDays` (§A) from the current wall clock, until none
remain. Each shard is processed independently, and the rule covers KMES
events, synthetic events and gap records alike.

## 3.6.2 Size

Size is measured as **logical live size**, not file size:

```text
logical_live_bytes = (page_count - freelist_count) * page_size
```

from `PRAGMA page_count`, `PRAGMA freelist_count` and
`PRAGMA page_size`, taken after attempting a passive WAL checkpoint. The
event store's total is the sum across every shard.

Pages freed by retention do not count, because they are reusable by
future inserts. Counting them would make retention chase its own tail:
each deletion would free pages that still counted against the limit,
prompting more deletion.

When `EventRetentionMaxBytes` is non-zero and the total exceeds it:

1. Identify every non-current boot ID present in the shards.
2. Order them by their newest event timestamp, oldest boot first.
3. Delete each of those boots entirely, across all shards, one boot at a
   time, until the total is within the limit or no non-current boots
   remain.
4. If still over, delete the oldest events of the **current** boot by
   timestamp, across all shards, until within the limit.

Size pressure prefers boot boundaries. Deleting a whole old boot removes
a self-contained unit — its sequence numbers, its gap records and its
startup event go together — and it preserves recent events across the
boundary, which is what an operator investigating a reboot needs. Only
when whole boots are exhausted does eventd start on the current one.

## 3.6.3 Running it

Retention runs on a background thread of its own, never on a writer or
drain thread, using a separate read-write connection per store. It
processes the event store first, then the log store (§4.4), then the
metric store (§5.5). The interval is `RetentionCheckIntervalMinutes`
(§A).

WAL mode lets a reader run alongside a writer, but not a second writer,
so the retention thread coordinates with each shard's writer thread — a
shard-level mutex taken before writing, which the writer briefly yields
to.

Deletion is batched. Each transaction deletes at most
`RetentionDeleteBatchRows` and commits before the next, and between
batches the retention thread releases the coordination primitive and
rechecks writer pressure. A single unbatched `DELETE` over a month of
events would hold a write transaction for as long as it took, blocking
the writer thread and, behind it, the drain thread and the ring buffer.

## 3.6.4 Reclamation

Deleting rows does not shrink a SQLite file. Freed pages are reused by
later inserts, and reclaiming filesystem space needs `VACUUM`, which
rewrites the whole database.

eventd never runs `VACUUM` automatically. Reclamation is an explicit
administrative operation.

In steady state it is not needed. Where ingestion and retention run at
comparable rates, the file settles at roughly the high-water mark of
retained data and the freed pages are recycled without further growth —
which is also why logical live size, rather than file size, is the right
thing to measure.

---

# 3.7 Boot Partitioning

_Peios / Advanced Peios / eventd / Event Storage_

> Every stored record carries a boot_id — what it is for, where it is stored, how uniqueness is assured, and how a boundary is detected.

Every record eventd stores — every event, every log line, every raw
metric sample — carries a `boot_id`: a 16-byte GUID identifying the boot
that produced it. peinit assigns it at each boot and eventd reads it at
startup.

Derived metric rollups are the exception. They are boot-agnostic
aggregates and carry no `boot_id` (§5.6).

## 3.7.1 What it is for

**Disambiguation.** KMES per-CPU sequence numbers restart at zero each
boot. Without a boot ID, sequence 42 from one boot is indistinguishable
from sequence 42 from the next, and every gap calculation across a
reboot would be wrong.

**Lifecycle.** Retention can delete a whole boot as a unit rather than
scanning by timestamp, and a boot is the natural unit to delete: its
events, its gaps and its startup record go together (§3.6).

## 3.7.2 Where it is stored

| Store | Column |
|---|---|
| Event | `events.boot_id`, every row |
| Log | `logs.boot_id`, every row |
| Metric | `samples.boot_id`, every row |

The log store records it because log output can mean different things
across boots — a service configured differently at boot time says
different things.

The metric store records it **per sample** but does not make it part of
series identity (§5.2). A time series stays continuous across a reboot,
which is what a chart of CPU usage across a restart should show, and a
query that wants one boot's worth filters for it explicitly
(PSPU §3.25). Rollups stay boot-agnostic scalars, so a boot-filtered
metric query is served from raw samples and never from a rollup.

## 3.7.3 Uniqueness

Within one boot an event is uniquely identified by
`(cpu_id, sequence)`. Across boots, `boot_id` supplies the
disambiguating dimension, and the triple
`(boot_id, cpu_id, sequence)` is globally unique.

## 3.7.4 Detecting the boundary

At startup eventd reads the current boot ID from peinit, then searches
every readable event shard database — historical shards included — for
committed rows carrying it.

**No committed rows for this boot.** This is the boot's first eventd
start. eventd resets every per-CPU sequence tracker to 0, records the
new boot ID for all subsequent writes to all three stores, and emits
`synthetic.startup` with `restart` false.

**Committed rows exist.** eventd crashed and peinit restarted it within
the same boot. eventd restores each CPU's tracker from the maximum
non-null `sequence` for `(boot_id, cpu_id)` across every readable shard,
with CPUs having no rows resuming at 0; continues writing under the
existing boot ID; and emits `synthetic.startup` with `restart` true.

The two cases are distinguished by the data itself rather than by any
flag eventd persisted. That is the point: a flag would have to be
written at a moment eventd might not reach, and a crash is precisely the
case where it did not.

Committed rows are the authority throughout. The metadata database's
sequence checkpoints and the previous `synthetic.shutdown` payload
record the same numbers, and both are diagnostic — neither is consulted
for resumption (§2.2, §3.5).

---

# 4.1 The Log Writer

_Peios / Advanced Peios / eventd / Log Storage_

> One thread reads the log socket and writes the store, independent of the event path — batching, durability and what it adds to a record.

One thread reads datagrams from the log socket and writes log records to
the log store. It is independent of the event drain and writer threads,
so log ingestion never contends with event ingestion.

The wire contract — socket type, datagram ceiling, record format, and
exactly which malformations cost what — is PSPU §3.6 to §3.8. What
follows is what eventd does with a record once it has one.

## 4.1.1 One thread does both jobs

The log thread performs both the socket reads and the SQLite writes.

The consequence is direct: **during a batch commit the socket is not
being drained**, and datagrams arriving in that window occupy the
receive queue until it fills, after which the kernel discards them. The
queue — `SO_RCVBUF` — is sized at four times the datagram ceiling, and it
is the whole cushion.

> [!NOTE]
> Linux's default `SO_RCVBUF` for a Unix datagram socket is around
> 212 KB, roughly a thousand typical log records. A batch commit takes
> one to ten milliseconds, and that buffer is the only thing absorbing
> arrivals in the window. A service dumping a stack trace will lose
> datagrams, which is the intended degradation for a loss-tolerant path
> (PSPU §3.4) — the alternatives being backpressure or unbounded
> buffering, and the design forbids both.

Splitting into a reader and a writer with a bounded handoff — the shape
the event path uses (§2.3) — would decouple them. It is not done, and
the reasoning is that log loss is tolerable by design (PSPU §3.4), log
volume is normally well below event volume, and the single-thread model
avoids a handoff channel and its backpressure semantics entirely.

Where log throughput does become the constraint, sharding the log store
the way the event store is sharded is the larger lever; splitting the
thread only moves the stall.

## 4.1.2 Batching

The writer batches on the same adaptive principle as the event writer
(§2.4), with the socket receive queue as its input. A transaction opens
when the first valid record is available and commits when any of these
holds:

- no further datagram is immediately available in the receive queue
- the batch holds `LogMaxBatchSize` records
- `LogMaxBatchLatencyMs` has elapsed since the first record entered it

If a datagram yields more valid records than fit in the remaining space,
the writer commits, then continues with the same datagram in a new
transaction. A transaction never exceeds the size cap and never stays
open past the latency cap — a batched datagram cannot smuggle a larger
transaction past either.

The defaults (§A) are 5000 records and 500 milliseconds. The latency is
five times the event writer's, because log loss on power failure is
acceptable where event loss is not, and larger, less frequent
transactions are more efficient at the moderate volumes logs normally
run at.

## 4.1.3 Durability

The log store runs in WAL mode with `synchronous=NORMAL`, not FULL.

NORMAL syncs at checkpoint time rather than at every commit. It is
durable against process crashes — the write-ahead log survives — but not
against power loss, where commits since the last checkpoint may be gone.

This is a deliberate divergence from the event store, and it is the
single clearest expression of the hierarchy the whole daemon is
organised around: events are sacred, logs are not. Paying an fsync per
transaction to protect data whose loss is defined as acceptable would be
paying for nothing.

## 4.1.4 Adding to the record

eventd supplies the `boot_id` and, where the producer omitted
`timestamp`, its own clock at receipt. Everything else is stored as
given — `message` byte for byte (PSPU §3.8).

---

# 4.2 The Logs Table

_Peios / Advanced Peios / eventd / Log Storage_

> A single database rather than a directory of shards, its schema, its write-time indexes, and why there is no adaptive indexing here.

The log store is a single SQLite database — not a directory of shards.
There is no sharding here: one ingestion thread produces the writes, so
splitting the target would give a single writer several files to switch
between rather than several writers working in parallel.

It holds one `logs` table.

| Column | Type | Contents |
|---|---|---|
| `id` | INTEGER PRIMARY KEY | SQLite rowid, monotonic. |
| `boot_id` | BLOB NOT NULL | 16-byte boot ID GUID in PCDS binary layout. |
| `timestamp` | INTEGER NOT NULL | Nanoseconds since the Unix epoch — the producer's value if it supplied one, otherwise eventd's clock at receipt. |
| `origin` | TEXT NOT NULL | The producing program's name, as the producer declared it. |
| `is_error` | INTEGER NOT NULL | 1 for standard error or an explicitly marked error, 0 otherwise. |
| `message` | TEXT NOT NULL | The log text. |
| `job_id` | BLOB | 16-byte correlation GUID when the producer supplied one; null otherwise. |

The schema is deliberately narrow. A log record is text with light
metadata: what produced it, whether it was an error, when, and
optionally which execution it belongs to. There is no payload blob and
no origin class, and the only identity-like field is the optional
correlation key — which is not an identity at all, since `origin` is
self-asserted and unverified (PSPU §3.28).

A program needing more structure than this emits events.

## 4.2.1 `is_error` is an integer here and a boolean there

The column stores 0 or 1; the query language exposes a boolean, and
accepts either `WHERE is_error == true` or `WHERE is_error == 1`
(PSPU §3.22). `ERROR ONLY` is sugar for the first.

## 4.2.2 Write-time indexes

Three indexes are created with the table:

- `idx_logs_timestamp` on `logs(timestamp)` — time-range filtering, as
  everywhere.
- `idx_logs_origin` on `logs(origin)` — "show me logs from X", which is
  the dominant log query.
- `idx_logs_job_id` on `logs(job_id) WHERE job_id IS NOT NULL` — a
  partial index for "show me logs for job X". Partial because
  directly-submitted lines carry no correlation key, so only correlated
  lines are worth indexing.

The origin index costs write amplification beyond the timestamp index,
and the cost is modest in practice: `origin` has low cardinality, tens
of distinct names on a normal system, so its index pages stay in
SQLite's page cache and insertion stays cheap. The trade is accepted
deliberately — the two dominant log queries must not become full table
scans.

## 4.2.3 No adaptive indexing

The log store does not participate in adaptive indexing (§3.4). Its
field set is closed and small, and the three write-time indexes already
cover the access patterns; there is no space of candidate fields for a
policy to discover.

The same follows for query frequency counters: log queries do not
increment them (§6.5).

## 4.2.4 Schema version

The log store holds a `metadata` table with the same two-column
structure as a shard's (§3.1). Its version is in §B.

eventd checks it at startup and applies the lifecycle rules of §4.3,
and does not migrate.

---

# 4.3 Database Lifecycle

_Peios / Advanced Peios / eventd / Log Storage_

> The log store is a file rather than a directory — its path, creation, opening, concurrency and checkpointing.

## 4.3.1 Path

The log store is the file named by `LogStorePath` (§A) — a file path,
unlike the event store's directory. There is no compiled-in default: a
missing or invalid value is a startup failure.

eventd creates the file and any absent parent directories.

## 4.3.2 Creation

A log store that does not exist is created with:

1. WAL mode, `PRAGMA journal_mode=WAL`
2. `PRAGMA synchronous=NORMAL`
3. the `logs` and `metadata` tables (§4.2)
4. the `idx_logs_timestamp`, `idx_logs_origin` and `idx_logs_job_id`
   indexes
5. the `schema_version` and `created_at` entries

## 4.3.3 Opening

1. Open in WAL mode.
2. Set synchronous to NORMAL.
3. Verify the schema version. Missing or unrecognised is a **startup
   failure**; no migration is attempted.
4. Verify structural integrity — required tables and indexes present.
   Failing this, with SQLite reporting no corruption, is a **startup
   failure**.
5. On SQLite reporting corruption, quarantine and replace.

Quarantine works exactly as for a shard (§3.3): the database, `-wal` and
`-shm` files are renamed with a shared `.corrupt.<timestamp_ns>` suffix,
`.N` appended with the lowest positive integer if the name is taken, and
a fresh empty log store is created at the configured path.

The log store is a **required** store. There is no degraded mode in
which eventd runs without one (§8.2), which is why steps 3 and 4 fail
startup rather than proceeding without logs.

## 4.3.4 Concurrency

One read-write connection owned by the log writer thread, and any number
of read-only connections owned by query handlers. WAL mode lets them run
concurrently.

## 4.3.5 Checkpointing

The log writer checkpoints when its write-ahead log reaches
`WalCheckpointPages` (§A), in passive mode, and does not block if
readers hold pages — it keeps writing and retries after a later commit.

Checkpointing matters more here than in the event store, because
`synchronous=NORMAL` makes the checkpoint the durability boundary rather
than merely a space-reclamation event: data committed since the last
checkpoint is what a power cut takes (§9.5).

---

# 4.4 Retention

_Peios / Advanced Peios / eventd / Log Storage_

> Log retention works exactly as event retention does, on the same thread, with a shorter default and its own reclamation.

Log retention works exactly as event retention does (§3.6), on the same
background thread, running after the event store and before the metric
store. As there, the v0.23 model is an early simplification and both
limits are enforced with the more aggressive one winning.

## 4.4.1 Age and size

Rows older than `LogRetentionDays` (§A) are deleted from `logs` until
none remain.

If `LogRetentionMaxBytes` is non-zero and the store's logical live size
exceeds it, the oldest entries by timestamp are deleted until it is
within the limit. Logical live size is the same measure as §3.6 —
`(page_count - freelist_count) * page_size` after attempting a passive
checkpoint — and freed pages do not count.

There is no boot-boundary preference here. Event size retention prefers
to drop whole old boots because a boot is a self-contained unit of
sequence-numbered records; a log line has no such structure, so oldest
first is the whole rule.

## 4.4.2 The default is shorter than events'

Fourteen days against the event store's thirty (§A).

Historical log data is worth less than historical audit data, and it is
usually bulkier per unit of value. The metric store's default is longer
than either at ninety days, for the opposite reason: a metric sample is
tiny and trend data is worth more the further back it goes (§5.5).

## 4.4.3 Batching

Deletion is batched at `RetentionDeleteBatchRows` per transaction, with
a commit between batches. Between them the retention thread releases any
writer coordination primitive and rechecks writer pressure.

The stall this avoids is the log ingestion thread's, and that thread is
also the one draining the socket (§4.1) — so a retention pass holding a
long write transaction would not merely delay writes, it would stop the
socket being read and lose the datagrams that arrived meanwhile.

## 4.4.4 Reclamation

`VACUUM` is never run automatically, as everywhere. Freed pages are
recycled by later inserts and are excluded from the size measure.

---

# 5.1 The Metric Writer

_Peios / Advanced Peios / eventd / Metric Storage_

> One thread reads the metric socket and writes samples — processing a record, out-of-order samples, batching and durability.

One thread reads datagrams from the metric socket and writes samples to
the metric store, independent of both the event and log paths. It has
the same single-thread shape as the log writer, with the same
consequence during a commit (§4.1).

The wire contract is PSPU §3.9 to §3.13.

## 5.1.1 Processing a record

For each valid record:

1. **Resolve the series** from name and labels — and, for a histogram,
   bucket boundaries — through the in-memory series cache (§5.3). A
   series that does not exist is inserted into `series` with the
   record's type, and the cache is updated.
2. **Check the type.** If the record's type differs from the resolved
   series' type, the record is dropped silently. The type is set at
   creation and is immutable; a series never changes type.
3. **Insert the sample** into `samples` with the resolved `series_id`,
   the timestamp and the value. SQLite assigns `samples.id`, which is
   the deterministic tiebreaker among samples sharing a timestamp
   (§5.2). A histogram's data is encoded as the canonical MessagePack
   sample map and stored in `histogram_data`.

Step 2 is the failure that leaves no trace. A producer that changes a
metric's type has silently stopped emitting it — every sample discarded,
no event, no counter, nothing in any log — and the only symptom is a
series that stopped advancing. The reason nothing is emitted is
PSPU §3.4: ingestion is unauthenticated, and reacting to input at all
is an amplification vector.

## 5.1.2 Out-of-order samples

The writer stores a valid sample whose timestamp precedes samples
already held for that series.

Producers batch, clocks step, and sweeps get retried. Refusing late
samples would convert any of those into silent loss, so eventd accepts
them and defines every ordering it performs over `(timestamp, id)`
rather than over insertion order — which is what makes rollup
computation and `RATE` evaluation deterministic regardless of arrival
(§5.6, §6.2).

## 5.1.3 Batching

The same adaptive algorithm as the event and log writers (§2.4), with
the socket receive queue as input. A transaction opens at the first
valid sample and commits when any of these holds:

- no further datagram is immediately available
- the batch holds `MetricMaxBatchSize` samples
- `MetricMaxBatchLatencyMs` has elapsed since the first sample entered
  it

A datagram yielding more samples than fit is split across transactions,
the writer committing before continuing with the same datagram. Neither
cap is ever exceeded.

The defaults (§A) are 5000 samples and 1000 milliseconds — the longest
latency of the three writers. Metrics are typically sampled every
fifteen seconds, so a one-second commit window accumulates a whole
sweep's worth without any latency that a dashboard could notice. Under a
burst, where a collection agent submits every core, disk and interface
at once, the size cap is what forces a timely commit.

## 5.1.4 Durability

WAL mode with `synchronous=NORMAL`, as the log store (§4.1). Metric loss
on power failure is acceptable, so per-transaction fsync buys nothing.

---

# 5.2 Series and Samples

_Peios / Advanced Peios / eventd / Metric Storage_

> The metric store is organised around series rather than records — the series table, the canonical label string and the boundaries blob.

The metric store is a single SQLite database, and unlike the event and
log stores it is organised around **series** rather than records.
Individual samples are appended to a series that already has an
identity.

## 5.2.1 The series table

| Column | Type | Contents |
|---|---|---|
| `id` | INTEGER PRIMARY KEY | Series identifier; the foreign key `samples` uses. |
| `name` | TEXT NOT NULL | The metric name. |
| `labels` | TEXT NOT NULL | Canonical label representation. Empty string for no labels. |
| `type` | INTEGER NOT NULL | 0 counter, 1 gauge, 2 histogram. |
| `label_hash` | INTEGER NOT NULL | Hash of the canonical label string. |
| `boundaries_hash` | INTEGER | Hash of the canonical boundary blob. Null for counters and gauges. |
| `boundaries` | BLOB | Canonical boundary blob. Null for counters and gauges. |

### 5.2.1.1 The canonical label string

Labels are sorted by key in unsigned UTF-8 byte order, each pair written
`key=value`, and the pairs joined with commas: `core=0,host=server1`.
The empty label set is the empty string.

No escaping is performed and none is needed, because ingestion rejects
`=` and `,` inside a key or a value (PSPU §3.10). That prohibition
exists precisely to make this encoding unambiguous, and it is the reason
the constraint binds the producer rather than being handled internally.

### 5.2.1.2 The boundaries blob

A fixed binary encoding, not MessagePack:

1. `boundary_count`, `u32` little-endian
2. that many IEEE-754 `f64` values, each little-endian, in the validated
   order the producer sent

It exists only to identify histogram series and to resolve hash
collisions, and is never returned in a query result.

### 5.2.1.3 Hashes narrow, they do not decide

`label_hash` and `boundaries_hash` are 64-bit FNV-1a over the exact
bytes of the canonical string or blob, with offset basis
`0xcbf29ce484222325` and prime `0x100000001b3`. The high bit is cleared
before storage, `hash & 0x7fff_ffff_ffff_ffff`, so the value always fits
SQLite's signed `INTEGER`.

A lookup always verifies the full `labels` string, and for a histogram
the full `boundaries` blob, after narrowing by hash. A hash is an index
key, never an identity: two label sets that collide are still two
series.

### 5.2.1.4 Uniqueness

The table carries `UNIQUE(name, labels, boundaries_hash)`.

For counters and gauges `boundaries_hash` is null, and SQLite treats
nulls as distinct in a unique constraint — so the constraint does not
enforce uniqueness for them. What does is the single-writer resolution
logic (§5.3), which checks before inserting. The constraint is a
defensive backstop against a future change that introduces a second
write path, not the primary mechanism.

`type` is **not** part of the identity. A record resolving to an
existing series with a different type resolves successfully and is then
dropped for the mismatch (§5.1).

## 5.2.2 The samples table

| Column | Type | Contents |
|---|---|---|
| `id` | INTEGER PRIMARY KEY | Internal row identifier; the tiebreaker for samples sharing a series and timestamp. |
| `series_id` | INTEGER NOT NULL | References `series(id)`. |
| `boot_id` | BLOB NOT NULL | 16-byte boot ID GUID. |
| `timestamp` | INTEGER NOT NULL | Nanoseconds since the Unix epoch. |
| `value` | REAL NOT NULL | The raw value for counters and gauges. Stores 0 for histograms. |
| `histogram_data` | BLOB | Canonical MessagePack histogram sample map. Null for counters and gauges. |

For a histogram, `histogram_data` is a canonical MessagePack map
(PSPU §3.5) with exactly four keys: `boundaries`, an array of `float64`
in the producer's order; `counts`, an array of unsigned integers;
`total_count`; and `sum`, a finite `float64`.

`value` is a placeholder for histogram rows and is never returned as a
metric query value. Storing 0 rather than null keeps the column
`NOT NULL` and keeps the row layout uniform.

Canonical encoding is required here because a stored sample map must be
byte-stable: two equal histograms encode identically, which is what
makes them comparable without decoding.

`boot_id` is per sample and is not part of the series identity, so a
series stays continuous across a reboot (§3.7).

## 5.2.3 Ordering

Query execution order within a series is always `(timestamp, id)`
ascending, never insertion order alone. Duplicate timestamps are
permitted and `id` gives them a stable order.

`id` is internal. It is never exposed as a query field, a result field,
an access-control field, or a reserved label key (PSPU §3.28).

## 5.2.4 The rollups table

The database also holds `rollups`, defined in §5.6. Rollups are
boot-agnostic scalar aggregates and carry no `boot_id`, which is why a
boot-filtered metric query is never served from one.

## 5.2.5 Write-time indexes

- `idx_samples_series_timestamp` on `samples(series_id, timestamp, id)`
  — the dominant pattern is "samples for series X over range Y in
  deterministic order", and this one composite index serves the series
  lookup, the range filter and the `(timestamp, id)` ordering in a
  single scan.
- `idx_series_name` on `series(name)` — name lookups.
- `idx_series_label_hash` on `series(label_hash)` — series resolution on
  the ingestion path.
- `idx_rollups_series_function_window` on
  `rollups(series_id, function, window_seconds, window_start)`.

## 5.2.6 Schema version

A `metadata` table with the same structure as the other stores' (§3.1).
Version 1 comprises `series`, `samples`, `rollups` and `metadata`; the
current value is in §B. eventd checks it at startup, applies the
lifecycle rules of §5.4, and does not migrate.

---

# 5.3 Series Resolution

_Peios / Advanced Peios / eventd / Metric Storage_

> Turning each arriving sample into a series_id once, on the single ingestion thread — the cache, and how to size it.

Every arriving sample must be turned into a `series_id` before it can be
inserted. This happens once per sample on the single metric ingestion
thread, so it is the hottest lookup in the daemon and the reason a cache
exists at all.

## 5.3.1 Resolving

1. Compute the canonical label string: sort by key in unsigned UTF-8
   byte order, encode each pair `key=value`, join with commas. The empty
   label set encodes as the empty string. No escaping — ingestion has
   already rejected the delimiters (§5.2).
2. Hash it.
3. For a histogram, compute the canonical boundary blob and its hash
   from the **validated, producer-supplied order**. eventd never sorts
   boundaries. For counters and gauges both are null.
4. Look up `series` by `name`, `label_hash`, and for histograms
   `boundaries_hash`.
5. On a match, verify the full `labels` string, and for histograms the
   full boundary blob. If the record's type differs from the existing
   series' type, drop the record (§5.1). Otherwise use the existing
   `series_id`.
6. On no match, insert a new `series` row and use the new identifier.

A histogram whose boundaries changed takes step 6: it is a new series
(PSPU §3.13). The old one keeps its historical samples and the new one
starts accumulating.

## 5.3.2 The cache

Resolution runs through a bounded in-memory cache mapping
`(name, canonical labels, boundaries hash, boundaries blob)` to
`series_id`. For counters and gauges the boundary components are absent.

A hit is a hash table lookup with no SQLite involvement. A miss costs
one `SELECT` on `name` and `label_hash`, after which the result is
inserted, evicting the least recently used entry if the cache is full.

The bound is `MetricSeriesCacheSize` (§A), default 50000, with LRU
eviction. It bounds **memory**, not the number of series: the `series`
table is uncapped and a new series is always created in the database.
A system with a million series and a 50000-entry cache uses memory
proportional to the cache, and at roughly 200 to 300 bytes an entry the
default costs 10 to 15 MB.

The cache starts empty after a restart and is warmed on demand — within
one collection cycle, typically fifteen seconds, every active series is
cached. There is no pre-warming pass, because reading a million-row
`series` table at startup to populate a 50000-entry cache would be work
spent to discard most of its result.

## 5.3.3 Sizing it

The cache is sized for the set of actively reporting series, and
behaves badly below it.

Below that, LRU does not help, because every series is equally hot: each
collection cycle evicts the overflow and reloads it, producing a fixed
number of `SELECT`s every cycle, permanently. A system with 55000 active
series and a 50000-entry cache incurs about 5000 cache misses every
fifteen seconds, indefinitely.

This interacts badly with label cardinality (PSPU §3.10). Labels with
unbounded values — request identifiers, user-supplied strings — grow the
series table without limit, and once the active set exceeds the cache
every cycle pays the eviction cost on the one thread that also drains
the metric socket. The failure presents as metric loss, because the
thread stops reading while it queries.

eventd does not defend against this and cannot: at the interface, a
producer creating a genuinely new series is indistinguishable from one
creating garbage, and every available defence would break a correct
producer to inconvenience an incorrect one (PSPU §3.13).

---

# 5.4 Database Lifecycle

_Peios / Advanced Peios / eventd / Metric Storage_

> The metric store file — its path, creation, opening, concurrency and checkpointing.

## 5.4.1 Path

The file named by `MetricStorePath` (§A). No compiled-in default; a
missing or invalid value is a startup failure. eventd creates the file
and any absent parent directories.

## 5.4.2 Creation

1. WAL mode.
2. `PRAGMA synchronous=NORMAL` — the log store's reasoning, for the same
   reason: metric loss on power failure is acceptable (§4.1).
3. The `series`, `samples`, `rollups` and `metadata` tables (§5.2,
   §5.6).
4. Every write-time index.
5. The `schema_version` and `created_at` entries.

## 5.4.3 Opening

1. Open in WAL mode with synchronous NORMAL.
2. Verify the schema version. Missing or unrecognised is a **startup
   failure**; no migration.
3. Verify structural integrity — required tables and indexes present,
   including `rollups` and its lookup index. Failing this, with SQLite
   reporting no corruption, is a **startup failure**.
4. On SQLite reporting corruption, quarantine and replace, exactly as
   for a shard (§3.3): matching `-wal` and `-shm` files renamed with the
   same `.corrupt.<timestamp_ns>` suffix, `.N` appended if the name is
   taken, and a fresh empty store created at the configured path.

The metric store is a required store; there is no degraded mode without
one (§8.2).

After opening or creation the series cache is empty and fills on demand
(§5.3).

## 5.4.4 Concurrency

One read-write connection owned by the metric writer thread, and any
number of read-only query connections. WAL mode permits both
concurrently.

The single-writer property is load-bearing here in a way it is not for
the other stores: series resolution checks for an existing row and then
inserts, without a transaction spanning both, and only one writer makes
that safe (§5.2).

## 5.4.5 Checkpointing

The metric writer checkpoints at `WalCheckpointPages` (§A) in passive
mode, and does not block when readers hold pages.

As with the log store, the checkpoint is the durability boundary under
`synchronous=NORMAL`, not merely space reclamation (§9.5).

---

# 5.5 Retention

_Peios / Advanced Peios / eventd / Metric Storage_

> The metric retention pass, its long default, why downsampling is not yet part of it, and how space is reclaimed.

Metric retention runs on the same background thread as the other two,
after the log store (§3.6, §4.4). Both limits are enforced and the more
aggressive wins.

## 5.5.1 The pass

1. Delete rows from `samples` older than `MetricRetentionDays` (§A)
   until none remain.
2. If `MetricRetentionMaxBytes` is non-zero and the store's logical live
   size exceeds it, delete the oldest samples by timestamp until it is
   within the limit.
3. Track every `series_id` whose samples were deleted by either step.
4. Delete every `rollups` row for those series.
5. Delete every `series` row with no remaining samples.

Logical live size is the same measure as §3.6.

Step 4 is not optional bookkeeping. A rollup is a pre-computed aggregate
of raw samples, so a rollup outliving its inputs would be served to a
query as an exact answer computed from data the store no longer has —
and the query engine, finding a matching rollup, would never look at the
raw samples to notice (§5.6). Rollups for the affected series can be
recomputed later from whatever samples remain.

Step 5 removes the definitions of series nobody produces any more, which
is the only mechanism that ever removes a `series` row. A series that
stopped receiving samples persists until its last sample ages out —
ninety days by default — so a burst of short-lived series from a
high-cardinality producer stays in the table for that long.

## 5.5.2 The longest default

Ninety days, against fourteen for logs and thirty for events (§A).

A metric sample is small and its value grows with age: a year of CPU
utilisation is a capacity-planning input in a way that a year of log
lines is not. A thousand series sampled every fifteen seconds produce
about 5.7 million samples a day, which is a few hundred megabytes in
SQLite.

## 5.5.3 Not yet: downsampling

The v0.23 model deletes; it does not downsample. A later retention
engine is expected to aggregate high-resolution data into
lower-resolution rollups as it ages — per-second samples becoming
five-minute averages after a week, hourly averages after a month —
which is what makes long-term metric retention affordable while
preserving the trend. The `rollups` table is the mechanism that would
serve it, but nothing currently promotes raw samples into rollups as a
retention action.

## 5.5.4 Batching and reclamation

Batched at `RetentionDeleteBatchRows` per transaction, with the
coordination primitive released and writer pressure rechecked between
batches — the same rule and the same reason as §4.4, since the metric
writer is also the thread draining the metric socket.

`VACUUM` is never run automatically. Freed pages are recycled and are
excluded from the size measure.

---

# 5.6 Adaptive Rollups

_Peios / Advanced Peios / eventd / Metric Storage_

> Precomputed aggregates so a wide query does not scan millions of raw samples — the rollups table, the registry, and serving a query from one.

Aggregate metric queries read raw samples. Over a large range that means
scanning thousands or millions of rows to produce a handful of numbers,
and the same numbers over and over.

Rollups pre-compute them. The principle is adaptive indexing's (§3.4):
watch which query patterns recur, compute their results in the
background, and serve from the results when they exist. What differs is
that a rollup is an answer rather than an access path, so it has to be
exactly the answer the raw samples would have given.

## 5.6.1 The rollups table

| Column | Type | Contents |
|---|---|---|
| `id` | INTEGER PRIMARY KEY | SQLite rowid. |
| `series_id` | INTEGER NOT NULL | References `series(id)`. |
| `function` | INTEGER NOT NULL | Function identifier (§B). |
| `window_seconds` | INTEGER NOT NULL | Window size. |
| `window_start` | INTEGER NOT NULL | Window start, nanoseconds since the epoch. |
| `value` | REAL NOT NULL | The pre-computed value. |
| `sample_count` | INTEGER NOT NULL | Scalar inputs that contributed. For AVG/MIN/MAX/SUM, raw samples; for RATE/DELTA, valid sample pairs. |
| `covered_ns` | INTEGER NOT NULL | For RATE and DELTA, the elapsed nanoseconds the contributing pairs covered. Zero for AVG, MIN, MAX and SUM. |

A unique constraint and an index both cover
`(series_id, function, window_seconds, window_start)`.

Every row satisfies: `sample_count` greater than zero, `value` finite,
and `covered_ns` greater than zero for RATE and DELTA and exactly zero
for the other four.

`sample_count` and `covered_ns` are what make rollups composable. A
window built from one sample is not as good as one built from sixty, and
combining sub-windows correctly needs their weights — an average
composes weighted by `sample_count`, a rate composes weighted by
`covered_ns`. Without them a rollup could only serve a query whose
window matched it exactly.

Rollups are **per series** and carry no `boot_id`. Cross-series
aggregation composes per-series rollup rows and then applies the query's
terminal aggregation across them.

## 5.6.2 What is not rolled up

**Percentiles.** P50, P95 and P99 are not composable: the P95 of twelve
five-minute P95 values is not the P95 of the hour. Percentile queries
always compute from raw histogram samples.

A later revision could add histogram rollups storing merged bucket
counts, computing percentiles from the rolled-up distribution — but that
is a different storage model, not a row in this scalar table.

**Histogram samples in scalar rollups.** AVG, MIN, MAX and SUM roll up
raw counter and gauge values only.

**Non-window RATE and DELTA scalar aggregations.** A stored RATE or
DELTA row is a *window-level* rate, whereas a scalar aggregation over a
transformed series operates on the per-pair values (PSPU §3.25). Serving
one from the other would give a different answer, so these are not
recorded and always fall back to raw samples.

## 5.6.3 The registry

eventd maintains a global set of `(function, window)` pairs worth
pre-computing, derived from query frequency exactly as the desired index
set is (§3.4), with its counters in the metadata database (§3.5).

Each metric query with a rollup-eligible aggregation records a pair:

- `AVG_OVER`, `MIN_OVER`, `MAX_OVER` and `SUM_OVER` record AVG, MIN, MAX
  and SUM respectively, when no RATE or DELTA transform is present, with
  the query's window duration.
- With RATE or DELTA present alongside a window aggregation, the
  *transform* is the recorded function and the terminal aggregation is
  applied afterward to the per-series values. The window duration is
  again the query's.
- Scalar AVG, MIN, MAX and SUM over raw counter or gauge samples with a
  `SINCE` clause record the same function using
  `AdaptiveRollupScalarWindowSeconds` (§A) as the window — a scalar
  query has no window of its own, so a base window is chosen for it and
  composition covers the rest.

A pair crossing `AdaptiveRollupCreateThreshold` over the rolling window
joins the registry; one falling below `AdaptiveRollupDropThreshold`
leaves it. Both thresholds are lower than the indexing ones, because
rollup computation is cheaper — it proceeds window by window rather than
building a whole B-tree — and the speedup is larger, twenty-four rows
instead of eighty-six thousand for a daily query at one-second
resolution.

The registry is global: if hourly averages are queried often for
anything, they are computed for every compatible series. Incompatible
series are skipped.

## 5.6.4 Computation

On a background thread, during low write activity. For each registry
pair, the thread finds windows with raw samples but no rollup row, reads
those samples, computes, and inserts.

**Only completed windows.** The current, still-accumulating window is
never pre-computed and is always computed from raw samples at query
time.

AVG, MIN, MAX and SUM take the raw counter or gauge values whose
timestamps fall in the window. RATE and DELTA are computed for counter
series only, and computation skips any series whose type the function
does not fit.

RATE and DELTA use the same counter-window rule the query engine uses
(PSPU §3.25): consecutive pairs in `(timestamp, id)` order whose later
sample falls inside the window; the immediately preceding sample before
the first in-window one as the baseline for the first pair, where it
exists; pairs with non-positive elapsed time ignored. DELTA's `value` is
the sum of reset-adjusted deltas, and RATE's is that divided by
`covered_ns` in seconds. A window with no contributing inputs — or, for
RATE, zero `covered_ns` — gets no row at all rather than a zero.

Computation is **cancellable** on rising write pressure and resumes
later, through the same mechanism as adaptive index creation (§3.4).

## 5.6.5 Serving a query from rollups

When a metric query carries a rollup-eligible aggregation, the engine
checks for matching rollups. It uses them when a rollup covers the
requested function, window size and time range **and the query does not
filter by `boot_id`** — rollups are boot-agnostic, so a boot-filtered
query cannot be answered from one (§3.7).

Partial coverage is handled rather than refused. Where rollups exist for
the complete windows fully inside the effective range, the engine reads
those and computes the remaining prefix or suffix from raw samples. That
edge handling is required for exactness whenever a `SINCE` or `UNTIL`
bound is not aligned to the window.

With no matching rollup, or a boot filter, or a percentile, or a
non-window RATE or DELTA scalar aggregation, the query falls back to raw
samples entirely. **The result is identical either way** — rollups are a
transparent optimisation and never a different answer.

## 5.6.6 Composition

A rollup window need not match the query window for composable
functions. Smaller windows serve larger queries; the reverse never
works.

| Function | Composes by |
|---|---|
| AVG | weighted average by `sample_count` |
| MIN, MAX | min or max across sub-windows |
| SUM, DELTA | addition |
| RATE | `sum(subrate × sub_covered_ns) / sum(sub_covered_ns)` |

`AVG_OVER 1h` is served from twelve five-minute AVG rollups by weighting
each by its `sample_count`.

Counter resets are handled once, during computation: a stored RATE or
DELTA already reflects reset-adjusted deltas, so composition operates on
adjusted values and never has to consider a reset again.

For cross-series unbracketed window queries, composition happens per
series first and the terminal aggregation is applied across series
afterward. The weighting differs between two cases that look alike:

- **Without a transform**, `AVG_OVER` across series combines per-series
  AVG rollups **weighted by `sample_count`**, because the query means
  the average of every scalar sample value in the window.
- **With RATE or DELTA**, the terminal average is an **unweighted** mean
  of the per-series window values, because under PSPU §3.25 each series
  contributes at most one scalar per window, and weighting one-value
  contributions by their pair counts would silently favour the
  busiest series.

## 5.6.7 Retention and departure

Rollup rows follow raw sample retention. When retention deletes samples
it deletes the rollups for the affected series (§5.5).

When a pair leaves the registry, existing rows are **not** deleted. They
remain available to queries until they age out through normal retention;
only new computation stops. Deleting them would discard work already
done in exchange for nothing — the rows are correct, and a query that
can use one still can.

## 5.6.8 Persistence

The registry and its counters live in the metadata database (§3.5) and
survive restarts. Existing rollup rows are discovered in the table
itself; eventd resumes computation from whatever state it finds, exactly
as it resumes index convergence (§3.4).

---

# 6.1 Parsing and Planning

_Peios / Advanced Peios / eventd / Query Execution_

> The four phases between a query string and an answer, what can only fail after planning, and when payloads are decoded.

A query arrives as one string (PSPU §3.15). Turning it into an answer
has four phases before any data is read: parse, plan, authorize,
execute.

## 6.1.1 Parsing

The string is parsed into a syntax tree. The parser:

1. Identifies the mode from the first token — `EVENTS`, `LOGS` or
   `METRIC`.
2. Extracts the primary selector: a type pattern, a `FROM` list, or a
   metric name with an optional label selector.
3. Collects every clause, in whatever order they appear.
4. Validates that the clauses suit the mode — `CONTAINING` only in log
   mode, `RATE` only in metric mode, `SELECT` only where a result schema
   is not fixed.

Parse errors are returned immediately, before anything is opened, read
or authorized. A malformed query costs a decode and a parse.

## 6.1.2 What can only fail later

Some failures need data. The parser cannot know whether a metric name
resolves to a counter or a histogram, how many series a selector
matches, or whether the effective range exceeds the cross-type lookback
limit — those depend on the store, so they surface at planning or
execution time (PSPU §3.B).

The practical consequence is that the same query string can parse
everywhere and fail on one machine: a metric selector matching one
series on a two-core box matches two on a four-core one.

## 6.1.3 Planning

Planning resolves what the query will actually touch:

- which concrete identifiers the data could carry — event types, log
  origins, metric names — because access control resolves per identifier
  and a broad selector authorizes nothing by itself (§7.4)
- which series a metric selector matches, and whether they are
  type-homogeneous
- which fields the query references, for both authorization and
  frequency accounting (§6.5)
- which stores are involved, including any cross-type source

The identifier discovery step is the expensive one and its cost is not
bounded by the query. `EVENTS SINCE 30d ago` with no type pattern has to
establish every distinct event type in that range before it can
authorize anything, and whether that is an index scan or a table scan
depends on whether `event_type` currently has an index — which is an
adaptive decision that pressure may have reversed (§3.4).

## 6.1.4 When payloads are decoded

Event payloads are stored as opaque MessagePack and are never decoded on
the write path (§3.1). Decoding happens here, on the read path, and only
where a query needs it: to evaluate a payload predicate, to build a flat
result record, or to compute a payload expression index's key on insert.

At high result counts this dominates the query path. Constructing flat
maps from thousands of events means decoding thousands of payloads and
applying the flattening rules of PSPU §3.22 to each. Partial extraction
is the lever: with a `SELECT` present, only the named paths need
decoding, and without one a streaming decoder that emits flattened pairs
avoids materialising the payload at all (§C).

## 6.1.5 Read connections

Execution uses read-only SQLite connections, which in WAL mode do not
block writer threads.

An event query opens one connection per shard database in the directory
(§6.4). A log or metric query opens one. eventd supports concurrent
queries up to its admission limit (§6.5), subject to the operating
system actually having the descriptors and memory; where it cannot
allocate what an admitted query needs, it fails that query rather than
blocking a writer or exceeding the limit.

---

# 6.2 Ordering and Tiebreakers

_Peios / Advanced Peios / eventd / Query Execution_

> Making every result order total and deterministic so paging works — the tiebreakers, and why insertion order is never the answer.

PSPU §3.21 requires every result order to be total and deterministic for
a fixed set of stored records, so that `SKIP` and `TAKE` page reliably.
This is how eventd achieves it.

## 6.2.1 The tiebreakers

Where the query's explicit `SORT` keys — or the mode's default ordering
— do not uniquely order two records, eventd appends internal keys until
the order is total:

| Mode | Appended, in order |
|---|---|
| Events | `timestamp` descending, shard index ascending, `events.id` descending |
| Logs | `timestamp` descending, `logs.id` descending |
| Metrics | `timestamp` ascending, metric name ascending, canonical labels ascending, and `samples.id` ascending where the row corresponds to a raw sample or a derived sample pair |

These are not query-language fields. They never appear in a result
record, cannot be named in a `SORT` or a `SELECT`, and have no
access-control identity (PSPU §3.28).

The **shard index** is the numeric identifier from the `shard-NNNN.db`
filename. It appears in the event tiebreaker because rowids are
per-database: two events in different shards can share a rowid, and
without the shard index the pair would be genuinely unordered.

The metric tiebreaker includes name and labels because an unbracketed
query merges several series into one output stream, where the timestamp
alone does not separate rows from different series.

## 6.2.2 Why insertion order is never the answer

`samples.id` and `events.id` break ties within one database, but neither
is a substitute for the timestamp ordering they follow.

Metric samples may arrive out of timestamp order (§5.1), so insertion
order and time order genuinely differ. Every metric computation —
`RATE`'s consecutive pairs, rollup window membership, cross-type
interval construction — is defined over `(timestamp, id)` ascending
precisely so that a late-arriving sample lands where its timestamp says
it belongs rather than where it happened to be written.

Events are less prone to it, since a drain thread reads one ring buffer
in order, but a shard receiving from several CPUs interleaves them
arbitrarily and a clock step can invert two events from the same CPU.

## 6.2.3 Value ordering is not SQLite's

Sorting, grouping and equality all use the query language's semantics,
never the storage engine's dynamic-type rules (PSPU §3.20, §3.21).

The divergences are not edge cases. SQLite compares an integer and a
real by converting; the query language compares them mathematically and
exactly, including beyond binary64's exact integer range. SQLite orders
text by byte; the query language folds ASCII case and then falls back to
the original bytes only to break a fold-equal tie. SQLite has its own
type-affinity ordering across storage classes; the query language fixes
its own type order, nulls first, arrays last.

Where a SQL construct cannot reproduce those semantics, eventd uses it
only to narrow candidates and applies the real comparison after loading
the row (§6.3).

## 6.2.4 Canonical representatives

A group whose members are equal under the language's rules but not
byte-identical emits the **smallest** member rather than the first
(PSPU §3.21).

Smallest rather than first is what makes the representative a property
of the set. An event query merges results from every shard in an order
that depends on which shard answered first, so "first" would be
nondeterministic across runs of the identical query — which is exactly
the property this chapter exists to prevent.

---

# 6.3 SQL Translation

_Peios / Advanced Peios / eventd / Query Execution_

> What the query language translates to directly, what does not translate, where access control sits, and how aggregation is handled.

Events and logs are translated to SQL. Metrics are translated to SQL
against `series` and `samples`. Clients never see any of it — the
translation is entirely internal and carries no guarantees.

## 6.3.1 What translates directly

**Event header fields** are columns, so a predicate on `event_type`,
`process_guid` or `cpu_id` becomes a SQL `WHERE` comparison over an
indexable column (§3.1).

**Log fields** are all columns; log mode has no payload and its field
set is closed (§4.2).

**Metric selection** resolves names and labels through `series` — from
the in-memory cache where possible — and reads `samples` for the range,
ordered by the composite index that already provides `(timestamp, id)`
(§5.2).

## 6.3.2 What does not

**Event payload predicates** have no column. They become eventd-internal
payload extraction predicates, and may use an adaptive payload
expression index to narrow candidates (§3.4).

The rule governing every such translation is that **SQL narrows, the
query language decides**. Where a SQL construct cannot reproduce a
predicate's comparison semantics exactly, eventd uses it only to reduce
the candidate set and then applies the real predicate after loading the
row.

SQLite's native dynamic-type equality and ordering never substitute for
the query language's ASCII case folding, exact numeric comparison,
binary comparison, array comparison, or null and missing-field handling
(§6.2). An index that returns a smaller set faster is useful; one that
returns a *different* set is a wrong answer arriving quickly.

## 6.3.3 Where access control sits

Access filtering is part of the logical execution, not a filter over the
output (PSPU §3.18, §3.28). eventd may push authorization predicates
down into SQL when the concrete identifier set is known at planning
time, or read candidate rows and discard them before aggregating.

Which it does is a performance decision. What is fixed is that the
externally visible result is identical to the one filtering-first would
produce — aggregates, ordering and pagination included, since all three
would otherwise leak the existence of rows the caller cannot read.

## 6.3.4 Aggregation

Aggregation is pushed into SQL wherever the storage engine can express
it, which is most of the time for simple grouping over columns and none
of the time for grouping over payload paths whose comparison semantics
SQL cannot reproduce.

For an event query the push-down matters twice over, because it also
determines what crosses the shard boundary (§6.4): a shard returning
per-group partial aggregates sends a result proportional to the group
cardinality, where a shard returning rows sends a result proportional to
the row count.

---

# 6.4 Cross-Shard Fan-Out

_Peios / Advanced Peios / eventd / Query Execution_

> Event queries run against every database in the store, active and historical — how results merge, and the unbounded case.

Event queries execute against **every** database in the event store
directory — active shards and historical ones alike (§3.3). Log and
metric queries touch one database each and need none of this.

Shards carry no meaning for the query path (§2.3). A shard holds
whatever CPUs routed to it during whatever lifetimes wrote it, so there
is no shard a query can skip on the basis of its contents, and a
predicate on `cpu_id` scans all of them.

## 6.4.1 Merging

How results combine depends on the query.

**Non-aggregating queries.** Each shard produces rows sorted by the
effective sort key, tiebreakers included (§6.2), and the coordinator
performs an N-way merge of the sorted streams.

With `TAKE` present, each shard returns at most `SKIP + TAKE` rows — or
`TAKE` rows when there is no `SKIP` — and the coordinator applies
`SKIP` and `TAKE` after merging. The total read is therefore at most
`(SKIP + TAKE) × shard_count`, which is the price of not knowing in
advance which shard holds the winning rows.

With `TAKE` absent, each shard streams every matching row until the
query completes or times out.

**Aggregating queries.** Each shard computes a partial aggregate and the
coordinator combines them:

| Query | Shard returns | Coordinator does |
|---|---|---|
| `COUNT` | its local count | sums |
| `COUNT BY`, `TOP N BY`, `GROUP … COUNT` | per-group counts | sums per group key, sorts by count descending, applies `TAKE` |
| `GROUP … SUM` | per-group sums | sums per group |
| `GROUP … AVG` | per-group **sum and count** | computes the average from the combined pair |
| `GROUP … MIN` / `MAX` | per-group min or max | takes the min or max |
| `DISTINCT` | local distinct values | unions |

`AVG` is the one that cannot be composed from its own output. Averaging
per-shard averages weights each shard equally regardless of how many
rows it held, so a shard is asked for the sum and the count and the
coordinator divides once — the same reasoning that makes rollup
composition carry `sample_count` (§5.6).

Pushing aggregation down bounds the coordinator's memory to the group
key cardinality times the shard count, rather than to the total row
count.

## 6.4.2 The unbounded case

A non-aggregating query without `TAKE` has no implicit row limit.
`EVENTS SINCE 7d ago` may match millions of rows, all of which pass
through the merge.

The query timeout is the only backstop (§6.5). Streaming merged results
to the client incrementally, rather than materialising the whole set
before sending, is what keeps the memory cost proportional to the merge
frontier instead of to the result.

## 6.4.3 Descriptors

An event query opens a read-only connection per database, and each
SQLite connection holds one or two descriptors for the database and its
write-ahead log. With many historical shards this adds up quickly across
concurrent queries.

Active shard writer connections stay open for the process lifetime and
are not negotiable. Historical shard read connections are the pool worth
bounding — opened when a query touches them, closed after a period of
inactivity (§C).

---

# 6.5 Accounting and Limits

_Peios / Advanced Peios / eventd / Query Execution_

> What every query records for the adaptive indexer, and the concurrency and timeout limits it runs under.

## 6.5.1 Recording what was asked

Every event query is recorded by the adaptive indexing system (§3.4).
For each `WHERE` predicate:

- a header column reference increments that column's frequency counter
- a payload field reference increments that path's counter

Cross-type `WHERE` predicates are counted like any other, since they
narrow the same data by the same fields.

This applies to **event queries only**. The log and metric stores have
fixed write-time indexes and no candidate space for a policy to explore
(§4.2, §5.2), so their queries increment nothing.

Counters are in-memory and are flushed to the metadata database at each
policy interval (§3.5). Query handlers never write to that database
directly, which is what keeps the once-per-query update off any lock a
writer thread contends for.

Metric queries feed the parallel rollup registry counters instead
(§5.6), which record `(function, window)` pairs rather than fields.

## 6.5.2 Concurrency

eventd bounds concurrent queries — streaming and non-streaming together
— at `MaxConcurrentQueries` (§A). Beyond it a query is rejected with an
error rather than queued.

The per-query cost that limit is protecting is real: read-only SQLite
connections, one per shard for an event query (§6.4), memory for the
merge, and CPU for execution and payload decoding.

`MaxStreamingQueries` is enforced separately and is lower. A streaming
query holds its resources for as long as its client stays connected,
where an ordinary one holds them for at most a timeout, so the two
populations need different bounds.

Both are global rather than per-caller. eventd cannot attribute
connections to a caller beyond the token it holds, so one client can
occupy every slot — and the interim protection is that queries and
ingestion are separate channels, so exhausting the query side cannot
exhaust ingestion (PSPU §3.3).

> [!NOTE]
> Per-caller limits would need a way to identify the connecting process
> beyond its token — a process GUID from the connection. That is the same
> class of missing primitive as the datagram peer identity that leaves
> `origin` unverifiable (PSPU §3.28), and the global limit is what stands
> in for it.

## 6.5.3 Timeouts

Every query has a maximum execution time, `QueryTimeoutMs` (§A).

The clock starts once the request has been decoded and the caller's
token obtained, and covers everything after: parsing, planning, access
checks, cross-type pre-computation, SQL execution, merging, aggregation,
pagination, projection, and transmitting the initial result set.

It bounds the **initial result set only**. A non-streaming query sends
`end` before it expires; a streaming query sends `watch`. Past `watch`
the stream is not time-limited, and what bounds it instead is
`MaxStreamingQueries`, `MaxDistinctStreamValues` and backpressure
(§6.6).

On expiry eventd cancels the query and sends an error. Any result
messages already sent are discarded by the client, since no terminal
message arrived (PSPU §3.16).

Cancellation has to reach two kinds of work. SQLite work is interrupted
through `sqlite3_interrupt` or an equivalent progress-handler check —
the same responsiveness problem as cancelling an index build (§3.4).
Non-SQL work — MessagePack flattening, the cross-shard merge — checks
the same deadline periodically, because a query can spend most of its
time in neither the database nor the kernel.

Large scans over unindexed fields are the main timeout risk, and the
adaptive indexing system reduces it over time by indexing whatever keeps
being filtered on — which is also why a timeout is a signal worth
watching rather than merely an error to retry.

---

# 6.6 The Streaming Machinery

_Peios / Advanced Peios / eventd / Query Execution_

> How a live watch is implemented — commit generations, latency, the DISTINCT seen set, cross-type re-evaluation and backpressure.

The externally visible behaviour of a streaming query — what may be
streamed, what applies during the watch phase, how `DISTINCT` streams
behave, and when a slow client is dropped — is PSPU §3.27. This is the
machinery underneath.

## 6.6.1 Commit generations

eventd keeps a monotonic `u64` commit generation counter for each
streamable store: one for the event store **as a whole**, and one for
the log store. Metric queries do not stream, so the metric store has
none.

After a writer commits a batch it increments the counter for its store
and wakes the streaming handlers waiting on it. A handler records the
last generation it processed and waits until the counter exceeds it.

The event counter covers the whole store rather than one per shard.
Several writer threads increment it, so a wake is "something committed
somewhere" and a handler re-examines every shard it cares about — which
is what it would have to do anyway, since a shard means nothing to the
query path (§6.4).

The counter is process-local and never persisted; it has no meaning
across a restart, and a streaming query does not survive one. On
wraparound the next increment is treated as a wake for every handler and
operation continues. At any commit rate a machine can sustain,
wraparound of a 64-bit counter is not reachable.

## 6.6.2 Latency

Delivery latency is bounded below by the commit interval of the store
concerned, because a record is not streamable until it is committed.

| Store | Approximate floor | From |
|---|---|---|
| Events | `MaxBatchLatencyMs`, default 100 ms | §2.4 |
| Logs | `LogMaxBatchLatencyMs`, default 500 ms | §4.1 |

Under light load the actual latency is lower, because the adaptive
batcher commits as soon as its input drains rather than waiting out the
cap (§2.4). Under sustained load it converges on the cap.

A consumer needing better than this is not served by eventd at all: the
KMES ring buffers are the low-latency path, they are specified in PSPK,
and attaching to them directly costs the per-event access control that
eventd exists to apply (§7).

## 6.6.3 The DISTINCT seen set

A `DISTINCT` stream holds a per-query set of the values it has already
emitted, initialised from the initial result set and added to as new
values appear (PSPU §3.27).

It is bounded by `MaxDistinctStreamValues` (§A), and exceeding the bound
terminates the query with an error rather than evicting. Eviction would
make the output wrong rather than merely truncated: a forgotten value
would be re-emitted as newly seen, and "newly seen" is the entire
meaning of the result.

The set is per query and in memory, which is why the bound exists and
why it is separate from the general query concurrency limit — sixty-four
streams each holding a hundred thousand values is a different memory
profile from sixty-four ordinary queries.

## 6.6.4 Cross-type re-evaluation

Pre-computed cross-type ranges describe the past and are discarded when
the watch phase begins (PSPU §3.27).

A **metric** condition costs one index seek per batch: the selector has
already been constrained to exactly one series, so finding the active
sample at the batch's latest candidate timestamp is a single lookup on
`idx_samples_series_timestamp` (§5.2).

An **existence** condition is evaluated per candidate record rather than
per batch, because the centred window is relative to each record's own
timestamp and a matching record may be near some of a batch and not the
rest.

The per-batch metric evaluation is an approximation, and the reason it
is acceptable is the ratio between the two intervals: a commit batch
spans a fraction of a second and a metric sample fifteen, so every
record in a batch normally maps to the same sample. At sub-second metric
resolution it filters more coarsely, and records near a threshold
crossing are included or excluded as a group.

## 6.6.5 Backpressure

Backpressure is detected on the socket send buffer. When a result
message cannot be sent because the buffer is full, the query is
terminated immediately; eventd never blocks on the send.

Blocking would put a slow reader in the path of eventd's own work, and
the write path is what would suffer. A streaming client is the
lowest-priority consumer of eventd's time, and dropping it is the same
principle as dropping an ingestion datagram (PSPU §3.4), applied on the
way out.

---

# 7.1 The Model

_Peios / Advanced Peios / eventd / Access Control_

> eventd enforces access on the read path only, through KACS descriptors, and decides nothing itself — caller identity, rights and timing.

eventd enforces access on the **read path only**, using KACS Security
Descriptors and the KACS AccessCheck API. Every query is evaluated
against descriptors that determine which records — and which fields
within a record — the caller may see, and everything else is filtered
out silently (PSPU §3.28).

## 7.1.1 eventd decides nothing itself

eventd implements no access check logic of its own. Every decision is
delegated to `kacs_access_check` and `kacs_access_check_list`, which run
the full KACS AccessCheck pipeline: integrity checks, restricted token
evaluation, confinement, conditional ACE evaluation, and the SACL audit
walk.

What eventd contributes is the three things AccessCheck needs and cannot
know — which descriptor applies (§7.2), which object type list describes
the record (§7.3), and what the caller's token is (§7.4) — and then it
acts on the verdicts.

The alternative would be reimplementing an access check algorithm that
already exists, in a daemon that would then have to be kept in agreement
with it forever.

## 7.1.2 Enforcement is at query time

All events, logs and metrics are stored regardless of who will ever be
allowed to read them, and two callers querying the same store see
different results.

Three reasons make this the only workable arrangement:

- **Audit integrity.** An audit event has to be stored whether or not
  anyone can currently read it. Filtering at storage time would let a
  descriptor decide what gets recorded, which is the one thing an audit
  store must not permit.
- **Descriptors change.** An administrator can grant or revoke access
  retroactively, and only query-time evaluation makes that meaningful.
- **Callers differ.** Several principals with different access levels
  query the same store, and there is one copy of the data.

## 7.1.3 Caller identity

When a client connects to the query socket, eventd obtains its token by
calling `kacs_open_peer_token` on the connected descriptor. The token
represents the peer's identity as captured **at connection time**.

If the call fails, eventd denies the query entirely. It has no fallback
identification and no anonymous mode.

The snapshot property matters most for streaming queries, which may run
indefinitely: a client whose group memberships change, or whose access
is revoked, continues to be evaluated against the token it connected
with until it disconnects (§7.5).

## 7.1.4 Rights

| Right | Bit | Value | Meaning |
|---|---|---|---|
| `EVENTD_READ` | 0 | 0x0001 | Read records matching the pattern. |
| `EVENTD_CLEAR` | 1 | 0x0002 | Delete records matching the pattern. |
| `EVENTD_ADMINISTER` | 2 | 0x0004 | Change eventd's own policy — the `INDEX` command (§7.2). |

The generic mapping passed to AccessCheck is in §B.

`EVENTD_ADMINISTER` is distinct from `EVENTD_READ` deliberately.
Accelerating a field costs write throughput on every record thereafter,
so `INDEX` is a way for a caller to degrade the system for everybody,
and a caller permitted only to read has not been permitted to do that
(PSPU §3.23).

> [!NOTE]
> `EVENTD_CLEAR` is defined and nothing yet uses it: no operation deletes
> records on a caller's behalf, and retention deletes on nobody's behalf
> (§3.6). It is reserved for administrative deletion. The consequence
> worth knowing is that a descriptor written today with `GENERIC_WRITE`
> already grants it, and will start granting a real capability the moment
> one exists.

## 7.1.5 What is not controlled here

**Event emission** is KMES's: `kmes_emit` and `kmes_emit_batch` require
SeAuditPrivilege, and eventd is not involved.

**Log and metric ingestion** has no per-record control at all. The
Security Descriptor on each ingestion socket is the entirety of it
(§7.6).

---

# 7.2 Patterns and Descriptors

_Peios / Advanced Peios / eventd / Access Control_

> Access is defined on named patterns each carrying a descriptor — matching, resolution, storage, and the first-boot defaults.

Access is defined on **named patterns**, each standing for a category of
observability data and each carrying a descriptor. The three data types
have independent pattern namespaces:

| Namespace | Patterns match |
|---|---|
| Events | event type |
| Logs | log origin |
| Metrics | metric name |

## 7.2.1 Matching

A pattern matches by **dot-delimited prefix**. The pattern `kacs` matches
the exact string `kacs` and any string beginning `kacs.` — and matches
neither `kacs_extended` nor `kacsfoo`, because the dot is the hierarchy
separator and not a mere character.

`*` is the wildcard default and matches everything.

Ingestion constrains origins and metric names to the identifier grammar
(PSPU §3.7, §3.10), which is what keeps a producer from choosing a name
containing the wildcard or a registry path separator — a name that would
otherwise match a rule its producer was never meant to satisfy, or store
its descriptor somewhere other than where the administrator who wrote it
believes.

## 7.2.2 Resolution

For a concrete identifier, eventd resolves the applicable descriptor by
walking up the hierarchy:

1. Look for an exact match on the full identifier, `kacs.access_denied`.
2. Remove the last dot-separated component and look again, `kacs`.
3. Repeat.
4. Fall back to the wildcard, `*`.

The first match wins; a more specific pattern overrides a less specific
one.

## 7.2.3 Storage

Descriptors are registry values under the eventd security subtree:

```text
Machine\System\eventd\Security\Events\*
Machine\System\eventd\Security\Events\kacs
Machine\System\eventd\Security\Events\kacs.access_denied
Machine\System\eventd\Security\Logs\*
Machine\System\eventd\Security\Logs\loregd
Machine\System\eventd\Security\Metrics\*
Machine\System\eventd\Security\Metrics\cpu
```

Each type's wildcard default is **load-bearing**. If a default is
missing, eventd denies access to all data of that type: resolution that
reaches the end of the hierarchy without a match is a denial, not a
grant (PSPU §3.28).

Storing them in the registry rather than in eventd's own databases means
the registry's access control protects them, and an administrator edits
them with the ordinary registry tools rather than through eventd.

## 7.2.4 Defaults on first boot

eventd creates the three wildcard keys if they do not exist:

| Key | Default |
|---|---|
| `…\Security\Events\*` | SYSTEM and Administrators: `EVENTD_READ` on all fields. |
| `…\Security\Logs\*` | SYSTEM, Administrators and Authenticated Users: `EVENTD_READ` on all fields. |
| `…\Security\Metrics\*` | SYSTEM, Administrators and Authenticated Users: `EVENTD_READ` on all fields. |

The asymmetry reflects sensitivity. Events include security audit data
and are restricted to administrators; logs and metrics are operational
data and are readable by any authenticated user. An administrator can
tighten either.

## 7.2.5 Conditional ACEs

Descriptors on eventd security objects may carry conditional ACEs, and
KACS evaluates them as it would anywhere.

eventd passes **no eventd-specific local claims** to AccessCheck:
`local_claims_ptr` is null and `local_claims_len` is zero. Conditions
referencing token claims that KACS itself supplies still evaluate;
conditions referencing eventd-local claims observe them as absent.

A stable set of eventd-local claims — the event's type, the log's
origin, the time of day — is a plausible later addition, and defining
one is a commitment to keep those claim names meaningful thereafter.

## 7.2.6 The administrative descriptor

`INDEX` (PSPU §3.23) is checked against `admin_sd`, held in the metadata
database rather than the registry (§3.5), with `EVENTD_ADMINISTER` as
the desired access. Its default grants SYSTEM and Administrators.

eventd refuses `INDEX` outright if no administrative descriptor exists,
by the same fail-closed rule as the read path.

> [!NOTE]
> This is the one piece of access control state outside the registry, and
> the protection of the directory holding it is not otherwise specified
> (§3.3). A process able to write the event store directory can rewrite
> the descriptor governing eventd's own policy.

---

# 7.3 Per-Field Control

_Peios / Advanced Peios / eventd / Access Control_

> Granting some fields of a record and not others through object ACEs — how field GUIDs are derived rather than registered.

A descriptor can grant read access to some fields of a record and not
others, using KACS **object ACEs** and object type lists. A caller
authorized for a pattern but not for a field receives the records with
that field absent — indistinguishable from a record that never carried
it (PSPU §3.28).

## 7.3.1 Object type lists

For each access check eventd builds an object type list: a two-level
tree with the data type's root at level 0 and one field per node at
level 1.

```text
Level 0: root GUID for the data type
  Level 1: timestamp
  Level 1: event_type
  Level 1: cpu_id
  Level 1: origin_class
  Level 1: effective_token_guid
  Level 1: true_token_guid
  Level 1: process_guid
  Level 1: granted_access
  Level 1: target_sid
  Level 1: source.name
```

`kacs_access_check_list` returns a verdict per node, and eventd uses
them to include or exclude each field.

The three root GUIDs — one for events, one for logs, one for metrics —
are in §B.

## 7.3.2 Field GUIDs are derived, not registered

A field's GUID is computed deterministically with UUID v5 (RFC 4122):

```text
field_guid = uuid_v5(EVENTD_FIELD_NAMESPACE, field_name)
```

with the namespace UUID in §B, and `field_name` the field's
query-language name as UTF-8.

There is **no registry of field GUIDs and no allocation step**. The same
name always yields the same GUID, so an administrator writing a
descriptor computes the GUID from the field name with the same algorithm
that eventd will use when it builds the list. This is the one part of
eventd's access control that a third party reproduces rather than merely
consumes.

Derivation rather than registration is what makes payload fields
tractable at all: event payload schemas belong to the emitting
subsystems, eventd has no catalogue of them, and a new event type
carrying a new field needs no registration anywhere before a descriptor
can name it.

Which names are used:

| Data | `field_name` |
|---|---|
| Event header field | the column name — `timestamp`, `event_type`, `cpu_id` |
| Event payload field | the flattened dot path — `granted_access`, `target_sid`, `source.name` |
| Log field | the column name — `origin`, `message`, `is_error` |
| Fixed metric field | `timestamp`, `boot_id`, `name`, `type`, `value` |
| Metric label | the label key — `core`, `device` |

Payload fields suppressed by flattening or by a header collision are not
query-language fields, so they get no GUID (PSPU §3.22). Metric label
keys cannot collide with the fixed metric fields, because ingestion
rejects records whose labels do.

## 7.3.3 The GUID does not encode scope

A field GUID names a field and nothing else. `granted_access` produces
the same GUID whatever event type carries it.

Scoping comes from the descriptor hierarchy: an object ACE naming the
`granted_access` GUID inside the descriptor for pattern `kacs` means
"the `granted_access` field of KACS events". The same ACE in a different
pattern's descriptor means the same field of that pattern's records.

## 7.3.4 Writing one

An object ACE with **no** object type GUID applies to the root and
therefore to every field. One **with** a field GUID applies to that
field.

To grant a security team full read access to KACS events, and a
monitoring team only the timestamp, type and CPU:

- Allow SecurityAdmins, `EVENTD_READ`, no object GUID
- Allow MonitoringTeam, `EVENTD_READ`, object GUID = `timestamp`
- Allow MonitoringTeam, `EVENTD_READ`, object GUID = `event_type`
- Allow MonitoringTeam, `EVENTD_READ`, object GUID = `cpu_id`

MonitoringTeam querying KACS events receives records containing exactly
those three keys. Payload fields, identity GUIDs and the remaining
header fields are absent.

## 7.3.5 Building the list per record

The list is constructed from the fields actually present in the record
being checked: the root node, then a level-1 node per field.

**Event records** contribute every header field plus every non-suppressed
flattened payload field present in that particular payload. Different
event types produce different lists, because they carry different
payloads — and two events of the same type can too, since a payload is
opaque MessagePack and nothing requires two of a type to agree.

**Log records** have a fixed field set — `timestamp`, `origin`,
`is_error`, `message`, `job_id`, `boot_id` — so the list is the same for
every log record.

**Metric records** contribute the five fixed fields plus the series'
label keys, which vary per series.

Derived aggregate outputs — `count`, `sum`, `avg`, `min`, `max` — are
omitted from the list entirely. They are not source fields, have no GUID
and no access identity of their own, and are visible when the caller is
authorized for the records and source fields they were computed from
(PSPU §3.28).

---

# 7.4 Enforcement

_Peios / Advanced Peios / eventd / Access Control_

> Access control is the third phase of query evaluation — which clauses count as referencing a field, and why denial is silent.

The order in which eventd evaluates a query is fixed (PSPU §3.18), and
access control is the third phase — before predicates, transforms,
grouping, aggregation, ordering and pagination. What follows is the
sequence within that phase.

1. **Obtain the caller's token** from the connection (§7.1). Failure
   denies the query.
2. **Parse** the query to establish its data sources and filters (§6.1).
3. **Discover the concrete identifiers** the query could touch — event
   type strings, log origin strings, metric name strings. A broad
   selector authorizes nothing by itself: `EVENTS`, `EVENTS kacs.*`,
   `LOGS` without `FROM` and `METRIC cpu.*` are each resolved identifier
   by identifier.
4. **Resolve and check** each discovered identifier: find its descriptor
   by hierarchical matching (§7.2), build the object type list for the
   fields the query references (§7.3), call `kacs_access_check_list`,
   and cache the verdicts for this `(token, identifier, field set)`
   (§7.5).
5. **Apply root verdicts.** An identifier whose root is denied is
   invisible: its records are excluded from the logical row set before
   aggregation, ordering, pagination and formatting. For a cross-type
   source, a denied identifier is treated as having no matching data.
6. **Apply field verdicts to predicates.** Where the query references a
   field in a predicate or shaping clause and a matching identifier does
   not grant it, that identifier's records contribute nothing — exactly
   as if its root had been denied. The query is **not** rejected.
7. **Cross-type sources** get the same treatment. A denied root, or a
   denied field needed to evaluate the condition, makes the condition
   evaluate as though no matching cross-source data existed.
8. **Execute**, with root filtering already part of the logical row set.
9. **Re-resolve per result identifier.** For each distinct concrete
   identifier in the resulting rows, resolve its descriptor, build the
   object type list with field GUIDs, call `kacs_access_check_list` with
   the token, the descriptor, `EVENTD_READ`, the list and an audit
   context naming the identifier, and cache the per-field results.
10. **Shape each record.** Look up the cached results for its
    identifier; exclude the record entirely if the root was denied;
    otherwise include it, and include each field only if its node was
    granted.

## 7.4.1 Which clauses count as referencing a field

Step 6 applies to every clause that reads a value rather than merely
displaying one:

- ordinary `WHERE` predicates
- metric label filters in a primary selector
- `GROUP`, `COUNT BY`, `TOP N BY`, `SORT` and `DISTINCT` fields
- event and log aggregation arguments — `SUM`, `AVG`, `MIN`, `MAX`
- metric transforms and terminal aggregations, all of which read the
  fixed `value` field: `RATE`, `DELTA`, `P50`, `P95`, `P99`, `AVG`,
  `MIN`, `MAX`, `SUM`, `AVG_OVER`, `MIN_OVER`, `MAX_OVER`, `SUM_OVER`
- a metric boot filter, which reads `boot_id`; and an explicit metric
  type predicate, grouping, sort or distinct, which reads `type`

`SELECT` is not on this list. It shapes output and is applied last, so a
field it omits was still available to every earlier phase — and
conversely, selecting a field the caller may not read removes the field,
not the record.

## 7.4.2 Denial is silent, not fatal

Step 6 excludes rather than rejects, and this is the decision most worth
being explicit about, because rejecting is the more informative
behaviour and that is exactly the objection to it.

A rejection would tell the caller that some identifier exists, matches
its query, and carries a field it may not read — three facts about data
it was not permitted to see, delivered by the mechanism meant to
withhold them, and enumerable by trying queries and watching which ones
fail. Excluding costs the caller a result narrower than it appears;
rejecting costs the model the property it rests on.

Cross-source fields already worked this way (step 7); primary-source
fields now match them.

## 7.4.3 Field authorization does not depend on presence

A field's authorization is resolved from the name **as written**,
against each concrete identifier, whether or not any record of that
identifier actually carries it.

Payload fields vary between records of the same type, so a rule turning
on presence would require the scan that authorization is meant to
precede.

## 7.4.4 Internal values are not fields

Row identifiers, series identifiers, ordering tiebreakers and series
type checks are not query-language source fields unless the mode exposes
them as fixed fields or the query names them (§6.2). Errors raised by
internal checks never carry a denied field's value.

A metric result's `value` **is** a source field, because it is either a
raw sample or a scalar derived from raw samples.

## 7.4.5 Filtering is part of the logical result

Access filtering is not a presentation step. Aggregating, ordering or
paginating over unreadable records would leak them through counts,
through ordering, and through the gaps in pagination.

eventd may push authorization predicates into SQL when the identifier
set is known, or read candidates and filter them before aggregating
(§6.3). The externally visible result is identical to filtering first,
and `COUNT`, `COUNT BY`, `TOP N BY` and every other aggregate reflect
only what the caller may see.

## 7.4.6 The audit trail

Every access check produces a KACS audit event through the SACL audit
walk in the AccessCheck pipeline.

eventd passes an `audit_context` blob naming the security pattern being
accessed — `"events:kacs.access_denied"`, `"logs:loregd"` — so the audit
trail records exactly which observability data was read, by whom, rather
than merely that eventd performed a check.

Those audit events are themselves KMES events, which eventd consumes and
stores, and which are governed by the `synthetic`-independent event
patterns like any other. Reading the audit store is auditable.

---

# 7.5 Caching

_Peios / Advanced Peios / eventd / Access Control_

> An access check is a syscall, and a large result cannot afford one per record — the record-level, field-level and descriptor caches.

An access check is a syscall through the full AccessCheck pipeline. A
query returning ten thousand records cannot afford ten thousand of them,
so results are cached — at two levels, plus the descriptor resolution
underneath both.

## 7.5.1 Record-level

When a pattern's descriptor contains **no object ACEs**, the check is a
plain grant or deny on the root, and the result is cached per
`(token, pattern)`.

A query returning ten thousand events across twenty distinct event types
performs at most twenty checks.

## 7.5.2 Field-level

When the descriptor **does** contain object ACEs, the verdict depends on
which fields the record carries, since different payloads produce
different object type lists (§7.3). The result is cached per
`(token, pattern, field set)`.

In practice events of one type carry the same fields, so this is
effectively one check per `(token, event type)`. Log records have a
fixed field set, so log queries reach one check per origin. Metric
records vary by series label keys.

The pathological case is an event type whose payload fields differ from
record to record, which produces a distinct field set — and a distinct
cache entry, and a distinct syscall — for each shape encountered.

## 7.5.3 Descriptor resolution

Resolving a pattern to a descriptor is itself cached, across queries
rather than within one, since it costs a registry read and a hierarchy
walk (§7.2).

eventd watches the security registry subtree and invalidates cached
resolutions and cached check results when a descriptor changes. That is
what makes a revocation take effect on the next query rather than at the
next restart.

If the registry watch fails after startup, eventd **discards the
descriptor cache and operates fail-closed for new resolutions** until
the watch is re-established. A cache it cannot trust to be current is
worse than none: continuing to serve from stale entries would make a
revocation silently ineffective, and the failure would be invisible.
This is a degraded state, not a failure — eventd keeps ingesting, and
keeps answering queries for descriptors already resolved (§9.3).

## 7.5.4 During a stream

Verdicts reached for a streaming query's initial result set are reused
through the watch phase, with two exceptions.

A **new concrete identifier** appearing in a streamed batch — an event
type or log origin not present in the initial results — is resolved and
checked before the record or its distinct value is used. It has never
been authorized, and inheriting a verdict from a sibling pattern would
be a grant nobody made.

A **descriptor change** invalidates the cache as it does anywhere, and
subsequent batches are re-checked against the new one.

The **token** is not re-examined. It was captured at connection (§7.1),
so a client whose memberships change mid-stream continues under what it
connected with, and a client whose access is revoked keeps receiving
records until it disconnects. The bound on that exposure is the client's
own connection lifetime, which for a dashboard may be days.

> [!NOTE]
> Re-obtaining the peer token periodically would narrow the window, and
> would also mean a streaming query could change what it returns halfway
> through for reasons the client cannot see. The snapshot is the simpler
> contract and is what PSPU §3.14 states; the exposure is the cost.

---

# 7.6 The Write Path

_Peios / Advanced Peios / eventd / Access Control_

> There is no per-record access control on the way in — what stands in its place, and what that leaves open for an operator.

There is no per-record access control on the way in. This article
records what stands in its place and what that leaves open.

## 7.6.1 Events

Emission is KMES's business. `kmes_emit` and `kmes_emit_batch` require
SeAuditPrivilege, and eventd is not in the path — it consumes what KMES
delivers and applies no admission control of its own (§2.2).

The identity stamps on an event are the kernel's, captured from kernel
state at the moment of the write, and an emitting process cannot set,
influence or suppress them. That is what makes an event's `process_guid`
evidence in a way that a log's `origin` is not.

## 7.6.2 Logs and metrics

The Security Descriptor on each ingestion socket is the entirety of the
write-path control (PSPU §3.3). There is nothing per record: no token is
obtained, no identity is checked, and no field is verified.

The socket descriptor is therefore doing all the work, and it is worth
being precise about what it does and does not do on Peios. An access
decision is routed through the object's Security Descriptor, not through
POSIX mode bits, so setting a mode on a socket pathname restricts
nothing — and an inode created without a descriptor is denied to every
caller, so binding a socket into a directory carrying no inheritable
ACEs produces a socket that nothing, including the service manager, can
reach. eventd establishes the descriptor on each socket before it begins
receiving on it.

## 7.6.3 Origin and name are claims

`origin` in a log record and `name` in a metric record are self-reported
(PSPU §3.7, §3.10). Any process that can reach an ingestion socket can
write under any origin or metric name it likes, including one belonging
to another program.

The consequences are the obvious ones. A compromised service can inject
log lines attributed to another service, manufacturing a plausible
operational narrative. It can bury a real incident under noise
attributed elsewhere. It can create metric series under a name a
dashboard trusts.

Read-path descriptors limit who can *see* data written under a given
identifier; they do nothing about who wrote it. eventd never presents a
stored `origin` or metric `name` as evidence of provenance.

> [!NOTE]
> Closing this needs a primitive eventd does not have: a way to obtain
> the peer's token for a **datagram**, as `kacs_open_peer_token` does for
> a stream connection. With one, an ingestion socket could carry a
> descriptor governing which origins a sender may write, checked per
> datagram — which is the shape the read path already has. Until it
> exists, confining which processes can reach the socket at all is the
> only available control, and it is coarse: it is a decision about
> processes, where the thing worth controlling is names.

## 7.6.4 What this means for an operator

The interim posture is that the ingestion sockets are a trust boundary
that only separates "can reach eventd" from "cannot" — and on a system
where every service logs, nearly everything is on the inside.

Two things follow. Data whose provenance must be trustworthy belongs in
an event, where the kernel stamps the identity, rather than in a log
where the producer asserts it. And read-path descriptors on the origins
and metric names that matter are worth writing even so: they prevent an
unauthorized reader from *querying* data written under a spoofed
identifier, which is a smaller property than authenticity but not
nothing.

---

# 8.1 Dependencies

_Peios / Advanced Peios / eventd / Startup And Shutdown_

> The four subsystems eventd needs before it can do anything, the one that is not really a dependency, and where it sits in the boot.

eventd needs four subsystems before it can do anything.

| Subsystem | For |
|---|---|
| KMES | Event ingestion. Available as soon as PKM is loaded. |
| LCS and loregd | Configuration. eventd reads every setting from the registry. |
| KACS | Access control — the AccessCheck API for query authorization, and `kacs_open_peer_token` for caller identification. |
| peinit | The boot ID, and lifecycle management. |

eventd is a peinit-managed service, started after loregd is available —
it cannot read a single configuration value without the registry, and it
has no compiled-in defaults for the six paths it needs (§A).

## 8.1.1 The dependency that is not one

KACS is needed to *serve* queries and not to ingest. The drain, write
and retention paths never call it. That asymmetry is what lets eventd
keep ingesting through a KACS outage while refusing every query (§9.3),
and it is the right way round: losing the ability to read the audit
store is recoverable, losing the events is not.

## 8.1.2 Ordering in the boot

eventd is one of the platform daemons and is Critical: peinit reboots
the system rather than continuing without it. It comes up after loregd
and authd, and it stops before them on the way down — it is among the
last services shut down, because everything else's shutdown is worth
recording.

The window before eventd exists is real and peinit covers it by
buffering service output until the log socket appears. Events emitted
during that window are not lost either: they sit in the KMES ring
buffers, and eventd's first drain after attaching reads them from
`tail_pos` (§2.2).

---

# 8.2 The Bootstrap Sequence

_Peios / Advanced Peios / eventd / Startup And Shutdown_

> The seven startup phases, which either complete or fail entirely — configuration, KMES attachment, storage, sockets and the rest.

Startup proceeds in seven phases, and either completes or fails
entirely.

## 8.2.1 Phase 1 — Configuration

1. Read every configuration key under `Machine\System\eventd\`. The six
   required keys are `EventStorePath`, `LogStorePath`, `MetricStorePath`,
   `QuerySocketPath`, `LogSocketPath` and `MetricSocketPath`; a missing
   or invalid one fails startup.
2. Read the optional keys and apply compiled-in defaults for those
   absent (§A).
3. Arm a persistent watch on the subtree, for runtime changes (§8.3).

## 8.2.2 Phase 2 — KMES attachment and shard sizing

4. Discover the CPU count by calling `kmes_attach` with incrementing
   CPU identifiers from 0 until `EINVAL`. The call requires
   SeSecurityPrivilege in the effective token. Discovering no CPUs fails startup.
5. Map each per-CPU ring buffer.
6. Resolve the active shard count from `StorageShards` — the CPU count
   when it is 0, the configured value otherwise.
7. Compute the shard-to-CPU assignment (§2.3).

## 8.2.3 Phase 3 — Storage

8. Open or create each active event shard: verify the schema version,
   open in WAL mode with `synchronous=FULL`, create tables and indexes
   if new, and quarantine on reported corruption (§3.3). Discover
   historical shards matching the naming pattern, and open those with a
   recognised schema read-only; exclude the rest from the query path.
9. Open or create the log store — schema verified, WAL,
   `synchronous=NORMAL`, quarantine on corruption (§4.3).
10. Open or create the metric store, likewise (§5.4). The series cache
    starts empty and fills on demand.
11. Open or create `eventd-meta.db`. Load the index and rollup counters
    and desired sets, load the sequence checkpoints for diagnostics
    only, and discover each shard's material indexes from its schema
    (§3.5).

## 8.2.4 Phase 4 — The boot boundary

12. Read the current boot ID from peinit.
13. Search every readable event shard for committed rows carrying it.
14. **No rows** — the boot's first eventd start. Reset every per-CPU
    sequence tracker to 0 and record the new boot ID for subsequent
    writes.
15. **Rows exist** — a restart within the boot. Derive each CPU's resume
    point from the maximum committed `sequence` for
    `(boot_id, cpu_id)` across every readable shard; CPUs with no rows
    resume at 0 (§3.7).

## 8.2.5 Phase 5 — Sockets

16. Create the query socket at `QuerySocketPath`. A stale pathname left
    by a crash is unlinked first if it is an `AF_UNIX` socket; if the
    path exists and is not a socket, startup fails.
17. Create the log socket at `LogSocketPath`, same rule.
18. Create the metric socket at `MetricSocketPath`, same rule.
19. Establish the Security Descriptor on all three, before any of them
    accepts or receives anything (§7.6).

The stale-socket rule distinguishes the two cases deliberately.
Unlinking a leftover socket is recovery from eventd's own crash;
unlinking a regular file at a configured path would be destroying
something that is not eventd's, and a path pointing at the wrong thing
is a configuration error worth failing on.

## 8.2.6 Phase 6 — Threads

20. One drain thread per CPU, each beginning to read its ring buffer.
21. One writer thread per active shard.
22. The log ingestion thread.
23. The metric ingestion thread.
24. The retention thread.
25. The adaptive indexing and rollup policy thread.

## 8.2.7 Phase 7 — Ready

26. Write and **commit** a `synthetic.startup` event recording the boot
    ID, the shard count and the per-CPU resume points (§3.2). The commit
    happens before readiness is signalled.
27. Signal readiness to peinit.

Committing before signalling is what makes the startup record
trustworthy. A readiness signal sent before the commit could be followed
by a crash that loses the record, leaving a boot in which eventd
demonstrably ran and left no trace of having started.

## 8.2.8 Failure

If any phase fails, eventd does not signal readiness. It logs the
failure to standard error where standard error exists, and exits
non-zero. peinit's restart policy decides what happens next.

**Partial startup is not permitted.** There is no degraded mode in which
eventd runs without one of its three stores, or without KMES. It either
completes the sequence or fails.

The all-or-nothing rule is a simplification. A later revision might
allow log and metric ingestion to proceed with KMES unavailable, but
that means partial-failure state to manage in every subsequent path —
what a query against a store that was never opened does, what happens
when the missing subsystem returns — and the failure mode it protects
against is one peinit already handles by restarting.

---

# 8.3 Configuration at Runtime

_Peios / Advanced Peios / eventd / Startup And Shutdown_

> Which settings apply immediately, which wait for a restart, which do neither, and what SIGHUP does.

eventd watches `Machine\System\eventd\` and reacts to changes without
restarting — for the settings that can be changed that way.

Notifications arriving **during** startup are queued and processed after
readiness is signalled (§8.2). Applying a configuration reload to
half-initialised state would mean every phase having to tolerate its
inputs changing underneath it.

## 8.3.1 What applies immediately

Tuning parameters: batch sizes and latencies for all three writers,
retention periods and the delete batch size, adaptive index and rollup
thresholds and windows, the adaptive scalar rollup window, the WAL
checkpoint threshold, the query timeout, the cross-type window and
lookback limit, and the query and streaming concurrency limits.

## 8.3.2 What waits for a restart

| Change | Why |
|---|---|
| Socket paths | The sockets are bound and clients are connected to them. |
| Store paths | The databases are open, and moving a store is a data migration, not a setting. |
| `StorageShards` | Shard-to-CPU assignment, writer threads and handoff channels are all built from it at startup (§2.3). |

The watch notices these changes and eventd defers them rather than
attempting a live migration. Shard count changes in particular are
expected once in a machine's life, and rebalancing writer threads while
events are in flight is a large mechanism for a rare event.

## 8.3.3 What is neither

Security Descriptors under `Security\` are not configuration in this
sense. The registry watch invalidates the descriptor cache and the next
query resolves afresh (§7.5), so a grant or revocation takes effect
immediately without anything being "applied".

## 8.3.4 Recording it

eventd emits a `synthetic.config_change` event for every change applied
at runtime, carrying the key name and the old and new values rendered
deterministically (§3.2).

Invalid values are ignored and the previous value is retained — an
administrator who types a batch size outside its range does not get a
daemon that stops working, and the retained value is the one already in
use rather than the compiled-in default. Unknown keys in the subtree are
ignored entirely.

## 8.3.5 SIGHUP

`SIGHUP` re-reads the configuration, equivalent to a watch notification
(§8.5). It exists for the case where the watch itself is not delivering
— a registry outage, or a watch that failed and has not re-armed
(§9.3) — and gives an administrator a way to force the read rather than
restarting the daemon.

---

# 8.4 Shutdown

_Peios / Advanced Peios / eventd / Startup And Shutdown_

> Persisting as much in-flight data as possible without blocking indefinitely — the sequence, and the timeout that bounds it.

When peinit signals a stop, eventd persists as much in-flight data as it
can without blocking indefinitely.

## 8.4.1 The sequence

1. **Stop accepting.** Unlink all three socket paths so no new client
   can reach them, and stop accepting query connections. Existing
   streaming queries are terminated with an error. The log and metric
   socket descriptors stay **open**.
2. **Drain ingestion.** Read and process the datagrams still in the log
   and metric receive queues, then close those descriptors. This is
   bounded by the queue size — four times the datagram ceiling — so it
   completes quickly.
3. **Final event drain.** Each drain thread performs one last drain
   cycle from its ring buffer.
4. **Final commit.** Every writer commits its current batch immediately,
   whatever its size. The log and metric writers do the same.
5. **Record sequence state.** Derive the per-CPU last committed
   sequence from committed rows — the same rule startup uses — and write
   it to `sequence_checkpoints` for diagnostics (§3.5).
6. **Emit the shutdown event.** Write `synthetic.shutdown` with the
   per-CPU sequences, using the daemon-wide shard assignment rule
   (§2.6). If no shard is writable, the event is skipped and the failure
   logged to standard error.
7. **Close databases.** Close every connection, writer and reader.
   SQLite checkpoints the write-ahead log automatically on close.
8. **Unmap.** Unmap every ring buffer and close the per-CPU descriptors.
9. **Exit.**

Steps 1 and 2 are deliberately split. Unlinking the pathnames stops new
senders finding the socket while the descriptors stay open, so whatever
is already queued is still readable — closing them at step 1 would
discard the queue, which is the data most recently produced and
therefore most likely to explain why the system is being stopped.

The final checkpoint in step 7 matters most for the log and metric
stores, which run `synchronous=NORMAL` and whose durability boundary is
the checkpoint rather than the commit (§4.1).

## 8.4.2 The timeout

Shutdown is bounded by peinit's service stop timeout. If the sequence
has not finished, eventd aborts and exits immediately.

What an aborted shutdown costs:

- **Uncommitted event batches** are lost. Those events remain in the
  KMES ring buffers and are available at the next start, provided they
  have not been overwritten by then.
- **Uncommitted log and metric batches** are lost, which is acceptable
  by design.
- **The diagnostic sequence metadata** may be stale. It does not matter:
  startup derives resume points from committed rows and detects the
  difference between those rows and the ring buffer state as an ordinary
  gap (§2.5).

Every consequence is one the restart path already handles, which is why
aborting is safe rather than merely tolerable.

---

# 8.5 Crash Recovery and Signals

_Peios / Advanced Peios / eventd / Startup And Shutdown_

> What an ungraceful termination leaves behind, the signals eventd handles, and the diagnostic dump.

## 8.5.1 After a crash

An ungraceful termination — a segmentation fault, a kill, an
out-of-memory kill — leaves four things true.

**The ring buffers are unaffected.** KMES writes regardless of consumer
state, and events emitted while eventd was down accumulate there.

**The databases are consistent.** WAL mode guarantees committed
transactions survive, and SQLite rolls back the in-flight batch on the
next open.

**There is a sequence gap.** Events between the last committed batch and
the crash were never persisted. On restart eventd derives its resume
points from committed rows, sees the difference from the current ring
buffer state, and writes a gap record (§2.5).

**Socket-buffered data is gone.** The kernel discards a socket receive
queue on process exit, taking whatever logs and metrics were waiting.
Acceptable by the loss model.

No manual recovery is needed and none is offered. eventd restarts,
re-attaches, resumes draining, and records what was missed. The boot
boundary logic recognises the restart from the committed rows themselves
rather than from any flag written in advance (§3.7) — which is the
point, since a crash is precisely the case where nothing was written in
advance.

## 8.5.2 Signals

| Signal | Behaviour |
|---|---|
| `SIGTERM` | Begin graceful shutdown (§8.4). |
| `SIGINT` | Begin graceful shutdown. |
| `SIGQUIT` | Write a diagnostic dump to standard error, then begin graceful shutdown. |
| `SIGHUP` | Re-read configuration from the registry (§8.3). |

Every other signal keeps its default behaviour.

## 8.5.3 The diagnostic dump

`SIGQUIT` writes human-readable text to standard error before step 1 of
the shutdown sequence, so that it reflects the daemon's state while it
is still running rather than while it is tearing down. It includes at
least:

- the current boot ID
- the active shard count and the readable historical shard count
- the per-CPU last committed sequence numbers, derived from committed
  rows
- the current non-streaming and streaming query counts
- the metric series cache occupancy
- the last observed write error for each store, where one exists

The format is not a stable machine interface and its wording may change.

The set is chosen to answer the questions an operator has about a
misbehaving eventd that a query cannot: how far behind the writers are,
whether the series cache is thrashing (§5.3), whether query slots are
exhausted (§6.5), and whether a store has been failing writes quietly.
Standard error is the destination because peinit captures it, so the
dump reaches the log store by the ordinary path — and reaches standard
error directly when the log store is the thing that is broken.

---

# 9.1 Losing Events

_Peios / Advanced Peios / eventd / Failure Modes_

> The one failure that costs something unrecoverable, and why the whole ingestion pipeline is shaped around delaying it.

Every other failure in this chapter costs something recoverable. This
one does not, which is why the whole ingestion pipeline is shaped around
delaying it.

## 9.1.1 Ring buffer overrun

When events are emitted faster than eventd drains them, the per-CPU ring
buffers fill and KMES overwrites its oldest entries.

- eventd sees it as a sequence gap on the affected CPU (§2.5).
- A `synthetic.gap` record is written, naming the missing range.
- Draining resumes from the oldest survivor at `tail_pos`.

The events are gone. No other copy exists, and a gap record is a
tombstone rather than a recovery — it records what was lost and when,
which is the most that can be offered.

Four mechanisms delay it:

| Mechanism | Effect |
|---|---|
| Adaptive batch sizing (§2.4) | Commits as often as throughput allows, so the writer stays close to the drain rate. |
| Index shedding (§3.4) | Drops per-insert index cost under pressure, including all of it at once in the emergency case. |
| Sharding (§2.3) | Scales write throughput with the shard count. |
| Ring buffer capacity | The absorption window, sized by an administrator. |

The first three are eventd's and operate automatically. The fourth is
KMES's and is the one an operator can enlarge for a workload that bursts
predictably.

## 9.1.2 Query timeouts

A query exceeding `QueryTimeoutMs` is cancelled and the client receives
an error (§6.5). Read-only connections are released; nothing is lost,
and the query simply did not finish.

Streaming queries are bounded only up to `watch`; past that the watch
phase is not time-limited.

The main risk is a large scan over a field with no index, which adaptive
indexing reduces over time by indexing whatever keeps being filtered on
(§3.4). A timeout is therefore worth reading as a signal about the index
set rather than only as an error to retry.

## 9.1.3 Ingestion backpressure

When a log or metric socket's receive queue is full, the kernel discards
the datagram. Neither the sender nor eventd is notified, and eventd does
not count it.

This is by design and is not a failure to be tuned away (PSPU §3.4). The
one operational note is that the queue is not being drained *while a
batch is committing*, because the same thread does both jobs (§4.1) —
so a burst arriving during a commit is the common case for log loss.

---

# 9.2 Storage Failure

_Peios / Advanced Peios / eventd / Failure Modes_

> What a full disk does to each of the three stores, and how corruption is detected and contained.

## 9.2.1 Disk full

When the filesystem holding a store reaches capacity, SQLite writes
fail.

After any write failure consistent with disk-full or quota exhaustion,
eventd schedules an immediate retention run across every store whose
retention is enabled, under the ordinary bounded-batch rules (§3.6,
§4.4, §5.5). Retention is the only lever eventd has that frees space,
and waiting up to an hour for the next scheduled pass would waste the
window in which recovery is still cheap.

### 9.2.1.1 The event store

A failed `INSERT` or `COMMIT` does not crash the writer thread.

The batch is lost, and it is not recoverable: those events were already
consumed from the ring buffer, so KMES no longer has them. The writer
records the per-CPU sequence ranges of the failed batch in an in-memory
**lost-batch list**, and on its next successful commit emits
`synthetic.gap` records for every accumulated range before writing new
events.

That ordering matters. Emitting the gap records first means the store
never contains events written after a loss without the record of the
loss preceding them.

The writer also logs the failure to standard error immediately —
including the CPU identifiers and sequence ranges — which peinit
captures. That is the only visibility available while the disk is still
full and the gap record cannot yet be written.

If eventd crashes before the disk recovers, the in-memory list dies with
it. Nothing is silently lost even so: on restart, resume points are
derived from committed rows, and the difference from the current ring
buffer state is detected as an ordinary restart gap (§3.7). The record
is coarser — one gap rather than several — but the loss is still
recorded.

Meanwhile events accumulate in the ring buffers. If the disk stays full
long enough, they overrun and additional loss occurs, detected by the
same mechanism (§9.1).

### 9.2.1.2 The log and metric stores

A failed commit loses the batch. The writer retries on the next one.
Acceptable under the loss model, and no lost-batch accounting exists for
either — there is nothing to reconcile against, since neither has
sequence numbers.

## 9.2.2 Corruption

Corruption from a hardware error, a filesystem bug, or an incomplete
write during a kernel crash.

**Detection at startup** is structural: eventd verifies that the
required tables and indexes exist. It does **not** run
`PRAGMA integrity_check`, which scans the entire database and costs time
proportional to its size — unacceptable for a large event store on every
boot. Corruption that leaves the schema intact, such as a single bad
page, is found later, at query or write time, when SQLite touches it.

**At startup**, when SQLite reports corruption in a required active
store, eventd quarantines the files, creates a fresh empty database at
the original path, and continues (§3.3). It logs the corruption and
emits `synthetic.storage_error` once a shard is available to hold it.

**At write time**, eventd stops writing to the affected database, emits
`synthetic.storage_error` if it can, quarantines and replaces the
database, and resumes writes to the replacement.

**At query time**, a handler encountering corruption fails the affected
query with an error. It does **not** return the rows it managed to read:
partial data from a database SQLite has declared corrupt is
indistinguishable from complete data, and silently under-reporting an
audit query is worse than failing it.

A missing or unrecognised **schema version** is not corruption. It is
not repaired and not migrated: a required store with one fails startup,
and a historical shard with one is excluded from the query path
(§3.3).

Recovering data from a quarantined file is an administrative operation.
eventd never attempts automatic repair, and the quarantined file is the
only copy of whatever it held.

---

# 9.3 Losing Dependencies

_Peios / Advanced Peios / eventd / Failure Modes_

> Two of eventd's four dependencies can vanish after startup without stopping it — what survives, and what stops working.

Two of eventd's four dependencies can disappear after startup without
stopping it, and in both cases the ingestion path survives while the
query path does not. That asymmetry is deliberate: events that are not
collected are gone, and queries that cannot be answered can be asked
again.

## 9.3.1 The registry becomes unavailable

If LCS or loregd goes away after eventd has started:

- eventd keeps its last known configuration. Changes are not applied
  until the registry returns.
- Descriptor lookups fall back to the cache. A pattern the cache does
  not hold is **denied**, fail-closed (§7.5).
- eventd keeps ingesting and keeps serving queries for descriptors it
  already resolved, indefinitely.
- When the registry returns, the watch fires and eventd re-reads.

This is a degraded state, not a failure. eventd does not exit, and it
does not stop collecting.

The related case is the **watch failing** while the registry is
otherwise reachable. eventd discards the descriptor cache and operates
fail-closed for new resolutions until the watch is re-established
(§7.5), because a cache it cannot trust to be current would make a
revocation silently ineffective. `SIGHUP` forces a configuration re-read
in the meantime (§8.3).

## 9.3.2 KACS becomes unavailable

If KACS goes away after startup:

- `kacs_open_peer_token` fails on new query connections, so new queries
  are denied.
- `kacs_access_check` and `kacs_access_check_list` fail, so a query in
  progress that needs a fresh check is denied.
- Cached check results stay valid for the duration of the query that
  obtained them.
- **Event ingestion is unaffected.** Neither the drain nor the write
  path calls KACS (§8.1).
- Log and metric ingestion are unaffected.

eventd keeps collecting and cannot answer. Query service resumes when
KACS does.

## 9.3.3 KMES

There is no partial mode. eventd attaches to every per-CPU ring buffer
at startup and fails to start if it cannot (§8.2); there is no
subsequent state in which KMES is present but unusable, because the
mapping is established once and the read protocol has no call that can
fail afterwards. A ring buffer resize is handled as a generation change,
not as a failure (§2.2).

## 9.3.4 peinit

peinit supplies the boot ID at startup and manages the lifecycle. It has
no runtime role once eventd is running, so there is no failure mode
here — peinit going away means the system is going away.

---

# 9.4 Resource Exhaustion

_Peios / Advanced Peios / eventd / Failure Modes_

> Memory, query slots, descriptors and writer stalls — how each runs out, and what eventd does when it does.

## 9.4.1 Memory

If the out-of-memory killer takes eventd, it is treated exactly as a
crash (§8.5): the databases are consistent, the ring buffers are
untouched, peinit restarts the daemon, and the gap is recorded.

Three things bound eventd's memory, and each is worth knowing because
each has a configuration that governs it:

| Consumer | Proportional to | Bounded by |
|---|---|---|
| Series cache | distinct metric series held in memory | `MetricSeriesCacheSize` (§5.3) |
| Index and rollup counters | distinct fields and function-window pairs queried | the query surface itself |
| SQLite page caches | connections and active indexes | per-connection configuration (§C) |

The handoff channels are deliberately not on that list. They are bounded
by the batch size (§2.3), which is the entire point of the
ring-buffers-are-the-only-buffer rule: the thing that would otherwise
grow without limit under load is the thing that is fixed.

The consumer most likely to surprise is the series cache under a
high-cardinality producer — not because the cache grows, since it is
bounded, but because the `series` table does, and the cache then
thrashes on the thread that also drains the metric socket (§5.3).

## 9.4.2 Query slots

Reaching `MaxConcurrentQueries` or `MaxStreamingQueries` rejects new
queries with an error rather than queueing them (§6.5). Both bounds are
global, so one client can occupy every slot.

Ingestion is unaffected, because queries and ingestion are separate
channels and separate threads. That separation is what makes query-side
exhaustion an inconvenience rather than data loss.

## 9.4.3 Descriptors

An event query opens a read-only connection per shard database, and each
connection holds one or two file descriptors (§6.4). Many historical
shards multiplied by many concurrent queries reaches a process limit
faster than anything else eventd does.

Where eventd cannot allocate what an admitted query needs, it fails that
query rather than blocking a writer or exceeding its own limit (§6.1).

## 9.4.4 Writer stalls

Two things stall a writer thread briefly, and both are bounded by
design.

**An index build in progress.** Drain threads detect rising write
pressure and signal cancellation; the writer aborts the `CREATE INDEX`,
SQLite rolls back the partial index, and event writing resumes
immediately (§3.4).

**Retention holding a write lock.** The shard's writer blocks until
retention releases it. Retention works in small bounded batches and
releases the coordination primitive between them, and under sustained
write pressure it delays the next batch long enough for a waiting writer
to make progress (§3.6). Events accumulate in the ring buffer during the
stall, which is the ordinary absorption path.

Neither stall loses anything by itself. Both contribute to ring buffer
pressure, and prolonged enough, both end at §9.1.

---

# 9.5 Power Loss

_Peios / Advanced Peios / eventd / Failure Modes_

> The failure the three stores' durability settings were chosen against, and what a restart does afterwards.

Sudden power loss is the failure the three stores' durability settings
were chosen against, and it is where the hierarchy the whole daemon is
built on becomes visible as three different outcomes.

| Store | Setting | What survives |
|---|---|---|
| Event | `synchronous=FULL` | Every committed transaction. |
| Log | `synchronous=NORMAL` | Transactions up to the last checkpoint. |
| Metric | `synchronous=NORMAL` | Transactions up to the last checkpoint. |

**The event store** loses only the in-flight batch — the events
accumulated since the last commit, which the adaptive batcher keeps as
small as throughput allows (§2.4). On restart this appears as an
ordinary sequence gap and is recorded as one (§3.7).

**The log and metric stores** may lose everything committed since their
last write-ahead log checkpoint, which under `synchronous=NORMAL` is the
real durability boundary rather than the commit. How much that is
depends on `WalCheckpointPages` and the write rate, and it can be
considerably more than one batch.

The difference is bought and paid for deliberately. `FULL` costs an
fsync on every commit, which at ten thousand events a batch is amortised
and at three events a batch is not — and eventd commits small batches
constantly under light load. The event store pays it because an event
may be an audit record whose absence is itself the finding. The other
two do not, because their loss is defined as acceptable and paying for
durability they do not need would slow the paths that are most likely to
be bursty.

Events are sacred; logs and metrics are important but not fundamental.
Every durability decision in this manual is that sentence applied to a
particular store.

## 9.5.1 What restart does

Nothing manual. eventd starts, finds its databases consistent — WAL
recovery is SQLite's, and an incomplete transaction is rolled back on
open — derives its per-CPU resume points from the committed rows, and
records the difference from the ring buffer state as a gap.

The one case that differs from a crash is that events emitted *while the
machine was off* are gone from the ring buffers too, since those are
memory. A gap after a power cut therefore covers the downtime as well as
the uncommitted batch, and there is nothing anywhere that held those
events.

---

# Appendix A Configuration Keys

_Peios / Advanced Peios / eventd_

> Every configuration key under the eventd registry subtree, its type and default, and how an invalid value is treated.

Every key lives under `Machine\System\eventd\`. eventd ignores unknown
keys in the subtree. An invalid value is ignored and the value already
in use is retained, and eventd emits a `synthetic.config_change` event
for every change actually applied (§8.3).

## A.1 Required

No compiled-in defaults. A missing or invalid value fails startup
(§8.2).

| Key | Type | Description |
|---|---|---|
| `EventStorePath` | REG_SZ | Directory for the event shard databases and `eventd-meta.db`. |
| `LogStorePath` | REG_SZ | File path for the log store database. |
| `MetricStorePath` | REG_SZ | File path for the metric store database. |
| `QuerySocketPath` | REG_SZ | Unix socket path for queries. |
| `LogSocketPath` | REG_SZ | Unix socket path for log ingestion. |
| `MetricSocketPath` | REG_SZ | Unix socket path for metric ingestion. |

## A.2 SQLite storage

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `WalCheckpointPages` | REG_DWORD | 1000 | 100–100000 | WAL page threshold triggering a passive checkpoint, on shard, log, metric and metadata databases alike. |

## A.3 Event ingestion

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `StorageShards` | REG_DWORD | 0 | 0–256 | Number of event shards. 0 means the CPU count. |
| `MaxBatchSize` | REG_DWORD | 10000 | 100–100000 | Maximum events per writer transaction. |
| `MaxBatchLatencyMs` | REG_DWORD | 100 | 10–5000 | Maximum ms before an event batch commits. |

## A.4 Log ingestion

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `LogMaxBatchSize` | REG_DWORD | 5000 | 100–100000 | Maximum log records per transaction. |
| `LogMaxBatchLatencyMs` | REG_DWORD | 500 | 10–5000 | Maximum ms before a log batch commits. |
| `MaxLogDatagramBytes` | REG_DWORD | 262144 | 4096–1048576 | Maximum accepted log datagram size. |

## A.5 Metric ingestion

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `MetricMaxBatchSize` | REG_DWORD | 5000 | 100–100000 | Maximum metric samples per transaction. |
| `MetricMaxBatchLatencyMs` | REG_DWORD | 1000 | 10–5000 | Maximum ms before a metric batch commits. |
| `MaxMetricDatagramBytes` | REG_DWORD | 262144 | 4096–1048576 | Maximum accepted metric datagram size. |
| `MetricSeriesCacheSize` | REG_DWORD | 50000 | 1000–1000000 | Entries in the LRU series resolution cache. |

## A.6 Adaptive indexing

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `AdaptiveIndexWindowHours` | REG_DWORD | 24 | 1–168 | Rolling window over which query frequency is measured. |
| `AdaptiveIndexPolicyIntervalMinutes` | REG_DWORD | 60 | 60–1440 | How often the desired index set is recomputed. The minimum of 60 prevents index churn. |
| `AdaptiveIndexCreateThreshold` | REG_DWORD | 100 | 10–10000 | Queries on a field within the window needed to add it. |
| `AdaptiveIndexDropThreshold` | REG_DWORD | 10 | 1–1000 | Queries below which it is removed. Less than the create threshold, which is what supplies the hysteresis. |

## A.7 Index shedding

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `SheddingWindowSeconds` | REG_DWORD | 30 | 10–300 | Sliding window for graduated shedding. |
| `SheddingBatchPercent` | REG_DWORD | 75 | 50–100 | Percentage of batches in the window exceeding 75% of `MaxBatchSize` that triggers graduated shedding. |
| `EmergencySheddingBufferPercent` | REG_DWORD | 75 | 50–95 | Ring buffer fill percentage triggering emergency shedding. |

## A.8 Adaptive rollups

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `AdaptiveRollupWindowHours` | REG_DWORD | 48 | 1–168 | Rolling window for rollup query frequency. |
| `AdaptiveRollupScalarWindowSeconds` | REG_DWORD | 300 | 60–86400 | Base window used when a scalar range query triggers rollup creation. |
| `AdaptiveRollupCreateThreshold` | REG_DWORD | 50 | 10–10000 | Queries needed to trigger rollup computation. |
| `AdaptiveRollupDropThreshold` | REG_DWORD | 5 | 1–1000 | Frequency below which a pair leaves the registry. Less than the create threshold. |

## A.9 Retention

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `EventRetentionDays` | REG_DWORD | 30 | 1–3650 | Maximum age of events. |
| `EventRetentionMaxBytes` | REG_QWORD | 0 | 0–2^64−1 | Maximum total logical live size of the event shards. 0 means no limit. |
| `LogRetentionDays` | REG_DWORD | 14 | 1–3650 | Maximum age of log entries. |
| `LogRetentionMaxBytes` | REG_QWORD | 0 | 0–2^64−1 | Maximum logical live size of the log store. 0 means no limit. |
| `MetricRetentionDays` | REG_DWORD | 90 | 1–3650 | Maximum age of metric samples. |
| `MetricRetentionMaxBytes` | REG_QWORD | 0 | 0–2^64−1 | Maximum logical live size of the metric store. 0 means no limit. |
| `RetentionCheckIntervalMinutes` | REG_DWORD | 60 | 1–1440 | How often the retention thread runs. |
| `RetentionDeleteBatchRows` | REG_DWORD | 10000 | 100–100000 | Maximum rows deleted in one retention transaction. |

## A.10 Querying

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `QueryTimeoutMs` | REG_DWORD | 30000 | 1000–300000 | Maximum query execution time. |
| `MaxConcurrentQueries` | REG_DWORD | 128 | 1–4096 | Concurrent queries globally, streaming and non-streaming. |
| `MaxStreamingQueries` | REG_DWORD | 64 | 1–1024 | Concurrent streaming queries globally. |
| `MaxDistinctStreamValues` | REG_DWORD | 100000 | 1000–10000000 | Values tracked by one DISTINCT streaming query. |
| `MaxQueryMessageBytes` | REG_DWORD | 65536 | 1024–16777216 | Maximum query request or response payload. |

> [!NOTE]
> `MaxQueryMessageBytes` and the two datagram ceilings are related and
> nothing enforces the relation. A record larger than the query message
> ceiling cannot be returned and fails every query that reaches it
> (PSPU §3.15). At these defaults a producer can deposit a log line four
> times larger than any response that could carry it.

## A.11 Cross-type filtering

| Key | Type | Default | Range | Description |
|---|---|---|---|---|
| `CrossTypeWindowMs` | REG_DWORD | 15000 | 1000–300000 | Centred window for cross-type event and log existence checks. |
| `CrossTypeMaxLookbackSeconds` | REG_DWORD | 604800 | 3600–2592000 | Maximum range a cross-type filter may scan. |

## A.12 The security subtree

Read-path descriptors live under `Machine\System\eventd\Security\` and
are not configuration in the sense above (§7.2):

```text
Machine\System\eventd\Security\Events\*
Machine\System\eventd\Security\Events\<pattern>
Machine\System\eventd\Security\Logs\*
Machine\System\eventd\Security\Logs\<pattern>
Machine\System\eventd\Security\Metrics\*
Machine\System\eventd\Security\Metrics\<pattern>
```

The administrative descriptor is not here; it is `admin_sd` in
`eventd-meta.db` (§3.5).

## A.13 When a change takes effect

| Change | Effect |
|---|---|
| Every tuning parameter above | Applied immediately. |
| Socket paths | Restart. |
| Store paths | Restart. |
| `StorageShards` | Restart. |
| Security descriptors | Next query; the registry watch invalidates the cache. |

---

# Appendix B Constants

_Peios / Advanced Peios / eventd_

> eventd's own constants — access rights, generic mapping, the field GUID namespace, data type roots and origin names.

Wire-protocol constants — the framing, the ingestion limits, the query
message ceiling — belong to the interfaces rather than to eventd and are
in PSPU §3.A.

## B.1 Access rights

| Right | Bit | Value | Meaning |
|---|---|---|---|
| `EVENTD_READ` | 0 | 0x0001 | Read records matching the pattern. |
| `EVENTD_CLEAR` | 1 | 0x0002 | Delete records matching the pattern. Reserved; nothing uses it yet (§7.1). |
| `EVENTD_ADMINISTER` | 2 | 0x0004 | Change eventd's own policy — the `INDEX` command. |

## B.2 Generic mapping

Passed to AccessCheck in the `generic_read`, `generic_write`,
`generic_execute` and `generic_all` fields.

| Generic right | Value | Composed of |
|---|---|---|
| `GENERIC_READ` | 0x00020001 | `EVENTD_READ` \| `READ_CONTROL` |
| `GENERIC_WRITE` | 0x00020006 | `EVENTD_CLEAR` \| `EVENTD_ADMINISTER` \| `READ_CONTROL` |
| `GENERIC_EXECUTE` | 0x00020001 | `EVENTD_READ` \| `READ_CONTROL` |
| `GENERIC_ALL` | 0x000F0007 | `EVENTD_READ` \| `EVENTD_CLEAR` \| `EVENTD_ADMINISTER` \| `DELETE` \| `READ_CONTROL` \| `WRITE_DAC` \| `WRITE_OWNER` |

`EVENTD_ADMINISTER` is in `GENERIC_WRITE` and deliberately not in
`GENERIC_READ` or `GENERIC_EXECUTE` (§7.1).

## B.3 Field GUID namespace

```text
EVENTD_FIELD_NAMESPACE = {e7d3a1b0-5c2f-4e8a-9b1d-0a6f3c8e2d4b}
```

Field GUIDs are `uuid_v5(EVENTD_FIELD_NAMESPACE, field_name)` with
`field_name` as UTF-8 (§7.3).

## B.4 Data type root GUIDs

The level-0 node of an object type list.

| Data type | GUID |
|---|---|
| Events | `{a1b2c3d4-0001-4000-8000-000000000001}` |
| Logs | `{a1b2c3d4-0001-4000-8000-000000000002}` |
| Metrics | `{a1b2c3d4-0001-4000-8000-000000000003}` |

## B.5 Field names

Field GUIDs are **computed from the algorithm, never hardcoded**. The
names they are computed from are these.

**Event header fields.** `timestamp`, `cpu_id`, `sequence`,
`origin_class`, `event_type`, `effective_token_guid`,
`true_token_guid`, `process_guid`, `boot_id`.

**Log fields.** `timestamp`, `origin`, `is_error`, `message`, `job_id`,
`boot_id`.

**Fixed metric fields.** `timestamp`, `boot_id`, `name`, `type`,
`value`.

**Event payload fields** use the flattened dot path (PSPU §3.22).
Suppressed paths and paths colliding with a header name are not
query-language fields and have no GUID.

**Metric label keys** use the key itself: `core` produces
`uuid_v5(EVENTD_FIELD_NAMESPACE, "core")`. A label key can never be one
of the five fixed metric field names, because ingestion rejects records
whose labels collide with them.

## B.6 Origin class

| Value | Origin |
|---|---|
| 0 | userspace |
| 1 | KMES |
| 2 | KACS |
| 3 | LCS |

The query language accepts these names as aliases (PSPU §3.23).

## B.7 Synthetic event types

| Type | Emitted when |
|---|---|
| `synthetic.startup` | eventd starts and attaches to KMES. |
| `synthetic.shutdown` | Graceful shutdown begins. |
| `synthetic.gap` | A sequence gap is detected on a CPU. |
| `synthetic.config_change` | A configuration value is applied at runtime. |
| `synthetic.storage_error` | A write to any store fails. |

Payload schemas are in §3.2.

## B.8 Metric types

| Value | Type |
|---|---|
| 0 | counter |
| 1 | gauge |
| 2 | histogram |

Stored in `series.type`. The query language exposes the names, not the
numbers (PSPU §3.22).

## B.9 Rollup functions

| Value | Function |
|---|---|
| 0 | AVG |
| 1 | MIN |
| 2 | MAX |
| 3 | SUM |
| 4 | RATE |
| 5 | DELTA |

These name **per-series** rollup functions. Window aggregation keywords
have no identifiers of their own: `AVG_OVER`, `MIN_OVER`, `MAX_OVER` and
`SUM_OVER` map to AVG, MIN, MAX and SUM when no transform is present,
and RATE and DELTA rollups carry `covered_ns` for exact composition
(§5.6).

P50, P95 and P99 have no identifiers because percentiles are not
composable and are never rolled up.

## B.10 Log severity

| Value | Meaning |
|---|---|
| 0 | Normal — standard output. |
| 1 | Error — standard error, or explicitly marked. |

Stored as an integer in `logs.is_error`; exposed as a boolean by the
query language, which accepts both forms (§4.2).

## B.11 Series hashing

FNV-1a, 64-bit, over the exact bytes of the canonical label string or
the boundary blob.

| Parameter | Value |
|---|---|
| Offset basis | `0xcbf29ce484222325` |
| Prime | `0x100000001b3` |
| Stored as | `hash & 0x7fff_ffff_ffff_ffff` |

The high bit is cleared so the value fits SQLite's signed `INTEGER`.
Hashes narrow lookups; identity is always confirmed against the full
string or blob (§5.2).

## B.12 Schema versions

| Store | Version |
|---|---|
| Event shard | 1 |
| Log store | 1 |
| Metric store | 1 |
| `eventd-meta.db` | 1 |

An unrecognised version is never migrated. For a required store it fails
startup; for a historical shard it excludes the shard from the query
path; for the metadata database it recreates from defaults (§3.3, §3.5).

---

# Appendix C Recommended Optimisations

_Peios / Advanced Peios / eventd_

> Implementation techniques that affect no storage format, protocol or observable behaviour, offered as guidance rather than requirement.

None of the following affects the storage format, the wire protocol, the
query language, or any behaviour a client can observe. An
implementation omitting all of them is complete. Each buys measurable
throughput or latency with no behavioural trade-off, which is why they
are collected here rather than described as design.

## C.1 Arena allocation for event copies

Drain threads copy events out of the ring buffer at rates reaching
hundreds of thousands per second (§2.2). Using the system allocator for
each variable-sized copy costs freelist bookkeeping, potential lock
contention in a multi-threaded allocator, and occasional page faults
when it asks the kernel for more.

A per-drain-cycle arena avoids all of it: allocate a block at the start
of the cycle, hand out sequential chunks by bumping a pointer, and
release the whole block once the batch has been handed off. Per-event
allocation cost falls from tens of nanoseconds to one or two, and the
latency spikes disappear with it.

## C.2 Drain thread affinity

A drain thread reading a per-CPU ring buffer benefits from running on
the same NUMA node as that CPU. It is not necessary — the per-CPU design
eliminates write contention regardless of where the consumer runs — but
NUMA-local reads avoid cross-node traffic in the drain loop.

In the 1:1 case, pinning the drain thread to the CPU whose buffer it
reads gives the best cache locality available: the pages are likely
still in that CPU's L3 from the kernel's write.

## C.3 Partial payload extraction

Building flat result records means decoding payloads, and at thousands
of rows that is the dominant query cost (§6.1).

Where a `SELECT` names specific payload paths, only those need
extracting. A streaming MessagePack decoder that scans for the wanted
keys and skips over everything else avoids materialising the payload at
all, and for a payload with many fields where one or two are selected
this cuts per-row CPU by an order of magnitude.

Without a `SELECT`, a streaming decoder emitting flattened key-value
pairs still avoids building a full in-memory representation.

## C.4 Prepared statement pooling

Writer threads already prepare one `INSERT` each and reuse it (§2.4).
Query handlers executing translated SQL benefit from the same treatment:
a small LRU pool of prepared statements per read connection, on the
order of fifty to a hundred, covers the case where an operator repeats
similar queries and where a dashboard issues the same query on a timer.

SQLite caches the query plan in a prepared statement, so re-preparing
identical SQL spends CPU on parsing and planning that was already done.

## C.5 Batched socket reads

The log and metric ingestion threads read datagrams one at a time by
default. `recvmmsg` reads many in one kernel round trip — up to a
thousand or so — and at a batch of 64 to 256 it cuts syscall overhead by
that factor under sustained load, with no protocol or format change.

It also shortens the window in which the thread is not draining the
socket, which is the window that loses datagrams (§4.1).

## C.6 SQLite page cache tuning

Each connection has a page cache, around 2 MB by default. For a shard
writer connection it holds B-tree pages for the events table and its
indexes, and under sustained writes with several adaptive indexes the
hot working set can exceed the default — producing evictions and
re-reads on the write path.

A reasonable heuristic is 2 MB plus 1 MB per active secondary index for
a writer connection, and 512 KB to 1 MB for a read-only query
connection, whose queries are short-lived.

## C.7 Bounded shard connection pool

The query path opens a read-only connection to every database in the
event store directory, and each holds one or two descriptors (§6.4).
With many historical shards this becomes the binding resource.

Active shard writer connections stay open for the process lifetime and
are not candidates. Historical shard read connections are: opening them
lazily when a query touches them, and closing them after a period of
inactivity, bounds the descriptor count without affecting any result.