eventd

eventd is the observability daemon: the single persistent sink for everything a Peios system records about itself. Events, logs and metrics all end in eventd, and every query for any of them is answered by it.

It is one of the platform daemons the service manager starts at boot, signed at TCB level, and it is Critical — a system that loses it loses its audit trail.

The three data types are genuinely different and eventd treats them differently at every layer.

Events are structured, typed records carrying identity stamps the kernel applied and an emitter could not influence. They arrive through KMES, in per-CPU shared-memory ring buffers, and eventd is their primary consumer. They are the audit and security telemetry, and losing one is a real failure — so the event path is the one with sequence numbers, gap detection, per-transaction durability, and an eventd.events.lost record written whenever anything is lost.

Logs are text: a line a program wrote, with light metadata attached. They arrive on a datagram socket, mostly from the service manager forwarding what it read from a service's standard output and standard error. Losing one is an inconvenience, and the ingestion path is designed around that tolerance rather than against it.

Metrics are numeric measurements over time — dense series rather than discrete occurrences. They arrive on a second datagram socket, pushed by whatever is doing the measuring. eventd is a sink, not a collector: it scrapes nothing and polls nothing.

1.1.1 The shape of the daemon #

Three ingestion paths, three storage engines, one query surface.

The event path runs one drain thread per CPU, each attached to one ring buffer, handing events over a bounded channel to a writer thread that batches them into a SQLite shard. Shards are independent — separate files, separate write-ahead logs, separate writer threads, no shared write-path state — so write throughput scales with the shard count (§2.3).

The log and metric paths each run a single thread that reads datagrams and writes them, to one database each. Neither contends with the event path.

Underneath all three sits a decision that shapes the event pipeline entirely: the KMES ring buffers are the only buffer. eventd holds no large intermediate queue. Events move from the ring buffer through a small bounded handoff straight into a transaction, and when the writer falls behind, backpressure propagates backwards until the ring buffer absorbs it — and when the ring buffer cannot, the loss is detected and recorded rather than hidden (§2.5).

Adaptive indexing watches which fields event queries filter on and maintains indexes for them, shedding those indexes under write pressure because ingestion throughput outranks query latency (§3.4). Repeated wide metric-window queries can seed bounded adaptive rollups, but eventd proves each cached result current against raw samples and never maintains it on the ingestion path (§5.6).

Everything eventd holds is readable only through access checks that KACS performs, per event type, per log origin, per metric name, and per field within a record (§7).

1.1.2 What eventd is not #

It is not a log framework. eventd stores lines; it does not parse them, does not understand severity beyond a single error flag, and does not care whether the text happens to be JSON.

It is not a metric collector. Nothing in eventd reads /proc, scrapes an endpoint, or polls a service. Something else measures and pushes.

It is not a tracing system. Distributed tracing is out of scope entirely.

It is not the low-latency path to events. A consumer that needs events in microseconds attaches to the KMES ring buffers directly, as revstrm does. eventd sits above that transport, adds persistence and access control, and costs a batch commit interval in latency.

It is not the only KMES consumer, and holds no privileged position among them.

This is a Technical Reference Manual Proposal. eventd has not been written.

Everything in this manual is therefore a description of a design rather than of an artifact: the schemas, the thread structure, the algorithms, the constants and the failure behaviour are what eventd is specified to do, not observations of what it does. Nothing here has been checked against an implementation, because there is none to check against.

It is written in the indicative mood, exactly as a reference manual for existing software would be, and it makes no conformance demands. When the code lands, this document becomes eventd's Technical Reference Manual — corrected wherever the implementation and the design turn out to disagree, and with no change in kind. That is the whole reason for the proposal form: a design document that will be read as a manual is better written as one from the start.

The distinction a reader needs to keep is that a TRM's authority comes from the software and this one has none of that yet. Where a statement here is surprising, it is a design decision that has not survived contact with an implementation.

1.2.1 What is a contract and what is not #

eventd's three external interfaces are specified separately, in PSPU §3: the log ingestion channel, the metric ingestion channel, and the query channel with its query language. Those are contracts, binding on anything that speaks them, and they are normative where this manual is not.

This manual covers the other side of that boundary — how eventd fulfils them:

• how it consumes KMES and what it does when it falls behind (§2)
• what it stores, in what schema, and how it accelerates and expires it (§3, §4, §5)
• how a query in the language of PSPU §3 becomes an answer (§6)
• how access decisions are reached (§7)
• how it starts, stops, and behaves when something breaks (§8, §9)

Where a chapter touches the contract, it references PSPU §3 rather than restating it, and describes only what eventd adds.

The access control mechanism is here rather than in PSPU because it is behind the abstraction: a client sees only which records and fields it received (PSPU §3.28). The field GUID derivation in §7.3 is the one part of it that a third party may need to reproduce — an administrator writing a Security Descriptor computes the same GUIDs — and it is a candidate for promotion into a specification if that turns out to be a common thing to do.

1.2.2 Versions and constants #

Constants, configuration keys and catalogues are collected in the appendices at the end of the manual, so that a chapter's reference material does not interrupt the prose that explains it. Where an appendix defines a value, the body references it rather than repeating it.

Terms defined elsewhere are used with the same meaning and are not redefined here: event, header, payload, stamp, ring buffer, consumer, origin class and sequence number from the KMES chapters of the Peios Kernel TRM and PSPK; token, GUID, SID, Security Descriptor, ACL, ACE and privilege from the Peios Kernel TRM and PCDS; registry, hive, key, value and layer from the Peios Kernel TRM; producer, client, log record, metric sample, time series and concrete identifier from PSPU §3.2.

Syscall numbers and signatures for the kernel interfaces eventd calls — kmes_attach, the SOL_KACS peer-token option, kacs_access_check and kacs_access_check_list — are in the Peios Kernel TRM's generated ABI appendices, §2.A and §3.A. This manual names them and does not repeat their numbers.

The following are specific to eventd.

Drain thread: one of the threads that reads from a per-CPU KMES ring buffer. There is exactly one per CPU (§2.2).

Writer thread: the sole writer to one event shard. There is exactly one per shard, and no other thread writes to that database (§2.3).

Shard: one of the independent SQLite databases the event store is split across, each with its own file, write-ahead log and writer thread. A shard is a write-path construct only; the query path treats the whole directory as one store (§2.3).

Active shard: a shard in the current configuration's numbering. Historical shard: a shard database left behind by a previous configuration. It takes no new events and is still queried, through read-only connections; its one writer is the retention coordinator (§3.3, §3.6).

Handoff channel: the bounded queue between drain threads and a writer thread. It is a small startup-fixed scheduling handoff, bounded by slots and bytes independently of transaction batch size (§2.3).

Synthetic event: a record eventd generates itself and writes directly to a shard, bypassing KMES. Synthetic events carry no identity stamps and no sequence numbers, and are distinguished by their type, one of exactly five eventd reserves: eventd.daemon.started, eventd.daemon.stopped, eventd.events.lost, eventd.store.quarantined and eventd.config.changed (§2.6).

Gap record: the eventd.events.lost synthetic event recording that events were lost on one CPU, and which sequence numbers went missing (§2.5).

Event store directory: the directory holding every shard database and the metadata database. Metadata database: eventd-meta.db, the one database that is not a shard and survives shard reconfiguration (§3.5).

Desired index set: the global, priority-ordered list of fields eventd aims to have indexed across all shards.

Material indexes: the indexes a given shard actually has, which converge toward the desired set when the shard is quiet and diverge from it under pressure (§3.4).

Shedding: dropping secondary indexes to protect write throughput (§3.4).

Rollup: a disposable, pre-computed metric-window result whose freshness is proved against authoritative raw samples before reuse (§5.6).

Series cache: the bounded in-memory map from series identity to series row, which keeps metric ingestion off SQLite in the common case (§5.3).

Logical live size: (page_count - freelist_count) × page_size for a SQLite database — the space actually holding data, excluding pages freed by deletion and available for reuse. Retention is enforced against this rather than against the file size (§3.6).

Quarantine: renaming a database SQLite has reported corrupt, aside from the path eventd uses, and creating an empty one in its place (§3.3).

eventd is not a port of anything. It occupies a position several existing systems also occupy, and differs from each of them in ways worth being explicit about. PSPU §3.C compares the wire contracts; this compares the systems.

1.4.1 Windows Event Log and ETW #

The closest structural match. Windows splits the job in two: Event Tracing for Windows delivers events from kernel and application providers, and the Event Log service persists and serves them. Peios splits it the same way, with KMES in the ETW position and eventd in the Event Log service's.

Held in common: kernel-mediated delivery with metadata the emitter cannot forge, a userspace service owning persistence, and access control by Security Descriptor on a named channel — which in eventd becomes a descriptor per event type pattern (§7.2).

Different: eventd unifies three data types where Windows separates them across the Event Log, ETL trace files and Performance Counters. eventd stores in SQLite rather than a proprietary binary format. And ETW's buffering is a kernel-managed trace session, where KMES exposes shared-memory ring buffers with a lock-free consumer protocol that eventd drains directly (§2.2).

1.4.2 journald #

journald is the systemd system journal: it captures service output and structured messages and stores them in an indexed binary journal.

Held in common: a single daemon for system-wide log ingestion, capturing standard output and standard error as records with metadata attached, stored in a binary format with indexes.

Different: journald is log-only, and a systemd system needs a separate stack for metrics. journald's access control is Unix file permissions and polkit, where eventd's is KACS Security Descriptors evaluated per record and per field (§7). And journald reads kernel messages from /dev/kmsg, a text interface with no identity, where eventd receives kernel events through KMES with the kernel's own identity stamps intact.

The similarity worth noting is that journald's storage is also its index, and its query surface is a matcher over fields rather than a language. eventd goes further in that direction — an actual query language, over all three data types — for the reason PSPU §3.C gives: computing on the collector's side is what lets access control constrain the computation.

1.4.3 Prometheus and OpenTelemetry #

Prometheus is a pull-based metrics system with a local time-series database; OpenTelemetry is a vendor-neutral collection framework spanning traces, metrics and logs.

eventd's metric store fills roughly the role of a local Prometheus TSDB, and takes the Prometheus data model — a name plus labels identifying a series, cumulative-bucket histograms — more or less wholesale (§5.2).

Different: eventd is pushed to rather than scraping. It implements no distributed tracing at all. And where OpenTelemetry's answer to the three-signal problem is a common collection framework in front of three backends, eventd's is one backend with one access control model and one query surface — which is the whole design bet.

1.4.4 Deliberately elsewhere #

eventd is the primary consumer of the KMES ring buffers. Events travel from shared memory to a committed database row in four stages.

1. Drain. One thread per CPU reads events from that CPU's ring buffer, following the lock-free read protocol PSPK specifies (§2.2).
2. Detect. The drain thread compares each event's sequence number against the last it saw for that CPU. A jump means events were lost, and the loss becomes a gap record (§2.5).
3. Hand off. The drain thread copies the event out of the mapped region and passes it to the writer thread that owns the shard it routes to (§2.3).
4. Write. The writer thread accumulates events into a transaction and commits, sizing the batch to whatever throughput allows (§2.4).

Two principles govern the whole pipeline, and most of its behaviour follows from them rather than from anything specific to a stage.

2.1.1 The ring buffers are the only buffer #

eventd holds no large intermediate queue between KMES and SQLite. The handoff channel has startup-fixed slot and byte bounds chosen by performance testing, independently of the writer's transaction batch size, and nothing else accumulates (§2.3).

When the writer falls behind, the channel fills; when the channel is full, the drain thread stops reading; when the drain thread stops reading, events accumulate in the ring buffer, which is exactly what a ring buffer is for. Backpressure propagates all the way back to the kernel, and the absorption capacity is the ring buffer's, which an administrator already sizes.

If the ring buffer also fills, KMES overwrites its oldest events, the drain thread notices the sequence jump when it resumes, and the loss is recorded (§2.5). That is the designed worst case: eventd loses events visibly rather than buffering without bound and dying.

The alternative — a large in-process queue — would move the same capacity into a place where losing it is invisible, where it competes with the page cache for memory, and where an out-of-memory kill takes the whole queue with no record that it existed.

2.1.2 Sharding scales writes linearly #

Each shard is a self-contained SQLite database with its own file, its own write-ahead log and its own writer thread. Shards share no write-path state, so the write path has no cross-shard lock, no shared counter and no coordination point (§2.3).

The consequence for the query path is that a shard means nothing to it. A shard database holds whatever CPUs happened to route to it in whatever eventd lifetime wrote it, so a query filtering by CPU reads every shard, and the query path never assumes a relationship between a shard and a CPU (§6.4).

2.2.1 Attachment #

At startup eventd calls kmes_attach(KMES_ATTACH_QUERY_SLOTS, …) to obtain the logical CPU slot count. It then walks every slot from zero to one below that count. A successful call returns one descriptor for that logical CPU's ring buffer; EINVAL means the slot is a hole and eventd continues rather than treating it as the end of enumeration. Each successful descriptor is mapped at the size derived from the returned capacity, exactly as PSPK §2.3 defines.

The attached logical CPU IDs may therefore be sparse. eventd assigns the successful attachments dense internal ordinals in ascending logical CPU-ID order for shard routing (§2.3), while preserving the logical cpu_id everywhere externally visible: event rows, receipt ranges, gap records and diagnostics.

Attachment requires SeSecurityPrivilege in the effective token, which is the privilege that grants an unfiltered view of every event on the system. eventd holds it because it is the party that then applies per-event access control on everything it stores (§7).

Discovering zero CPUs is a startup failure (§8.2). There is no configuration for the CPU count and no way to attach to a subset.

2.2.2 Drain threads #

There is one drain thread per CPU, and each reads exactly one ring buffer. A drain thread never reads another CPU's buffer, which is what makes per-CPU sequence tracking a thread-local variable rather than shared state.

Each thread follows the read protocol PSPK specifies:

• read_pos starts at tail_pos on first attachment, so eventd begins at the oldest surviving event rather than at the newest.
• The drain loop loads write_pos with acquire ordering, checks tail_pos for lapping, validates the event's structural integrity, and advances read_pos by event_size.
• After reading an event it re-reads tail_pos — the torn-read check — to detect that KMES overwrote the event while it was being copied.
• With nothing to read it uses the notification protocol, setting need_wake and waiting on the futex, rather than spinning.

2.2.3 Copying #

A drain thread reserves both one slot and the event's byte length in its shard handoff before copying the event — header and payload — into process-local memory and advancing read_pos. If either reservation is unavailable, it leaves the event in KMES and waits; it does not copy or advance first (§2.3).

Nothing derived from the mapped region ever reaches a writer thread. The region is producer-owned and KMES may overwrite any part of it the moment read_pos moves past, so a pointer handed across the channel would be a pointer into memory that another CPU is entitled to rewrite before the writer gets to it.

The copy is bounded by the event's own event_size, and the drain thread never reads beyond it.

2.2.4 Generation changes #

An administrator changing the ring buffer capacity causes KMES to replace the buffers, which it signals by changing the generation field. A drain thread checks it after each drain cycle, and on a change:

1. Records the sequence number of the last event it has successfully handed off.
2. Finishes draining the old buffer to its now-frozen write_pos, updating that sequence number as it goes.
3. Calls kmes_attach again for the same logical CPU, obtains the replacement descriptor, and maps it while the old mapping remains valid.
4. Scans the new buffer from tail_pos for the first event whose sequence is greater than the final sequence drained from the old buffer, and sets the new read_pos there.
5. Closes and unmaps the old buffer only after that scan succeeds.
6. Resumes draining from the new buffer.

Each drain thread handles this independently. There is no barrier, no coordination and no shared state, because each attaches only to its own CPU's buffer — so a resize is a per-CPU event that happens to occur on every CPU at roughly the same time.

Draining the frozen old buffer before switching prevents loss at the back of the old generation. The sequence scan prevents duplicates from the events KMES copied into the replacement. A shrink may still discard the oldest survivors; that loss appears as an ordinary sequence gap.

2.2.5 Sequence tracking #

During an uninterrupted run, each drain thread holds the last sequence number it saw for its logical CPU. It serves gap detection (§2.5) and a generation change.

After a restart, there is deliberately no single resume number. Different shard transactions may have committed out of sequence, so taking MAX(sequence) could skip an earlier stripe that never committed. The authority is the union of committed receipt_ranges from every readable active and historical shard (§3.1).

Startup merges those ranges per (boot_id, cpu_id), scans the surviving ring records, skips survivors already covered by a receipt, and re-ingests uncovered survivors. It emits a gap only for a sequence that is covered by neither a committed receipt nor a surviving ring record (§8.5). With no receipt for a CPU, coverage begins before sequence 1, so an overwritten prefix is detected correctly.

The metadata database's sequence checkpoints and the shutdown event are diagnostic only (§3.5, §8.4). Neither participates in recovery.

eventd depends on KMES sequence numbering remaining continuous while the kernel boot ID is unchanged, and so on KMES being initialised once for the whole boot and never unloaded or reloaded. A sequence regression under the same boot ID, other than duplicates deliberately encountered during replacement scanning, is a fatal incompatibility: eventd stops rather than confuse two KMES stream lifetimes under one receipt key.

2.3.1 The shard count #

Event writes are distributed across independent SQLite databases. An explicit StorageShards count (§A) is between one and 256. Zero means "as many shards as there are successfully attached KMES buffers", and is the default; that automatic count is not capped at 256.

Every shard costs file descriptors. An active shard's writer connection holds three for eventd's life — the database, its write-ahead log and its shared-memory index — as does the retention coordinator's connection to each historical shard (§3.6), and every event query that reads every shard holds about two more per shard while it runs (§6.4). eventd neither raises its own descriptor limit nor caps the shard count to fit it. The packaged eventd-service.reg sets LimitNOFILE to 262144. peinit applies it as both the soft and hard RLIMIT_NOFILE before launch; eventd does not raise the limit itself. Under an inherited 1024-descriptor soft limit, 256 shards do not start, and counts well below that leave little room for queries. Operators can override LimitNOFILE (DWORD) at Machine\System\Services\eventd and restart the service to apply it (peinit TRM §3.2).

The packaged value uses this conservative planning allowance:

4 × Q × max(E, 1) + 4 × Q + 3 × (E + 3) + 2 × C + 256

Here Q is the concurrent query count, E the total active and historical event databases visible to queries, and C the attached KMES buffer count. The terms allow four descriptors per query database (database, WAL, shared-memory index and temporary-file headroom), four per query for its socket, peer token and transient handles, three per persistent database connection (including log, metric and metadata stores), two per KMES buffer during replacement, and 256 for fixed handles and maintenance. Historical shards have persistent retention connections as well as query readers. Streaming queries are already included in Q; their separate limit is not added again.

At the default MaxConcurrentQueries=128, with at most 256 query-visible event databases and 256 attached KMES buffers, this gives 133129; 262144 rounds up with additional headroom. This is a sizing allowance, not a strict SQLite descriptor bound or a guarantee that the workload fits memory and thread limits. The limit does not preallocate handles.

Neither historical shard discovery nor the automatic shard count is globally capped by the explicit StorageShards range. More historical shards, more attached buffers or higher query concurrency require operator sizing and, where needed, a larger service limit. The host's fs.nr_open ceiling must also permit the chosen limit; setting the service value does not raise that system ceiling.

Two properties make a count perform well, and neither is enforced. A power of two lets routing use a bitwise AND rather than a modulo. A multiple of the attached-buffer count distributes shards evenly across drain threads. The default satisfies the second by construction.

2.3.2 Assignment #

Shard-to-CPU assignment is computed once at startup and is fixed for the process lifetime.

The successfully attached logical CPUs are assigned dense ordinals c = 0..attached_count-1 in logical-ID order (§2.2). For each ordinal c, eventd assigns every shard j where j % attached_count == c. If that produces nothing — which happens when there are fewer shards than attached buffers — ordinal c is instead assigned c % shard_count. Every drain thread ends with at least one write path. Logical cpu_id, not the ordinal, is stored.

The three cases behave differently:

A drain thread owning several shards distributes contiguous sequence stripes, not individual events round-robin. It sends a fixed positive number of successive events to one shard, then moves to the next shard it owns. The stripe length is selected once at startup and is not persisted; committed receipt ranges, rather than recomputing the routing choice, are the recovery authority (§2.2, §3.1).

Keeping a contiguous run together makes one receipt describe the run compactly. It preserves parallelism across CPUs and across successive stripes without requiring any cross-shard commit coordinator.

When the counts do not divide evenly, some CPUs carry one shard more than others, or some shards receive from one CPU more than others. The resulting imbalance is one shard's worth of throughput, which is negligible against the whole.

2.3.3 Shards are not a query-path concept #

Assignment is not persisted. A shard database accumulates events from whatever CPUs routed to it during whatever eventd lifetimes wrote it, so a single shard file may hold events from a different set of CPUs in different regions of its history.

The query path therefore assumes nothing: it reads every database in the directory, and a query filtering on event.cpu scans all of them (§6.4). Sharding is a write-path optimisation that the read path pays a fan-out for.

2.3.4 Writer threads #

Each shard has exactly one writer thread, and that thread is the only writer to that database. No other thread and no other connection writes to it, which is what makes the single-writer assumptions in §3.3 and §3.4 safe.

Drain threads never write to SQLite. When several drain threads share a shard they hand off concurrently, so the handoff channel is multi-producer and single-consumer.

2.3.5 The handoff channel #

Each writer thread has one bounded channel through which drain threads submit events. Its capacity is fixed at startup and independent of MaxBatchSize: the channel is bounded both by occupied slots and by the total bytes held by those slots. Both are internal implementation limits selected by performance testing, not live registry settings.

When either bound would be exceeded the drain thread stops reading from the ring buffer and waits. It reserves capacity before copying or advancing read_pos (§2.2), does not drop events to relieve pressure, and does not grow the channel. Events accumulate in the ring buffer instead, which is the designed path (§2.1):

writer slow → channel fills → drain pauses → ring buffer absorbs
            → KMES overwrites oldest if full → gap detected on resume

When the writer consumes slots and the channel has room, the drain thread resumes immediately.

Publication wakes a writer waiting on an empty channel; releasing capacity wakes producers waiting for room. The change itself supplies the wakeup, even when it races with a thread going to sleep. Neither side needs a later event or a timeout to notice the available work or capacity.

The channel is a small scheduling handoff, not storage for a complete transaction and not a second backlog buffer. A writer can consume and insert one slot while producers refill it, so a transaction may contain many more events than the channel can hold simultaneously. Changing MaxBatchSize changes only the writer's next commit threshold and never resizes or replaces a live channel (§8.3).

2.3.6 Lifecycle and reconfiguration #

Shard databases are created in the event store directory on first use. eventd never deletes or overwrites one left by a previous configuration: starting with fewer shards than exist leaves the excess in place, and the query path continues to read them (§3.3).

Changing StorageShards takes effect at the next restart. The configuration watch notices the change and eventd defers it rather than reassigning CPUs or creating shards while running (§8.3).

Shard count changes are expected to be rare — set once from the hardware profile, one for a small board and a multiple of the CPU count for a server, and then left alone. Live migration would mean rebalancing writer threads and channels while events are in flight, for a configuration change that happens once in a machine's life.

2.4.1 Transactions #

Each writer thread writes to its shard with explicit transactions: a BEGIN, one INSERT per event, a COMMIT. The commit is the durability boundary.

For every maximal contiguous sequence run represented by the event rows and eventd.events.lost rows in that transaction, the writer also inserts a receipt_ranges row for (boot_id, cpu_id, first_sequence, last_sequence) (§3.1). Events, gaps and their receipts commit together: there is no second durability operation and no global receipt writer.

The database runs in WAL mode with synchronous = FULL, so every commit fsyncs the write-ahead log. This is the strictest of the three stores' settings, and the only one where per-transaction durability is bought at per-transaction cost — because an event may be an audit record and losing the last second of them to a power cut is a real loss.

2.4.2 Adaptive batch sizing #

The writer sizes each batch to balance throughput against how much sits uncommitted at any moment.

Throughput is always the priority. If eventd falls behind the emission rate, ring buffers fill and events are overwritten, which is irrecoverable; a shorter power-loss window is not worth that trade. The algorithm maximises resilience within the constraint that throughput is maintained, never against it.

1. When the first event is available, the writer opens a transaction and records the start time.
2. It reads available events from its drain threads and inserts them.
3. After each group of inserts it commits if any of these holds:
   • no assigned drain thread currently has an event available
   • the batch holds MaxBatchSize events
   • MaxBatchLatencyMs has elapsed since the first event entered it
4. Otherwise it keeps reading and inserting.
5. With nothing available and an empty batch, it sleeps until a producer wakes it.

The first condition is what makes the algorithm adaptive. Under light load the input drains immediately, so a batch of three events commits at once and the exposure window is microseconds. Under sustained load batches grow until they hit the size cap or the latency cap, whichever comes first, and the per-commit fsync is amortised across thousands of rows.

The writer chooses its own insert-group size, subject to a group never letting a batch exceed the size cap or stay open past the latency cap.

Both bounds are configuration (§A). The defaults are 10000 events and 100 milliseconds — the tightest latency of the three stores, for the same reason the durability setting is the strictest.

2.4.3 WAL checkpointing #

WAL mode accumulates log data until a checkpoint copies it back into the main database file. Under sustained writes the log grows.

Each writer triggers a checkpoint when its write-ahead log reaches WalCheckpointPages (§A), in SQLITE_CHECKPOINT_PASSIVE mode — checkpointing as much as it can without blocking readers. If a passive checkpoint cannot make progress because readers hold pages, the writer does not block: it keeps writing and retries after a later commit.

Checkpointing runs on the writer thread and briefly serialises with insert work, which is inherent to SQLite rather than a choice — a database cannot be checkpointed and written concurrently. Passive mode is the lightest option available, yielding immediately when readers hold pages, and the per-checkpoint cost is bounded by the threshold.

2.4.4 Prepared statements #

Each writer prepares its INSERT once at startup and reuses it for every row, which keeps SQL parsing and planning off the hot path entirely.

The receipt insert and new-event-type catalogue insert are prepared and reused too. A known event type causes no catalogue SQL operation; the writer's in-memory intern table identifies only genuinely new names, which are inserted in the same transaction as their first event (§3.1).

2.4.5 Maintenance commands #

The writer owns the shard's only read-write connection. Retention, receipt compaction and index policy code may inspect through read-only connections, but they submit bounded maintenance commands to this writer rather than opening another writer or taking a shard mutex.

The writer considers low-priority maintenance only at transaction boundaries and rechecks ingestion pressure after one bounded action. A size-urgent retention delete may be included in an existing ingestion transaction so it adds no second commit. Index creation remains cancellable through SQLite's progress handler (§3.4). Under sustained pressure maintenance may lag indefinitely; accepting incoming events is the higher priority.

Every event carries a per-CPU, per-boot sequence number, and a drain thread knows what it last saw. That is the whole mechanism: an event whose sequence number exceeds the expected next one means the numbers in between belonged to events eventd never received.

During restart reconciliation, "expected" is the union of committed receipt coverage and surviving ring records rather than one last-seen number. A sequence already covered by either source is not a gap (§2.2).

2.5.1 Causes #

• Ring buffer overrun. KMES overwrote events before eventd read them. The most serious case: it means audit events were lost irrecoverably.
• Structural drops. KMES declined to write an event that exceeded its size limits.
• Downtime. Events emitted while eventd was not running.

All three appear identically at the consumer, which is why the record says what was lost rather than why.

2.5.2 Gap records #

On detecting a jump, the drain thread generates a gap record, an eventd.events.lost event, carrying:

• the CPU identifier
• the first missing sequence number, the last seen plus one
• the last missing sequence number, the revealing event's minus one
• the count of missing events
• the timestamp of the last event successfully processed on this CPU, where one is known

"Known" means the drain thread saw that event in this run. A gap found during restart reconciliation whose preceding events are accounted only by a committed receipt range carries no preceding timestamp, even though those events are stored: a receipt proves the sequence was accounted for, and eventd does not read the stored event back to recover its time.

Its own timestamp, event.time, is the timestamp of the event that revealed the gap, not the moment eventd noticed. The lost events were therefore written between the last processed timestamp and the record's own, and the record sorts beside the event that revealed it.

The record is written into the shard database through the normal write path — handed to the same writer thread, batched with ordinary events, committed in the same transaction. That transaction's receipt range covers the missing sequence interval as well as any real event rows, so a restart never emits the same gap twice (§3.1). It is not emitted through KMES, which would be circular: the mechanism for recording that the event transport lost something cannot depend on that transport.

The gap details are stored as a MessagePack map in the payload column (§3.2), under the catalogue's buffer.cpu and loss.* fields, and gap records are queryable exactly like any other event.

2.5.3 The CPU column #

A gap record populates cpu_id and leaves the other KMES header columns — sequence, origin_class, the identity GUIDs — null (§3.1).

cpu_id is populated deliberately, and it is the one place a synthetic event carries a header field. Without it, EVENTS WHERE event.cpu == 3 would return every event from CPU 3 except the record saying that events from CPU 3 went missing — which is the one record such a query most needs to return.

sequence stays null because a gap record has no place in the sequence: it describes numbers that were skipped, and giving it one of them would make it indistinguishable from the event that was lost. It is also what keeps gap records distinct from real events; receipt_ranges records the accounted sequence interval separately (§3.1).

2.5.4 Lapping #

If read_pos falls behind tail_pos, the consumer has been lapped and the PSPK read protocol advances it to tail_pos.

Lapping needs no special handling here. The next event read is the oldest survivor, its sequence number is far beyond what the thread expected, and ordinary gap detection records the difference. The lapping case and the restart case produce the same record by the same path.

Synthetic events are records eventd generates about itself. They are written straight to a shard database and never touch KMES.

They carry no KMES header: no identity stamps, no sequence number, no origin class. What they have is a timestamp and one of exactly five event types, all under eventd's own platform root eventd (PGSS §6.A), and the type is what distinguishes them in the events table — no separate record-type column exists (§3.1). The timestamp is eventd's clock when it generated the record, except for a gap record, whose timestamp is that of the event that revealed the gap (§2.5).

2.6.1 When they are generated #

Payload schemas for all five are in §3.2.

All five are tier essential (PGSS §6.8). eventd is their emitter as well as their store, and the emission policy never switches an essential type off (PGSS §6.9), so eventd writes each whenever its condition occurs and consults no policy for them. Their fields are those of eventd's catalogue fragment, installed as /usr/share/evman/eventd.evman (PGSS §6.10).

Note what is absent: there is no synthetic event for malformed ingestion input. Authentication does not make admitted input trustworthy, and emitting a durable record per bad datagram would hand a producer an amplification primitive (PSPU §3.4). These five are conditions eventd observed about itself, not reactions to what it was sent.

2.6.2 Which shard #

CPU-specific synthetic events — gap records — go to the shard assigned to the CPU that generated them, handed to that writer thread alongside that CPU's ordinary events. A gap record travels with the events it describes.

Daemon-wide ones — start, stop, configuration changes, quarantines — go to shard 0 when shard 0 is writable, and otherwise to the lowest-numbered writable active shard. If no shard is writable at all the event is skipped, with the failure logged to standard error.

A quarantine is the case that needs the fallback. It describes a failure on one particular shard but is itself a daemon-wide notification, so it is not written to the failing shard unless that shard has since been replaced and is writable again (§9.2) — writing the record of a shard's failure into that shard would lose it exactly when it matters.

These events are infrequent enough that concentrating them on shard 0 costs nothing measurable in balance.

2.6.3 Storage and ordering #

Synthetic events live in the same shard databases as KMES events and participate in the same batching, the same retention and the same queries. Access control treats their types like any other (§7.2), so a descriptor at Machine\System\eventd\Security\Events\eventd governs all five, and one at …\Events\eventd.daemon only the start and stop records.

They are ordered by their timestamp and take no part in per-CPU sequence numbering.

For the four daemon-wide types that timestamp is when eventd noticed, not when the condition occurred. A gap record is different: its timestamp is the revealing event's (§2.5), so it sorts beside that event, and with loss.preceding-time it bounds when the lost events were written. eventd may detect a gap long after the events were overwritten — after a restart, the first thing it writes in a boot can be the record of events lost before — and the record's time says when they were lost, not when eventd found out.

2.6.4 The reserved types #

A KMES event whose type is one of the five is not stored: eventd counts it and drops it, and still receipts its sequence (§3.1). Exactly those five names are reserved, not the eventd root or any prefix of it, so eventd remains free to emit other eventd.* types through KMES.

Every shard database holds one events table.

3.1.1 Header fields are columns #

Every KMES header field is extracted into its own column rather than left inside the payload blob. That is what lets a predicate on emitter.process.guid or event.type become a SQL comparison rather than a decode of every candidate row, and it is what makes those fields indexable by ordinary column indexes (§3.4).

The column names are internal to the store. A query names a header field by its path from PSPU §3.22, and eventd reads it from its column:

A column's name is not a field: cpu_id in a query names a payload field, as any other name outside these nine does.

event_type is the sole discriminator between real and synthetic records. No record-type column exists, because the five types eventd writes itself already partition the type namespace and a second column would be a second thing to keep consistent.

The partition holds because eventd reserves those five names, and only those (§2.6). A KMES event whose type is one of them is not stored: eventd counts it and receipts its sequence, so it is neither a row nor a gap. Any other type, including any other under eventd., is stored as usual. The count appears in the diagnostic dump (§8.5).

3.1.2 The payload is not touched #

For a KMES event the payload column holds the bytes KMES delivered, unmodified. eventd does not decode them on the write path, does not re-encode them, and does not validate them beyond what the ring-buffer protocol already checked.

The payload is a MessagePack value whose schema belongs to the emitting subsystem, and eventd has no catalogue of those schemas. It decodes on the read path when a query needs a payload field (§6.1), which is also the only point at which the flattening rules of PSPU §3.22 apply.

Storing the bytes verbatim is also what keeps a payload value at a header field's path recoverable: the value, or the map and everything beneath it, is suppressed from the query surface but remains in the blob.

3.1.3 The event-type catalogue #

Every shard also holds event_types:

The writer loads this small set into an in-memory intern table at startup. A known type adds no catalogue statement to an event insert. A genuinely new type is inserted once, in the same transaction as its first event, and enters the in-memory set only after that transaction commits. A transaction-local pending set prevents later events of that type in the same batch from repeating the catalogue statement; rollback discards the pending set.

Query planning unions these catalogues across relevant active and historical shards to discover concrete identifiers for access checks (§6.1). It never needs a DISTINCT event_type scan of the events table, so idx_events_event_type remains an adaptive index that may be shed without breaking planning (§3.4).

Retention records the distinct types touched by each delete batch and offers orphan checks as low-priority maintenance. The writer rechecks NOT EXISTS in the deletion transaction, and removes the type from its in-memory set only after that deletion commits. A stale catalogue row is safe — it causes an unnecessary access check but cannot expose data — so an unindexed or pressure-interrupted check is simply skipped and cleanup never delays ingestion. Catalogue pages count toward the store's logical live size (§3.6), preventing unique-name abuse from escaping the store-wide size accounting.

3.1.4 Committed receipt ranges #

Every shard also holds receipt_ranges:

The primary key is (boot_id, cpu_id, first_sequence, last_sequence), and every row satisfies first_sequence > 0 and last_sequence >= first_sequence.

A receipt says that every sequence in its range was accounted for by the same transaction: either the real event row was stored or an eventd.events.lost row durably records why it was absent. The receipt is in that transaction, so its presence proves commit without a global checkpoint or another fsync. Receipt rows survive event retention.

Startup unions and merges overlapping or adjacent ranges across every readable shard (§2.2, §8.5). A read-only background pass may prepare a low-priority compaction plan while the store is quiet. Writer-owned mutations first insert a merged range and only later delete ranges it subsumes, including ranges in other shards. Insert-before-delete makes every intermediate state conservative and crash-safe.

Compaction mutations piggyback transactions a writer would commit anyway; they never cause a standalone commit or fsync, acquire no hot-path coordination primitive, and may lag indefinitely. Startup merge correctness never depends on physical compaction.

3.1.5 Identity may be absent two ways #

effective_token_guid, which a query reads as emitter.token.guid, distinguishes two cases that would otherwise look alike. Null means the record is synthetic and never had an identity. The null GUID — sixteen zero bytes — means the record is a real KMES event whose identity was not available at emission time, because it was emitted before or outside a context that had one.

The distinction matters for audit: "eventd wrote this" and "the kernel emitted this and could not attribute it" are different facts.

3.1.6 Write-time indexes #

One index is created with the table:

• idx_events_timestamp on events(timestamp)

Time-range filtering is the foundational access pattern — nearly every query carries a SINCE — and it is the one index eventd never sheds, whatever the write pressure (§3.4). Every other index is the adaptive system's business.

3.1.7 Schema version #

Each shard holds events, event_types, receipt_ranges, and a metadata table:

with two required entries: schema_version, and created_at as a UTC timestamp formatted YYYY-MM-DDTHH:MM:SSZ. The current version is in §B.

eventd checks the version at startup and applies the lifecycle rules of §3.3. It does not migrate: an unrecognised version is a startup failure for an active shard and an exclusion for a historical one. Migration is an administrative operation, deliberately not an automatic one — a daemon that silently rewrote an audit store's schema on first start after an upgrade would be doing the one thing an audit store must not do unattended.

Each of the five synthetic event types (§2.6) carries a MessagePack map in payload, with the schema below. The fields are catalogue fields, defined in eventd's fragment /usr/share/evman/eventd.evman or in the kernel's (PGSS §6.10), and each is stored as nested maps, one per segment of its path: store.shard-count is the value at {store: {shard-count: …}}. A query names a field by its dotted path, and reads an array field whole.

No field is ever nil. A field with no value is left out of the map (PGSS §6.5), and the table below says when each one can be.

None of them is at a header field's path, so none is suppressed. The boot and the CPU are read from the columns as event.boot.guid and event.cpu (§3.1), and no payload repeats the boot.

3.2.1 eventd.daemon.started #

There is no boot ID in the payload: the record's own event.boot.guid is the boot it describes.

store.restarted is the boot-boundary decision of §3.7 recorded as data, which makes "did eventd crash during this boot, and how often" answerable by query rather than by inference from gaps.

3.2.2 eventd.daemon.stopped #

When eventd cannot read its committed receipts at shutdown, both arrays are left out and the payload is an empty map. It never writes zeroes in their place, so a 0 is always a real value.

Diagnostic only. Startup derives recovery coverage from committed receipt ranges, never from this payload (§2.2).

3.2.3 eventd.events.lost #

The time of the event that revealed the gap is not a payload field: it is the record's own event.time (§2.5).

The CPU appears both here, as buffer.cpu, and in the cpu_id column (§2.5). The column is what a WHERE event.cpu == N predicate matches and what a query presents as event.cpu; the payload field is what WHERE buffer.cpu == N matches, and what a reader of the stored payload sees without joining anything.

3.2.4 eventd.config.changed #

Every value eventd reloads at runtime is a REG_DWORD or a REG_QWORD (§A), so both values are integers, and a query can compare them as numbers.

A side is absent when no value is stored, even though eventd's compiled default then applies. A stored value of the wrong type is reported as the type eventd expects, holding the integer it actually uses.

3.2.5 eventd.store.quarantined #

outcome.detail is diagnostic text and its wording is not stable. store.kind and store.shard are the fields worth alerting on. The metadata database is never reported here: a damaged one is rebuilt (§3.5) and writes no record.

3.3.1 The event store directory #

Every shard database and the metadata database live in one directory, named by EventStorePath (§A). There is no compiled-in default: a missing or invalid value is a startup failure, and eventd writes event databases nowhere else.

The standard path is /var/state/eventd/events/. The eventd package ships /var/state/eventd/ and its events/, logs/, and metrics/ children. The three store directories are also declared as required peinit provisioned directories, so peinit establishes their descriptor before eventd starts; /var/state/eventd/ itself is not a provisioned directory. Each of the four carries an explicit protected, inheritable descriptor granting full control only to SYSTEM, Administrators, and the service SID of eventd's own virtual Service identity, which is what the daemon runs as:

O:SYG:SYD:P(A;OICI;GA;;;SY)(A;OICI;GA;;;BA)(A;OICI;GA;;;S-1-5-80-1963885778-1835409261-1671587836-2279113866-1994761124)

A deployment choosing another configured path provisions it with this descriptor before eventd starts.

eventd does not create store directories. It opens every path component without following symbolic links, retains the resulting directory descriptor, and opens, creates, renames and quarantines database, WAL and shared-memory files relative to that descriptor. A missing path, a non-directory component, a symbolic-link component, or a store directory whose owner, group and DACL are not exactly the descriptor above — which is what keeps an untrusted principal from replacing children — is a startup failure. SQLite's database, -wal, and -shm files inherit the directory's protection.

3.3.2 Naming #

Active shards are shard-NNNN.db, with the shard index zero-padded to four digits — the index assigned at startup, which is not a CPU number (§2.3).

Starting with more shards than exist creates the new ones. Starting with fewer leaves the excess in place: they become historical shards, are never deleted, and remain available to the query path.

3.3.3 Creation #

A shard database that does not exist, or that holds no schema at all (what a power cut leaves when it takes the uncheckpointed creating transaction), is created, in one transaction, with:

1. WAL mode, PRAGMA journal_mode=WAL
2. PRAGMA synchronous=FULL
3. the events, event_types, receipt_ranges, and metadata tables (§3.1)
4. the idx_events_timestamp index
5. the schema_version and created_at entries

3.3.4 Opening an active shard #

1. Open in WAL mode.
2. Set synchronous to FULL.
3. Read and verify schema_version. Missing or unrecognised is a startup failure. No migration is attempted.
4. Verify structural integrity — the required tables and write-time indexes exist. Failing this, with SQLite reporting no corruption, is a startup failure.
5. If SQLite reports corruption while opening or verifying, quarantine and replace (below). A database holding schema objects but no metadata table with entries is not a shard eventd could have written; it is quarantined and replaced like a corrupt one.

Steps 3 and 4 fail rather than repair because an active shard is required: eventd has no degraded mode that runs without one (§8.2).

3.3.5 Quarantine #

When SQLite reports corruption in a required store, eventd renames the database aside and starts a fresh one at the original path. The main database file and any matching -wal and -shm files are renamed with the suffix .corrupt.<timestamp_ns>, all three using the same suffix from one operation, and a new empty shard-NNNN.db is created.

If a target name is taken, eventd appends .N with the lowest positive integer that makes it unique — which happens when two quarantines land in the same nanosecond, and when a previous quarantine already used the name.

Quarantining rather than deleting is the point: the corrupt file is the only copy of whatever it held, recovering data from it is an administrative operation, and eventd attempts no automatic repair.

The corruption is logged and an eventd.store.quarantined event is emitted once a shard is available to write it to (§9.2).

3.3.6 Historical shards #

A historical shard is never required for startup. If one has a missing or unrecognised schema version, fails structural verification, or cannot be opened read-only, eventd logs the error and excludes it from the query path for this run — it does not fail startup and does not quarantine it.

The asymmetry is deliberate. An active shard that will not open means eventd cannot do its job; a historical one that will not open means some old data is unreadable, which is a smaller problem than refusing to boot the audit daemon over it.

3.3.7 Query path discovery #

The query path opens every file in the directory matching shard-NNNN.db that has a recognised schema and passes structural verification — active and historical alike. It assumes no particular number of them.

It explicitly does not treat every .db file in the directory as a shard: eventd-meta.db is excluded by the naming pattern, along with anything else that happens to be there.

Each is opened with a read-only connection. Read-only connections in WAL mode do not contend with the writer's connection.

3.3.8 Concurrency #

Each shard has exactly one read-write connection, owned by its writer thread — for a historical shard, the retention coordinator (§3.6) — and any number of read-only connections owned by query handlers. WAL mode permits concurrent readers alongside one writer without blocking either.

Writer threads never share connections. Each creates and owns its own for the process lifetime, which is what makes the prepared statement in §2.4 a per-thread object with no locking around it.

Secondary indexes make queries fast and writes slow. Which indexes are worth that trade depends on what a particular deployment actually queries, which varies between systems and over time, and which nobody wants to tune by hand.

eventd observes the queries and maintains the indexes they imply — subject throughout to the rule that throughput outranks query latency.

3.4.1 Three decoupled parts #

Query frequency counters. Query handlers increment a per-field counter when a field appears in a WHERE predicate. This is the write-heavy path — once per query, per predicate. Counters live in memory and are flushed periodically to the metadata database (§3.5), never to a shard, because they are global state that must survive shard reconfiguration.

Index policy. A periodic process reads the counters, applies the creation and removal thresholds, and computes the desired index set — an ordered list of fields, highest priority first. It runs every AdaptiveIndexPolicyIntervalMinutes (§A), and also at once on every applied configuration change (§8.3) and every INDEX command, and once more at shutdown; it is the only writer to the desired set.

Each run reads the current thresholds and WAL checkpoint setting after its wait ends, including the final run on shutdown. An applied threshold change is therefore judged using the new configuration; it does not require a second change to take effect. Pending wakes may coalesce, so this is not a promise of one run per registry update.

Shard convergence. Writer threads read the desired set and move their material indexes toward it. They never read the counters and never write the desired set.

The separation exists so that the high-frequency counter updates never contend with the writer threads. The policy is the bridge between them and runs on the order of once an hour, so it is never a contention point.

The desired set is global — one list applying to every shard. Individual shards do not make independent decisions; they differ only in how far they have got.

3.4.2 Convergence #

A shard converges when it is quiet: its writer thread has no pending events, and no batch within the last SheddingWindowSeconds exceeded 75% of MaxBatchSize, the same measure that sheds indexes (below). When it is quiet and its material indexes do not match the desired set, it takes one convergence action — creating the highest-priority missing index, or dropping the lowest-priority material index no longer wanted — then rechecks write pressure before considering another. It does so without waiting for the next policy run.

Creation uses CREATE INDEX IF NOT EXISTS; removal uses DROP INDEX IF EXISTS. Both run on the shard's writer thread, which is the only thread permitted to write that database (§2.3).

Index creation is cancellable. If drain threads detect rising write pressure during a build, they signal the writer to abort; the writer cancels the CREATE INDEX, SQLite rolls back the partial index cleanly, and the writer returns to event batches immediately. The abandoned build is retried at the next quiet period.

Cancellation responsiveness matters more than it looks. sqlite3_interrupt sets a flag checked at SQL VM opcode boundaries, and during B-tree construction for a large index the gap between checks can be tens of milliseconds — long enough to overrun a ring buffer at a high event rate. sqlite3_progress_handler(), registering a callback invoked every thousand opcodes that checks a cancellation flag and returns non-zero to abort, gives cancellation that tracks the pressure signal rather than lagging it.

Shards converge at their own pace. One under sustained pressure may lag the desired set indefinitely, and that is the correct outcome: it is prioritising throughput.

3.4.3 Shedding #

Under sustained pressure a shard drops indexes to cut per-insert cost.

Graduated shedding. If more than SheddingBatchPercent of a shard's batches within a SheddingWindowSeconds sliding window exceeded 75% of MaxBatchSize, the shard drops its lowest-priority secondary index — the one whose column has the lowest query frequency in the desired set. If pressure persists, the next-lowest goes, and so on. The check runs once per batch commit.

Emergency shedding. If a shard is at maximum batch size and its drain thread signals rising ring buffer pressure, it drops all secondary indexes at once. DROP INDEX is a metadata operation measured in milliseconds, so it is safe to do under pressure in a way that creation is not.

The pressure signal comes from the drain thread watching the gap between write_pos and its own read_pos. Exceeding EmergencySheddingBufferPercent of ring buffer capacity raises it. It is a distinct signal from the index-build cancellation one: this triggers shedding whether or not a build is in progress.

idx_events_timestamp is exempt. It is never shed at any pressure, because time-range queries are the access pattern everything else is built on and the store is unusable without it.

When pressure subsides, shedding reverses: the shard rebuilds toward the desired set under the same quiet-period scheduling and the same cancellability, highest priority first.

3.4.4 Candidates #

Any field that can appear in a WHERE predicate is a candidate.

Header columns. The columns of event.type, emitter.class, event.cpu, emitter.token.guid, emitter.true-token.guid, emitter.process.guid and event.boot.guid (§3.1). The counters and the desired set name each by its field path. timestamp, the column of event.time, is always indexed and is not adaptively managed; a predicate on event.time is not counted.

Payload fields. Any queryable flattened path that appears in a predicate is a candidate for an expression index. A path suppressed by the flattening rules of PSPU §3.22 — a header field's path or one beneath it, a key that is not a valid segment, a duplicate path — never receives one, because it is not a query-language field at all. A path beside a header field's, such as emitter.process.pid beside emitter.process.guid, is an ordinary candidate.

Counters and a desired set written by an earlier eventd, which named header fields by their columns (event_type, process_guid), are read back at startup under the fields' paths. Such a name could only have meant the header field then, because a payload key spelled like a column was reserved, so the renaming loses no index and builds no spurious one.

The raw payload column never receives a plain column index. Indexing an opaque blob accelerates nothing.

3.4.5 Payload indexes are an optimisation, not an authority #

An expression index extracts a field from the payload on every insert and indexes a deterministic private key for it. The exact key bytes are internal to eventd and are not part of the storage contract.

eventd may implement these as SQLite expression indexes with deterministic extraction functions, as generated columns, or by any equivalent SQLite-backed means. What every mechanism has in common is that it implements the same field resolution and flattening as PSPU §3.22 — and that where the index cannot reproduce the query language's comparison semantics exactly, it is used only to narrow candidate rows, with the real predicate applied after the row is loaded (§6.3).

SQLite's native dynamic-type equality and ordering never substitute for the query language's string case folding, numeric comparison, binary comparison, array comparison, or null and missing-field handling. Getting a smaller answer faster is worthless if it is a different answer.

Rows where the field is absent, unqueryable or suppressed index as null, or are otherwise excluded in a way that preserves those semantics.

Payload indexes are otherwise ordinary members of the desired set, with the same priority ordering, shedding and convergence.

3.4.6 Naming #

Header column indexes are idx_events_<column>, named for the column rather than the field — idx_events_event_type for event.type, idx_events_process_guid for emitter.process.guid.

Payload expression indexes are named from the field GUID (§7.3), to avoid both collisions and characters SQLite will not accept in an identifier:

idx_events_payload_<field_guid_hex>

where field_guid_hex is the UUID v5 field GUID for the flattened path, as 32 lowercase hexadecimal digits with braces and hyphens stripped. The path source.name yields idx_events_payload_ followed by the 32 hex digits of uuid_v5(EVENTD_FIELD_NAMESPACE, "source.name").

Deriving the name from the GUID rather than from the path means the same path always produces the same index name, and no path — however it is spelled — can produce a name that collides with another's or that SQLite rejects.

One database in the event store directory is not a shard: eventd-meta.db. It holds the state that is global to eventd rather than to any shard — adaptive index state and diagnostic sequence checkpoints — and it is the one database that survives shard reconfiguration untouched.

It is created on first startup if absent or holding no schema, opened in WAL mode with synchronous=NORMAL. It is written at each run of the index policy (§3.4) — every policy interval, on every applied configuration change and every INDEX command, and once more at shutdown — and read at startup, so per-transaction durability buys nothing: losing the counters since the last run costs some adaptation, not any data.

The query path excludes it explicitly, since it is in the same directory as the shards (§3.3).

3.5.1 Tables #

index_counters — query frequency per field (§3.4).

desired_indexes — the computed desired index set.

sequence_checkpoints — diagnostic only.

Primary key (boot_id, cpu_id). Startup recovery uses committed receipt ranges and never this table (§2.2). The table exists so that an operator can see what eventd believed at shutdown and compare it with receipt coverage — the two disagreeing is itself diagnostic.

meta — key-value.

with two required entries: schema_version (§B) and created_at as a UTC YYYY-MM-DDTHH:MM:SSZ string. The administrative descriptor lives with every other eventd descriptor at Machine\System\eventd\Security\Admin (§7.2); no access-control state is stored in this reconstructible database.

3.5.2 Concurrency #

One writer connection, owned by the index policy thread. No other thread opens the database read-write.

Query handlers write only to the in-memory counters; the policy thread flushes them at each interval. Writer threads and query handlers read the desired sets from memory, never from the database.

Graceful shutdown writes sequence_checkpoints through the same connection, after policy activity has stopped (§8.4).

With a single writer and no other database-level access, SQLite's WAL mode is the whole of the concurrency control needed.

The policy thread checkpoints the write-ahead log at WalCheckpointPages in passive mode, and does not block when readers hold pages — the same rule as every other store (§2.4).

3.5.3 Recovery is cheap #

If SQLite cannot read the file, or the schema version is missing or unrecognised, or any required table or meta entry is missing or malformed, eventd logs an error and recreates the database from defaults.

This is the opposite of the rule for a shard, which fails startup (§3.3), and the difference is what is at stake. A shard holds the only copy of audit data. This database holds an optimisation policy and some diagnostics — all of it reconstructible, none of it irreplaceable. Losing it costs the adaptation eventd had accumulated, and the counters begin refilling immediately. Security policy is unaffected because it lives in the registry.

3.5.4 Startup #

1. Open eventd-meta.db, creating it if absent.
2. Verify the schema version and required meta entries; recreate from defaults on failure.
3. Load index_counters into memory.
4. Load desired_indexes into memory.
5. Load sequence_checkpoints, for diagnostics only.
6. Discover the material indexes in each shard from its schema and compare against the desired set.

eventd resumes convergence from wherever each shard happens to be. It neither drops nor rebuilds indexes at startup: a shard's material set is a fact to be observed, not a state to be restored.

Retention bounds disk growth. eventd deletes on two axes, age and size, and enforces both — an event goes when it exceeds either threshold.

The retention model is deliberately minimal, and a later one is expected to support rules resembling queries: retain KACS events for ninety days, synthetic events for seven, userspace-origin events for fourteen; and to prune during ingestion rather than only in arrears. What is here is the least that prevents unbounded growth.

3.6.1 Age #

Rows are deleted from events where timestamp is older than EventRetentionDays (§A) from the current wall clock, until none remain. Each shard is processed independently, and the rule covers KMES events, synthetic events and gap records alike.

3.6.2 Size #

Size is measured as logical live size, not file size:

logical_live_bytes = (page_count - freelist_count) * page_size

from PRAGMA page_count, PRAGMA freelist_count and PRAGMA page_size, taken after attempting a passive WAL checkpoint. The event store's total is the sum across every shard.

Pages freed by retention do not count, because they are reusable by future inserts. Counting them would make retention chase its own tail: each deletion would free pages that still counted against the limit, prompting more deletion.

When EventRetentionMaxBytes is non-zero and the total exceeds it:

1. Identify every non-current boot ID present in the shards.
2. Order them by their newest event timestamp, oldest boot first.
3. Delete each of those boots entirely, across all shards, one boot at a time, until the total is within the limit or no non-current boots remain.
4. If still over, delete the oldest events of the current boot by timestamp, across all shards, until within the limit.

Size pressure prefers boot boundaries. Deleting a whole old boot removes a self-contained unit — its sequence numbers, its gap records and its startup event go together — and it preserves recent events across the boundary, which is what an operator investigating a reboot needs. Only when whole boots are exhausted does eventd start on the current one.

3.6.3 Running it #

A background retention coordinator processes the event store first, then the log store (§4.4), then the metric store (§5.5). Each store is processed whether or not the one before it failed, and the pass reports every store that did. The interval is RetentionCheckIntervalMinutes (§A); an applied configuration change (§8.3) and a write refused for want of space (§9.2) each request a pass at once. After a failed pass, requests are held for one second, doubling to a minute while passes keep failing, so a full disk does not drive retention into a tight retry loop. It uses read-only connections to measure and plan. The only read-write connections it owns are to historical shards, which have no ingestion writer: it opens each one read-write for the life of the process and deletes from it and checkpoints it directly, as that shard's one writer.

Every other database has exactly one read-write connection, owned by its ingestion writer. The coordinator submits low-priority maintenance commands to that owner. A command runs only at a transaction boundary, deletes at most RetentionDeleteBatchRows, and yields to ingestion before another command runs. Under urgent size pressure the owner may append one bounded deletion to an ingestion transaction it already has open, avoiding an additional commit and fsync.

The passive checkpoint attempted before a size measurement is also a writer-owned command. The coordinator requests it, then reads the page and freelist counts through its read-only connection. It never tries to checkpoint through that connection or temporarily promotes it.

There is no retention writer mutex and no second SQLite writer. A single unbatched DELETE over a month of events could otherwise hold a write transaction for as long as it took, blocking the writer and, behind it, the drain thread and KMES ring.

3.6.4 Reclamation #

Deleting rows does not shrink a SQLite file. Freed pages are reused by later inserts, and reclaiming filesystem space needs VACUUM, which rewrites the whole database.

eventd never runs VACUUM automatically. Reclamation is an explicit administrative operation.

In steady state it is not needed. Where ingestion and retention run at comparable rates, the file settles at roughly the high-water mark of retained data and the freed pages are recycled without further growth — which is also why logical live size, rather than file size, is the right thing to measure.

Every record eventd stores — every event, every log line, every raw metric sample — carries a boot_id: a 16-byte GUID identifying the boot that produced it. Linux assigns it at boot; eventd reads /proc/sys/kernel/random/boot_id at startup, validates the UUID text, and stores the corresponding PCDS-layout GUID.

3.7.1 What it is for #

Disambiguation. KMES per-CPU sequence numbers restart at zero each boot. Without a boot ID, sequence 42 from one boot is indistinguishable from sequence 42 from the next, and every gap calculation across a reboot would be wrong.

Lifecycle. Retention can delete a whole boot as a unit rather than scanning by timestamp, and a boot is the natural unit to delete: its events, its gaps and its startup record go together (§3.6).

3.7.2 Where it is stored #

The log store records it because log output can mean different things across boots — a service configured differently at boot time says different things.

The metric store records it per sample but does not make it part of series identity (§5.2). A time series stays continuous across a reboot, which is what a chart of CPU usage across a restart should show, and a query that wants one boot's worth filters for it explicitly (PSPU §3.25).

3.7.3 Uniqueness #

Within one boot an event is uniquely identified by (cpu_id, sequence). Across boots, boot_id supplies the disambiguating dimension, and the triple (boot_id, cpu_id, sequence) is globally unique.

3.7.4 Detecting the boundary #

At startup eventd reads the kernel boot ID, then searches every readable event shard database — historical shards included — for committed rows or receipt ranges carrying it.

No committed rows or receipts for this boot. This is the boot's first successful eventd start. Coverage for every CPU begins before sequence 1, the new boot ID is used for subsequent writes to all three stores, and eventd.daemon.started carries store.restarted false.

Committed rows or receipts exist. eventd was previously ready in the same boot. It merges receipt coverage per (boot_id, cpu_id), reconciles that coverage with surviving ring records (§2.2), continues under the same boot ID, and emits eventd.daemon.started with store.restarted true.

The two cases are distinguished by the data itself rather than by any flag eventd persisted. That is the point: a flag would have to be written at a moment eventd might not reach, and a crash is precisely the case where it did not.

Committed receipt ranges are the sequence authority. Committed event rows still establish that eventd previously ran when no receipt was yet needed. The metadata database's sequence checkpoints and the previous eventd.daemon.stopped payload are diagnostic — neither is consulted for recovery (§2.2, §3.5).

The receipt key is sound only while KMES sequence numbers do not restart under one kernel boot ID, so eventd depends on KMES remaining initialised for the whole boot. A future KMES ABI may add a stream generation GUID to make module reloads distinguishable without relying on that lifecycle invariant.

One thread reads datagrams from the log socket and writes log records to the log store. It is independent of the event drain and writer threads, so log ingestion never contends with event ingestion.

The wire contract — socket type, datagram ceiling, record format, and exactly which malformations cost what — is PSPU §3.6 to §3.8. What follows is what eventd does with a record once it has one.

4.1.1 One thread does both jobs #

The log thread performs both the socket reads and the SQLite writes.

The consequence is direct: during a batch commit the socket is not being drained, and datagrams arriving in that window occupy the receive queue until it fills. A Unix datagram socket's receive queue is bounded in datagrams, by the kernel's net.unix.max_dgram_qlen, and it is the whole cushion. Once it is full the kernel refuses further sends rather than accepting and discarding them: a non-blocking sender receives EAGAIN, and a blocking one waits until the queue drains. eventd sets SO_RCVBUF to four times the datagram ceiling, but for this socket type the kernel charges a queued datagram to its sender's send buffer, and the receive buffer bounds nothing.

Splitting into a reader and a writer with a bounded handoff — the shape the event path uses (§2.3) — would decouple them. It is not done, and the reasoning is that log loss is tolerable by design (PSPU §3.4), log volume is normally well below event volume, and the single-thread model avoids a handoff channel and its backpressure semantics entirely.

Where log throughput does become the constraint, sharding the log store the way the event store is sharded is the larger lever; splitting the thread only moves the stall.

4.1.2 Batching #

The writer batches on the same adaptive principle as the event writer (§2.4), with the socket receive queue as its input. A transaction opens when the first valid record is available and commits when any of these holds:

• no further datagram is immediately available in the receive queue
• the batch holds LogMaxBatchSize records
• LogMaxBatchLatencyMs has elapsed since the first record entered it

If a datagram yields more valid records than fit in the remaining space, the writer commits, then continues with the same datagram in a new transaction. A transaction never exceeds the size cap and never stays open past the latency cap — a batched datagram cannot smuggle a larger transaction past either.

The defaults (§A) are 5000 records and 500 milliseconds. The latency is five times the event writer's, because log loss on power failure is acceptable where event loss is not, and larger, less frequent transactions are more efficient at the moderate volumes logs normally run at.

The writer keeps the known origin names in memory. Before inserting the first log row for an origin not in that set, it executes INSERT OR IGNORE into log_origins in the same transaction (§4.2).

A transaction-local pending set ensures that insert runs only once per new origin in a batch. The main in-memory set is updated only after commit and the pending set is discarded on rollback. Thus discovery never needs to scan the hot logs table, and a rolled-back first row cannot make the cache claim that its catalogue entry exists.

4.1.3 Durability #

The log store runs in WAL mode with synchronous=NORMAL, not FULL.

NORMAL syncs at checkpoint time rather than at every commit. It is durable against process crashes — the write-ahead log survives — but not against power loss, where commits since the last checkpoint may be gone.

This is a deliberate divergence from the event store, and it is the single clearest expression of the hierarchy the whole daemon is organised around: events are sacred, logs are not. Paying an fsync per transaction to protect data whose loss is defined as acceptable would be paying for nothing.

4.1.4 Adding to the record #

eventd supplies the boot_id and, where the producer omitted timestamp, its own clock at receipt. Everything else is stored as given — message byte for byte (PSPU §3.8).

4.1.5 Rejected origins are counted #

A record whose origin is outside the origin grammar (§4.2) is discarded where it is parsed, before it can join a batch. eventd counts every such discard and keeps the most recent origin, escaped and truncated to 64 characters; both appear in the diagnostic dump (§8.5) as log_ingress and last_rejected_log_origin.

It also writes one line to standard error on the first discard and at most one a minute after that. peinit captures eventd's own standard error, so that line is stored as an ordinary log record under the origin eventd and is queryable with everything else — and it still reaches the console directly when the log store is the broken thing.

The rate limit is what makes the report usable. A producer whose vocabulary has drifted from this one usually uses the same origin for every line it sends, so an unlimited report would turn a silent total loss into a flood; the count is the quantity that matters and the text is there to name the vocabulary that drifted.

The log store is a single SQLite database — not a directory of shards. There is no sharding here: one ingestion thread produces the writes, so splitting the target would give a single writer several files to switch between rather than several writers working in parallel.

It holds the logs, log_origins and metadata tables.

The schema is deliberately narrow. A log record is text with light metadata: what produced it, whether it was an error, when, and optionally which execution it belongs to. There is no payload blob and no origin class. origin is broker-attested rather than a service's self-assertion: the log socket admits peinit and excludes service-logon tokens (§7.6). The optional correlation key is not an identity.

4.2.1 What an origin looks like #

An origin is one or two components (PSPU §3.7):

origin    := component | component "/" producer
producer  := component | component "[" [0-9]+ "]"
component := [A-Za-z0-9_][A-Za-z0-9_.-]*

One component is a service — loregd, jellyfin — and is what a main process's output carries. Two name a producer within a service: peinit tags a pre-start hook jellyfin/ExecStartPre[0], a reload command jellyfin/ExecReload, a health check jellyfin/HealthCheck, and a submitted job jobs/<guid> (peinit TRM §11.1). The bracketed index appears only after a slash, and there is never more than one slash.

Ingestion discards a record whose origin is anything else, before the record reaches a batch (§4.1). Nothing wider is accepted: * would impersonate the access-control wildcard, and a backslash, a quote or whitespace would reach into the registry path an origin's descriptor lives at (§7.2).

A program needing more structure than this emits events.

4.2.2 The origin catalogue #

log_origins has one column, origin TEXT PRIMARY KEY. The log writer inserts a previously unseen origin in the same transaction as its first log row (§4.1), so every committed row is discoverable immediately.

Query planning enumerates origins from this compact catalogue rather than running DISTINCT origin over the retained log rows (§6.1).

Catalogue entries deliberately survive ordinary retention. An origin may therefore remain discoverable after its final row expires; this safe superset avoids write work on the ingestion path.

Low-priority maintenance may remove an entry after proving through a read-only scan that no row refers to it, with the deletion submitted to the log writer. The writer rechecks NOT EXISTS in that transaction and removes the origin from its in-memory set only after commit, so a log arriving between planning and execution cannot become undiscoverable.

Catalogue pages count toward logical live size.

4.2.3 is_error is an integer here and a boolean there #

The column stores 0 or 1; the query language exposes a boolean, and accepts either WHERE is_error == true or WHERE is_error == 1 (PSPU §3.22). ERROR ONLY is sugar for the first.

4.2.4 Write-time indexes #

Three indexes are created with the table:

• idx_logs_timestamp on logs(timestamp) — time-range filtering, as everywhere.
• idx_logs_origin on logs(origin) — "show me logs from X", which is the dominant log query.
• idx_logs_job_id on logs(job_id) WHERE job_id IS NOT NULL — a partial index for "show me logs for job X". Partial because ordinary service output may carry no correlation key, so only correlated lines are worth indexing.

The origin index costs write amplification beyond the timestamp index, and the cost is modest in practice: origin has low cardinality, tens of distinct names on a normal system, so its index pages stay in SQLite's page cache and insertion stays cheap. The trade is accepted deliberately — the two dominant log queries must not become full table scans.

4.2.5 No adaptive indexing #

The log store does not participate in adaptive indexing (§3.4). Its field set is closed and small, and the three write-time indexes already cover the access patterns; there is no space of candidate fields for a policy to discover.

The same follows for query frequency counters: log queries do not increment them (§6.5).

4.2.6 Schema version #

The log store holds a metadata table with the same two-column structure as a shard's (§3.1). Schema version 1 comprises logs, log_origins and metadata; its version number is in §B.

eventd checks it at startup and applies the lifecycle rules of §4.3, and does not migrate.

4.3.1 Path #

LogStorePath (§A) names a provisioned directory, conventionally /var/state/eventd/logs. The database itself has the fixed name logs.db inside it. A missing, invalid or unsafe directory is a startup failure.

The directory has the provisioning, descriptor and path-resolution requirements defined for the event store (§3.3). eventd does not create it or any parent directory, never follows a symbolic-link component, and creates or opens the database relative to an already validated directory handle.

4.3.2 Creation #

A log store that does not exist, or whose database holds no schema at all (what a power cut leaves when it takes the uncheckpointed creating transaction), is created, in one transaction, with:

1. WAL mode, PRAGMA journal_mode=WAL
2. PRAGMA synchronous=NORMAL
3. the logs, log_origins and metadata tables (§4.2)
4. the idx_logs_timestamp, idx_logs_origin and idx_logs_job_id indexes
5. the schema_version and created_at entries

4.3.3 Opening #

1. Open in WAL mode.
2. Set synchronous to NORMAL.
3. Verify the schema version. Missing or unrecognised is a startup failure; no migration is attempted.
4. Verify structural integrity — required tables and indexes present. Failing this, with SQLite reporting no corruption, is a startup failure.
5. On SQLite reporting corruption, quarantine and replace. A database holding schema objects but no metadata table with entries is not a store eventd could have written; it is quarantined and replaced like a corrupt one.

Quarantine works exactly as for a shard (§3.3): the database, -wal and -shm files are renamed with a shared .corrupt.<timestamp_ns> suffix, .N appended with the lowest positive integer if the name is taken, and a fresh empty log store is created at the configured path.

The log store is a required store. There is no degraded mode in which eventd runs without one (§8.2), which is why steps 3 and 4 fail startup rather than proceeding without logs.

4.3.4 Concurrency #

Exactly one read-write connection, owned by the log writer thread, and any number of read-only query and maintenance connections. WAL mode lets readers run concurrently. Retention and catalogue cleanup submit bounded commands to the writer and never open another read-write connection (§3.6).

4.3.5 Checkpointing #

The log writer checkpoints when its write-ahead log reaches WalCheckpointPages (§A), in passive mode, and does not block if readers hold pages — it keeps writing and retries after a later commit.

Checkpointing matters more here than in the event store, because synchronous=NORMAL makes the checkpoint the durability boundary rather than merely a space-reclamation event: data committed since the last checkpoint is what a power cut takes (§9.5).

Log retention works exactly as event retention does (§3.6), on the same background thread, running after the event store and before the metric store. As there, the model is an early simplification and both limits are enforced with the more aggressive one winning.

4.4.1 Age and size #

Rows older than LogRetentionDays (§A) are deleted from logs until none remain.

If LogRetentionMaxBytes is non-zero and the store's logical live size exceeds it, the oldest entries by timestamp are deleted until it is within the limit. Logical live size is the same measure as §3.6 — (page_count - freelist_count) * page_size after attempting a passive checkpoint — and freed pages do not count.

There is no boot-boundary preference here. Event size retention prefers to drop whole old boots because a boot is a self-contained unit of sequence-numbered records; a log line has no such structure, so oldest first is the whole rule.

4.4.2 The default is shorter than events' #

Fourteen days against the event store's thirty (§A).

Historical log data is worth less than historical audit data, and it is usually bulkier per unit of value. The metric store's default is longer than either at ninety days, for the opposite reason: a metric sample is tiny and trend data is worth more the further back it goes (§5.5).

4.4.3 Batching #

The retention coordinator plans with a read-only connection and submits low-priority commands to the log writer. Each writer-owned transaction deletes at most RetentionDeleteBatchRows, and ingestion is rechecked before the next command. Under urgent size pressure the writer may append one bounded delete to a transaction already open. Retention never takes a writer mutex or opens a second read-write connection (§3.6).

The stall this avoids is the log ingestion thread's, and that thread is also the one draining the socket (§4.1) — so a retention pass holding a long write transaction would not merely delay writes, it would stop the socket being read and lose the datagrams that arrived meanwhile.

4.4.4 Reclamation #

VACUUM is never run automatically, as everywhere. Freed pages are recycled by later inserts and are excluded from the size measure.

One thread reads datagrams from the metric socket and writes samples to the metric store, independent of both the event and log paths. It has the same single-thread shape as the log writer, with the same consequence during a commit (§4.1).

The wire contract is PSPU §3.9 to §3.13. The same thread also writes eventd's own health samples, which skip step 1 below (§5.7).

5.1.1 Processing a record #

For each valid record:

1. Authorize the name. The datagram's KACS token must hold EVENTD_PUBLISH on the resolved metric-name descriptor (§7.6).
2. Resolve the series from name and labels — and, for a histogram, bucket boundaries — through the in-memory series cache (§5.3). A series that does not exist is inserted into series with the record's type, and the cache is updated.
3. Check the type. If the record's type differs from the resolved series' type, the record is dropped without replying to the producer. The type is set at creation and is immutable; a series never changes type.
4. Insert the sample into samples with the resolved series_id, the timestamp and the value. SQLite assigns samples.id, which is the deterministic tiebreaker among samples sharing a timestamp (§5.2). A histogram's data is encoded as the canonical MessagePack sample map and stored in histogram_data.

The transaction result counts step-3 failures and carries the most recent conflict's metric name and expected and received types. The metric thread adds the count to an in-memory diagnostic total, retains that conflict, and emits a coalesced standard-error warning at most once per minute. SIGQUIT includes both the total and latest conflict (§8.5).

There is still no response to the producer and no event per failure. Authentication identifies the producer; it does not make reactions proportional to its bad input safe. The one-minute global bound is what keeps the warning from becoming the amplification vector PSPU §3.4 forbids. A valid transaction takes only the existing zero-mismatch branch and performs no diagnostic allocation or synchronization.

5.1.2 Out-of-order samples #

The writer stores a valid sample whose timestamp precedes samples already held for that series.

Producers batch, clocks step, and sweeps get retried. Refusing late samples would convert any of those into silent loss, so eventd accepts them and defines every ordering it performs over (timestamp, id) rather than over insertion order. That makes raw RATE and DELTA evaluation deterministic regardless of arrival (§6.2).

5.1.3 Batching #

The same adaptive algorithm as the event and log writers (§2.4), with the socket receive queue as input. A transaction opens at the first valid sample and commits when any of these holds:

• no further datagram is immediately available
• the batch holds MetricMaxBatchSize samples
• MetricMaxBatchLatencyMs has elapsed since the first sample entered it

A datagram yielding more samples than fit is split across transactions, the writer committing before continuing with the same datagram. Neither cap is ever exceeded.

The defaults (§A) are 5000 samples and 1000 milliseconds — the longest latency of the three writers. Metrics are typically sampled every fifteen seconds, so a one-second commit window accumulates a whole sweep's worth without any latency that a dashboard could notice. Under a burst, where a collection agent submits every core, disk and interface at once, the size cap is what forces a timely commit.

5.1.4 Durability #

WAL mode with synchronous=NORMAL, as the log store (§4.1). Metric loss on power failure is acceptable, so per-transaction fsync buys nothing.

The metric store is a single SQLite database, and unlike the event and log stores it is organised around series rather than records. Individual samples are appended to a series that already has an identity.

5.2.1 The series table #

5.2.1.1 The canonical label string #

Labels are sorted by key in unsigned UTF-8 byte order, each pair written key=value, and the pairs joined with commas: core=0,host=server1. The empty label set is the empty string.

No escaping is performed and none is needed, because ingestion rejects = and , inside a key or a value (PSPU §3.10). That prohibition exists precisely to make this encoding unambiguous, and it is the reason the constraint binds the producer rather than being handled internally.

5.2.1.2 The boundaries blob #

A fixed binary encoding, not MessagePack:

1. boundary_count, u32 little-endian
2. that many IEEE-754 f64 values, each little-endian, in the validated order the producer sent

It exists only to identify histogram series and to resolve hash collisions, and is never returned in a query result.

5.2.1.3 Hashes narrow, they do not decide #

label_hash and boundaries_hash are 64-bit FNV-1a over the exact bytes of the canonical string or blob, with offset basis 0xcbf29ce484222325 and prime 0x100000001b3. The high bit is cleared before storage, hash & 0x7fff_ffff_ffff_ffff, so the value always fits SQLite's signed INTEGER.

A lookup always verifies the full labels string, and for a histogram the full boundaries blob, after narrowing by hash. A hash is an index key, never an identity: two label sets that collide are still two series.

5.2.1.4 Uniqueness #

The table carries UNIQUE(name, labels, boundaries), on the full boundary blob. boundaries_hash is a lookup key with its own index, and two histograms whose boundaries collide on it are two series.

For counters and gauges boundaries is null, and SQLite treats nulls as distinct in a unique constraint — so the constraint does not enforce uniqueness for them. What does is the single-writer resolution logic (§5.3), which checks before inserting. The constraint is a defensive backstop against a future change that introduces a second write path, not the primary mechanism.

type is not part of the identity. A record resolving to an existing series with a different type resolves successfully and is then dropped for the mismatch (§5.1).

5.2.2 The samples table #

For a histogram, histogram_data is a canonical MessagePack map (PSPU §3.5) with exactly four keys: boundaries, an array of float64 in the producer's order; counts, an array of unsigned integers; total_count; and sum, a finite float64.

value is a placeholder for histogram rows and is never returned as a metric query value. Storing 0 rather than null keeps the column NOT NULL and keeps the row layout uniform.

Canonical encoding is required here because a stored sample map must be byte-stable: two equal histograms encode identically, which is what makes them comparable without decoding.

boot_id is per sample and is not part of the series identity, so a series stays continuous across a reboot (§3.7).

5.2.3 Ordering #

Query execution order within a series is always (timestamp, id) ascending, never insertion order alone. Duplicate timestamps are permitted and id gives them a stable order.

id is internal. It is never exposed as a query field, a result field, an access-control field, or a reserved label key (PSPU §3.28).

5.2.4 Write-time indexes #

• idx_samples_series_timestamp on samples(series_id, timestamp, id) — the dominant pattern is "samples for series X over range Y in deterministic order", and this one composite index serves the series lookup, the range filter and the (timestamp, id) ordering in a single scan.
• idx_series_name on series(name) — name lookups.
• idx_series_label_hash on series(label_hash) — series resolution on the ingestion path.
• idx_series_boundaries_hash on series(boundaries_hash) — narrowing a histogram's lookup by its boundary hash.

5.2.5 Schema version #

A metadata table with the same structure as the other stores' (§3.1). Version 1 comprises series, samples and metadata. Version 2 adds the disposable rollups query cache (§5.6); it does not change raw series or sample storage. Version 3 changes only the series table's uniqueness, from the boundary hash to the full boundary blob, and adds idx_series_boundaries_hash. The current value is in §B. eventd checks it at startup and applies the lifecycle and migration rules of §5.4.

Every arriving sample must be turned into a series_id before it can be inserted. This happens once per sample on the single metric ingestion thread, so it is the hottest lookup in the daemon and the reason a cache exists at all.

5.3.1 Resolving #

1. Compute the canonical label string: sort by key in unsigned UTF-8 byte order, encode each pair key=value, join with commas. The empty label set encodes as the empty string. No escaping — ingestion has already rejected the delimiters (§5.2).
2. Hash it.
3. For a histogram, compute the canonical boundary blob and its hash from the validated, producer-supplied order. eventd never sorts boundaries. For counters and gauges both are null.
4. Look up series by name, label_hash, and for histograms boundaries_hash.
5. On a match, verify the full labels string, and for histograms the full boundary blob. If the record's type differs from the existing series' type, drop the record (§5.1). Otherwise use the existing series_id.
6. On no match, insert a new series row and use the new identifier.

A histogram whose boundaries changed takes step 6: it is a new series (PSPU §3.13). The old one keeps its historical samples and the new one starts accumulating.

5.3.2 The cache #

Resolution runs through a bounded in-memory cache mapping (name, canonical labels, boundaries hash, boundaries blob) to series_id. For counters and gauges the boundary components are absent.

A hit is a hash table lookup with no SQLite involvement. A miss costs one SELECT on name and label_hash, after which the result is inserted, evicting the least recently used entry if the cache is full.

The bound is MetricSeriesCacheSize (§A), default 50000, with LRU eviction. It bounds memory, not the number of series: the series table is uncapped and a new series is always created in the database. A system with a million series and a 50000-entry cache uses memory proportional to the cache, and at roughly 200 to 300 bytes an entry the default costs 10 to 15 MB.

The cache starts empty after a restart and is warmed on demand — within one collection cycle, typically fifteen seconds, every active series is cached. There is no pre-warming pass, because reading a million-row series table at startup to populate a 50000-entry cache would be work spent to discard most of its result.

5.3.3 Sizing it #

The cache is sized for the set of actively reporting series, and behaves badly below it.

Below that, LRU does not help, because every series is equally hot: each collection cycle evicts the overflow and reloads it, producing a fixed number of SELECTs every cycle, permanently. A system with 55000 active series and a 50000-entry cache incurs about 5000 cache misses every fifteen seconds, indefinitely.

This interacts badly with label cardinality (PSPU §3.10). Labels with unbounded values — request identifiers, user-supplied strings — grow the series table without limit, and once the active set exceeds the cache every cycle pays the eviction cost on the one thread that also drains the metric socket. The failure presents as metric loss, because the thread stops reading while it queries.

eventd does not cap series creation: at the interface, a producer creating a genuinely new series is indistinguishable from one creating garbage, and rejecting either would break a correct producer to inconvenience an incorrect one (PSPU §3.13). The default 1 GiB metric retention ceiling bounds the resulting disk growth (§5.5), but it does not prevent a hot working set from exceeding this cache and thrashing the ingestion thread before retention runs.

5.4.1 Path #

MetricStorePath (§A) names a provisioned directory, conventionally /var/state/eventd/metrics. The database itself has the fixed name metrics.db inside it. A missing, invalid or unsafe directory is a startup failure.

The directory has the provisioning, descriptor and path-resolution requirements defined for the event store (§3.3). eventd does not create it or any parent directory, never follows a symbolic-link component, and creates or opens the database relative to an already validated directory handle.

5.4.2 Creation #

A metric store that does not exist, or whose database holds no schema at all (what a power cut leaves when it takes the uncheckpointed creating transaction), is created, in one transaction, with:

1. WAL mode.
2. PRAGMA synchronous=NORMAL — the log store's reasoning, for the same reason: metric loss on power failure is acceptable (§4.1).
3. The series, samples, rollups and metadata tables (§5.2, §5.6).
4. Every write-time index.
5. The schema_version and created_at entries.

5.4.3 Opening #

1. Open in WAL mode with synchronous NORMAL.
2. Verify the schema version. Version 1 is migrated transactionally to version 2 by adding the adaptive-rollup cache. Version 2 is migrated transactionally to version 3 by rebuilding the series table under its new uniqueness constraint, foreign keys off for the rebuild and every identifier kept; a version 2 store missing a required table or index fails startup instead. Missing or unrecognised versions are a startup failure.
3. Verify structural integrity — required tables and indexes present. Failing this, with SQLite reporting no corruption, is a startup failure.
4. On SQLite reporting corruption, quarantine and replace, exactly as for a shard (§3.3): matching -wal and -shm files renamed with the same .corrupt.<timestamp_ns> suffix, .N appended if the name is taken, and a fresh empty store created at the configured path. A database holding schema objects but no metadata table with entries is not a store eventd could have written; it is quarantined and replaced like a corrupt one.

The metric store is a required store; there is no degraded mode without one (§8.2).

After opening or creation the series cache is empty and fills on demand (§5.3).

5.4.4 Concurrency #

Exactly one read-write connection, owned by the metric writer thread, and any number of read-only query and maintenance connections. WAL mode permits readers alongside the writer. Retention and adaptive rollups plan or compute with read-only connections and submit bounded commands to the writer; they never open another read-write connection (§3.6, §5.6).

The single-writer property is load-bearing here in a way it is not for the other stores: series resolution checks for an existing row and then inserts, without a transaction spanning both, and only one writer makes that safe (§5.3).

5.4.5 Checkpointing #

The metric writer checkpoints at WalCheckpointPages (§A) in passive mode, and does not block when readers hold pages.

As with the log store, the checkpoint is the durability boundary under synchronous=NORMAL, not merely space reclamation (§9.5).

Metric retention runs on the same background thread as the other two, after the log store (§3.6, §4.4). Both limits are enforced and the more aggressive wins.

5.5.1 The pass #

1. Delete rows from samples older than MetricRetentionDays (§A) until none remain.
2. If MetricRetentionMaxBytes is non-zero and the store's logical live size exceeds it, delete the oldest samples by timestamp until it is within the limit. The mainline default is 1073741824 bytes (1 GiB); zero is an explicit opt-out rather than the default.
3. Track every series_id whose samples were deleted by either step.
4. Before deleting a sample batch, delete all adaptive rollups belonging to any affected series (§5.6).
5. Delete every series row with no remaining samples.

Logical live size is the same measure as §3.6.

Step 5 removes the definitions of series nobody produces any more, which is the only mechanism that ever removes a series row. A series that stopped receiving samples persists until its last sample is removed. In the absence of size pressure that is ninety days by default; the byte ceiling can remove it earlier.

5.5.2 The longest default #

Ninety days, against fourteen for logs and thirty for events (§A).

A metric sample is small and its value grows with age: a year of CPU utilisation is a capacity-planning input in a way that a year of log lines is not. A thousand series sampled every fifteen seconds produce about 5.7 million samples a day, which is a few hundred megabytes in SQLite.

The long age limit therefore does not make storage unbounded. The 1 GiB size default is the safety boundary for a publisher that continually creates new label combinations: eventd still accepts new series, as PSPU requires, but retires the oldest metric data rather than allowing one metric namespace to consume the shared eventd volume. Installations with a dedicated, externally bounded metric volume may set zero deliberately.

The size decision is made by the retention coordinator, never while a datagram is parsed or a sample is committed. Normal metric persistence therefore pays no size-check cost.

5.5.3 Rollups are not downsampling #

Retention deletes raw samples; it does not replace them with lower-resolution data. Adaptive rollups accelerate repeated queries while their raw sources remain present and valid. They are invalidated before retention removes any source from their series, so they cannot extend the apparent lifetime of data or answer a query after its authoritative samples have expired (§5.6).

A future retention engine may aggregate high-resolution data as it ages — per-second samples becoming five-minute aggregates after a week, hourly aggregates after a month — but that would require a separate versioned storage and query contract.

5.5.4 Batching and reclamation #

The retention coordinator plans with a read-only connection and submits low-priority delete commands to the metric writer. Each writer-owned transaction deletes at most RetentionDeleteBatchRows, and ingestion is rechecked before the next command. Under urgent size pressure the writer may append one bounded retention delete to a transaction it already has open; retention never takes a writer mutex or opens another read-write connection (§3.6).

VACUUM is never run automatically. Freed pages are recycled and are excluded from the size measure.

Raw rows in samples are always authoritative. The rollups table is a disposable query cache: deleting every rollup can make a query slower, but cannot change its answer. A miss or a stale row is computed directly from raw samples and returned immediately; query execution never waits for the cache to be repaired.

The design protects eventd's primary job. Metric ingestion does not inspect, update or coordinate with rollups. A query computes a candidate while doing work it already needed for its own answer, and submits that candidate through a bounded non-blocking channel. The sole metric writer considers at most one such command only after the receive queue is idle and the current sample batch is committed. A full command channel, writer pressure or any ordinary cache-write failure drops the candidate rather than delaying ingestion.

5.6.1 Eligible queries #

A query can read or seed rollups only when all of these are true:

• it has SINCE and an explicit AVG_OVER, MIN_OVER, MAX_OVER or SUM_OVER function;
• the selected output is per series — the selector is bracketed, or it resolves to exactly one series;
• it has no WHERE predicate or cross-type filter; and
• AdaptiveRollupMaxRows is non-zero.

The label selector has already selected concrete series and is therefore safe. An unbracketed multi-series query is not eligible because its window result combines series; reusing independently cached averages or extrema would not in general reproduce that result. Predicate and cross-filter shapes remain on the raw path because their result depends on more than the cached series window.

Only complete windows aligned to Unix-epoch multiples of the requested width are cached. Any partial window at either end of the requested range is always read from raw samples. A query uses the cache only when every complete interior window for the series has a valid row, including explicit empty-window rows; otherwise it evaluates that series from raw samples. This all-or-raw rule keeps the fallback simple and prevents a cache miss from changing transform pairing.

Streaming continuations do not use or seed rollups. They preserve their normal incremental raw-query semantics.

5.6.2 Schema and identity #

Metric schema version 2 adds:

CREATE TABLE rollups (
    series_id INTEGER NOT NULL REFERENCES series(id) ON DELETE CASCADE,
    window_start INTEGER NOT NULL,
    window_width INTEGER NOT NULL CHECK (window_width > 0),
    transform INTEGER NOT NULL CHECK (transform IN (0, 1, 2, 50, 95, 99)),
    function INTEGER NOT NULL CHECK (function BETWEEN 0 AND 3),
    value REAL,
    overflow INTEGER NOT NULL CHECK (overflow IN (0, 1)),
    source_max_sample_id INTEGER NOT NULL CHECK (source_max_sample_id >= 0),
    source_baseline_sample_id INTEGER CHECK (source_baseline_sample_id > 0),
    CHECK (overflow = 0 OR value IS NULL),
    PRIMARY KEY
        (series_id, window_start, window_width, transform, function)
) WITHOUT ROWID;

transform distinguishes no transform, RATE, DELTA, P50, P95 and P99. function distinguishes AVG_OVER, MIN_OVER, MAX_OVER and SUM_OVER. value is null only for an empty window or a percentile overflow; overflow distinguishes those cases. Empty rows are necessary to prove that a complete cached range really has no missing window.

Opening a version-1 metric store creates the table and its pruning index in one immediate transaction, then records version 2, and continues to version 3 (§5.4). Unknown versions still fail startup (§5.4, §B).

5.6.3 Validity proof #

Each row records the greatest raw sample id observed inside its window as source_max_sample_id. Before serving it, the query engine proves that the same series and window has no sample with a greater id. A later sample therefore makes the row stale even when its timestamp was backfilled into an old window. An empty window records zero, so its first raw sample also invalidates it.

RATE and DELTA additionally depend on the sample immediately preceding the window. Their rows record that sample's id in source_baseline_sample_id, with null meaning no preceding sample existed. The query engine compares this with the current immediately preceding sample. A new, changed or deleted baseline makes the row stale.

Retention invalidates all rollups for every series from which it deletes a raw sample before it deletes the sample (§5.5). This is deliberately conservative. The query-time proof remains mandatory: maintenance ordering alone is not a freshness guarantee.

Non-finite cached values are rejected. As everywhere in metric evaluation, a numeric result must be finite; percentile overflow is represented by the explicit null-and-overflow state instead (§6, PSPU §3.25).

5.6.4 Seeding and bounds #

An eligible raw evaluation may enqueue candidates only after it has scanned at least AdaptiveRollupMinSamples raw inputs. It enqueues no more than AdaptiveRollupBatchRows missing windows, and submission uses try_send on a fixed-capacity channel. Repeating a wide query gradually fills its cache in bounded pieces without starting a background scanner.

The writer upserts a candidate under the complete cache identity and prunes the oldest window starts until no more than AdaptiveRollupMaxRows remain. The cap is global to the metric store. It limits this optional acceleration state; it does not delete series or raw samples.

All three controls are live (§A). Setting AdaptiveRollupMaxRows to zero immediately disables cache reads and writes without changing query results.

Adaptive rollups are not downsampling. Raw samples retain their ordinary age and size policy, and every supported query remains answerable from them (§5.5).

eventd records its own health as metrics named eventd.*. They answer the questions an operator asks of a running collector — is it keeping up, is it losing anything, how big are the stores, are queries being turned away — through the ordinary query channel, so they can be charted and compared over time like any other metric.

5.7.1 How they are written #

Every HealthMetricIntervalSeconds (§A), the metric thread takes one sample of every series below and commits it to the metric store. The samples do not travel through the metric socket: there is no datagram, no KACS token and no EVENTD_PUBLISH check (§7.6), in the same way synthetic events never touch KMES (§2.6). An interval of 0 turns them off.

Health samples obey MetricMaxBatchSize (§5.1). They fill any pending batch up to the size cap captured when it opened; further transactions use the current configured cap (§8.3). The final partial batch is committed immediately. Splitting an interval's samples does not drop or reorder them.

Each sample carries eventd's clock and the current boot ID, like a metric record that omits its timestamp (PSPU §3.11).

The counters count from eventd's start, and so restart from zero whenever eventd does. A reader asks for their RATE or DELTA, which read a decrease as a restart (PSPU §3.10, §3.25).

5.7.2 The series #

eventd.store.bytes is the space on disk, which is what fills a volume. Size retention compares a store's logical live size instead, which leaves out free pages and the write-ahead log (§3.6), so the two can differ.

5.7.3 Who can read and publish them #

They are read like any metric, through the Metrics descriptors (§7.2). eventd creates Metrics\eventd on first boot, granting EVENTD_READ to SYSTEM, Administrators and Authenticated Users and EVENTD_PUBLISH to nobody. Without it the wildcard's grant would let an administrator's process send samples under eventd.* through the socket and pass them off as eventd's. An administrator who wants them narrower edits that descriptor.

5.7.4 What is left out #

Rejected ingestion input is not counted here: log records with an invalid origin, and metric datagrams without a token, truncated, denied by name policy or carrying the wrong type. Counting that input where a query client can see it is exactly what PSPU §3.4 and §3.12 forbid. Those counts stay in memory and reach only the diagnostic dump (§8.5).

Neither are datagrams the kernel refuses at a full socket queue. eventd is never told about them (§9.1), so it has nothing to count.

A query arrives as one string (PSPU §3.15). Turning it into an answer has four phases before any data is read: parse, plan, authorize, execute.

6.1.1 Parsing #

The string is parsed into a syntax tree. The parser:

1. Identifies the mode from the first token — EVENTS, LOGS or METRIC.
2. Extracts the primary selector: a type pattern, a FROM list, or a metric name with an optional label selector.
3. Collects every clause, in whatever order they appear.
4. Validates that the clauses suit the mode — CONTAINING only in log mode, RATE only in metric mode, SELECT only where a result schema is not fixed.

Parse errors are returned immediately, before anything is opened, read or authorized. A malformed query costs a decode and a parse.

6.1.2 What can only fail later #

Some failures need data. The parser cannot know whether a metric name resolves to a counter or a histogram, how many series a selector matches, or whether the effective range exceeds the cross-type lookback limit — those depend on the store, so they surface at planning or execution time (PSPU §3.B).

The practical consequence is that the same query string can parse everywhere and fail on one machine: a metric selector matching one series on a two-core box matches two on a four-core one.

6.1.3 Planning #

Planning resolves what the query will actually touch:

• which concrete identifiers the data could carry — event types, log origins, metric names — because access control resolves per identifier and a broad selector authorizes nothing by itself (§7.4)
• which series a metric selector matches, and whether they are type-homogeneous
• which fields the query references, for both authorization and frequency accounting (§6.5)
• which stores are involved, including any cross-type source

Identifier discovery reads compact catalogues: the union of every shard's event_types, the log store's log_origins, and the metric store's series rows (§3.1, §4.2, §5.2). The primary selector filters that set before access checks. Discovery therefore never runs a DISTINCT scan over a hot records table and does not depend on an adaptive event index being present.

Catalogues may conservatively contain a name whose final retained row was deleted. That safe superset can cause an extra access check but cannot expose a row or omit a concrete identifier that committed successfully. Low-priority cleanup is optional and is never query-critical.

6.1.4 Header literal types #

Literal conversion uses the namespace of the record being compared. emitter.class aliases apply only to event headers. GUID forms apply to the event GUID headers, log boot_id and job_id, and metric boot_id. The same spelling in an ordinary payload field or metric label remains a string; it does not borrow another namespace's header type (PSPU §3.22).

The rule holds for comparisons, membership and compound predicates, metric bracket selectors, and metric selectors inside historical or streaming cross-type conditions. A metric condition uses the metric namespace even when the query's primary source is events or logs.

6.1.5 When payloads are decoded #

Event payloads are stored as opaque MessagePack and are never decoded on the write path (§3.1). Decoding happens here, on the read path, and only where a query needs it: to evaluate a payload predicate, to build a flat result record, or to compute a payload expression index's key on insert.

At high result counts this dominates the query path. Constructing flat maps from thousands of events means decoding thousands of payloads and applying the flattening rules of PSPU §3.22 to each. Partial extraction is the lever: with a SELECT present, only the named paths need decoding, and without one a streaming decoder that emits flattened pairs avoids materialising the payload at all (§C).

6.1.6 Read connections #

Execution uses read-only SQLite connections, which in WAL mode do not block writer threads. The one exception is SQLite's own. A reader that catches the wal-index header while a writer is updating it takes the write lock for the moment it needs to read the header again. A writer therefore waits up to 25 milliseconds for the write lock rather than failing a commit at once, and fails only if the lock is held longer.

An event query opens one connection per shard database in the directory (§6.4). A log or metric query opens one. eventd supports concurrent queries up to its admission limit (§6.5), subject to the operating system actually having the descriptors and memory; where it cannot allocate what an admitted query needs, it fails that query rather than blocking a writer or exceeding the limit.

PSPU §3.21 requires every result order to be total and deterministic for a fixed set of stored records, so that SKIP and TAKE page reliably. This is how eventd achieves it.

6.2.1 The tiebreakers #

Where the query's explicit SORT keys — or the mode's default ordering — do not uniquely order two records, eventd appends internal keys until the order is total:

These are not query-language fields. They never appear in a result record, cannot be named in a SORT or a SELECT, and have no access-control identity (PSPU §3.28).

A metric's canonical-label key is captured before field filtering. Hiding a label leaves this internal key intact, including on per-series transformed and aggregate rows, so it does not change their order or pagination. The denied label remains absent from the result. Explicit query references to that label still require field access (§7.4).

The shard index is the numeric identifier from the shard-NNNN.db filename. It appears in the event tiebreaker because rowids are per-database: two events in different shards can share a rowid, and without the shard index the pair would be genuinely unordered.

The metric tiebreaker includes name and labels because an unbracketed query merges several series into one output stream, where the timestamp alone does not separate rows from different series.

6.2.2 Why insertion order is never the answer #

samples.id and events.id break ties within one database, but neither is a substitute for the timestamp ordering they follow.

Metric samples may arrive out of timestamp order (§5.1), so insertion order and time order genuinely differ. Every metric computation — RATE's consecutive pairs, aggregate window membership, cross-type interval construction — is defined over (timestamp, id) ascending precisely so that a late-arriving sample lands where its timestamp says it belongs rather than where it happened to be written.

Events are less prone to it, since a drain thread reads one ring buffer in order, but a shard receiving from several CPUs interleaves them arbitrarily and a clock step can invert two events from the same CPU.

6.2.3 Value ordering is not SQLite's #

Sorting, grouping and equality all use the query language's semantics, never the storage engine's dynamic-type rules (PSPU §3.20, §3.21).

The divergences are not edge cases. SQLite compares an integer and a real by converting; the query language compares them mathematically and exactly, including beyond binary64's exact integer range. SQLite orders text by byte; the query language folds ASCII case and then falls back to the original bytes only to break a fold-equal tie. SQLite has its own type-affinity ordering across storage classes; the query language fixes its own type order, nulls first, arrays last.

Where a SQL construct cannot reproduce those semantics, eventd uses it only to narrow candidates and applies the real comparison after loading the row (§6.3).

6.2.4 Canonical representatives #

A group whose members are equal under the language's rules but not byte-identical emits the smallest member rather than the first (PSPU §3.21).

Smallest rather than first is what makes the representative a property of the set. An event query merges results from every shard in an order that depends on which shard answered first, so "first" would be nondeterministic across runs of the identical query — which is exactly the property this chapter exists to prevent.

Events and logs are translated to SQL. Metrics are translated to SQL against series and samples. Clients never see any of it — the translation is entirely internal and carries no guarantees.

6.3.1 What translates directly #

Event header fields are columns, so a predicate on one can become a SQL WHERE comparison over an indexable column (§3.1). Today one predicate does per shard: the first of event.type == a string, which becomes a comparison of the event_type column, or a comparison of event.cpu or emitter.class with an integer, which compares cpu_id or origin_class, falling back to an indexed payload field (below). Other header predicates, emitter.process.guid among them, are applied after loading. A column's own name in a predicate is a payload field, not the column.

Timestamps bound every read already, through SINCE and UNTIL. A top-level comparison of the record's time with an integer — event.time for events, timestamp for logs — narrows that range further, so a page of older records — WHERE event.time <= T — starts reading at T rather than at the newest record. One inside an OR does not narrow, since it need not hold.

Log fields are all columns; log mode has no payload and its field set is closed (§4.2).

Metric selection resolves names and labels through series — from the in-memory cache where possible — and reads samples for the range, ordered by the composite index that already provides (timestamp, id) (§5.2).

6.3.2 What does not #

Event payload predicates have no column. They become eventd-internal payload extraction predicates, and may use an adaptive payload expression index to narrow candidates (§3.4).

HAS, array containment, narrows nothing. The payload expression index stores one key per field, and a field holding an array of group SIDs has no single key to store, so a HAS predicate is answered by decoding each candidate row and testing its array in full. The operator is correct on every row and costs a scan of whatever the rest of the query left; a query using it wants a time range or another predicate beside it.

The rule governing every such translation is that SQL narrows, the query language decides. Where a SQL construct cannot reproduce a predicate's comparison semantics exactly, eventd uses it only to reduce the candidate set and then applies the real predicate after loading the row.

SQLite's native dynamic-type equality and ordering never substitute for the query language's ASCII case folding, exact numeric comparison, binary comparison, array comparison, or null and missing-field handling (§6.2). An index that returns a smaller set faster is useful; one that returns a different set is a wrong answer arriving quickly.

6.3.3 Where access control sits #

Access filtering is part of the logical execution, not a filter over the output (PSPU §3.18, §3.28). eventd may push authorization predicates down into SQL when the concrete identifier set is known at planning time, or read candidate rows and discard them before aggregating.

Which it does is a performance decision. What is fixed is that the externally visible result is identical to the one filtering-first would produce — aggregates, ordering and pagination included, since all three would otherwise leak the existence of rows the caller cannot read.

6.3.4 Aggregation #

Aggregation is never pushed into SQL. SQL reads the candidate rows, and eventd folds them into groups as they are read (§6.4), under the language's own equality (§6.2) and after access filtering has discarded what the caller cannot read — grouping over columns included, not only grouping over payload paths whose comparison semantics SQL cannot reproduce.

For an event query this means every shard yields its matching rows, so the work is proportional to the row count, while what the aggregation holds is proportional to the group cardinality (§6.4).

Event queries execute against every database in the event store directory — active shards and historical ones alike (§3.3). Log and metric queries touch one database each and need none of this.

Shards carry no meaning for the query path (§2.3). A shard holds whatever CPUs routed to it during whatever lifetimes wrote it, so there is no shard a query can skip on the basis of its contents, and a predicate on event.cpu scans all of them.

6.4.1 Merging #

How results combine depends on the query.

Queries in the default order. A non-aggregating query with no SORT reads every shard at once, each newest first through idx_events_timestamp, and the coordinator takes whichever shard's next row comes first under the tiebreakers of §6.2: an N-way merge of the sorted streams. Each row that passes access control and every predicate is sent as soon as it is taken. Once SKIP + TAKE rows have passed, the merge stops. What the query holds is one row per shard, the merge frontier, whatever the size of its result.

Sorted queries. A SORT on any other field reads the shards in turn and gathers the visible rows. With TAKE it keeps only the best SKIP + TAKE of them, trimming to that whenever it holds twice as many or a thousand, whichever is more. Without TAKE it holds every visible row. Either way it holds them within MaxQueryHeldBytes (§6.5).

Aggregating queries. Rows from every shard fold into one set of groups as they are read, and no row is kept. Each group keeps:

GROUP … SUM, AVG, MIN and MAX groups are sorted by their keys ascending, in the order the fields are named, before SKIP and TAKE apply. An explicit SORT overrides that order, with group-key order breaking its ties (PSPU §3.21).

AVG is the one that cannot be composed from its own output: averaging partial averages weights each part equally, regardless of how many rows it held, so the sum and the count are kept and divided once.

A group is found by hashing a key that every pair of language-equal values shares — numbers by their nearest binary64, text with ASCII case folded — and then by the language's own equality against each group's representative in the order the groups appeared (§6.2). The hash only narrows the search; equality decides.

What an aggregation holds is bounded by the number of its groups rather than by its rows, within MaxQueryHeldBytes (§6.5).

6.4.2 The unbounded case #

A non-aggregating query without TAKE has no implicit row limit. EVENTS SINCE 7d ago may match millions of rows, all of which pass through the merge.

In the default order that costs time and no memory, so the query timeout is the only backstop (§6.5). The timeout covers sending as well as reading, so a client that reads slowly can run out of time on a result the merge would have finished.

6.4.3 Descriptors #

An event query opens a read-only connection per database, and each SQLite connection holds one or two descriptors for the database and its write-ahead log. With many historical shards this adds up quickly across concurrent queries.

Active shard writer connections stay open for the process lifetime and are not negotiable. Historical shard read connections are the pool worth bounding — opened when a query touches them, closed after a period of inactivity (§C).

6.5.1 Recording what was asked #

Every event query is recorded by the adaptive indexing system (§3.4). For each WHERE predicate:

• a header field reference increments the frequency counter of that field's column, kept under the field's path (event.type, emitter.process.guid); event.time, whose column is always indexed, is not counted
• a payload field reference increments that path's counter

Cross-type WHERE predicates are counted like any other, since they narrow the same data by the same fields.

This applies to event queries only. The log and metric stores have fixed write-time indexes and no candidate space for a policy to explore (§4.2, §5.2), so their queries increment nothing.

Counters are in-memory and are flushed to the metadata database at each policy interval (§3.5). Query handlers never write to that database directly, which is what keeps the once-per-query update off any lock a writer thread contends for.

6.5.2 Concurrency #

eventd bounds concurrent queries — streaming and non-streaming together — at MaxConcurrentQueries (§A). Beyond it a query is rejected with an error rather than queued.

The per-query cost that limit is protecting is real: read-only SQLite connections, one per shard for an event query (§6.4), memory for the merge, and CPU for execution and payload decoding.

MaxStreamingQueries is enforced separately and is lower. A streaming query holds its resources for as long as its client stays connected, where an ordinary one holds them for at most a timeout, so the two populations need different bounds.

Both are global. Every authenticated caller may connect (§7.1), so eventd also bounds the queries one caller may have running at once, at MaxQueriesPerUser. The caller is the user SID of the token read at connect. The slot is taken as soon as the token is read, before the request, so connections held open without a query count too. SYSTEM's queries are not counted: SYSTEM is the machine, not one caller among others. Over the limit, the query is rejected with an error, as over the global ones.

The per-user limit counts users, not processes or sessions: two windows of the same person share one budget, and two people never share one.

Queries and ingestion are separate channels, so exhausting the query side cannot exhaust ingestion (PSPU §3.3).

6.5.3 Memory #

eventd may not be killed: it runs with oom_score_adj −1000, so if its memory alone fills the machine, the kernel has nothing left to reclaim and panics. What bounds a query's memory therefore has to hold however many queries run at once.

A query in the default order holds one row per shard and nothing more (§6.4), and is never refused for the size of its result (PSPU §3.16).

Everything else a query must hold — a sorted query's rows, an aggregation's groups, a watch batch — counts against MaxQueryHeldBytes (§A), one budget for every running query together. A query takes from it in 64 KiB granules as it grows, gives back what it no longer needs when a sorted query trims, and gives back all of it when it ends. A query that would take more than is left fails with an error, rather than being truncated or answered from part of the data.

What a row or group costs is estimated from its fields and values, not measured from the allocator, so the budget is approximate. It is still a bound: with the default of 256 MiB, no number of concurrent queries holds much more than that.

A metric query reads each series' samples in order and holds none of them: a transform keeps only the sample before, and an aggregation folds each sample into its window, or its series, as it is read. What it holds is what its result needs, and that counts against the same budget: the matched series, one fold per window or per series, the latest point of each series, and the result rows. A query whose result is the samples themselves — a range without an aggregation — holds them all, and is refused when they pass the budget like a sorted query. A window aggregation over the same range holds one fold per window, however many samples fall in it.

6.5.4 Request and response sizes #

eventd rejects an inbound frame above MaxQueryRequestBytes before allocating or reading its payload. Outbound result records are grouped at record boundaries toward QueryResponseTargetBytes (§A). The target is not a limit: a complete record that exceeds it is sent alone, never split, truncated, skipped or failed merely for exceeding the target (PSPU §3.15–§3.16). Only the protocol's u32 frame length is a hard outbound bound.

6.5.5 Timeouts #

Every query has a maximum execution time, QueryTimeoutMs (§A).

The clock starts once the request has been decoded and the caller's token obtained, and covers everything after: parsing, planning, access checks, cross-type pre-computation, SQL execution, merging, aggregation, pagination, projection, and transmitting the initial result set.

It bounds the initial result set only. A non-streaming query sends end before it expires; a streaming query sends watch. Past watch the stream is not time-limited, and what bounds it instead is MaxStreamingQueries, MaxDistinctStreamValues and backpressure (§6.6).

On expiry eventd cancels the query and sends an error. Any result messages already sent are discarded by the client, since no terminal message arrived (PSPU §3.16).

Cancellation has to reach two kinds of work. SQLite work is interrupted through sqlite3_interrupt or an equivalent progress-handler check — the same responsiveness problem as cancelling an index build (§3.4). Non-SQL work — MessagePack flattening, the cross-shard merge — checks the same deadline periodically, because a query can spend most of its time in neither the database nor the kernel.

Large scans over unindexed fields are the main timeout risk, and the adaptive indexing system reduces it over time by indexing whatever keeps being filtered on — which is also why a timeout is a signal worth watching rather than merely an error to retry.

The externally visible behaviour of a streaming query — what may be streamed, what applies during the watch phase, how DISTINCT streams behave, and when a slow client is dropped — is PSPU §3.27. This is the machinery underneath.

6.6.1 Commit generations #

eventd keeps a monotonic u64 commit generation counter for each streamable store: one for the event store as a whole, and one for the log store. Metric queries do not stream, so the metric store has none.

After a writer commits a batch it increments the counter for its store and wakes the streaming handlers waiting on it. A handler records the last generation it processed and waits until the counter exceeds it.

The event counter covers the whole store rather than one per shard. Several writer threads increment it, so a wake is "something committed somewhere" and a handler re-examines every shard it cares about — which is what it would have to do anyway, since a shard means nothing to the query path (§6.4).

The counter is process-local and never persisted; it has no meaning across a restart, and a streaming query does not survive one. On wraparound the next increment is treated as a wake for every handler and operation continues. At any commit rate a machine can sustain, wraparound of a 64-bit counter is not reachable.

6.6.2 Record order within a watch batch #

The initial result set uses the query's usual sorting and pagination. During the watch phase, newly visible event records are delivered by shard ordinal ascending, then database record ID ascending; log records are delivered by database record ID ascending. SORT, SKIP and TAKE do not reorder or trim these watch records.

DISTINCT watches consider values in this same order. The first unseen value is emitted using its first record's representation; equivalent later values are suppressed by the stream's seen set. The initial distinct result retains its usual value ordering.

Record IDs preserve insertion order within each store. Event shards commit independently, so there is no durable total chronological commit order across shards. Shard ordinal supplies a deterministic order within each observed batch; successive batches follow the stream cursor. A batch may include several commits observed together. Producer timestamps do not change delivery order, even when they arrive out of order.

6.6.3 Latency #

Delivery latency is bounded below by the commit interval of the store concerned, because a record is not streamable until it is committed.

Under light load the actual latency is lower, because the adaptive batcher commits as soon as its input drains rather than waiting out the cap (§2.4). Under sustained load it converges on the cap.

A consumer needing better than this is not served by eventd at all: the KMES ring buffers are the low-latency path, they are specified in PSPK, and attaching to them directly costs the per-event access control that eventd exists to apply (§7).

6.6.4 The DISTINCT seen set #

A DISTINCT stream holds a per-query set of the values it has already emitted, initialised from the initial result set and added to as new values appear (PSPU §3.27).

It is bounded by MaxDistinctStreamValues (§A), and exceeding the bound terminates the query with an error rather than evicting. Eviction would make the output wrong rather than merely truncated: a forgotten value would be re-emitted as newly seen, and "newly seen" is the entire meaning of the result.

The set is per query and in memory, which is why the bound exists and why it is separate from the general query concurrency limit — sixty-four streams each holding a hundred thousand values is a different memory profile from sixty-four ordinary queries.

6.6.5 Cross-type re-evaluation #

Pre-computed cross-type ranges describe the past and are discarded when the watch phase begins (PSPU §3.27).

A metric condition costs one index seek per batch: the selector has already been constrained to exactly one series, so finding the active sample at the batch's latest candidate timestamp is a single lookup on idx_samples_series_timestamp (§5.2).

An existence condition is evaluated per candidate record rather than per batch, because the centred window is relative to each record's own timestamp and a matching record may be near some of a batch and not the rest.

The per-batch metric evaluation is an approximation, and the reason it is acceptable is the ratio between the two intervals: a commit batch spans a fraction of a second and a metric sample fifteen, so every record in a batch normally maps to the same sample. At sub-second metric resolution it filters more coarsely, and records near a threshold crossing are included or excluded as a group.

6.6.6 Backpressure #

Backpressure is detected on the socket send buffer. When a result message cannot be sent because the buffer is full, the query is terminated immediately; eventd never blocks on the send.

Blocking would put a slow reader in the path of eventd's own work, and the write path is what would suffer. A streaming client is the lowest-priority consumer of eventd's time, and dropping it is the same principle as dropping an ingestion datagram (PSPU §3.4), applied on the way out.

eventd enforces access on the read path only, using KACS Security Descriptors and the KACS AccessCheck API. Every query is evaluated against descriptors that determine which records — and which fields within a record — the caller may see, and everything else is filtered out silently (PSPU §3.28).

7.1.1 eventd decides nothing itself #

eventd implements no access check logic of its own. Every decision is delegated to kacs_access_check and kacs_access_check_list, which run the full KACS AccessCheck pipeline: integrity checks, restricted token evaluation, confinement, conditional ACE evaluation, and the SACL audit walk.

What eventd contributes is the three things AccessCheck needs and cannot know — which descriptor applies (§7.2), which object type list describes the record (§7.3), and what the caller's token is (§7.4) — and then it acts on the verdicts.

The alternative would be reimplementing an access check algorithm that already exists, in a daemon that would then have to be kept in agreement with it forever.

7.1.2 Enforcement is at query time #

All events, logs and metrics are stored regardless of who will ever be allowed to read them, and two callers querying the same store see different results.

Three reasons make this the only workable arrangement:

• Audit integrity. An audit event has to be stored whether or not anyone can currently read it. Filtering at storage time would let a descriptor decide what gets recorded, which is the one thing an audit store must not permit.
• Descriptors change. An administrator can grant or revoke access retroactively, and only query-time evaluation makes that meaningful.
• Callers differ. Several principals with different access levels query the same store, and there is one copy of the data.

7.1.3 Who may connect #

The query socket lets every authenticated caller connect: its descriptor is D:P(A;;GA;;;SY)(A;;GA;;;BA)(A;;GA;;;OW)(A;;FW;;;AU), set in place of the one it would inherit from eventd's closed runtime directory. What a caller may read is decided by eventd, per identifier and per field, against the descriptors below, not by the socket. A socket that admitted fewer callers would only leave a grant to anyone else unusable. How many queries one caller may hold is bounded per user (§6.5).

7.1.4 Caller identity #

When a client connects to the query socket, eventd obtains its token by reading the peer-token socket option (getsockopt(SOL_KACS, KACS_SO_PEER_TOKEN)) on the connected descriptor. The token represents the peer's identity as captured at connection time.

If the call fails, eventd denies the query entirely. It has no fallback identification and no anonymous mode. A client that connects at the Anonymous impersonation level conveys the Anonymous token (S-1-5-7), and eventd answers it with an error and closes the connection, whatever a descriptor would grant Everyone.

The snapshot property matters most for streaming queries, which may run indefinitely: a client whose group memberships change, or whose access is revoked, continues to be evaluated against the token it connected with until it disconnects (§7.5).

7.1.5 Rights #

The generic mapping passed to AccessCheck is in §B.

EVENTD_ADMINISTER is distinct from EVENTD_READ deliberately. Accelerating a field costs write throughput on every record thereafter, so INDEX is a way for a caller to degrade the system for everybody, and a caller permitted only to read has not been permitted to do that (PSPU §3.23).

7.1.6 What is not controlled here #

Event emission is KMES's: kmes_emit and kmes_emit_batch require SeAuditPrivilege, and eventd is not involved.

Log and metric ingestion has no per-record control at all. The Security Descriptor on each ingestion socket is the entirety of it (§7.6).

Access is defined on named patterns, each standing for a category of observability data and each carrying a descriptor. The three data types have independent pattern namespaces:

7.2.1 Matching #

A pattern matches by dot-delimited prefix. The pattern kacs matches the exact string kacs and any string beginning kacs. — and matches neither kacs_extended nor kacsfoo, because the dot is the hierarchy separator and not a mere character.

* is the wildcard default and matches everything.

Ingestion constrains metric names to the identifier grammar and origins to that grammar plus a producer component (PSPU §3.7, §3.10), which is what keeps a producer from choosing a name containing the wildcard or a backslash — a name that would otherwise match a rule its producer was never meant to satisfy, or store its descriptor somewhere other than where the administrator who wrote it believes.

An origin naming a producer within a service — jellyfin/HealthCheck, jobs/<guid> — resolves from the part before the slash, so the matching above starts at jellyfin and at jobs. A service's hooks, reload commands and health checks therefore answer to the descriptor written against the service itself, and every submitted job's output answers to one descriptor at Logs\jobs. The slash never enters a descriptor path, which is what keeps it out of the registry namespace it would otherwise be a separator in.

7.2.2 Resolution #

For a concrete identifier, eventd resolves the applicable descriptor by walking up the hierarchy:

1. Look for an exact match on the full identifier, kacs.audit.access.checked.
2. Remove the last dot-separated component and look again, kacs.audit.access.
3. Repeat: kacs.audit, then kacs.
4. Fall back to the wildcard, *.

The first match wins; a more specific pattern overrides a less specific one.

7.2.3 Storage #

Descriptors are registry values under the eventd security subtree:

Machine\System\eventd\Security\Events\*
Machine\System\eventd\Security\Events\kacs
Machine\System\eventd\Security\Events\kacs.audit.access.checked
Machine\System\eventd\Security\Logs\*
Machine\System\eventd\Security\Logs\loregd
Machine\System\eventd\Security\Metrics\*
Machine\System\eventd\Security\Metrics\cpu
Machine\System\eventd\Security\Admin

Each type's wildcard default is load-bearing. If a default is missing, eventd denies access to all data of that type: resolution that reaches the end of the hierarchy without a match is a denial, not a grant (PSPU §3.28).

Storing them in the registry rather than in eventd's own databases means the registry's access control protects them, and an administrator edits them with the ordinary registry tools rather than through eventd.

7.2.3.1 Who may change them #

A descriptor is stored whole, as one value, SACL included. Whoever may write that value may therefore replace the SACL as well as the DACL, and removing a SACL this way needs no SeSecurityPrivilege: the registry checks the right to set the value, not the right to change audit policy. Write access to the Security subtree is therefore audit-policy authority, and only those trusted to change audit policy should hold it. A tool that edits a descriptor's DACL must write the SACL back unchanged.

eventd gives Machine\System\eventd\Security a protected DACL: SYSTEM and Administrators full control, Authenticated Users read, which is what eventd and its clients need to resolve descriptors. Protection stops a broader grant on Machine\System\eventd, such as one made to let someone tune eventd, from reaching the audit policy. eventd sets it only while the key's DACL is still wholly inherited, so an administrator's own choice there is never overwritten.

7.2.4 Defaults on first boot #

eventd creates the three wildcard keys, the descriptor for its own health metrics and the administrative descriptor if they do not exist:

Every one of the five also carries a SACL with one failure-audit ACE for Everyone, covering EVENTD_READ, EVENTD_ADMINISTER and EVENTD_PUBLISH, and nothing else: a check under a default descriptor that is denied any of them is audited, whoever made it, and a granted one is not (§7.4).

The asymmetry reflects sensitivity. Events include security audit data and are restricted to administrators; logs and metrics are operational data and are readable by any authenticated user. Publication is fail-closed for ordinary services until a more-specific pattern grants their service SID EVENTD_PUBLISH (§7.6). An administrator can tighten either side independently.

On upgrade, eventd replaces a stored descriptor only when its binary form exactly matches a default an earlier eventd shipped at that key: the Metrics wildcard's former read-only default, and each key's former default without a SACL. Those are replaced by the current default, SACL and all. A value an administrator changed is never treated as a default and never rewritten.

7.2.5 Conditional ACEs #

Descriptors on eventd security objects may carry conditional ACEs, and KACS evaluates them as it would anywhere.

eventd passes no eventd-specific local claims to AccessCheck: local_claims_ptr is null and local_claims_len is zero. Conditions referencing token claims that KACS itself supplies still evaluate; conditions referencing eventd-local claims observe them as absent.

A stable set of eventd-local claims — the event's type, the log's origin, the time of day — is a plausible later addition, and defining one is a commitment to keep those claim names meaningful thereafter.

7.2.6 The administrative descriptor #

INDEX (PSPU §3.23) is checked against Machine\System\eventd\Security\Admin, with EVENTD_ADMINISTER as the desired access. Its default grants SYSTEM and Administrators.

eventd refuses INDEX outright if no administrative descriptor exists, by the same fail-closed rule as the read path. Keeping it in the registry means the registry's own descriptor protects the policy and a recreated eventd-meta.db cannot reset it (§3.5).

A descriptor can grant read access to some fields of a record and not others, using KACS object ACEs and object type lists. A caller authorized for a pattern but not for a field receives the records with that field absent — indistinguishable from a record that never carried it (PSPU §3.28).

7.3.1 Object type lists #

For each access check eventd builds an object type list: a tree with the data type's root at level 0 and, beneath it, the fields' dotted paths, one node per segment. subject is at level 1, subject.token at level 2, subject.token.sid at level 3. A field's node is the one at the end of its path, and every node above it is a prefix of the path, ending before a ., named as a field would be (below): the node for subject.token has the GUID of the name subject.token. A prefix that several fields share is one node, so the list names no GUID twice, and the list is in preorder, each node's subtree directly after it, as KACS requires.

Level 0: root GUID for the data type
  Level 1: event
    Level 2: event.time
    Level 2: event.type
    Level 2: event.cpu
  Level 1: emitter
    Level 2: emitter.class
    Level 2: emitter.token
      Level 3: emitter.token.guid
    Level 2: emitter.true-token
      Level 3: emitter.true-token.guid
    Level 2: emitter.process
      Level 3: emitter.process.guid
  Level 1: access
    Level 2: access.granted
  Level 1: subject
    Level 2: subject.token
      Level 3: subject.token.sid
  Level 1: source
    Level 2: source.name

The list is as deep as the deepest path, with no level limit. MS-DTYP's object type lists stop at level 4; KACS sets no depth limit, only one on the number of nodes, 1024 to a list (KACS_ACCESS_CHECK_MAX_OBJECT_TYPE_COUNT). Every prefix counts toward it. A record whose list would be longer cannot be checked, and a query reaching one fails rather than showing it.

A name with no . — a log field, a fixed metric field, most metric labels — is a node at level 1. A metric label key may contain ., and is then a path like any other: a label disk.read is beneath the node disk.

kacs_access_check_list returns a verdict per node, and eventd uses the verdict on each field's node to include or exclude that field. The verdicts on the prefix nodes are not used.

The three root GUIDs — one for events, one for logs, one for metrics — are in §B.

7.3.2 Field GUIDs are derived, not registered #

A field's GUID is computed deterministically with UUID v5 (RFC 4122):

field_guid = uuid_v5(EVENTD_FIELD_NAMESPACE, field_name)

with the namespace UUID in §B, and field_name the field's query-language name as UTF-8.

There is no registry of field GUIDs and no allocation step. The same name always yields the same GUID, so an administrator writing a descriptor computes the GUID from the field name with the same algorithm that eventd will use when it builds the list. This is the one part of eventd's access control that a third party reproduces rather than merely consumes.

Derivation rather than registration is what makes payload fields tractable at all: event payload schemas belong to the emitting subsystems, eventd has no catalogue of them, and a new event type carrying a new field needs no registration anywhere before a descriptor can name it.

Which names are used:

Payload fields suppressed by flattening or by a header collision — a value or map at a header field's path — are not query-language fields, so they get no GUID (PSPU §3.22). A GUID naming a header field's path is the header field's, whatever the payload holds there.

A descriptor written before header fields were named by path, granting timestamp or cpu_id by GUID, now grants the payload field of that name rather than the header field, because the GUID is the name's. Such a grant must be rewritten with the path to keep its meaning. Metric label keys cannot collide with the fixed metric fields, because ingestion rejects records whose labels do.

7.3.3 The GUID does not encode scope #

A field GUID names a field and nothing else. access.granted produces the same GUID whatever event type carries it.

Scoping comes from the descriptor hierarchy: an object ACE naming the access.granted GUID inside the descriptor for pattern kacs means "the access.granted field of KACS events". The same ACE in a different pattern's descriptor means the same field of that pattern's records.

7.3.4 Granting a subtree #

An object ACE applies to its node and to every node beneath it, so an ACE naming a prefix applies to every field whose path it begins: subject covers subject.token.sid, subject.process.pid and any subject field an emitter adds later. The header fields are paths like the rest: event covers every event.* header field, and emitter every emitter.* header field together with the payload fields beside them, such as emitter.process.pid.

KACS decides each node once, first ACE first. A field may therefore be read when, of the ACEs that match the caller and name its GUID, the GUID of any prefix of its path, the data type's root, or no object type at all, the first in DACL order to decide EVENTD_READ allows it. Neither the more specific nor the more general ACE wins as such:

A canonically ordered DACL (PCDS §5.5) puts every deny before every allow, so there a deny anywhere on a field's path hides it.

A deny on a field does not reach its siblings: denying subject.token.sid leaves subject.token.auth-id and subject.process.pid as the rest of the DACL decides them, and denying subject.token hides every token field and leaves subject.process. KACS does carry a deny up to the node's ancestors, and a grant up to a node all of whose children are granted, but only onto the prefix nodes and the root, never onto another field.

A field can also be a prefix of another in the same record, since a payload is opaque: a payload value at emitter sits beside the header's emitter.class. Its node in the record's list then has nodes beneath it, whose grants and denies KACS carries up to it. eventd therefore checks such a field again in a list of its own, holding only its prefixes and itself, and uses that verdict, so that the same first-ACE rule decides it as decides any other field.

A descriptor written before the list was a tree keeps its meaning for every field it names by full path: that field's GUID, and so its node, is unchanged. What is new is that an ACE naming a prefix, which named a node in no list before, now covers the fields beneath it.

7.3.5 Writing one #

An object ACE with no object type GUID applies to the root and therefore to every field. One with a field GUID applies to that field.

To grant a security team full read access to KACS events, and a monitoring team only the time, type and CPU:

• Allow SecurityAdmins, EVENTD_READ, no object GUID
• Allow MonitoringTeam, EVENTD_READ, object GUID = event.time
• Allow MonitoringTeam, EVENTD_READ, object GUID = event.type
• Allow MonitoringTeam, EVENTD_READ, object GUID = event.cpu

MonitoringTeam querying KACS events receives records containing exactly those three keys. Payload fields, identity GUIDs and the remaining header fields are absent.

To let an auditing team see who acted but not the token's groups:

• Deny AuditTeam, EVENTD_READ, object GUID = subject.token.groups
• Allow AuditTeam, EVENTD_READ, object GUID = event
• Allow AuditTeam, EVENTD_READ, object GUID = subject

AuditTeam receives every event.* header field and every subject field except subject.token.groups.

7.3.6 Building the list per record #

The list is constructed from the fields actually present in the record being checked: the root node, then the tree of their paths.

Event records contribute every header field plus every non-suppressed flattened payload field present in that particular payload. Different event types produce different lists, because they carry different payloads — and two events of the same type can too, since a payload is opaque MessagePack and nothing requires two of a type to agree.

Log records have a fixed field set — timestamp, origin, is_error, message, job_id, boot_id — so the list is the same for every log record.

Metric records contribute the five fixed fields plus the series' label keys, which vary per series.

Derived aggregate outputs — count, sum, avg, min, max — are omitted from the list entirely. They are not source fields, have no GUID and no access identity of their own, and are visible when the caller is authorized for the records and source fields they were computed from (PSPU §3.28).

7.3.7 Metric aggregate headers #

A metric aggregate or window result does not restore timestamp, name or type if source-field authorization removed that header. The private time used for window placement and ordering is independent of the returned record, so hiding timestamp does not change the numeric result or move samples into another window.

When an output combines several contributing series or metric identifiers, each returned header must be visible in every contributing source. A first contributor with broader access cannot restore a header denied by another. Per-series output retains each series' own visibility. This also applies to latest-value aggregates, transformed windows, and cached rollup results: a cached value does not bypass the current source-field check. Explicit references to a denied field still filter the source records as specified by PSPU §3.28.

The order in which eventd evaluates a query is fixed (PSPU §3.18), and access control is the third phase — before predicates, transforms, grouping, aggregation, ordering and pagination. What follows is the sequence within that phase.

1. Obtain the caller's token from the connection (§7.1). Failure denies the query.
2. Parse the query to establish its data sources and filters (§6.1).
3. Discover the concrete identifiers the query could touch — event type strings, log origin strings, metric name strings. A broad selector authorizes nothing by itself: EVENTS, EVENTS kacs.*, LOGS without FROM and METRIC cpu.* are each resolved identifier by identifier.
4. Resolve and check each discovered identifier: find its descriptor by hierarchical matching (§7.2), and build the object type list for the fields the query references (§7.3). A query that references none asks instead about every field the descriptor's allowing object ACEs name, since those may make its records visible though the query names none of them; a GUID does not say its path, so each is a level-1 node of that list. Call kacs_access_check_list, and cache the verdict for this (token, identifier, field set) (§7.5).
5. Apply record verdicts. An identifier is invisible when the caller may read neither its root nor any field: its records are excluded from the logical row set before aggregation, ordering, pagination and formatting. One whose root is denied but some field granted stays, for its records to be shaped to those fields (step 10). For a cross-type source, a denied identifier is treated as having no matching data.
6. Apply field verdicts to predicates. Where the query references a field in a predicate or shaping clause and a matching identifier does not grant it, that identifier's records contribute nothing — exactly as if its root had been denied. The query is not rejected.
7. Cross-type sources get the same treatment. An invisible identifier, or a denied field needed to evaluate the condition, makes the condition evaluate as though no matching cross-source data existed.
8. Execute, with root filtering already part of the logical row set.
9. Re-resolve per result identifier. For each distinct concrete identifier in the resulting rows, resolve its descriptor, build the object type list with field GUIDs, call kacs_access_check_list with the token, the descriptor, EVENTD_READ, the list and an audit context naming the pattern the descriptor was resolved from (below), and cache the per-field results.
10. Shape each record. Look up the cached results for its identifier and fields. Exclude the record entirely if neither the root nor any of its fields was granted; otherwise include it, with each field only if its node was granted. A grant of the root alone grants every field, since an ACE with no object type applies to the whole tree (§7.3). A grant of some fields alone gives records holding exactly those fields, and a record carrying none of them is excluded.

7.4.1 Which clauses count as referencing a field #

Step 6 applies to every clause that reads a value rather than merely displaying one:

• ordinary WHERE predicates
• metric label filters in a primary selector
• GROUP, COUNT BY, TOP N BY, SORT and DISTINCT fields
• event and log aggregation arguments — SUM, AVG, MIN, MAX
• metric transforms and terminal aggregations, all of which read the fixed value field: RATE, DELTA, P50, P95, P99, AVG, MIN, MAX, SUM, AVG_OVER, MIN_OVER, MAX_OVER, SUM_OVER
• a metric boot filter, which reads boot_id; and an explicit metric type predicate, grouping, sort or distinct, which reads type

SELECT is not on this list. It shapes output and is applied last, so a field it omits was still available to every earlier phase — and conversely, selecting a field the caller may not read removes the field, not the record.

7.4.2 Denial is silent, not fatal #

Step 6 excludes rather than rejects, and this is the decision most worth being explicit about, because rejecting is the more informative behaviour and that is exactly the objection to it.

A rejection would tell the caller that some identifier exists, matches its query, and carries a field it may not read — three facts about data it was not permitted to see, delivered by the mechanism meant to withhold them, and enumerable by trying queries and watching which ones fail. Excluding costs the caller a result narrower than it appears; rejecting costs the model the property it rests on.

Cross-source fields already worked this way (step 7); primary-source fields now match them.

7.4.3 Field authorization does not depend on presence #

A field's authorization is resolved from the name as written, against each concrete identifier, whether or not any record of that identifier actually carries it.

Payload fields vary between records of the same type, so a rule turning on presence would require the scan that authorization is meant to precede.

7.4.4 Internal values are not fields #

Row identifiers, series identifiers, ordering tiebreakers and series type checks are not query-language source fields unless the mode exposes them as fixed fields or the query names them (§6.2). Errors raised by internal checks never carry a denied field's value.

A metric result's value is a source field, because it is either a raw sample or a scalar derived from raw samples.

7.4.5 Filtering is part of the logical result #

Access filtering is not a presentation step. Aggregating, ordering or paginating over unreadable records would leak them through counts, through ordering, and through the gaps in pagination.

eventd may push authorization predicates into SQL when the identifier set is known, or read candidates and filter them before aggregating (§6.3). The externally visible result is identical to filtering first, and COUNT, COUNT BY, TOP N BY and every other aggregate reflect only what the caller may see.

7.4.6 The audit trail #

Whether an access check produces a KACS audit event is decided by the SACL audit walk in the AccessCheck pipeline, against the descriptor's SACL. Every default descriptor eventd creates carries a failure-audit ACE for Everyone at every eventd right (§7.2), so out of the box every denied check produces a kacs.audit.access.checked record, and a granted one produces none unless an administrator adds a success ACE. KACS writes such a record from the AccessCheck syscall only when the caller holds SeAuditPrivilege enabled. eventd's service definition requires that privilege, and its token holds it enabled; without it the check would be made and nothing recorded.

eventd passes each check an audit context naming the object it guards, in the form PGSS §6.7 defines: a MessagePack map of the object's kind and, under the kind's own name, the pattern whose descriptor was checked.

The pattern is the one the descriptor was found under (§7.2), not the identifier asked about, and for a log origin it never holds the producer after a /. KACS copies the map into its kacs.audit.access.checked record as object.kind and object.<kind>.pattern, and marks the record with fields.attestation.userspace, since those values are eventd's claim. The audit trail therefore records exactly which observability data was read, by whom — the client, as subject.* — rather than merely that eventd performed a check. Reading and publishing a metric name the same object; the record's access.requested says which was checked.

Those audit events are themselves KMES events, which eventd consumes and stores, and which are governed by the event patterns like any other, under kacs (§7.2). Reading the audit store is auditable.

An access check is a syscall through the full AccessCheck pipeline. A query returning ten thousand records cannot afford ten thousand of them, so results are cached — at two levels, plus the descriptor resolution underneath both.

7.5.1 Record-level #

Verdicts are cached for the life of one query — for a stream, the whole stream — under the connection's token (§7.1), and keyed by concrete identifier, not by pattern: two event types under one pattern are two keys. When a descriptor contains no object ACEs, the check is a plain grant or deny on the root, so every check of an identifier reaches the same verdict.

Each identifier costs one pre-check before any record is read, for the fields the query references (§7.4 step 4), and one result check for each distinct field set its records carry (§7.4 step 9). A query returning ten thousand events across twenty event types, each type's records of one shape, performs at most forty checks.

7.5.2 Field-level #

When the descriptor does contain object ACEs, the verdict depends on which fields the record carries, since different payloads produce different object type lists (§7.3). The result check is therefore cached per (identifier, field set), and is so whatever the descriptor holds.

In practice events of one type carry the same fields, so this is effectively one result check per event type. Log records have a fixed field set, so a log query reaches two checks per origin: the pre-check and one result check. Metric records vary by series label keys.

The pathological case is an event type whose payload fields differ from record to record, which produces a distinct field set — and a distinct cache entry, and a distinct syscall — for each shape encountered.

7.5.3 Descriptor resolution #

Resolving a pattern to a descriptor is itself cached, across queries rather than within one, since it costs a registry read and a hierarchy walk (§7.2).

eventd watches the security registry subtree and invalidates cached resolutions and cached check results when a descriptor changes. That is what makes a revocation take effect on the next query rather than at the next restart.

If the registry watch fails after startup, eventd discards the descriptor cache and operates fail-closed for new resolutions until the watch is re-established. A cache it cannot trust to be current is worse than none: continuing to serve from stale entries would make a revocation silently ineffective, and the failure would be invisible. With the cache discarded every resolution is new, so the descriptors resolved before the failure are not kept either. This is a degraded state, not a failure — eventd keeps ingesting and keeps answering queries, which see no records until the watch is re-established (§9.3).

7.5.4 During a stream #

Verdicts reached for a streaming query's initial result set are reused through the watch phase, with two exceptions.

A new concrete identifier appearing in a streamed batch — an event type or log origin not present in the initial results — is resolved and checked before the record or its distinct value is used. It has never been authorized, and inheriting a verdict from a sibling pattern would be a grant nobody made.

A descriptor change invalidates the cache as it does anywhere, and subsequent batches are re-checked against the new one.

The token is not re-examined. It was captured at connection (§7.1), so a client whose memberships change mid-stream continues under what it connected with, and a client whose access is revoked keeps receiving records until it disconnects. The bound on that exposure is the client's own connection lifetime, which for a dashboard may be days.

7.5.5 Metric publication #

The metric ingestion thread owns a separate bounded cache for EVENTD_PUBLISH checks (§7.6). It is keyed by the conveyed token's token_id and modified_id plus concrete metric name. It caches denials as well as grants, clears as a unit when full, and is invalidated by the same descriptor generation as the read caches.

This cache is deliberately thread-local. A recurring metric performs a borrowed string lookup without allocation, shared locking or AccessCheck; the only per-datagram identity operation left is querying the fresh token fd's stable statistics before that fd is closed.

The three write paths establish provenance differently because their inputs come from different places.

7.6.1 Events #

Emission is KMES's business. kmes_emit and kmes_emit_batch require SeAuditPrivilege, and eventd is not in the admission path — it consumes what KMES delivers (§2.2).

The identity stamps on an event are the kernel's, captured from kernel state at the write. An emitting process cannot set, influence or suppress them. That is what makes an event's emitter.process.guid evidence, and why a payload value at that path never shadows it (PSPU §3.22).

7.6.2 Logs use peinit as a broker #

Service processes do not reach the log socket. eventd replaces the inherited DACL on LogSocketPath with this protected DACL before its first receive:

D:P(D;;0x2;;;SU)(A;;GA;;;SY)(A;;GA;;;OW)

It sets the DACL alone. The socket stays owned by the service SID of eventd's own virtual Service identity, which created it, and the OW allow lets that owner manage it; making SYSTEM the owner would need a privilege the long-running daemon deliberately does not hold.

SU is the Service logon group, S-1-5-6. Every phase-2 service token, including a SYSTEM service token, carries it. peinit's bootstrap SYSTEM token does not. The deny is evaluated before the SYSTEM allow, so a service cannot write merely because its user SID is SYSTEM; peinit can.

peinit determines origin from the output pipe it is draining and may batch records from several origins in one datagram. eventd therefore does not request or check a KACS token for each log datagram. The socket admission attests the broker and the broker attests the origin, without token-fd churn or an AccessCheck on the log hot path.

A log origin proves which service context peinit associated with the line. It does not prove that the line's message is truthful.

7.6.3 Metrics carry the producer token #

Every authenticated caller may send to the metric socket: eventd replaces the descriptor it would inherit from its closed runtime directory with D:P(A;;GA;;;SY)(A;;GA;;;BA)(A;;GA;;;OW)(A;;FW;;;AU). Who may publish what is decided below, per metric name, against the token each datagram carries, so a socket that admitted fewer callers would only leave a grant of EVENTD_PUBLISH to anyone else unusable.

The cost of an open socket is that anyone signed in can send datagrams, and the socket is lossy (§9.1): a caller sending in bulk can crowd out other producers' samples, though none of its own unauthorized records is stored. An operator who wants the socket narrower sets another descriptor on it after eventd starts, for example with sd set. eventd sets its own again each time it starts.

A metric producer enables KACS_SO_PASS_TOKEN once on its persistent sending socket. KACS attaches the producer's effective identity to each datagram as KACS_SCM_TOKEN. eventd uses recvmsg with room for exactly one token and no ordinary file descriptors.

eventd discards the whole datagram before MessagePack parsing when:

• no token arrived
• the data was truncated
• the ancillary data was truncated
• querying the token or resolving policy failed

For a valid datagram, eventd resolves each record's metric name through the ordinary hierarchical Metrics descriptor namespace (§7.2) and runs AccessCheck against the conveyed token for EVENTD_PUBLISH. A denied record is discarded; authorized sibling records in the same datagram continue.

The wildcard Metrics descriptor grants EVENTD_PUBLISH to SYSTEM and Administrators, not Authenticated Users. A package that owns a metric prefix installs a more-specific descriptor granting its service SID that right. Existing wildcard descriptors that exactly match eventd's old read-only default are upgraded in place; administrator-modified descriptors are never rewritten.

Authorization covers the metric name, not its labels or value. An authorized producer can create arbitrarily many valid label sets under that name (§5.3).

7.6.4 The publication cache #

A full AccessCheck is not performed per sample or per datagram. The metric ingestion thread owns a bounded verdict cache keyed by:

(token_id, modified_id, concrete metric name)

KACS reuses the captured token object while a persistent sender socket keeps the same effective identity, so ordinary traffic repeatedly hits the same entry. A hit performs no allocation and no AccessCheck.

MetricAuthorizationCacheSize bounds the total entry count. When full, eventd clears the cache rather than maintaining an LRU list on the write path. This makes churn expensive for the producer causing it without adding pointer updates to every successful lookup.

The descriptor cache carries a generation. Any security-registry change advances it and clears all local publication verdicts before they are reused. The cache stores denials as well as grants, so repeatedly sending an unauthorized name does not repeatedly invoke KACS.

7.6.5 Rejected input #

Missing identity, truncation, denied records and policy errors increment in-memory diagnostic counters. Policy errors may produce rate-limited standard-error text. None produces a durable event or log record of eventd's own: doing work proportional to hostile input would create an amplification path (PSPU §3.4). The one record a denial can cause is KACS's: under a descriptor whose SACL audits failure, as every default does (§7.2), the publication check that denies a name writes a kacs.audit.access.checked record. Denials are cached like grants, so that is one record per token and name until the cache is cleared, not one per datagram.

eventd needs four subsystems before it can do anything.

eventd is a peinit-managed service, started after loregd is available — it cannot read a single configuration value without the registry, and it has no compiled-in defaults for the six paths it needs (§A).

The boot ID is not a service-manager API. eventd reads Linux's /proc/sys/kernel/random/boot_id, validates its UUID text, and converts it to PCDS GUID layout (§3.7).

8.1.1 The dependency that is not one #

KACS is needed to serve queries and not to ingest. The drain, write and retention paths never call it. That asymmetry is what lets eventd keep ingesting through a KACS outage while refusing every query (§9.3), and it is the right way round: losing the ability to read the audit store is recoverable, losing the events is not.

8.1.2 Ordering in the boot #

eventd is one of the platform daemons and is Critical: peinit reboots the system rather than continuing without it. It comes up after loregd and authd, and it stops before them on the way down — it is among the last services shut down, because everything else's shutdown is worth recording.

The window before eventd exists is real and peinit covers it by buffering service output until the log socket appears. Events emitted during that window are not lost either: they sit in the KMES ring buffers, and eventd's first drain after attaching reads them from tail_pos (§2.2).

Startup proceeds in seven phases, and either completes or fails entirely. The phases group the work by concern rather than fixing its order: eventd reads the boot ID (step 13) before it opens any store, so that each shard can be searched for it as it is opened; it opens eventd-meta.db (step 12) before the shards; and it binds each ingestion socket as soon as its store is open, so the log socket (step 18) is bound before metrics.db (step 11) is opened.

8.2.1 Phase 1 — Configuration #

1. Read every configuration key under Machine\System\eventd\. The six required keys are EventStorePath, LogStorePath, MetricStorePath, QuerySocketPath, LogSocketPath and MetricSocketPath; a missing or invalid one fails startup.
2. Open and validate the three provisioned store directories without following symbolic links. A missing directory, unsafe component or descriptor that is not equivalent to the protected platform default fails startup (§3.3).
3. Read the optional keys and apply compiled-in defaults for those absent (§A).
4. Arm a persistent watch on the subtree, for runtime changes (§8.3).

8.2.2 Phase 2 — KMES attachment and shard sizing #

5. Query the KMES logical slot count with kmes_attach(KMES_ATTACH_QUERY_SLOTS, …), walk every slot, skip EINVAL holes, and assign dense internal ordinals to successful logical CPU IDs (§2.2). The calls require SeSecurityPrivilege in the effective token. Discovering no buffers fails startup.
6. Map each per-CPU ring buffer.
7. Resolve the active shard count from StorageShards — the attached KMES-buffer count when it is 0, the configured value otherwise.
8. Compute the shard-to-CPU assignment (§2.3).

8.2.3 Phase 3 — Storage #

9. Open or create each active event shard: verify the schema version, open in WAL mode with synchronous=FULL, create tables and indexes if new, and quarantine on reported corruption (§3.3). Discover historical shards matching the naming pattern, and open those with a recognised schema read-only; exclude the rest from the query path.
10. Open or create logs.db in the log store directory — schema verified, WAL, synchronous=NORMAL, quarantine on corruption (§4.3).
11. Open or create metrics.db in the metric store directory, likewise (§5.4). The series cache starts empty and fills on demand.
12. Open or create eventd-meta.db. Load the index counters and desired set, load the sequence checkpoints for diagnostics only, and discover each shard's material indexes from its schema (§3.5).

8.2.4 Phase 4 — The boot boundary #

13. Read /proc/sys/kernel/random/boot_id, require valid UUID text, and convert it to PCDS GUID layout. An unavailable or malformed value fails startup.
14. Read and merge every committed receipt_ranges row for that boot from every readable active and historical shard, grouped by logical CPU ID (§2.2, §3.1).
15. Search those shards for any committed row or receipt carrying the boot ID. None means the boot's first successful eventd start; otherwise this is a restart within the boot (§3.7).
16. Initialise each drain's recovery coverage from the merged receipts. It begins at the ring's oldest survivor, skips covered sequences, re-ingests uncovered survivors, and records as gaps only sequences in neither source (§8.5).

8.2.5 Phase 5 — Sockets #

17. Create the query socket at QuerySocketPath. A stale pathname left by a crash is unlinked first if it is an AF_UNIX socket; if the path exists and is not a socket, startup fails. Its inherited descriptor is replaced with the protected one that lets every authenticated caller connect (§7.1).
18. Create the log socket at LogSocketPath, then replace its inherited DACL with the protected deny-Service, allow-SYSTEM broker DACL, leaving it owned by eventd's service SID (§7.6).
19. Create the metric socket at MetricSocketPath, then replace its inherited descriptor with the protected one that lets every authenticated caller send (§7.6).
20. Establish and verify the Security Descriptor on all three, before any of them accepts or receives anything (§7.6).

The stale-socket rule distinguishes the two cases deliberately. Unlinking a leftover socket is recovery from eventd's own crash; unlinking a regular file at a configured path would be destroying something that is not eventd's, and a path pointing at the wrong thing is a configuration error worth failing on.

Recognising a leftover socket needs no right on it. If an operator has narrowed its descriptor so that eventd may no longer delete it, eventd, which still owns it, first puts its own descriptor back, so a narrowed socket never stops the next start.

8.2.6 Phase 6 — Threads #

21. One writer thread per active shard.
22. One drain thread per attached logical CPU, each beginning to reconcile and read its ring buffer.
23. The log ingestion thread.
24. The metric ingestion thread.
25. The retention coordinator thread.
26. The adaptive indexing policy thread.

8.2.7 Phase 7 — Ready #

27. Write and commit an eventd.daemon.started event in the boot recording whether this start is a restart, the shard count and the per-CPU recovered coverage points (§3.2). The commit happens before readiness is signalled.
28. Signal readiness to peinit.

Committing before signalling is what makes the startup record trustworthy. A readiness signal sent before the commit could be followed by a crash that loses the record, leaving a boot in which eventd demonstrably ran and left no trace of having started.

8.2.8 Failure #

If any phase fails, eventd does not signal readiness. It logs the failure to standard error where standard error exists, and exits non-zero. peinit's restart policy decides what happens next.

Partial startup is not permitted. There is no degraded mode in which eventd runs without one of its three stores, or without KMES. It either completes the sequence or fails.

The all-or-nothing rule is a simplification. A later revision might allow log and metric ingestion to proceed with KMES unavailable, but that means partial-failure state to manage in every subsequent path — what a query against a store that was never opened does, what happens when the missing subsystem returns — and the failure mode it protects against is one peinit already handles by restarting.

eventd watches Machine\System\eventd\ and reacts to changes without restarting — for the settings that can be changed that way.

Notifications arriving during startup are queued and processed after readiness is signalled (§8.2). Applying a configuration reload to half-initialised state would mean every phase having to tolerate its inputs changing underneath it.

8.3.1 What applies immediately #

Tuning parameters: batch sizes and latencies for all three writers, retention periods and the delete batch size, adaptive index thresholds and windows, the WAL checkpoint threshold, the query request and response sizes, the query timeout, the cross-type window and lookback limit, and the query and streaming concurrency limits.

A live MaxBatchSize change is an atomic update to the event writers' next commit threshold. It does not resize, replace or drain their startup-fixed handoff channels (§2.3). The log and metric batch settings likewise affect only subsequent transaction thresholds.

Every applied change, whichever key it touches, also requests an index policy run (§3.4), which writes the metadata database (§3.5), and a retention pass (§3.6), so a changed threshold or a shortened retention period takes effect without waiting for either interval.

8.3.2 What waits for a restart #

The watch notices these changes and eventd defers them rather than attempting a live migration. Shard count changes in particular are expected once in a machine's life, and rebalancing writer threads while events are in flight is a large mechanism for a rare event.

8.3.3 What is neither #

Security Descriptors under Security\ are not configuration in this sense. The registry watch invalidates the descriptor cache and the next query resolves afresh (§7.5), so a grant or revocation takes effect immediately without anything being "applied".

8.3.4 Recording it #

eventd emits an eventd.config.changed event for every change applied at runtime, carrying the value's name and its old and new values as integers, each left out when no value was stored (§3.2).

Invalid values are ignored and the previous value is retained — an administrator who types a batch size outside its range does not get a daemon that stops working, and the retained value is the one already in use rather than the compiled-in default. Unknown keys in the subtree are ignored entirely.

8.3.5 SIGHUP #

SIGHUP re-reads the configuration, equivalent to a watch notification (§8.5). It exists for the case where the watch itself is not delivering — a registry outage, or a watch that failed and has not re-armed (§9.3) — and gives an administrator a way to force the read rather than restarting the daemon.

When peinit signals a stop, eventd persists as much in-flight data as it can without blocking indefinitely.

8.4.1 The sequence #

1. Stop accepting. Unlink all three socket paths so no new client can reach them, and stop accepting query connections. Existing streaming queries are terminated with an error. The log and metric socket descriptors stay open, and eventd goes on reading them until every query connection still being served has ended: a sender that reached either socket before its path was unlinked can still deliver to it.
2. Drain ingestion. Shut the log and metric sockets for reading, so that a later send fails with EPIPE and its sender keeps the datagram (peinit buffers it and replays it to the next eventd), rather than having it queued where nothing will read it. Then read and process the datagrams still in the receive queues, and close those descriptors. This is bounded by the receive queue, at most net.unix.max_dgram_qlen datagrams plus one (§4.1), so it completes quickly.
3. Final event drain. Each drain thread performs one last drain cycle from its ring buffer.
4. Final commit. Every writer commits its current batch immediately, whatever its size. The log and metric writers do the same.
5. Record sequence state. Derive each logical CPU's highest contiguously covered sequence from committed receipt ranges and write it to sequence_checkpoints for diagnostics (§3.5).
6. Emit the shutdown event. Write eventd.daemon.stopped with the per-CPU sequences, using the daemon-wide shard assignment rule (§2.6). If the receipt ranges could not be read in step 5, the record carries no sequences rather than zeroes (§3.2). If no shard is writable, the event is skipped and the failure logged to standard error.
7. Close databases. Close every connection, writer and reader. SQLite checkpoints the write-ahead log automatically on close.
8. Unmap. Unmap every ring buffer and close the per-CPU descriptors.
9. Exit.

Steps 1 and 2 are deliberately split. Unlinking the pathnames stops new senders finding the socket while the descriptors stay open, so whatever is already queued is still readable — closing them at step 1 would discard the queue, which is the data most recently produced and therefore most likely to explain why the system is being stopped. Shutting the sockets for reading before the drain is what makes the drain final: nothing can join the queue behind it to be discarded when the descriptors close.

The final checkpoint in step 7 matters most for the log and metric stores, which run synchronous=NORMAL and whose durability boundary is the checkpoint rather than the commit (§4.1).

8.4.2 A failed event writer #

When an event-shard writer exits, it closes its handoff queue and releases the pending control requests. A caller waiting for a final-commit barrier, synthetic-event commit or maintenance result then observes failure instead of waiting for a reply from a writer that has stopped. These requests are not acknowledged as committed. Producers waiting for queue capacity wake with a closed-queue error; already admitted handoffs are released once their producer publishes or cancels them. Healthy writers retain their normal backpressure and commit acknowledgements.

A reservation racing with closure either joins the admitted handoffs or fails with a closed-queue error. The final drained check cannot miss a reservation that succeeds, including on processors with weaker memory ordering. Closing also wakes threads that are entering a queue wait; it does not depend on another publication to release them.

8.4.3 The timeout #

Shutdown is bounded by peinit's service stop timeout. If the sequence has not finished, eventd aborts and exits immediately.

What an aborted shutdown costs:

• Uncommitted event batches are lost. Those events remain in the KMES ring buffers and are available at the next start, provided they have not been overwritten by then.
• Uncommitted log and metric batches are lost, which is acceptable by design.
• The diagnostic sequence metadata may be stale. It does not matter: startup derives coverage from committed receipt ranges and reconciles it with the ring buffer (§2.2).

Every consequence is one the restart path already handles, which is why aborting is safe rather than merely tolerable.

8.5.1 After a crash #

An ungraceful termination — a segmentation fault, a kill, an out-of-memory kill — leaves four things true.

The ring buffers are unaffected. KMES writes regardless of consumer state, and events emitted while eventd was down accumulate there.

The databases are consistent. WAL mode guarantees committed transactions survive, and SQLite rolls back the in-flight batch on the next open.

There may be uncovered sequences. Events in an uncommitted batch were not persisted, but some may still survive in KMES. On restart eventd merges committed receipt ranges from all shards, re-ingests each uncovered survivor, and writes a gap only for a sequence present in neither receipts nor the ring (§2.2, §2.5).

Socket-buffered data is gone. The kernel discards a socket receive queue on process exit, taking whatever logs and metrics were waiting. Acceptable by the loss model.

No manual recovery is needed and none is offered. eventd restarts, re-attaches, resumes draining, and records only what was truly missed. The boot-boundary logic recognises the restart from committed rows or receipts rather than a flag written in advance (§3.7) — which is the point, since a crash is precisely the case where nothing was written in advance.

8.5.2 Signals #

Every other signal keeps its default behaviour.

8.5.3 The diagnostic dump #

SIGQUIT writes human-readable text to standard error before step 1 of the shutdown sequence, so that it reflects the daemon's state while it is still running rather than while it is tearing down. It includes at least:

• the current boot ID
• the active shard count and the readable historical shard count
• the per-CPU committed receipt coverage and highest covered sequence
• the current non-streaming and streaming query counts
• the metric series cache occupancy
• KMES events discarded for claiming one of the five types eventd writes itself (§3.1)
• log records discarded for an origin outside the origin grammar, and the most recent such origin where one exists (§4.1)
• metric datagrams missing identity, truncated, denied by name policy or rejected by an authorization error
• the cumulative metric type-mismatch count and the latest conflicting metric name and expected and received types, where one exists
• the last observed write error for each store, where one exists

The format is not a stable machine interface and its wording may change.

Most of what an operator asks of a running eventd — whether the writers keep up, whether the series cache is thrashing (§5.3), whether query slots are exhausted (§6.5), whether a store has been failing writes — is answered over time by its health metrics (§5.7). The dump is for what those cannot carry: the counts of rejected ingestion input, which no query client may see (PSPU §3.4), the latest error text and rejected names, and the receipt coverage. It also still works when the metric store is the thing that is broken. Standard error is the destination because peinit captures it, so the dump reaches the log store by the ordinary path — and reaches standard error directly when the log store is broken.

Every other failure in this chapter costs something recoverable. This one does not, which is why the whole ingestion pipeline is shaped around delaying it.

9.1.1 Ring buffer overrun #

When events are emitted faster than eventd drains them, the per-CPU ring buffers fill and KMES overwrites its oldest entries.

• eventd sees it as a sequence gap on the affected CPU (§2.5).
• An eventd.events.lost record is written, naming the missing range.
• The range is added to that CPU's eventd.events.lost (§5.7).
• Draining resumes from the oldest survivor at tail_pos.

The events are gone. No other copy exists, and a gap record is a tombstone rather than a recovery — it records what was lost and when, which is the most that can be offered.

Four mechanisms delay it:

The first three are eventd's and operate automatically. The fourth is KMES's and is the one an operator can enlarge for a workload that bursts predictably.

9.1.2 Query timeouts #

A query exceeding QueryTimeoutMs is cancelled and the client receives an error (§6.5). Read-only connections are released; nothing is lost, and the query simply did not finish.

Streaming queries are bounded only up to watch; past that the watch phase is not time-limited.

The main risk is a large scan over a field with no index, which adaptive indexing reduces over time by indexing whatever keeps being filtered on (§3.4). A timeout is therefore worth reading as a signal about the index set rather than only as an error to retry.

9.1.3 Ingestion backpressure #

When a log or metric socket's receive queue is full, the kernel refuses the send: a non-blocking sender receives EAGAIN, and a blocking one waits (§4.1). eventd is not notified and does not count the refusal, so no health metric shows it (§5.7).

This is by design and is not a failure to be tuned away (PSPU §3.4). The one operational note is that the queue is not being drained while a batch is committing, because the same thread does both jobs (§4.1) — so a burst arriving during a commit is the common case for log loss.

9.2.1 Disk full #

When the filesystem holding a store reaches capacity, SQLite writes fail.

After any write failure consistent with disk-full or quota exhaustion, eventd schedules an immediate retention run across every store whose retention is enabled, under the ordinary bounded-batch rules (§3.6, §4.4, §5.5). Retention is the only lever eventd has that frees space, and waiting up to an hour for the next scheduled pass would waste the window in which recovery is still cheap.

A write refused for want of space emits no eventd.store.quarantined; that event reports only a store found corrupt and quarantined (§2.6).

Capacity diagnostics name the operation that failed. A synthetic-event, quarantine-report, maintenance or rollup-maintenance failure is not by itself a report of a discarded ingestion batch. Actual failed batches are reported as discarded, with event-batch loss ranges as described below.

9.2.1.1 The event store #

A failed INSERT or COMMIT does not crash the writer thread.

The batch is lost, and it is not recoverable: those events were already consumed from the ring buffer, so KMES no longer has them. The writer records the per-CPU sequence ranges of the failed batch in an in-memory lost-batch list, and on its next successful commit emits eventd.events.lost records for every accumulated range before writing new events. Each record's timestamp is that of the last lost event in its range, since no later event revealed it (§2.5).

That ordering matters. Emitting the gap records first means the store never contains events written after a loss without the record of the loss preceding them.

The writer also logs the failure to standard error immediately — including the CPU identifiers and sequence ranges — which peinit captures. That is the only visibility available while the disk is still full and the gap record cannot yet be written.

If eventd crashes before the disk recovers, the in-memory list dies with it. Nothing is silently lost even so: on restart, committed receipt ranges are reconciled with the current ring survivors. Uncovered survivors are ingested again, and sequences present in neither source become ordinary restart gaps (§2.2, §3.7). The record may be coarser — one gap rather than several — but the loss is still recorded.

Meanwhile events accumulate in the ring buffers. If the disk stays full long enough, they overrun and additional loss occurs, detected by the same mechanism (§9.1).

9.2.1.2 The log and metric stores #

A failed commit loses the batch. The writer retries on the next one. Acceptable under the loss model, and no lost-batch accounting exists for either — there is nothing to reconcile against, since neither has sequence numbers.

9.2.2 Corruption #

Corruption from a hardware error, a filesystem bug, or an incomplete write during a kernel crash.

Detection at startup is structural: eventd verifies that the required tables and indexes exist. It does not run PRAGMA integrity_check, which scans the entire database and costs time proportional to its size — unacceptable for a large event store on every boot. Corruption that leaves the schema intact, such as a single bad page, is found later, at query or write time, when SQLite touches it.

At startup, when SQLite reports corruption in a required active store, eventd quarantines the files, creates a fresh empty database at the original path, and continues (§3.3). It logs the corruption and emits eventd.store.quarantined once a shard is available to hold it.

At write time, eventd stops writing to the affected database, emits eventd.store.quarantined if it can, quarantines and replaces the database, and resumes writes to the replacement.

At query time, a handler encountering corruption fails the affected query with an error. It does not return the rows it managed to read: partial data from a database SQLite has declared corrupt is indistinguishable from complete data, and silently under-reporting an audit query is worse than failing it.

A missing or unrecognised schema version is not corruption. It is not repaired and not migrated: a required store with one fails startup, and a historical shard with one is excluded from the query path (§3.3).

Recovering data from a quarantined file is an administrative operation. eventd never attempts automatic repair, and the quarantined file is the only copy of whatever it held.

Two of eventd's four dependencies can disappear after startup without stopping it, and in both cases the ingestion path survives while the query path does not. That asymmetry is deliberate: events that are not collected are gone, and queries that cannot be answered can be asked again.

9.3.1 The registry becomes unavailable #

If LCS or loregd goes away after eventd has started:

• eventd keeps its last known configuration. Changes are not applied until the registry returns.
• Descriptor lookups fall back to the cache. A pattern the cache does not hold is denied, fail-closed (§7.5).
• eventd keeps ingesting and keeps serving queries for descriptors it already resolved, indefinitely.
• When the registry returns, the watch fires and eventd re-reads.

This is a degraded state, not a failure. eventd does not exit, and it does not stop collecting.

The related case is the watch failing while the registry is otherwise reachable. eventd discards the descriptor cache and operates fail-closed for new resolutions until the watch is re-established (§7.5), because a cache it cannot trust to be current would make a revocation silently ineffective. SIGHUP forces a configuration re-read in the meantime (§8.3).

9.3.2 KACS becomes unavailable #

If KACS goes away after startup:

• reading the peer token (KACS_SO_PEER_TOKEN) fails on new query connections, so new queries are denied.
• kacs_access_check and kacs_access_check_list fail, so a query in progress that needs a fresh check is denied.
• Cached check results stay valid for the duration of the query that obtained them.
• Event ingestion is unaffected. Neither the drain nor the write path calls KACS (§8.1).
• Log ingestion is unaffected.
• Metric ingestion continues, but every record needs an EVENTD_PUBLISH check (§7.6). A record whose verdict is cached for the sender's token and name is stored as usual; one needing a fresh check is refused and counted as an authorization error, and the rest of its datagram is still taken.

eventd keeps collecting and cannot answer. Query service resumes when KACS does.

9.3.3 KMES #

There is no partial mode. eventd attaches to every per-CPU ring buffer at startup and fails to start if it cannot (§8.2); there is no subsequent state in which KMES is present but unusable, because the mapping is established once and the read protocol has no call that can fail afterwards. A ring buffer resize is handled as a generation change, not as a failure (§2.2).

9.3.4 peinit #

peinit manages eventd's lifecycle but supplies no runtime service to it. The boot ID comes from /proc/sys/kernel/random/boot_id (§8.1). There is therefore no separate peinit-loss mode once eventd is running — peinit going away means the system is going away.

9.4.1 Memory #

If the out-of-memory killer takes eventd, it is treated exactly as a crash (§8.5): the databases are consistent, the ring buffers are untouched, peinit restarts the daemon, and the gap is recorded.

The principal long-lived memory consumers are:

The handoff limits do not follow live MaxBatchSize changes. The channels are small scheduling handoffs; KMES remains the real event backlog.

High-cardinality producers are the likely surprise. The series cache cannot grow past its limit, but the series table can grow and make it thrash on the thread that also drains the metric socket (§5.3). Identifier intern sets intentionally trade memory for zero catalogue work on known-name ingestion. The default 1 GiB metric retention ceiling and low-priority orphan cleanup bound persistent growth indirectly, but do not bound the number of active series between retention passes.

9.4.2 Query slots #

Reaching MaxConcurrentQueries, MaxStreamingQueries or MaxQueriesPerUser rejects new queries with an error rather than queueing them (§6.5). The per-user bound keeps one caller from occupying every slot; several callers together, or SYSTEM, still can.

Ingestion is unaffected, because queries and ingestion are separate channels and separate threads. That separation is what makes query-side exhaustion an inconvenience rather than data loss.

9.4.3 Descriptors #

An event query opens a read-only connection per shard database, and each connection holds one or two file descriptors (§6.4). Many historical shards multiplied by many concurrent queries reaches a process limit faster than anything else eventd does.

Where eventd cannot allocate what an admitted query needs, it fails that query rather than blocking a writer or exceeding its own limit (§6.1).

9.4.4 Writer stalls #

Two things stall a writer thread briefly, and both are bounded by design.

An index build in progress. Drain threads detect rising write pressure and signal cancellation; the writer aborts the CREATE INDEX, SQLite rolls back the partial index, and event writing resumes immediately (§3.4).

A writer-owned maintenance command. Retention and adaptive-index maintenance execute only as bounded, low-priority commands at transaction boundaries. Ingestion wins before the next command; there is no competing read-write connection or writer mutex (§3.6).

Neither stall loses anything by itself. Both contribute to ring buffer pressure, and prolonged enough, both end at §9.1.

Sudden power loss is the failure the three stores' durability settings were chosen against, and it is where the hierarchy the whole daemon is built on becomes visible as three different outcomes.

The event store loses only the in-flight batch — the events accumulated since the last commit, which the adaptive batcher keeps as small as throughput allows (§2.4). Unlike a process crash, a power cycle destroys the old KMES rings and changes the boot ID. eventd can neither recover that batch nor determine its final sequence range, so it cannot honestly manufacture an eventd.events.lost record for it. The last receipt coverage and the following boot's eventd.daemon.started bound the unknown loss.

The log and metric stores may lose everything committed since their last write-ahead log checkpoint, which under synchronous=NORMAL is the real durability boundary rather than the commit. How much that is depends on WalCheckpointPages and the write rate, and it can be considerably more than one batch.

The difference is bought and paid for deliberately. FULL costs an fsync on every commit, which at ten thousand events a batch is amortised and at three events a batch is not — and eventd commits small batches constantly under light load. The event store pays it because an event may be an audit record whose absence is itself the finding. The other two do not, because their loss is defined as acceptable and paying for durability they do not need would slow the paths that are most likely to be bursty.

Events are sacred; logs and metrics are important but not fundamental. Every durability decision in this manual is that sentence applied to a particular store.

9.5.1 What restart does #

Nothing manual. eventd starts, finds its databases consistent — WAL recovery is SQLite's, and an incomplete transaction is rolled back on open — reads the new kernel boot ID and starts that boot's per-CPU coverage before sequence 1 (§3.7). It does not compare a new boot's ring against the previous boot's receipts: those are different sequence namespaces.

This is the irreducible difference from a process crash. In the latter, KMES survives and receipt/ring reconciliation can distinguish committed, recoverable and truly missing sequences (§8.5). Across power loss, the old volatile source no longer exists.

evctl is eventd's native command-line client. Its normal operation has no subcommand: the one positional argument is a PSPU observability query.

evctl 'EVENTS kacs.* SINCE 1h ago TAKE 50'
evctl 'LOGS FROM authd ERROR ONLY STREAM'
evctl 'METRIC cpu.usage SINCE 10m ago AVG'

Quote the complete query. Without the quotes, the shell can split string literals or expand * patterns before evctl sees them. Query syntax and semantics remain those of PSPU §3.18–§3.27; evctl does not maintain a second flag-based query language.

10.1.1 Commands and queries #

Exact standalone words such as help and version occupy the command namespace. Anything whose first word is EVENTS, LOGS or METRIC, matched without regard to ASCII case, is a query. This leaves room for administrative commands without requiring the common query operation to be written as evctl query ....

The query can instead be read from a UTF-8 file or standard input:

evctl --file denied-events.evq
printf '%s\n' 'EVENTS eventd.* SINCE 1d ago' | evctl -

The default socket is /run/eventd/query.sock. --socket PATH selects a nonstandard configured socket. The server obtains the caller from the Unix socket peer token and performs every identifier and field access check; the client never opens an eventd database or substitutes its own authorization decision.

10.1.2 Output #

Data is written to standard output. Diagnostics and query errors are written to standard error, so redirecting or piping the result does not mix the two.

Records do not have a uniform schema. A field absent from one line is not necessarily absent from later lines. Scripts should select the fields they need in the query or tolerate missing object keys.

10.1.3 Result commitment #

An ok response is not, by itself, a completed result. A non-streaming query only succeeds when eventd sends end; a streaming query's initial result only succeeds when eventd sends watch (PSPU §3.16). If eventd reports an error before that point, every preceding record belongs to an invalid partial result and must be discarded.

evctl therefore renders initial records into an anonymous temporary spool. It publishes the spool to standard output only after end or watch. The spool bounds heap use independently of the result count and has no pathname to clean up after a crash.

After watch, new records are written directly as they arrive. Closing evctl, including with Ctrl-C, closes the query connection and ends the watch.

As with every streaming client, a blocked output pipeline eventually applies socket backpressure. eventd may terminate that stream rather than allowing a slow observer to interfere with ingestion.

eventd-client is a Rust library, in the eventd source tree, for programs that query eventd: evctl itself, Event Viewer, and any third-party tool. It speaks the query channel of PSPU §3.14–§3.17 and adds what every client would otherwise have to get right on its own.

It depends on the peios crate and nothing of eventd's own, so a program using it brings in neither SQLite nor eventd's storage.

10.2.1 Whole results or none #

PSPU §3.16 says an error that arrives before the terminal message voids every result message before it. The library keeps that rule itself, so a program cannot show part of a result as if it were all of it.

• query(socket, text) runs a query that does not stream and returns every record once end has arrived, or the error and no records.
• Tail::start(socket, text, capacity) follows a STREAM query. It reports the initial result once, complete, when watch arrives; then each live batch; then why the tail ended. If the query fails before watch, the only report is that it ended.

A query blocks until eventd's terminal message, for as long as eventd's query timeout (§6.5), so a program with a window to keep responsive runs it on another thread.

10.2.2 A tail that stops rather than grows #

A tail reads the socket on a thread of its own, as fast as eventd writes, and passes reports through a channel that holds at most capacity of them.

If the program stops taking reports, the channel fills, the thread stops reading, the socket's buffer fills, and eventd ends the query, since it never waits for a slow reader (§6.6). The program's next report is that the tail ended. Nothing grows without bound, in eventd or in the program.

Dropping the Tail closes the connection, which eventd treats as cancelling the query (PSPU §3.14).

10.2.3 Writing query text #

The text module writes a value into query text so that it is always exactly one value (PSPU §3.19).

• string writes a quoted string: the quote and backslash escaped, and every control character as \n, \r, \t or \uXXXX.
• binary writes x"…"; duration writes 90s, 2h and so on, in the largest unit that holds the duration exactly.

Every place the language takes an identifier or a pattern also takes a quoted string, so text a person typed should always go through string. Written bare, an event type called STREAM or WHERE would be read as the clause, since the pattern after EVENTS is optional. After LOGS FROM, which always takes an origin, the same word is read as the origin, but quoting is never wrong there either.

eventd's own tests parse the library's quoted text back through eventd's parser for strings chosen to break naive quoting, and require the value to come back unchanged.

10.2.4 What a caller may read #

eventd removes what a caller may not read and says nothing about it (PSPU §3.28). A program that wants to tell its user "you can't read events here", rather than show an empty list, has to work it out from the same descriptors eventd checks (§7). The access module does that.

• The rights, eventd's generic mapping, the three namespaces with their root GUIDs, and the walk from an identifier to its pattern come from this module, and eventd uses the same definitions, so a client and eventd cannot disagree on them.
• field_guid derives a field's GUID (§7.3). eventd-core computes the same derivation for itself, and a test holds the two equal.
• resolve finds the descriptor eventd would check an identifier against: the identifier, each shorter dotted prefix, then *, with an origin's producer left off (§7.2).
• access checks a descriptor the way eventd does: the calling process's own token, eventd's mapping, and the object type list of the data type's root and the paths of any fields asked about (§7.3), then every field the descriptor grants by name. Records are visible with EVENTD_READ on the root or on any field, as eventd shows them (§7.4).
• field_tree builds that list for a set of field names, and field_reads makes the check over it, answering per field; eventd uses both, so the tree a client asks about is the one eventd builds.
• field_grants lists the fields a descriptor grants by name: the object GUIDs of its allowing object ACEs. eventd uses it too, to find which identifiers a field-only grant makes visible.
• readable says how much of events, logs or metrics the caller may read across every pattern written for them, since a more specific pattern can grant what * denies, and the other way round: everything, some (and which patterns it may not read), nothing, or unknown.

When the policy itself can't be read, readable says unknown and why, never "nothing": a caller denied the registry keys may still be granted the data.

Every key lives under Machine\System\eventd\. eventd ignores unknown keys in the subtree. An invalid value is ignored and the value already in use is retained, and eventd emits an eventd.config.changed event for every change actually applied (§8.3).

A.1 Required #

No compiled-in defaults. A missing or invalid value fails startup (§8.2).

A.2 SQLite storage #

A.3 Event ingestion #

A.4 Log ingestion #

A.5 Metric ingestion #

A.6 Adaptive indexing #

A.7 Index shedding #

A.8 Retention #

A.9 Querying #

A.10 Adaptive metric rollups #

A.11 Cross-type filtering #

A.12 The security subtree #

Read-path descriptors live under Machine\System\eventd\Security\ and are not configuration in the sense above (§7.2):

Machine\System\eventd\Security\Events\*
Machine\System\eventd\Security\Events\<pattern>
Machine\System\eventd\Security\Logs\*
Machine\System\eventd\Security\Logs\<pattern>
Machine\System\eventd\Security\Metrics\*
Machine\System\eventd\Security\Metrics\<pattern>
Machine\System\eventd\Security\Admin

Security\Admin governs EVENTD_ADMINISTER, especially INDEX, and defaults to SYSTEM and Administrators. It is registry-protected policy, not data in eventd-meta.db (§3.5, §7.2).

A.13 When a change takes effect #

Wire-protocol constants — framing, ingestion limits, the query request ceiling and response target — belong to the interfaces rather than to eventd and are in PSPU §3.A.

B.1 Access rights #

B.2 Generic mapping #

Passed to AccessCheck in the generic_read, generic_write, generic_execute and generic_all fields.

EVENTD_ADMINISTER and EVENTD_PUBLISH are in GENERIC_WRITE and deliberately not in GENERIC_READ or GENERIC_EXECUTE (§7.1, §7.6).

B.3 Field GUID namespace #

EVENTD_FIELD_NAMESPACE = {e7d3a1b0-5c2f-4e8a-9b1d-0a6f3c8e2d4b}

Field GUIDs are uuid_v5(EVENTD_FIELD_NAMESPACE, field_name) with field_name as UTF-8 (§7.3).

B.4 Data type root GUIDs #

The level-0 node of an object type list.

B.5 Field names #

Field GUIDs are computed from the algorithm, never hardcoded. The names they are computed from are these.

Event header fields. event.time, event.cpu, event.sequence, emitter.class, event.type, emitter.token.guid, emitter.true-token.guid, emitter.process.guid, event.boot.guid, the paths of PSPU §3.22. The events table's column names are not field names (§3.1).

Log fields. timestamp, origin, is_error, message, job_id, boot_id.

Fixed metric fields. timestamp, boot_id, name, type, value.

Event payload fields use the flattened dot path (PSPU §3.22). Suppressed paths, and payload paths at or beneath a header field's path, are not query-language fields and have no GUID.

Metric label keys use the key itself: core produces uuid_v5(EVENTD_FIELD_NAMESPACE, "core"). A label key can never be one of the five fixed metric field names, because ingestion rejects records whose labels collide with them.

B.6 Origin class #

The origin class is the header field emitter.class, held in the origin_class column, and the query language accepts the names of classes 0 to 3 as aliases for their values (PSPU §3.23 names no alias for NTFE).

B.7 Synthetic event types #

All five are tier essential. These five names, and no others, are reserved: a KMES event of one of these types is not stored (§3.1). Payload schemas are in §3.2.

B.8 Metric types #

Stored in series.type. The query language exposes the names, not the numbers (PSPU §3.22).

B.9 Log severity #

Stored as an integer in logs.is_error; exposed as a boolean by the query language, which accepts both forms (§4.2).

B.10 Series hashing #

FNV-1a, 64-bit, over the exact bytes of the canonical label string or the boundary blob.

The high bit is cleared so the value fits SQLite's signed INTEGER. Hashes narrow lookups; identity is always confirmed against the full string or blob (§5.2).

B.11 Schema versions #

An unrecognised version is never migrated. For a required store it fails startup; for a historical shard it excludes the shard from the query path; for the metadata database it recreates from defaults (§3.3, §3.5).

None of the following affects the storage format, the wire protocol, the query language, or any behaviour a client can observe. An implementation omitting all of them is complete. Each buys measurable throughput or latency with no behavioural trade-off, which is why they are collected here rather than described as design.

C.1 Arena allocation for event copies #

Drain threads copy events out of the ring buffer at rates reaching hundreds of thousands per second (§2.2). Using the system allocator for each variable-sized copy costs freelist bookkeeping, potential lock contention in a multi-threaded allocator, and occasional page faults when it asks the kernel for more.

A pool of arena chunks avoids all of it: hand out sequential space by bumping a pointer, transfer the owning chunk with the channel slot, and recycle it only after the writer consumes the last referenced event. The pool's charged bytes are part of the channel byte bound (§2.3), so the optimisation cannot create a hidden queue. Per-event allocation cost falls from allocator bookkeeping to a pointer bump.

C.2 Drain thread affinity #

A drain thread reading a per-CPU ring buffer benefits from running on the same NUMA node as that CPU. It is not necessary — the per-CPU design eliminates write contention regardless of where the consumer runs — but NUMA-local reads avoid cross-node traffic in the drain loop.

In the 1:1 case, pinning the drain thread to the CPU whose buffer it reads gives the best cache locality available: the pages are likely still in that CPU's L3 from the kernel's write.

C.3 Partial payload extraction #

Building flat result records means decoding payloads, and at thousands of rows that is the dominant query cost (§6.1).

Where a SELECT names specific payload paths, only those need extracting. A streaming MessagePack decoder that scans for the wanted keys and skips over everything else avoids materialising the payload at all, and for a payload with many fields where one or two are selected this cuts per-row CPU by an order of magnitude.

Without a SELECT, a streaming decoder emitting flattened key-value pairs still avoids building a full in-memory representation.

C.4 Prepared statement pooling #

Writer threads already prepare one INSERT each and reuse it (§2.4). Query handlers executing translated SQL benefit from the same treatment: a small LRU pool of prepared statements per read connection, on the order of fifty to a hundred, covers the case where an operator repeats similar queries and where a dashboard issues the same query on a timer.

SQLite caches the query plan in a prepared statement, so re-preparing identical SQL spends CPU on parsing and planning that was already done.

C.5 Batched socket reads #

The log and metric ingestion threads read datagrams one at a time by default. recvmmsg reads many in one kernel round trip — up to a thousand or so — and at a batch of 64 to 256 it cuts syscall overhead by that factor under sustained load, with no protocol or format change.

It also shortens the window in which the thread is not draining the socket, which is the window that loses datagrams (§4.1).

C.6 SQLite page cache tuning #

Each connection has a page cache, around 2 MB by default. For a shard writer connection it holds B-tree pages for the events table and its indexes, and under sustained writes with several adaptive indexes the hot working set can exceed the default — producing evictions and re-reads on the write path.

A reasonable heuristic is 2 MB plus 1 MB per active secondary index for a writer connection, and 512 KB to 1 MB for a read-only query connection, whose queries are short-lived.

C.7 Bounded shard connection pool #

The query path opens a read-only connection to every database in the event store directory, and each holds one or two descriptors (§6.4). With many historical shards this becomes the binding resource.

Active shard writer connections stay open for the process lifetime and are not candidates. Historical shard read connections are: opening them lazily when a query touches them, and closing them after a period of inactivity, bounds the descriptor count without affecting any result.