9.5 Power Loss

Sudden power loss is the failure the three stores' durability settings were chosen against, and it is where the hierarchy the whole daemon is built on becomes visible as three different outcomes.

StoreSettingWhat survives
Eventsynchronous=FULLEvery committed transaction.
Logsynchronous=NORMALTransactions up to the last checkpoint.
Metricsynchronous=NORMALTransactions up to the last checkpoint.

The event store loses only the in-flight batch — the events accumulated since the last commit, which the adaptive batcher keeps as small as throughput allows (§2.4). Unlike a process crash, a power cycle destroys the old KMES rings and changes the boot ID. eventd can neither recover that batch nor determine its final sequence range, so it cannot honestly manufacture an eventd.events.lost record for it. The last receipt coverage and the following boot's eventd.daemon.started bound the unknown loss.

The log and metric stores may lose everything committed since their last write-ahead log checkpoint, which under synchronous=NORMAL is the real durability boundary rather than the commit. How much that is depends on WalCheckpointPages and the write rate, and it can be considerably more than one batch.

The difference is bought and paid for deliberately. FULL costs an fsync on every commit, which at ten thousand events a batch is amortised and at three events a batch is not — and eventd commits small batches constantly under light load. The event store pays it because an event may be an audit record whose absence is itself the finding. The other two do not, because their loss is defined as acceptable and paying for durability they do not need would slow the paths that are most likely to be bursty.

Events are sacred; logs and metrics are important but not fundamental. Every durability decision in this manual is that sentence applied to a particular store.

9.5.1 What restart does #

Nothing manual. eventd starts, finds its databases consistent — WAL recovery is SQLite's, and an incomplete transaction is rolled back on open — reads the new kernel boot ID and starts that boot's per-CPU coverage before sequence 1 (§3.7). It does not compare a new boot's ring against the previous boot's receipts: those are different sequence namespaces.

This is the irreducible difference from a process crash. In the latter, KMES survives and receipt/ring reconciliation can distinguish committed, recoverable and truly missing sequences (§8.5). Across power loss, the old volatile source no longer exists.

Edit this page