8.4 Shutdown

4 min read

When peinit signals a stop, eventd persists as much in-flight data as it can without blocking indefinitely.

8.4.1 The sequence #

  1. Stop accepting. Unlink all three socket paths so no new client can reach them, and stop accepting query connections. Existing streaming queries are terminated with an error. The log and metric socket descriptors stay open, and eventd goes on reading them until every query connection still being served has ended: a sender that reached either socket before its path was unlinked can still deliver to it.
  2. Drain ingestion. Shut the log and metric sockets for reading, so that a later send fails with EPIPE and its sender keeps the datagram (peinit buffers it and replays it to the next eventd), rather than having it queued where nothing will read it. Then read and process the datagrams still in the receive queues, and close those descriptors. This is bounded by the receive queue, at most net.unix.max_dgram_qlen datagrams plus one (§4.1), so it completes quickly.
  3. Final event drain. Each drain thread performs one last drain cycle from its ring buffer.
  4. Final commit. Every writer commits its current batch immediately, whatever its size. The log and metric writers do the same.
  5. Record sequence state. Derive each logical CPU's highest contiguously covered sequence from committed receipt ranges and write it to sequence_checkpoints for diagnostics (§3.5).
  6. Emit the shutdown event. Write eventd.daemon.stopped with the per-CPU sequences, using the daemon-wide shard assignment rule (§2.6). If the receipt ranges could not be read in step 5, the record carries no sequences rather than zeroes (§3.2). If no shard is writable, the event is skipped and the failure logged to standard error.
  7. Close databases. Close every connection, writer and reader. SQLite checkpoints the write-ahead log automatically on close.
  8. Unmap. Unmap every ring buffer and close the per-CPU descriptors.
  9. Exit.

Steps 1 and 2 are deliberately split. Unlinking the pathnames stops new senders finding the socket while the descriptors stay open, so whatever is already queued is still readable — closing them at step 1 would discard the queue, which is the data most recently produced and therefore most likely to explain why the system is being stopped. Shutting the sockets for reading before the drain is what makes the drain final: nothing can join the queue behind it to be discarded when the descriptors close.

The final checkpoint in step 7 matters most for the log and metric stores, which run synchronous=NORMAL and whose durability boundary is the checkpoint rather than the commit (§4.1).

8.4.2 A failed event writer #

When an event-shard writer exits, it closes its handoff queue and releases the pending control requests. A caller waiting for a final-commit barrier, synthetic-event commit or maintenance result then observes failure instead of waiting for a reply from a writer that has stopped. These requests are not acknowledged as committed. Producers waiting for queue capacity wake with a closed-queue error; already admitted handoffs are released once their producer publishes or cancels them. Healthy writers retain their normal backpressure and commit acknowledgements.

A reservation racing with closure either joins the admitted handoffs or fails with a closed-queue error. The final drained check cannot miss a reservation that succeeds, including on processors with weaker memory ordering. Closing also wakes threads that are entering a queue wait; it does not depend on another publication to release them.

8.4.3 The timeout #

Shutdown is bounded by peinit's service stop timeout. If the sequence has not finished, eventd aborts and exits immediately.

What an aborted shutdown costs:

  • Uncommitted event batches are lost. Those events remain in the KMES ring buffers and are available at the next start, provided they have not been overwritten by then.
  • Uncommitted log and metric batches are lost, which is acceptable by design.
  • The diagnostic sequence metadata may be stale. It does not matter: startup derives coverage from committed receipt ranges and reconciles it with the ring buffer (§2.2).

Every consequence is one the restart path already handles, which is why aborting is safe rather than merely tolerable.

Edit this page