Skip to main content

Module barrier

Module barrier 

Source
Expand description

Barrier cuts: an exact, reproducible point in every input at once.

A krabka broker injects an epoch-stamped marker into every partition of a named barrier group and publishes the resulting cut, one marker offset per partition, to the internal topic __barrier_state. The marker is a Kafka control record, so no consumer ever receives one. A task aligns on a cut by comparing its consumed offsets against the cut’s offsets, never by watching for a marker.

§Key Types

  • BarrierCut — one epoch’s cut, with the marker offset of each partition.
  • CutReader — reads published cuts from __barrier_state.
  • BarrierAlignment — tells a runtime which group to align on, and where to keep each cut’s snapshot.
  • BarrierListener — the callback that fires when a barrier is reached.

§Examples

use std::sync::Arc;

use krabka_client_streams::{barrier::BarrierAlignment, store::snapshot::FileSnapshotStore};

let snapshots = Arc::new(FileSnapshotStore::new("/var/lib/app/state"));
BarrierAlignment::on("transactions", snapshots)

Structs§

Barrier
The barrier a stream thread reached: the cut, the tasks it snapshotted, and the offsets it committed.
BarrierAlignment
Tells a stream thread which barrier group to align on.
BarrierCut
One epoch’s cut across every partition of a barrier group.
CutReader
Reads the cuts a barrier coordinator published to __barrier_state.

Enums§

CutStatus
Whether the coordinator reached every partition of a barrier group.

Constants§

BARRIER_STATE_TOPIC
The internal topic a krabka broker publishes barrier state to.

Traits§

BarrierListener
Receives a barrier after a stream thread commits at a cut.

Functions§

decode_barrier_cut
Decodes one record of BARRIER_STATE_TOPIC into a cut.