Package io.krabka.streams.columnar.barrier

package io.krabka.streams.columnar.barrier

Barrier-aligned, epoch-keyed state snapshots for columnar runners.

A krabka broker injects epoch-stamped markers into every partition of a named barrier group and publishes the resulting cut to the internal __barrier_state topic. A cut is an exact, reproducible point in every input at once. The markers are Kafka control records, so a JVM consumer never delivers one; this package reads the published cut instead and compares consumed offsets against it.

BarrierCutDecoder decodes the topic's records, BarrierCutReader reads a group's complete cuts, and BarrierAlignment tells a group runner to align on them. The runner snapshots each partition at the cut, keys the snapshot by epoch, and can restore back to it later.

Example

var alignment = BarrierAlignment.on("transactions", new BarrierCutReader(cutConsumer));
try (var runner = ColumnarRunner.group(topology, consumer, producer,
        ColumnarErrorPolicy.fail(), new FileColumnarStateStore(directory),
        new ColumnarMetrics(), alignment)) {
  runner.runOnce(Duration.ofSeconds(1));
  runner.restoreToLatestCut();
}

Related Packages

Package	Description
io.krabka.streams.columnar	Apache Arrow batch codecs, topology operators, runners, and state handling.
io.krabka.streams.columnar.schema	Avro and Protobuf bridges into the Arrow columnar runtime.

Class	Description
BarrierAlignment	Tells a group runner which barrier group to align on.
BarrierCut	One epoch's cut across every partition of a barrier group.
BarrierCutDecoder	Decodes the cut records of the internal __barrier_state topic.
BarrierCutReader	Reads published cuts from the internal __barrier_state topic.
BarrierCutStatus	Whether a barrier coordinator reached every partition of a cut.
BarrierListener	Receives a cut when a runner reaches the barrier.