Interface ColumnarStateStore

All Known Implementing Classes:
FileColumnarStateStore
public interface ColumnarStateStore
Persists operator snapshots for one logical Kafka partition and one epoch.

A group runner saves a partition's snapshots when the partition is revoked and loads them when it is assigned, so stateful operators survive rebalances and restarts. That rebalance state uses LIVE_EPOCH. A runner that aligns on a barrier group also saves a snapshot under the epoch of each cut it reaches, and ColumnarRunner.GroupRunner.restoreToEpoch reads it back. Snapshots are keyed by operator name inside the map, which is why node names should stay stable across application versions. FileColumnarStateStore persists to local files; none() disables persistence.

Example

var stateStore = new FileColumnarStateStore(Path.of("/var/lib/app/state"));
var runner = ColumnarRunner.group(
    topology, consumer, producer,
    ColumnarErrorPolicy.fail(), stateStore, new ColumnarMetrics());

Map<String, byte[]> live = stateStore.load(0, ColumnarStateStore.LIVE_EPOCH);

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static interface 
    ColumnarStateStore.EpochLease
    A closeable snapshot-retention lease without checked exceptions.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final long
    LIVE_EPOCH
    The epoch of the running state, which no barrier cut owns.

  • Method Summary

    Modifier and Type
    Method
    Description
    Map<String,byte[]>
    load(int partition, long epoch)
    Loads the stored snapshots for a partition and epoch.
    static ColumnarStateStore
    none()
    Returns a store that keeps nothing.
    default ColumnarStateStore.EpochLease
    retain(long epoch)
    Prevents an epoch from being reclaimed while a restore is reading it.
    default Optional<List<Long>>
    retainedEpochs(int partition)
    Lists the barrier epochs currently available for a partition.
    void
    save(int partition, long epoch, Map<String,byte[]> snapshot)
    Stores the snapshots for a partition and epoch, replacing what was stored before.

  • Field Details

    • LIVE_EPOCH

      static final long LIVE_EPOCH
      The epoch of the running state, which no barrier cut owns.

      Rebalance saves and loads use it. A barrier epoch is never negative, so the two never collide.

      See Also:

  • Method Details

    • load

      Map<String,byte[]> load(int partition, long epoch)
      Loads the stored snapshots for a partition and epoch.
      Parameters:
      partition - the logical partition number
      epoch - the barrier epoch, or LIVE_EPOCH for the running state
      Returns:
      operator name to snapshot bytes; empty when nothing is stored

    • save

      void save(int partition, long epoch, Map<String,byte[]> snapshot)
      Stores the snapshots for a partition and epoch, replacing what was stored before.
      Parameters:
      partition - the logical partition number
      epoch - the barrier epoch, or LIVE_EPOCH for the running state
      snapshot - operator name to snapshot bytes

    • retainedEpochs

      default Optional<List<Long>> retainedEpochs(int partition)
      Lists the barrier epochs currently available for a partition.

      An empty optional means the store cannot enumerate epochs.

      Parameters:
      partition - the logical partition number
      Returns:
      the retained epochs in ascending order, when enumeration is supported

    • retain

      default ColumnarStateStore.EpochLease retain(long epoch)
      Prevents an epoch from being reclaimed while a restore is reading it.
      Parameters:
      epoch - the barrier epoch in use
      Returns:
      a lease that releases the epoch when closed

    • none

      static ColumnarStateStore none()
      Returns a store that keeps nothing.

      Loads are empty and saves are discarded, so operator state is ephemeral and lost on rebalance or restart.

      Returns:
      the no-op store