Crate krabka_hlc 

Hybrid logical clock stamps that ride on Kafka record headers.

A hybrid logical clock (HLC) gives every event in the cluster one timestamp that is close to real time and that still respects cause and effect. The stamp holds a wall time, a logical counter, and the node that minted it, so it sorts totally and every node computes the same order. Kulkarni et al., Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases (OPODIS 2014), define the algorithm this crate implements.

Carriage

A stamp travels on one Kafka record header, HLC_HEADER. A producer calls HlcClock::now and attaches Hlc::header. A consumer calls extract_from_headers and feeds the result to HlcClock::observe.

Like krabka-trace-context, this crate does not depend on krabka-protocol. It stays type-erased over (&str, impl AsRef<[u8]>) header pairs, so any producer or consumer header type works. The caller converts to and from its own Header at the edge.

Four rules govern that boundary:

1. The crate never stores a peer’s bytes. It parses them into an Hlc, and it renders the header again from the parsed value.
2. A failure to read a stamp is silent at the carrier layer. extract_from_headers gives None for an absent, short, or invalid value, because a consumer that cannot read the stamp must still be able to apply the record. Hlc::decode returns a Result for a caller that wants the reason.
3. A record with no stamp carries no header, so it costs zero bytes.
4. No HlcError variant holds input bytes, so hostile input cannot reach a log line.

Byte layout

The header value is 16 bytes, big-endian and fixed-width: 8 bytes of wall time as an i64, 4 bytes of counter as a u32, and 4 bytes of node id as a u32. There is no varint framing. Java and Go clients read this header by hand, and a plain big-endian layout is what DataInputStream.readLong and binary.BigEndian.Uint32 already give them. A varint would make every such client write a decoder first.

Self-fence

HlcClock refuses a stamp whose wall time is more than its configured maximum offset ahead of the local physical clock, and it holds DEFAULT_MAX_OFFSET by default. A hybrid logical clock adopts the highest wall time it sees, so one host with a broken clock would drag every node that reads its records into the future, and logical time never comes back. The fence keeps the damage inside the broken host. A stamp behind the local clock is always safe, and the clock always accepts it.

Clock-confidence signal

HlcClock::observation returns what the clock has learned about skew: the current drift of the logical clock from the physical clock, how many records pulled the clock forward, how many stamps the fence refused, and the largest peer-ahead delta. A record whose stamp pulls the local clock forward by 40 milliseconds is direct evidence of 40 milliseconds of skew between this node and the node that wrote the record, measured on the data path with no probe and no extra round trip. The snapshot is a plain struct: this crate has no metrics dependency and it must not gain one.

Example

use krabka_hlc::{HlcClock, ManualClock, WallMicros, extract_from_headers};
use krabka_ids::NodeId;

let physical = ManualClock::new(WallMicros(1_700_000_000_000_000));

// The producer stamps a record it writes.
let producer = HlcClock::with_clock(NodeId(1), &physical);
let (key, value) = producer.now()?.header()?;

// The consumer reads the stamp back and merges it into its own clock.
let consumer = HlcClock::with_clock(NodeId(2), &physical);
if let Some(stamp) = extract_from_headers([(key, value.as_slice())]) {
    consumer.observe(stamp)?;
}

Structs

ClockObservation

What an HlcClock has learned about clock skew in the cluster.

Hlc

One hybrid logical clock stamp: a wall time, a logical counter, and the node that minted it.

HlcClock

A hybrid logical clock for one node.

ManualClock

A physical clock a test moves by hand.

SystemClock

The host’s wall clock, read through SystemTime.

WallMicros

A wall-clock instant, in microseconds since the Unix epoch.

Enums

HlcError

Why a hybrid logical clock operation failed.

Constants

DEFAULT_MAX_OFFSET

The maximum offset an HlcClock accepts from a peer by default.

HLC_ENCODED_LEN

The byte length of an encoded Hlc.

HLC_HEADER

The Kafka record header key that carries a hybrid logical clock stamp.

Traits

PhysicalClock

The source of physical time for an HlcClock.

Functions

extract_from_headers

The stamp on a record, or None when the record carries none.