Struct KafkaMetadataEventLog 

pub struct KafkaMetadataEventLog { /* private fields */ }

Production MetadataEventLog backed by an internal Kafka topic.

Implementations

impl KafkaMetadataEventLog

pub async fn start( cfg: KafkaMetadataLogConfig, ) -> Result<Arc<Self>, MetadataLogError>

Provision the topic if it is missing, connect the producer and the raw client, learn the topic id, and return the log.

Errors

Returns MetadataLogError::Other on admin / producer / client construction failures.

pub async fn open_read_only( cfg: KafkaMetadataLogConfig, ) -> Result<Arc<Self>, MetadataLogError>

Open an existing topic for reads only, and return the log.

The log neither provisions the topic nor changes its cleanup policy, whatever cfg.provision_topic and cfg.compacted say, and it builds no producer. Its reads therefore need only DESCRIBE and READ on the topic. MetadataEventLog::publish and MetadataEventLog::publish_keyed return MetadataLogError::Publish without contacting the broker.

Errors

Returns MetadataLogError::Other when the topic does not exist, and on admin or client construction failures.

pub async fn shutdown(&self)

Cancel the fetch tasks of every active subscription. A drop also cancels them.

Trait Implementations

impl Drop for KafkaMetadataEventLog

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl MetadataEventLog for KafkaMetadataEventLog

fn partition_count(&self) -> i32

Number of partitions the log holds. It is stable for the lifetime of the log. The manager hashes user partitions into [0, partition_count()).

fn publish<'life0, 'async_trait>( &'life0 self, partition: i32, event: Bytes, ) -> Pin<Box<dyn Future<Output = Result<i64, MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Append event to partition. Resolves to the assigned offset.

fn publish_keyed<'life0, 'async_trait>( &'life0 self, partition: i32, key: Bytes, event: Option<Bytes>, ) -> Pin<Box<dyn Future<Output = Result<i64, MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Append a keyed value or tombstone to a compacted metadata topic.

fn publish_keyed_group<'life0, 'async_trait>( &'life0 self, events: Vec<KeyedEvent>, ) -> Pin<Box<dyn Future<Output = Result<Vec<i64>, MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Append a group of keyed records and wait until every one of them is acknowledged. Resolves to their offsets, in the order given.

fn subscribe( &self, assignment: Vec<PartitionStart>, ) -> (MetadataEventStream, Arc<dyn AssignmentHandle>)

Start to consume the given partitions, each from its start offset, which is inclusive. Returns the event stream and a handle to mutate the live assignment.

fn low_water_marks<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<i64>, MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Earliest retained offset for each partition, indexed by partition.

fn high_water_marks<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<i64>, MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

One past the highest written offset for each partition, indexed by partition.

fn visit_range<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, partition: i32, start: i64, end: i64, visit: &'life1 mut RangeVisitor<'life2>, ) -> Pin<Box<dyn Future<Output = Result<(), MetadataLogError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Pass every retained record of partition whose offset lies in [start, end) to visit, in offset order.