Skip to main content

Module manager

Module manager 

Source
Expand description

TopicBasedRemoteLogMetadataManager: the production RemoteLogMetadataManager implementation backed by a publish and subscribe MetadataEventLog.

The manager keeps the canonical in-memory view in an InmemoryRemoteLogMetadataManager, so the lifecycle state machine is the single source of truth for cache mutation. It uses the MetadataEventLog as the durable event log.

Lifecycle:

  • TopicBasedRemoteLogMetadataManager::start: load any on-disk snapshot into the cache and spawn the consumer pump subscribed to NOTHING. The broker then drives the consumed set with TopicBasedRemoteLogMetadataManager::reconcile_assignment. That call adds only the __remote_log_metadata partitions that cover the user-partitions this broker leads or follows. NotReady gates a newly-added partition until the pump reaches the HWM observed at assignment time. A partition this broker does not consume is a genuine Ok(None), and the manager never serves it from any stale cache.
  • Mutation calls, which are add, update, and put_partition_delete: serialize, publish, and wait until the consumer pump has applied the published offset to the inner cache. The sync return means “the event has been recorded and is visible to local reads”.
  • Read calls: pure local lookups against the inner cache.
  • Drop / TopicBasedRemoteLogMetadataManager::shutdown: cancel the consumer pump.

Structs§

MetadataPartitionReadiness
Readiness of one assigned __remote_log_metadata partition, as TopicBasedRemoteLogMetadataManager::metadata_partition_readiness reports it.
TopicBasedRemoteLogMetadataManager
Production RemoteLogMetadataManager backed by the __remote_log_metadata topic through a MetadataEventLog adapter.