Skip to main content

RemoteStorageManager

krabka_remote_storage

Trait RemoteStorageManager 

Source 
pub trait RemoteStorageManager: Send + Sync {
    // Required methods
    fn copy_log_segment_data(
        &self,
        metadata: &RemoteLogSegmentMetadata,
        data: &LogSegmentData,
    ) -> Result<Option<CustomMetadata>, RemoteStorageError>;
    fn fetch_log_segment(
        &self,
        metadata: &RemoteLogSegmentMetadata,
        start_position: u32,
        end_position: Option<u32>,
    ) -> Result<Vec<u8>, RemoteStorageError>;
    fn fetch_index(
        &self,
        metadata: &RemoteLogSegmentMetadata,
        index_type: IndexType,
    ) -> Result<Vec<u8>, RemoteStorageError>;
    fn delete_log_segment_data(
        &self,
        metadata: &RemoteLogSegmentMetadata,
    ) -> Result<(), RemoteStorageError>;
}
Expand description

SPI for the remote object store that holds offloaded segment data.

Implementations are synchronous and blocking. They mirror Kafka’s RemoteStorageManager, which the broker drives from a dedicated thread pool, and the broker wraps these calls in spawn_blocking. Implementations must be Send + Sync so the broker can share one instance across tasks.

Required Methods§

Source

fn copy_log_segment_data( &self, metadata: &RemoteLogSegmentMetadata, data: &LogSegmentData, ) -> Result<Option<CustomMetadata>, RemoteStorageError>

Copies a segment’s data and all of its indexes to the remote tier.

Returns optional CustomMetadata, for example an object-store key or a version id. The broker records it on the segment and passes it back on every later fetch or delete for that segment.

§Errors

Returns RemoteStorageError if any underlying store operation fails.

Source

fn fetch_log_segment( &self, metadata: &RemoteLogSegmentMetadata, start_position: u32, end_position: Option<u32>, ) -> Result<Vec<u8>, RemoteStorageError>

Fetches a byte range of a segment’s .log data.

start_position is the inclusive starting byte offset within the segment. end_position, when Some, is the inclusive last byte offset; when None, the read runs to the end of the segment.

§Errors

Returns RemoteStorageError::SegmentNotFound if the segment is not present, or RemoteStorageError::Io on a store failure.

Source

fn fetch_index( &self, metadata: &RemoteLogSegmentMetadata, index_type: IndexType, ) -> Result<Vec<u8>, RemoteStorageError>

Fetches one of a segment’s indexes in full.

§Errors

Returns RemoteStorageError::SegmentNotFound if the segment or the requested index is not present, or RemoteStorageError::Io on a store failure.

Source

fn delete_log_segment_data( &self, metadata: &RemoteLogSegmentMetadata, ) -> Result<(), RemoteStorageError>

Deletes a segment’s data and all of its indexes from the remote tier.

Implementations must be idempotent: a delete of an absent segment succeeds — or return an error, when the backend is an immutable archive.

§Errors

Returns RemoteStorageError::Io on a store failure.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§