Struct LogConfig 

pub struct LogConfig {

    pub read_buffer_cap: ByteSize,
    pub read_ahead_max: ByteSize,
    pub timestamp_scan_window: ByteSize,
    pub segment_size: ByteSize,
    pub segment_roll_interval: Time,
    pub segment_jitter: Time,
    pub file_delete_delay: Time,
    pub start_offset_checkpoint_interval: Time,
    pub flush_messages: Option<u64>,
    pub flush_interval: Option<Time>,
    pub compression_gzip_level: i32,
    pub compression_lz4_level: i32,
    pub compression_zstd_level: i32,
    pub retention: Option<Time>,
    pub retention_size: Option<ByteSize>,
    pub max_message_size: ByteSize,
    pub max_decompressed_record: Option<ByteSize>,
    pub index_interval: ByteSize,
    pub segment_index_size: ByteSize,
    pub flush_on_append: bool,
    pub segment_allocation: SegmentAllocation,
    pub tail_cache_size: ByteSize,
    pub writeback_window: ByteSize,
    pub page_cache_budget: Option<ByteSize>,
    pub validate_on_open: bool,
    pub cleanup_policy: CleanupPolicy,
    pub min_compaction_lag: Time,
    pub max_compaction_lag: Option<Time>,
    pub min_cleanable_dirty_ratio: Ratio,
    pub message_timestamp_type: TimestampType,
    pub compression_type: Option<CompressionType>,
    pub remote_storage_enable: bool,
    pub remote_tier: RemoteTierFlags,
    pub local_retention: Option<Time>,
    pub local_retention_size: Option<ByteSize>,
    pub delete_retention: Time,
    pub delivery_policy: DeliveryPolicy,
    pub schedule_order: ScheduleOrder,
    pub delivery_clock_uncertainty: Time,
}

Tunables for Log behavior.

Defaults match Apache Kafka 4.2 for segment.bytes, segment.ms, retention.ms, index.interval.bytes, and the other tunables. Start from the Default impl. Most production deployments override only Self::retention and Self::retention_size.

Fields

read_buffer_cap: ByteSize

Cap the initial allocation used by decoded and raw segment reads.

read_ahead_max: ByteSize

How far past a verbatim read’s own window the read asks the kernel to read ahead, as a POSIX_FADV_WILLNEED hint, so a consumer that is behind finds its next fetch in the page cache. The hint covers the window and then up to one more window, capped at this. 0 hints only the window itself. The broker gives the hint as one WILLNEED call for each 128 KiB of the range, because Linux reads only about max(max_sectors_kb, read_ahead_kb) of one call. The hint is a no-op off Linux.

timestamp_scan_window: ByteSize

Read timestamp searches in windows of this size.

segment_size: ByteSize

Roll the active segment once it grows past this. Kafka’s segment.bytes; default 1 GiB.

segment_roll_interval: Time

Roll the active segment when its first record is older than this. Kafka’s segment.ms; default 7 days.

segment_jitter: Time

Maximum random subtraction from the segment roll interval, sampled per segment.

file_delete_delay: Time

Delay before retired segment files are unlinked. Readers stop seeing them immediately.

start_offset_checkpoint_interval: Time

How long a log start that only retention moved may wait before it is written to log-start-offset-checkpoint. Kafka’s broker-wide log.flush.start.offset.checkpoint.interval.ms; default one minute. Zero writes the start on the next maintenance pass. Retention moves the start to a segment boundary, which the segment names already record, so the write can wait: see Log::checkpoint_log_start_if_due.

flush_messages: Option<u64>

Offset span between forced syncs. None disables the threshold.

flush_interval: Option<Time>

Time between forced syncs of dirty data. None disables the timer.

compression_gzip_level: i32

Gzip level used when encoding owned batches; -1 selects the codec default.

compression_lz4_level: i32

Lz4 level used when encoding owned batches.

compression_zstd_level: i32

Zstd level used when encoding owned batches.

retention: Option<Time>

Delete sealed segments older than this. None = unlimited. Kafka’s retention.ms; default 7 days.

retention_size: Option<ByteSize>

Delete oldest sealed segments until the total .log size fits. None = unlimited. Kafka’s retention.bytes.

max_message_size: ByteSize

Largest single record batch this partition accepts, measured over the batch’s whole wire encoding including its 61-byte v2 header. Kafka’s max.message.bytes; default 1048588. A Produce carrying a larger batch is refused with MESSAGE_TOO_LARGE (10) rather than truncated, because a batch that lands in the log is a batch every consumer has to fetch whole.

max_decompressed_record: Option<ByteSize>

Kafka trunk’s max.decompressed.message.bytes: the largest record body a compressed batch may hold when the log decompresses it, which it does to compact a partition and to answer a by-timestamp ListOffsets. A record above it fails the lookup or the pass with LogError::RecordTooLarge, as DefaultRecord.readFrom throws InvalidRecordException. An uncompressed batch is never held to it. None is no limit, which is Kafka’s default (Records.SOFT_MAX_ARRAY_LENGTH) and the only setting Kafka 4.3.1 has: the broker sets it only under unstable.api.versions.enable.

index_interval: ByteSize

Write one .index/.timeindex entry per this much .log. Kafka’s index.interval.bytes; default 4 KiB.

segment_index_size: ByteSize

Roll the active segment once its .index or its .timeindex holds this many bytes of entries. Kafka’s segment.index.bytes; default 10 MiB. Kafka rounds the size down to whole entries, so the offset index holds size / 8 entries and the time index size / 12.

flush_on_append: bool

fsync after every append. Default off. The broker manages fsync separately.

segment_allocation: SegmentAllocation

Kafka’s preallocate. Defaults to SegmentAllocation::OnWrite, Kafka’s false. See SegmentAllocation.

tail_cache_size: ByteSize

How much of the newest data an active segment that writes through O_DIRECT keeps in memory to serve reads with. Those writes bypass the page cache and drop what it held of the range, so without this a consumer reading right behind the producer reads from disk. Every partition under SegmentAllocation::Preallocate holds up to this much, plus one batch; 0 keeps only the newest batch. Default 8 MiB: see DEFAULT_TAIL_CACHE_SIZE.

writeback_window: ByteSize

In a segment written through the page cache, start a background fdatasync of the active .log file after every this many appended bytes, and make an append wait while two such syncs are still running. The kernel then holds about twice this much dirty data for the segment at most, so neither its dirty-page throttling, which pauses a writer for up to 200 ms at a time, nor the sync at segment roll holds an append for long. A segment written through O_DIRECT needs none of this, and flush_on_append syncs every append anyway. 0 leaves write-back to the kernel. Default 8 MiB: see DEFAULT_WRITEBACK_WINDOW.

page_cache_budget: Option<ByteSize>

The most page cache the segments written through it keep, over every log in the process. Each background sync of an active segment records the whole pages it covered, and while the recorded bytes exceed this, the oldest recorded pages of any log are dropped from the page cache (posix_fadvise(POSIX_FADV_DONTNEED)), off the appending thread. A consumer further behind than this reads from the disk. Without it, a container’s memory limit is what evicts the pages, and at that limit the kernel reclaims a page inside every append that writes one: in 10 GiB containers taking 50K 1 KiB records/s at RF3, that doubled the publish p50 for minutes at a time. None keeps every page until the kernel evicts it. Logs with different values each trim the shared record to their own. Segments written through O_DIRECT keep no page cache and are not counted. Default None; the broker sets half of its memory limit.

validate_on_open: bool

On open, CRC every batch in the active segment and rebuild its sparse indexes.

cleanup_policy: CleanupPolicy

Cleanup policy. Defaults to Delete. See CleanupPolicy.

min_compaction_lag: Time

Kafka’s min.compaction.lag.ms: a record stays uncompacted for at least this long after it is written. The broker’s cleaner reads it and leaves a partition whose newest dirty record is younger than this alone. Default 0, which is Kafka’s.

max_compaction_lag: Option<Time>

Kafka’s max.compaction.lag.ms: however clean a partition looks, once its oldest dirty record is older than this the cleaner runs anyway. None is no bound, which is Kafka’s default (Long.MAX_VALUE).

min_cleanable_dirty_ratio: Ratio

Kafka’s min.cleanable.dirty.ratio: the share of a compacted partition’s log that must be uncleaned before the cleaner spends a pass on it. Default 0.5, which is Kafka’s.

message_timestamp_type: TimestampType

Kafka’s message.timestamp.type: whose clock the stored records carry. CreateTime is the producer’s own timestamp and is the default; LogAppendTime is the broker’s clock at append time.

compression_type: Option<CompressionType>

Broker-side recompression target. None is Kafka’s compression.type=producer, which is pass-through: the broker stores the batch exactly as the producer sent it. Some(c) re-encodes every batch the broker accepts on this partition to c before the write. This matches Kafka’s per-topic compression.type config. gzip, snappy, lz4, zstd, and uncompressed map to Some(_). producer, the default, maps to None.

remote_storage_enable: bool

When true, the broker’s RemoteLogManager may copy this partition’s sealed segments (KIP-405) to the remote tier. This maps to Kafka’s per-topic remote.storage.enable. Default false, which is also Kafka’s default, because tiered storage is opt-in per topic.

remote_tier: RemoteTierFlags

KIP-950’s two disablement controls, remote.log.copy.disable and remote.log.delete.on.disable. See RemoteTierFlags.

local_retention: Option<Time>

Local-disk time-retention window for tiered partitions (KIP-405). None inherits Self::retention. Default None.

local_retention_size: Option<ByteSize>

Local-disk size budget for tiered partitions (KIP-405). None inherits Self::retention_size. Default None.

delete_retention: Time

KIP-534. After a tombstone or transaction marker first becomes compaction-eligible, the log retains it for at least this long before deletion. This is the delete-horizon grace window. Default 24h.

delivery_policy: DeliveryPolicy

When a durable record becomes visible. Defaults to DeliveryPolicy::Immediate. See DeliveryPolicy.

schedule_order: ScheduleOrder

KFC-1 delivery.schedule.monotonic, read only under DeliveryPolicy::Scheduled. See ScheduleOrder.

The check belongs to the log because only the log serializes appends. A partition’s schedule is a property of what it already holds, so a test that reads it and an append that extends it have to be the same critical section: two producers that each pass a check taken before the append can still land out of order, and so can two jobs the writer batches into one group. Kafka runs its own record-shape rejections in the same place, under UnifiedLog.append’s lock.

delivery_clock_uncertainty: Time

Declared bound on how far this broker’s clock can be from true time. Default 250 ms. It has an effect only under DeliveryPolicy::Scheduled.

A batch is visible once max_timestamp + delivery_clock_uncertainty <= now. If the clock reads c while true time is somewhere in [c - e, c + e], then c >= activation + e proves true time has reached the activation instant. Provided the declared bound holds, this test does not deliver early. The clock test’s additional delay is at most twice the bound; polling, unavailable storage and an earlier pending batch can delay delivery further.

Changes are prospective for the published delivery prefix: increasing this bound or moving the clock backward does not retract offsets the in-memory watermark already released. The new bound governs subsequent advances and pending batches. A restart rebuilds that watermark from retained records using the current bound; the watermark is not durable.

Implementations

impl LogConfig

pub fn compression_level(&self, codec: CompressionType) -> Option<i32>

The configured level for a codec that supports compression levels.

Trait Implementations

impl Clone for LogConfig

fn clone(&self) -> LogConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LogConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LogConfig

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for LogConfig

fn eq(&self, other: &LogConfig) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LogConfig