pub const DEFAULT_WRITEBACK_WINDOW: ByteSize;Expand description
Default for LogConfig::writeback_window.
Linux lets a file’s dirty pages pile up to vm.dirty_ratio of memory
before it pauses the writer, and balance_dirty_pages then pauses it for
up to 200 ms at a time; a 1 GiB segment’s roll then syncs up to 1 GiB at
once. Three brokers writing 1 KiB records at the disk’s rate through the
page cache measured a produce p99 of 0.75–0.9 s, against 0.09 s through
O_DIRECT, on the OMB max-rate RF3 workloads. 2 MiB is about two
milliseconds of one partition’s writes at that rate, so two of them in
flight bound the wait an append can see to that. Against 8 MiB, 2 MiB
cut the publish p99 of 50K 1 KiB records/s at RF3 from 14–21 ms to 4–6 ms
and raised the 100-partition max rate by a quarter, for 4% of the max
rate of a single partition.