Benchmark Methodology & Performance

Empirical evaluation comparing Krabka against Strimzi (Apache Kafka on OpenJDK 21) across memory working set, cold-start time, saturated throughput, and tail latency.

Evaluation Philosophy & Goals

Krabka is engineered in native Rust to eliminate JVM runtime overhead, garbage collection pauses, and multi-gigabyte memory footprints in streaming architectures. To validate these characteristics, all macro-benchmarks follow four strict principles:

Identical Hardware & Topology: Both Krabka and Apache Kafka (Strimzi) run on identical Kubernetes clusters and GCP virtual machines.
Identical Storage Media: Direct I/O write-ahead logs backed by local NVMe SSDs (ext4 formatted).
Identical Load Driver: A shared Rust load generator (bench-driver) speaking standard Apache Kafka wire protocol executes unmodified against both targets.
Coordinated-Omission-Free Latency: Latencies are tracked via HdrHistogram using send timestamps embedded in payload headers rather than client send loops.

Test Environment Topology

All comparative benchmarks are executed against dedicated cloud infrastructure with fixed hardware allocation:

Compute Hardware: 3x GCP c3-standard-8 instances (8 vCPUs Intel Xeon, 32 GB RAM per node).
Storage Subsystem: Local NVMe SSD formatted with ext4, utilizing Direct I/O asynchronous write-ahead logs.
Benchmark Suite: Industry-standard OpenMessaging Benchmark (OMB) and bench-driver executing sustained workloads from 50,000 to 500,000 msg/sec.
Software Baseline: Krabka (native Rust, KRaft consensus) evaluated side-by-side against Strimzi 1.0 running Apache Kafka 4.3 (KRaft, OpenJDK 21).

Memory Working Set Footprint (RSS)

In production Kubernetes deployments, memory capacity and pod eviction decisions are governed by cgroup limits. We measure container_memory_working_set_bytes through Prometheus scrapers during steady-state produce and fanout consumption workloads.

Sustained Produce Workload Memory Footprint

Krabka Broker (RSS) 114.2 MiB

Strimzi Kafka JVM (Heap + Off-Heap) 4,280 MiB

Measurement Caveat: Strimzi's cgroup memory conflates active heap, non-heap buffers, and Linux page cache from mmap. We collect JMX metrics (jvm_memory_bytes_used) separately to verify heap versus off-heap consumption against Krabka's direct user-space allocation model.

How Krabka Achieves 114.2 MiB Working Set

Krabka operates at a fraction of JVM Kafka's memory footprint due to three architectural design decisions:

Zero-Copy Buffer Slicing: Slices of byte buffers (bytes::Bytes) pass through the network I/O layer, log segment appends, and consumer dispatch pipelines without memory duplication or re-allocation.
Deterministic Deallocation: Memory is immediately reclaimed when reference counts hit zero. There are no background GC compaction passes, heap fragmentation margins, or speculative memory reservations.
Compact Memory Representation: Rust structs avoid the 12-to-16-byte object header overhead, pointer indirection, and boxed primitive structures inherent to JVM runtimes.

How Strimzi Kafka Consumes ~4,280 MiB

Under an identical workload, Strimzi Kafka's memory footprint is composed of:

JVM Heap Space: Heap pre-allocations (2 GiB initial cap) required to prevent frequent Garbage Collection pauses under high throughput.
Metaspace & JVM Threads: JVM class metadata, JIT compiler caches, and 1 MiB per-thread stack allocations across dozens of thread pools.
Off-Heap & Netty Buffers: Direct byte buffers used by network transports and GC card tables.

Cold Start & Failover Recovery

Broker restart and failover agility determine cluster availability during node reboot, autoscaling, and rolling upgrades.

Cold Start & Recovery Duration

Krabka Startup Time 1.2 seconds

Strimzi JVM Startup Time 8.6 seconds

Measurement Methodology

Cold Start Time: Elapsed wall-clock time from Kubernetes Pod launch until the broker initializes its log directory, establishes KRaft quorum consensus, and accepts its first client connection.
Failover Recovery Time: Under an active produce workload at replication factor 3, the active partition leader pod is abruptly killed (SIGKILL). The load driver measures the elapsed duration until the new KRaft leader is elected and successfully acknowledges subsequent produce requests.

How Krabka Achieves 1.2s Cold Start & 1.8s Failover

As a compiled native binary, Krabka bypasses JVM classloading, bytecode verification, and JIT compilation phases. Startup involves only:

Immediate memory mapping of log segment headers via krabka-log without full-file scans.
Instant KRaft election convergence and metadata state machine snapshot replay via krabka-raft.

Scenario Matrix Results

The comparative test suite exercises real-world workloads under various partition counts, payload sizes, and failure events:

Scenario	Payload	Topology	Duration	Krabka Metric	Strimzi Metric	Advantage
small-msg-saturate	100 B	6p / RF=1	60 s	182.4 MB/s	74.8 MB/s	2.44x
fixed-rate-latency	1 KiB	6p / RF=1	120 s	1.4 ms (p99)	8.2 ms (p99)	82.9%
large-msg	100 KiB	6p / RF=1	60 s	890.2 MB/s	512.6 MB/s	1.74x
failover	1 KiB	12p / RF=3	180 s	1.8 s (failover)	12.4 s (failover)	6.89x
high-partition-saturate	100 B	100p / RF=3	60 s	245 MB/s	98.2 MB/s	2.49x

Reproducing Benchmarks Locally

Developers can run local microbenchmarks or complete cluster benchmark suites using the open-source recipes:

Microbenchmarks (Criterion)

# In krabka-protocol
cargo bench --bench codec

# In krabka-broker
cargo bench -p krabka-log --bench log

OpenMessaging Benchmark (OMB) Cluster Suite

git clone https://github.com/krabka-io/krabka-o11y-demo
cd krabka-o11y-demo/benchmarks
./run-suite.sh --cluster-type gke --target krabka --workload omb-standard