Skip to main content
krabka
GitHub

OpenMessaging Benchmark

Broker benchmarks

Compare throughput, tail latency and CPU use across published OpenMessaging Benchmark runs. Inspect release changes and the limits of each comparison.

These are our measurements using OMB, with source and configuration linked below.

Explore the charts Explore every captured sample Methodology and reproduction

Latest full OMB run · krabka 1.0.5

What the measurements show

Read the source results

Shared page cache

krabka and Kafka use the Linux page cache in this suite. Compare all max-rate workloads, then inspect the differences.

14 workload / replication-factor pairs. 1 measured round per workload. 1 omitted trial across the full suite.

103%of Kafka’s publish rateMedian; workload range 53% to 165%
1.17×Kafka’s CPU efficiency indexHigher in 10 of 14 pairs

CPU efficiency index = publish rate ÷ total broker CPU time, relative to Kafka. CPU capture includes startup, warm-up and shutdown. This is a proxy, not CPU cost per delivered message.

Across published releases

See where the gap changed

The replicated 1 KiB workloads improved relative to Kafka across these runs. Check the full set alongside this group.

Disk preparation can change between runs; each row records SSD trim. These results do not isolate the effect of code changes.

krabka relative to Kafka in the same run
Version1 KiB, RF3All max-rateCPU index
1.0.1Uncontrolled trim75%98%1.24×
1.0.3Uncontrolled trim97%99%1.12×
1.0.5Trimmed SSD103%103%1.17×

100% = Kafka’s rate. 1× = Kafka’s CPU index. Values are medians of workload ratios, not pooled throughput.

Direct I/O: within 3% of Redpanda’s rate in 2 reported rounds

This separate experiment used main with #1373 and #1376 with preallocate=true, against Redpanda 26.2.2.

One RF1, 16-partition, 1 KiB max-rate workload. Trials ran interleaved after SSD trim on ext4 and mirrored NVMe drives. Figures are rounded in the source.

Redpanda had lower tail latency and disk utilization. CPU efficiency was not established for this experiment.

Read the direct I/O investigation
Reported rounds, not a full release qualification
RoundkrabkaRedpanda
1931k/s81 ms E2E p99926k/s53 ms E2E p99
2818k/s95 ms E2E p99838k/s75 ms E2E p99
Inspect every available intervalRead the direct I/O qualification plan

Results apply to this host and these workloads. Similar throughput does not establish equal latency or crash durability.

Workload by workload

Read the pattern across releases

Each cell compares krabka with a broker in the same run. Select a cell to inspect the captured intervals.

100% is the comparison broker. Favorable by more than 5% · within 5% · unfavorable by more than 5%. These bands are visual guides, not statistical significance.

Rates on fixed-load workloads can be capped by offered load. Max-rate workloads give a clearer view of throughput limits.

krabka results as percentages of the selected broker in each release run
Workload1.0.11 round · trim uncontrolled1.0.31 round · trim uncontrolled1.0.51 round · SSD trimmed
max-rate-1-topic-1-partition-1p-1c-1kb74%116%112%
max-rate-1-topic-1-partition-1p-1c-100b100%105%103%
1-topic-3-partition-100b-3producers99%100%93%
max-rate-1-topic-16-partitions-1kb97%123%106%
max-rate-1-topic-16-partitions-100b113%92%53%
max-rate-1-topic-100-partitions-1kb95%110%112%
max-rate-1-topic-100-partitions-100b118%71%165%
max-rate-1-topic-1-partition-1p-1c-1kb75%72%101%
max-rate-1-topic-1-partition-1p-1c-100b92%92%96%
1-topic-3-partition-100b-3producers81%69%94%
max-rate-1-topic-16-partitions-1kb72%98%104%
max-rate-1-topic-16-partitions-100b105%104%115%
max-rate-1-topic-100-partitions-1kb100%97%103%
max-rate-1-topic-100-partitions-100b108%101%87%

Bars share a 0–200% scale; longer values stop at the edge and keep their numeric label. Every workload has equal weight in the median. Missing comparisons are excluded and counted.

Disk preparation changed in the latest run. Cross-release differences include test conditions. The full suite uses krabka’s shared page cache, not its direct I/O mode.

OpenMessaging Benchmark

The upstream OpenMessaging Benchmark workloads, run by its Kafka driver and Kafka 3.6.1 client against every broker: krabka-broker 1.0.5 (ac9deff, signed release), Apache Kafka 4.3.1, and Redpanda 26.2.2. The driver carries one patch, applied identically to every broker, that keeps at most one offset commit in flight, as Redpanda's fork of the benchmark does. All trials ran on one AMD EPYC 4344P 8-Core Processor host with 16 logical CPUs, the brokers one at a time, with fresh storage per trial, four CPUs and 10 GiB per broker, and a separate client. The run merges 3 invocations; RF1 and RF3 (9 workloads) came from invocations that did not finish cleanly, keeping only workloads whose trials had all completed and passed; anything left out was rerun for all three brokers, except the trials listed as not measured below. Each shard's source run and reason are in the provenance. Redpanda was not measured on 1-topic-3-partition-100b-3producers at RF3: Redpanda's trial reached the runner's 20 GiB disk reserve before its 15 minutes ended: with unlimited retention at RF3, its three brokers filled the about 670 GiB that was free above the reserve. The trial was not run again. 9 offset commits failed with a retriable error after a leader or coordinator move and were superseded by the next commit; these are counted, not failed. Run omb-local-2026-10-14, completed ; one round per workload.

Writes are buffered: Kafka and krabka acknowledge in-sync replication with acks=all; Redpanda acknowledges a Raft majority with write.caching=true. This does not establish equal crash durability. OMB reports rates and latency, not exact delivery or duplicate checks. Backlog workloads fill a 100 GB backlog before consumers drain it, so their end-to-end latency includes that wait. CPU and memory windows include client startup, warm-up, rate probing, measurement and shutdown. Results apply only to this runner, these images and these workloads.

Compare the results

Each chart compares the three brokers on one OMB workload. Bars start at zero and use a separate linear scale for each metric. Values are from one measured round.


1 topic · 16 partitions per topic · 1,024-byte messages · Maximum sustainable rate (probed) · 5-minute measurement

Publish rateHigher is better · messages/s
krabka
741,306 messages/s
Apache Kafka
697,199 messages/s
Redpanda
908,991 messages/s
Publish p99 latencyLower is better · ms
krabka
175.35 ms
Apache Kafka
174.07 ms
Redpanda
47.90 ms
End-to-end p99 latencyLower is better · ms
krabka
177.00 ms
Apache Kafka
177.00 ms
Redpanda
52.00 ms

Read the source report or the full tables below. These charts compare aggregate results. Explore interval charts, resources, and trial data. Raw OMB interval data is also retained for this run.

Replication factor 1

Workload Offered krabkaApache KafkaRedpanda
msg/sE2E p99msg/sE2E p99msg/sE2E p99
10,000/s 10,0022.0 ms10,0022.0 ms10,0032.0 ms
50,000/s 50,0042.0 ms50,0042.0 ms50,0052.0 ms
50,000/s 50,0062.0 ms50,0032.0 ms50,0042.0 ms
50,000/s 50,0032.0 ms50,0042.0 ms50,0042.0 ms
100,000/s 100,0051,037.8 s100,0041,038.2 s100,0061,038.0 s
100,000/s 100,0071,077.0 s100,0041,038.2 s100,0051,038.2 s
10,000,000/s 1,139,780104 ms1,016,55196 ms1,054,93243 ms
10,000,000/s 3,593,900111 ms3,497,97479 ms3,973,94145 ms
max (probed) 1,620,371197 ms1,748,596273 ms2,057,76350 ms
max (probed) 741,306177 ms697,199177 ms908,99152 ms
max (probed) 628,12341 ms1,178,43489 ms594,48533 ms
max (probed) 818,326146 ms729,900153 ms897,69753 ms
max (probed) 2,229,882141 ms1,353,44472 ms625,48917 ms

Replication factor 3

Workload Offered krabkaApache KafkaRedpanda
msg/sE2E p99msg/sE2E p99msg/sE2E p99
10,000/s 10,0032.0 ms10,0032.0 ms10,0022.0 ms
50,000/s 50,004476 ms50,004508 ms50,0042.0 ms
50,000/s 50,0042.0 ms50,0042.0 ms50,0072.0 ms
50,000/s 50,00460 ms50,00476 ms50,0042.0 ms
100,000/s 100,0091,037.6 s100,0081,038.9 s100,0041,037.7 s
100,000/s 100,0061,037.2 s100,0071,037.0 s100,0061,037.3 s
10,000,000/s 377,487694 ms374,241713 ms387,738105 ms
10,000,000/s 2,108,784674 ms2,203,941491 ms2,859,077134 ms
max (probed) 1,531,0341.7 s1,622,3711.5 snot measured—
max (probed) 292,399315 ms281,411333 ms371,910102 ms
max (probed) 313,798439 ms271,813468 ms350,888385 ms
max (probed) 336,115368 ms325,701325 ms380,969184 ms
max (probed) 287,511303 ms331,275297 ms364,312262 ms
Publish p99 and pinned images
RF1: publish p99 latency
WorkloadkrabkaApache KafkaRedpanda
1.3 ms1.2 ms1.3 ms
1.3 ms1.3 ms1.3 ms
1.2 ms1.2 ms1.2 ms
1.3 ms1.3 ms1.3 ms
10 ms45 ms1.4 ms
5.1 ms9.3 ms1.5 ms
104 ms94 ms39 ms
111 ms77 ms44 ms
131 ms173 ms2.6 ms
175 ms174 ms48 ms
1.7 ms79 ms1.5 ms
144 ms150 ms48 ms
137 ms67 ms1.6 ms
RF3: publish p99 latency
WorkloadkrabkaApache KafkaRedpanda
1.8 ms1.6 ms1.4 ms
457 ms475 ms1.5 ms
1.7 ms1.6 ms1.4 ms
36 ms48 ms1.6 ms
701 ms872 ms1.7 ms
215 ms146 ms2.1 ms
693 ms712 ms104 ms
673 ms490 ms134 ms
846 ms819 ms—
303 ms316 ms61 ms
3.1 ms20 ms2.0 ms
364 ms319 ms174 ms
3.0 ms51 ms2.1 ms
krabka-broker 1.0.5 (ac9deff, signed release)
ghcr.io/krabka-io/krabka-broker:v1.0.5
ghcr.io/krabka-io/krabka-broker@sha256:2a81d1ba38de4e8ffb032fedf67952845185158a7607ac76ba38fa162dc37e50
Apache Kafka 4.3.1
apache/kafka:4.3.1
apache/kafka@sha256:77e3df9054047a88b520d0cc46e16696d3b22022e1d580aeccd2632df6532837
Redpanda 26.2.2
docker.redpanda.com/redpandadata/redpanda:v26.2.2
docker.redpanda.com/redpandadata/redpanda@sha256:468bd13a9f2bd24794cb7fddc867c767fb1008b9a07b297b89fde48c564d7d96

The run summary lists every trial, and the provenance records the host, CPU sets, OMB commit, client jar hashes and effective driver settings.

Methodology and resource settings

Full runs preserve the upstream workload payloads, partition counts, offered rates, backlog sizes, and durations. The pinned OMB commit, driver patch, immutable broker images, host resources, and effective configurations are recorded in each run’s provenance. Smoke runs are wiring checks and cannot supply website figures.

Publish rates are means of OMB’s measurement intervals; p99 latency comes from OMB’s aggregate latency results. When a run has multiple rounds, the tables show medians across those rounds.

Proposed release gate · not yet automated

Qualify direct I/O before a release

The two reported rounds do not qualify a release. This plan tests each candidate against the previous release and the comparison brokers.

  1. Pin the comparison. Use immutable images for the candidate, previous release, Kafka and Redpanda. Record the OMB commit, hardware, kernel, filesystem and driver settings.
  2. Run both write modes. Run the full OMB catalog at RF1 and RF3, with krabka’s default settings and with preallocate=true. Verify O_DIRECT on open segment descriptors. Record fallback as a failed direct-mode qualification.
  3. Control disk state and repeat. Trim before every trial, confirm free space and use fresh clusters. Run at least five measured rounds per configuration. Rotate broker order across rounds. Keep ext4 and XFS results separate.
  4. Review the distributions. Publish every round, median and range for throughput, publish and end-to-end p99 and p99.9. Align CPU and device telemetry with measurement intervals. Keep full-capture totals separate.
  5. Check correctness and recovery. Check acknowledged records after process termination, leader failover, restart and disk-full errors. Include retention, segment roll, compaction and catch-up reads. Test host power loss separately before a power-loss durability claim.
  6. Make a recorded release decision. Set workload budgets before the run. Proposed review triggers are a throughput drop above 5% or a p99 or CPU-cost increase above 10% against the previous release in the same mode. Repeat flagged pairs; do not claim statistical significance from these thresholds. Block qualification for correctness failures, missing required trials or unresolved regressions.

Publish direct-mode results separately from the default-mode suite. Archive raw OMB files, measurement timestamps, resource samples and all failed trials. A release note must state any qualification gap.

Read the qualification protocol

Repeat the comparison

From this website repository, use Node 22.12 or later, JDK 17 or later, and native Linux/amd64 with local Docker. Check the benchmark instructions for resource requirements and deadlines.

# Resolve and check the pinned images and local prerequisites
npm run benchmark -- --suite openmessaging --dry-run

# Run the upstream OMB workload catalog on a Linux host with Docker
npm run benchmark -- --suite openmessaging

See the benchmark instructions for the upstream catalog, resource requirements, and publication checks.