Migrating from Apache Kafka

What changes, what keeps working, and how to move a running cluster.

krabka serves the Apache Kafka wire protocol, so applications move by changing their bootstrap address. The work is in what sits beside the protocol: JMX, broker plugins, and the data already in your cluster. From 1.0.0, a krabka-broker data directory stays readable across 1.x releases; check the KIP matrix for the features your applications use before you plan a cutover.

How krabka differs from Apache Kafka

Area	Apache Kafka	krabka
Wire protocol	Kafka protocol	The same protocol, byte-compatible. Clients, the admin CLI and librdkafka work unchanged.
Runtime	JVM	One native Rust binary. No heap tuning and no garbage collector.
Metadata	KRaft quorum	A native KRaft quorum (KIP-595 RPCs). No ZooKeeper.
Metrics	JMX MBeans	Prometheus on /metrics. No JMX port; each series names the MBean it replaces.
Broker plugins	JVM classes loaded into the broker	None. Custom authorizers, metrics reporters and interceptors have nothing to load into.
Correctness evidence	Tests	Tests plus Creusot-proved decision kernels and Stateright model checks.

For OpenMessaging publish rates and latency against Kafka and Redpanda, see the benchmarks.

Which tools keep working

The line is the interface a tool reads. A tool that speaks the Kafka protocol works: the kafka-*.sh admin commands, every client library, lag monitors such as Burrow and kafka-lag-exporter, dashboards such as AKHQ, Kafdrop and Redpanda Console, MirrorMaker 2, Kafka Connect workers and schema registries.

A tool that needs a broker-side JVM does not: Cruise Control (use krabka-rebalancer), JMX exporters and jmx_prometheus_javaagent (scrape /metrics), and the JMX views of CMAK or Conduktor. Tools with both a protocol path and a JMX path keep the protocol path. The full list is in ecosystem-support.md.

Move the data with MirrorMaker 2

MirrorMaker 2 treats krabka as an ordinary Kafka cluster: run the stock connect-mirror-maker.sh from a Kafka release and install nothing on either side. The broker's test suite runs this procedure with MirrorMaker 2 from Kafka 4.3.1 on every change.

What	Carried over?
Records, with keys, values, headers and timestamps	Yes
Topic partition counts and topic configs	Yes, with sync.topic.configs
Consumer-group positions	Yes, translated, with sync.group.offsets.enabled=true
Topic ACLs	Yes, when the source has an authorizer and sync.topic.acls is on
Offsets as numbers	No: target offsets differ; read positions from krabka after cutover
Transactional state and producer ids	No: restart transactional producers against krabka
SCRAM credentials, delegation tokens, quotas	No: create them on krabka first
Schema registry subjects	No: use the registry’s own export, or mirror _schemas and cut the registry over

A minimal mm2.properties mirroring a cluster aliased prod:

clusters = prod, krabka
prod.bootstrap.servers = kafka-1.example.com:9092
krabka.bootstrap.servers = krabka-1.example.com:9092

prod->krabka.enabled = true
prod->krabka.topics = .*
prod->krabka.groups = .*

sync.topic.configs.enabled = true
sync.group.offsets.enabled = true
sync.group.offsets.interval.seconds = 10
emit.checkpoints.interval.seconds = 10

Set sync.group.offsets.enabled to true; without it every migrated consumer starts from the beginning or the end. Mirrored topics carry the source alias as a prefix (prod.orders) unless you choose IdentityReplicationPolicy before the first record arrives.

Cut over

Wait until MirrorMaker 2's consumer lag on the source stays near zero.
Move consumers one group at a time: stop the group on Kafka, wait one sync.group.offsets.interval.seconds, check its position on krabka with kafka-consumer-groups.sh --describe, then start it against krabka and confirm its lag falls.
Move producers last: stop them on Kafka, let the mirror drain, start them against krabka, then stop MirrorMaker 2.

Moving producers before their consumers splits a topic across two clusters. The step-by-step procedure, including ACL sync and cleanup, is migrate-from-kafka.md.