Runtime Architecture
What runs where once the cluster is up.
Krabka is one broker core with a set of operational services around it. The broker speaks the Kafka wire protocol, stores records in Kafka-compatible log segments, and keeps metadata in a native KRaft quorum. Every other component connects to that core the way any Kafka client does. None of them reaches into broker internals.
The Ecosystem Architecture page covers the source layout, which repository holds what. This page covers the running system.
Inside the broker
| Subsystem | What it owns |
|---|---|
| Kafka request dispatch | Decodes every Kafka API request, authorizes it, applies quotas, and routes it to the subsystem that answers it. |
| Log segments | Kafka-compatible segment files, indexes and retention. Fetches are served zero-copy where the listener allows it. |
| Group and transaction coordinators | Consumer groups, share groups and the transaction state machine, all on internal topics. |
| KRaft metadata quorum | The native Raft quorum that owns cluster metadata. No ZooKeeper and no JVM. |
Kafka clients and the JVM kafka-*.sh tools connect
to request dispatch. Dispatch reads and writes the log, drives the coordinators, and reads
cluster metadata from the quorum.
Services around the broker
| Service | Speaks | What it does |
|---|---|---|
| krabka-operator | Kubernetes API and the broker | Turns custom resources into StatefulSets, Services, ConfigMaps, Secrets, certificates, broker configuration, topic and user reconciliation, rolls and rebalances. |
| krabka-schema-registry | Kafka wire protocol | A Confluent-API-compatible registry. It keeps every schema in the compacted _schemas topic, so the topic is the database. |
| krabka-rebalancer | Kafka wire protocol | Answers a rebalance request with a partition assignment plan, the way Cruise Control does for Apache Kafka. |
| krabka-connect | Kafka wire protocol | The connector framework, the PostgreSQL CDC source, the worker runtime and the cluster replicator. |
| krabka-client-rs | Kafka wire protocol | The native Rust producer, consumer and admin client. |
The Kubernetes control plane
On Kubernetes the operator is the only component that talks to the Kubernetes API. It watches
the namespaces you grant it and reconciles every Krabka resource it finds. A
Kafka resource owns the cluster. One or more
KafkaNodePool resources supply the brokers, and
the broker count is the sum of the pool replica counts.
| Custom resource | What it declares |
|---|---|
| Kafka | A broker cluster: version, cluster-wide configuration, and the owner of the Services, ConfigMap, Secrets and CA. |
| KafkaNodePool | A StatefulSet of brokers: roles, replicas, storage and resources, bound to a cluster. |
| KafkaTopic | A topic and its configuration. |
| KafkaUser | SCRAM or mTLS credentials, ACLs and quotas for one principal. |
| KafkaRebalance | A partition rebalance request. |
| SchemaRegistry | A schema registry bound to a cluster. |
The CRD reference lists the fields of each resource. The operator page covers the reconcile loop.
Where to go next
- Run a cluster locally: Quickstart.
- Run one on Kubernetes: Kubernetes Operator.
- Watch the quorum work: Consensus Playground.
- Pick a serde: Serdes and Data Formats.