Cluster Lab

The real brokers run as WebAssembly processes in Workers. Clients and apps run as state machines in the lab module. The page owns their clock and virtual network, so you can pause time, change link latency and cut connections.

The brokers speak the Kafka wire protocol to the clients. Their partition logs and KRaft metadata are kept in this browser's IndexedDB, so a reload reopens the cluster where you left it. The schema registry keeps its schemas where Confluent's does, in the _schemas topic on the brokers, and replays it on every start. Several tabs, on one machine or across the internet, can each host a share of the clients and apps over WebRTC data channels; the brokers stay in the tab that started them.

Try an experiment: when must a write stop?

Can an acks=all producer write when a topic needs three in-sync replicas but only two remain? Predict the result before opening the experiment below.

1. Open Build · Scenarios · Connect, then choose Scenarios to see the experiment checks. Set Speed to 5× to advance the two minutes of lab time faster.
2. Watch the checks: the ISR falls to two, and writes still succeed with the original minimum.
3. When the minimum rises to three, expect NOT_ENOUGH_REPLICAS and no acknowledged writes.
4. After the replica restarts and the ISR returns to three, expect acknowledged writes to resume. The experiment lists each check and its result.

The minimum protects the requested replication level by refusing writes until enough replicas are in sync. This is an experiment in simulated time; it does not measure throughput or disk durability. Next, run a broker locally and produce and consume a record, then use the operations guides before changing a cluster.

What to try

Rebalancer: grow a two-broker cluster starts with six partitions replicated on both brokers. Add a broker from the palette with KRaft voter unchecked to keep the controller quorum unchanged. The rebalancer detects the new broker, moves replicas onto it, and elects preferred leaders. Its inspector shows the plan and per-broker counts; use Pause, Plan now, and Execute now to control the moves.

Share group: three workers, one partition demonstrates workers sharing records from one partition. The broker acquires records for each worker and persists its acknowledgements. Share group: release and redelivery starts a worker that releases records; inspect its delivery counts, then select Accept to complete work or Reject to archive it. Pause keeps heartbeats running, and Close releases unfinished work before leaving. Share consumers use record acknowledgements instead of consumer offset commits.

For authenticated, encrypted virtual links, open Scenario settings and choose Kerberos authentication + encryption under SSPI / Kerberos transport. Pure Rust SSPI runs in the browser with an embedded KDC and lab-only node identities. The Events tab records authenticated peers and failures; Network → Frames shows Kerberos tokens and encrypted bytes. Changing the mode restarts nodes on their existing disks. These demonstration credentials do not use Windows single sign-on or Active Directory. In an ACL scenario, the adapter binds the verified node identity to the broker’s SASL principal before delivering requests. This transport uses SSPI message protection rather than TLS.

Open Broker ACLs: topic and group permissions to see a producer and billing consumer succeed while a consumer without permission for group audit receives GROUP_AUTHORIZATION_FAILED. Broker ACLs: prefixed allow, explicit deny adds a producer whose prefixed grant is overridden by a literal deny; the broker returns TOPIC_AUTHORIZATION_FAILED. Both use the broker’s default-deny authorizer and replicated Kafka ACL records. Edit Broker ACL policy (JSON) in Scenario settings or authorization.acls in an exported scenario; clearing the settings field disables ACL enforcement. Brokers and the scenario admin are superusers. Client traffic waits until every broker confirms the policy.

Load Three brokers, a producer and a consumer group from the presets and set the speed to 5×. The three brokers form one KRaft quorum; its leader becomes the active controller, registers the brokers, unfences them once they have caught up with the metadata log. The scenario admin creates orders. The producer's records flow through replicated partitions to two KIP-848 consumers that share them. Select a broker: its inspector shows process logs and runtime activity, and Browse disk opens its stored files. Select the leader of a partition and Kill it: once its 9 s session runs out, the controller fences it and moves the leadership to another replica in the ISR, and the consumers keep reading. Restart it and it catches up and rejoins the ISR.

Schema registry with an Avro producer and a decoding consumer starts a registry that creates its _schemas topic on the brokers; the producer registers a schema before the first record, and the consumer decodes every value by its schema id. Add a second registry: the two elect a primary through their group, and the secondary forwards the writes it takes to the primary. Kafka Streams word count joins a KIP-1071 streams group, counts words in a state store backed by the changelog topic the group creates, and answers a store query from the command bar. Other presets cover independent consumer groups, a broker observer, minimum in-sync replicas, slow links and a rack split.

Saving, sharing, hosting

The scenario autosaves to this browser; the Saved in this browser list on the Scenarios tab reopens it with its durable state. The Storage tab under the canvas shows what is kept per node and can forget it on request. Browse opens a real broker's disk, read from its committed IndexedDB bytes. Export writes the scenario as a JSON file, Copy link puts the whole document into the URL. Neither carries node state: a shared link starts the cluster fresh.

Invite a tab in the Session section of the Connect tab makes a link that another tab or another machine opens to join. That page shows an answer code; paste it back into the host and the two are connected without any server. The host then assigns clients and apps to peers from the inspector; frames between tabs travel over the WebRTC data channel, and each tab persists the state of the nodes it hosts. A real broker stays on the host, with its disk.

How the brokers run

krabka broker in the Build tab adds the real krabka-broker, compiled for wasm32-wasip1, as a node of the lab. It runs in a Web Worker on a browser WASI runtime: its listeners, its dials to other nodes and its clock are the lab's, so links, faults, pause and speed apply to it like to any node, and its disk is a volume in this browser's IndexedDB. Every preset has brokers, so the first visit downloads the broker module, about 14 MB, and the page shows the progress on each broker node; later visits take it, and the lab module, from this browser's cache until the site ships a new build. The runtime also needs a cross-origin isolated page. Any page of this site sets that up in the background, so a visit that starts elsewhere on the site opens the lab isolated; landing on the lab first, the page reloads once, then reopens the scenario. A real broker stays in the tab that runs it: its process cannot move to another tab of a session. If the broker module is missing from the site, or the browser cannot isolate the page, the node says so and refuses connections.

Logs

The Logs tab under the canvas shows what the broker processes write to stderr, one JSON object per line: the lab time, the level, the node, the target that logged it and its message. Click a row, or press Enter, to open the whole record as a JSON tree. Filter by minimum level, by node and by target, or search the message and fields; node:2 level:warn style terms match a field's value. The list follows the newest line until you scroll up, and a pill says how many new lines are waiting. Download saves the lines on screen as NDJSON. A line that is not JSON, such as a panic message, is shown as it is, marked raw. The page keeps up to 5,000 lines per broker and forgets them on reload. Starts, exits, kills and restarts show up in the stream as marker rows.

Log levels sets how much a broker logs: Quiet (warn), Normal (the default, info), Verbose (debug), Trace (trace), or a directive such as warn,krabka_broker=debug, for all brokers or for one. The level is the broker's KRABKA_LOG environment variable, read when its process starts, so applying a new level restarts the broker on the disk it has, after asking. It is a setting of this browser, not part of the node's configuration, which is why it does not wipe the disk.

Network

The Network tab records every frame the virtual network carries, from the moment the scenario starts: opens, closes and up to 64 KiB of each payload. It pairs each Kafka request with its response by connection and correlation id. Exchanges lists the pairs, Frames lists everything, and Statistics gives per-API counts, failures, bytes and round-trip percentiles with a histogram, and per-link bytes and throughput. Times are lab milliseconds. RTT runs from the request leaving the client to the response reaching it; server runs from the request reaching the broker to the response leaving it; the rest is the link, both ways.

A selected exchange decodes field by field from the Kafka message schemas krabka-protocol generates its codecs from: the headers, every request and response version, record batches with their CRC-32C recomputed and their records (gzip, snappy, lz4 and zstd opened in the page), and krabka's own controller RPCs by name. The filter takes words and terms such as api:fetch, node:broker-1, port:9093, err, slow:50 and pending. pcapng exports what the filter shows as raw IPv4 and TCP between the nodes' 10.0.x.y addresses, timestamped in lab milliseconds. Wireshark opens it and its Kafka dissector decodes port 9092 (Decode As maps 9093 too), up to the API versions that Wireshark release knows; JSON keeps every frame's bytes and CSV one row per exchange. The status line says how many frames the capture lost: dropped before the page read them, evicted to stay within its 64 MiB, or skipped while capture was paused.

Cluster state, charts and invariants

The scenario's admin node polls the brokers once a lab second. The Cluster tab shows what it last saw: the KRaft quorum with its leader, epoch and each voter's lag, the brokers with their rack, whether they are fenced and how many partitions each leads and hosts, and every partition with its leader, leader epoch, replicas, ISR, high watermark, log start and each consumer group's lag. Replicas that fell out of the ISR are struck through. Offline and under-replicated partitions sort first; internal topics such as __consumer_offsets hide behind a toggle. The canvas marks the quorum leader's card controller, counts the partitions each broker leads and hosts, and puts one chip per partition under a topic, coloured by health, with its leader's id.

The Charts tab samples the run once a lab second: records acknowledged and consumed per second, total consumer lag, Produce and Fetch round trips (p50 and p99, from the network capture), the smallest ISR and the number of under-replicated partitions. Every fault, kills and restarts included, is a labelled line on every chart, on the same lab-time axis, and hovering reads all the charts at one instant. Pin as baseline keeps the series in this browser; Rerun opens the scenario again from nothing with its current configuration, so you can change one setting, rerun and see the new run over the old one, dashed.

The same tab checks invariants on every snapshot, and the toolbar says whether they hold: no record a producer with acks=all saw acknowledged is lost (the partition's high watermark must stay above it; a new leader's may stand lower while its ISR catches up or shrinks, so it counts as a loss after 60 s of lab time, or at once when the partition is gone), consumer positions only move forward unless an admin reset them, one broker leads a partition per leader epoch, the KRaft quorum has one leader per epoch, and a read_committed consumer never receives a record of an aborted transaction. A violation is listed with its lab time and drawn as a red line on the charts.

Trace, on a row of a producer's Last records, follows that record through the capture: the Produce request that carried it, each follower's Fetch that copied it, the first Fetch response whose high watermark passed it, the consumer's Fetch that delivered it and the OffsetCommit past it, each with its lab time, its two nodes and a button that selects the exchange in the Network tab. A step that is missing says why: it has not happened yet, or it fell outside what the capture kept.

Process faults, experiments and forks

With one node selected, the Break things bar adds Pause: the node stops as under SIGSTOP or a long GC pause. A real broker's worker gets no input, socket readiness or timers; what is sent to it waits in its socket buffers, and Resume lets it catch up at the time it is. Paused for longer than the clients' 30 s request timeout, its clients drop their connections and reconnect. More… skews a real broker's wall clock (REALTIME; its monotonic clock is untouched, so the files it writes carry the skewed time) and sets its disk to slow (each fsync takes that many lab milliseconds), full (writes that grow a file fail with ENOSPC) or failing (writes and syncs fail with EIO). The krabka broker does not fsync on produce, so a slow disk shows when it starts and recovers its log; a full or failing disk makes it log the error and stop. With two nodes selected, Cut → and Cut ← drop the frames one way only: connections stay open until a peer times out. The canvas menu has the same faults.

A scenario can carry an experiment: timed faults and commands, and checks decided at a lab time or required by one, such as an ISR size, a consumer's lag, an event, a Kafka error code in a response, a dotted path into a node's state, or that the invariants above hold. The Scenarios tab edits it as JSON and validates it; Run experiment restarts the scenario on fresh disks, runs the script and lists each step and check with its time and result. The experiment travels with Copy link and Export. Fork here pauses the clock, copies the brokers' disks into a new saved scenario and opens it: the brokers boot from the copies, the clients start fresh. Three presets carry an experiment; each link opens it in the lab and runs it.

A broker's disk

Browse in the Storage tab, or Browse disk in a broker's inspector, opens its log directory by partition. A partition lists its segments and checks them together: every batch CRC, segment names against their first offsets, offsets across segment boundaries, and the leader-epoch checkpoint against the batches. A segment shows its batches as a byte map and a table, then the decoded batches and records; the KRaft metadata log decodes into its metadata records, and __consumer_offsets into offset commits. An offset or time index is checked entry by entry against the batch it points at. Producer-state snapshots, checkpoints and the quorum state decode too.

Connect kafkactl from your computer

Open the Connect tab in the left rail and expand Connect with kafkactl. This is a krabka build of fgrosse/kafkactl v1.4.0 with the bridge command added. Download it for your computer, extract it, and run kafkactl lab bridge. Enter the token it prints in the lab. The browser may ask to allow access to the local network. The bridge listens only on this computer. In a second terminal, add the krabka-lab context using the command shown in the panel, then copy the broker, topic, produce, and consume commands. The single broker quickstart has an orders topic ready to use. Keep the lab clock playing while the commands run.

What is real

The brokers run the krabka-broker process compiled for wasm32-wasip1. The page supplies their clock, network and disk. Clients and apps run in the lab's wasm32-unknown-unknown module.

What is simulated: the network (per-link latency, loss and cuts under the seed), disks (IndexedDB stands in for the log directory), and time. The seed fixes the network, the clients and the apps; the broker processes draw real randomness, so two runs on one seed need not play out the same way. The lab makes no performance claims; the numbers on screen are counts and simulated milliseconds.