Kafka KIP compatibility matrix

This matrix records what the tree claims about Kafka compatibility and where the evidence for each claim is: one row per KIP that the roster in kip_inventory lists or that any file under crates/ names, then the repository’s JVM differential evidence, which is the Kafka on-disk log contract and the ApiVersions table every client negotiates against.

KIP status

Every row comes from KIP_ANNOTATIONS in api_catalog, and two sources decide which rows have to be there. The generator scans crates/ for KIP-<n> mentions, except the crate CHANGELOGs, and fails when a KIP is named without a row here, so a compatibility claim cannot enter src/ without evidence beside it. It also reads kip_inventory, the checked-in roster of the KIPs the compatibility target covers, and fails when one of them has no row – the direction the scan cannot see, because code that mentions its KIP in no comment would otherwise go unadvertised here. A row for a KIP that neither the roster nor crates/ names fails as well, and so does a row whose owner or test does not exist, or a path::function entry naming a function the file does not define.

• Status is Implemented, Partial (the note says what is missing), or Out of scope (the note cites the decision).
• Owner is the module that holds the behavior.
• Tests establish the status. A test under src/ is a unit or model test.
• Kafka image is what the listed suites run against, read from each crate’s BUILD.bazel docker map and pinned below; in process means no suite in the row starts a Kafka container.
• librdkafka is the non-JVM client evidence, and it is read from the row’s own tests: a row is credited with a client exactly when a test it lists in librdkafka_conformance drives that client. The suite’s CLIENTS table names each one, and the cell names the test that ran it, so a row citing the test that drives only the librdkafka 2.x image is credited with that image and not the older one.

Kafka images

On-disk log contract

The suite covers timestamped record batches (KIP-32), segment and index discovery, and record key/value recovery in both directions. It ran against mirror.gcr.io/confluentinc/cp-kafka:6.1.1@sha256:537c303f182ff6ba665f324520105c44970c97d7ef94b306709a5d68b8c1aed7.

Advertised API versions

kafka-broker-api-versions read both tables below. api_versions_differential asserts krabka’s against api_catalog and records the join in divergence.json, which is where these two version columns come from. A range that moves, or a key that appears on one side only, changes that file and so changes this table.

The oracle is mirror.gcr.io/apache/kafka:4.3.1@sha256:ccd1314e47ec76909e01f86308b4dcf2064f19f7c89759234322314b0e319e26.

It is a stock single-node KRaft broker, read on its PLAINTEXT broker listener. krabka is read the same way: one client listener, no client-telemetry receiver, and both version switches at their defaults. unstable.api.versions.enable is Kafka’s own; legacy_request_versions_enable is krabka’s. With those defaults krabka advertises exactly Kafka 4.3.1’s table, so every row below should match. A krabka only or Kafka only verdict would mean the two brokers disagree on which keys exist.

What krabka serves past 4.3.1 is opt-in, and so it does not appear here. Kafka trunk’s ApiVersions v5, TxnOffsetCommit v6, the streams v1 pair, and api keys 93 and 94 need unstable.api.versions.enable. The pre-4.0 Fetch, ListOffsets and Produce versions need legacy_request_versions_enable. The KIP rows above name each one.

A row whose ranges differ carries the reason krabka means to differ, recorded beside the ranges themselves in RANGE_DIVERGENCE_INTENTS (divergence.rs). The suite panics on a range divergence with no entry there, so a range that moves cannot reach this page unexplained. Every other verdict leaves the column empty.

The Bazel Docker lane supplies each image from a digest-pinned OCI repository. CI regenerates this page and fails when the test names, lane, image tag, digest, recorded API versions, or checked-in output drift.

KIP-219 throttle-echo divergences

The broker reports a request-quota delay by patching the leading ThrottleTimeMs int32 of an already-encoded response, so it can only echo the delay on responses whose schema puts that field first. Every advertised API is audited by throttle_audit, which encodes each response at each advertised version and compares where ThrottleTimeMs lands against the broker’s table.

The APIs below carry ThrottleTimeMs behind another field, so the patch cannot reach it. That is a statement about the schema, not about what a client observes: whether a request-quota delay is ever applied to one of these APIs is decided separately, by its dispatch entry’s RequestQuotaPolicy. The two columns are kept apart for that reason.

• SelfAccounted entries charge the quota in the handler and set ThrottleTimeMs on the typed response before encoding, so the buried field costs nothing and the client does see the delay.
• ApplyFallbackAccounting entries are the ones the dispatch loop charges and delays. For the buried-field APIs below, the dispatch loop decodes the response, sets ThrottleTimeMs and encodes it again, so the client does see the delay.
• InlineExempt entries are the APIs Kafka answers through sendResponseExemptThrottle (WriteTxnMarkers, AlterPartition, FetchSnapshot, UpdateRaftVoter) and the SASL handshake APIs, which Kafka answers in the authenticator. They are never delayed by the request quota on the ordinary path.

recorded_reach_matches_the_dispatch_registry in the audit pins the last column against the assembled dispatch registry, so a policy change on any of these APIs fails the build until this page is regenerated.