Skip to main content
krabka
GitHub
Java 17+ · Release v1.4.2

Java Streams & Arrow

Kafka Streams 4.x compatible engine with columnar Apache Arrow operators and schema serdes via krabka-streams-java.

Browse Javadoc Reference

krabka-streams-java is the Java client library for stream processing with krabka. It exports the standard Kafka Streams API and adds native support for Confluent Schema Registry and Apache Arrow direct-memory processing. The library targets Java 17 bytecode, and continuous integration compiles and tests it on JDK 17 and JDK 21.

Architecture & Engineering Guides

In-depth technical guides maintained alongside the Java codebase:

  • Getting Started: Prerequisites, dependency coordinates, and your first streaming pipeline.
  • Architecture: Module graph, dependencies, and how the pieces fit together.
  • Configuration: KrabkaStreamsConfig defaults and the properties it applies.
  • API Reference: Every public type, grouped by module.
  • Columnar Processing: VectorSchemaRoot batch execution, codecs, and topology lifecycle.
  • Columnar Operators: Direct off-heap buffer allocation, try-with-resources, and zero-copy filters.
  • Serdes: Avro, Protobuf, and JSON Schema serdes with non-blocking schema caching.
  • Schema Registry: Confluent Schema Registry client integration, schema evolution, and wire protocol.
  • Barrier Alignment: Synchronous checkpoint cuts across multi-partition consumer topologies.
  • Coordination: Distributed leader election, fencing tokens, and lease succession semantics.
  • Testing: Mock topologies in-memory with ColumnarTestDriver and SchemaRegistryStub.
  • Runtime Constraints: What comes from the runtime and the services the library talks to.
  • Troubleshooting: Common JVM flags, buffer allocator diagnostics, and error resolutions.
  • Build and Release: Hermetic Bazel build targets, Maven Central publishing, and Javadoc generation.

Dependency Coordinates

Gradle Kotlin DSL 
dependencies {
    implementation(platform("io.krabka:krabka-streams-bom:1.4.2"))
    implementation("io.krabka:krabka-streams")
    implementation("io.krabka:krabka-streams-schema-serde")
    implementation("io.krabka:krabka-streams-columnar")
    testImplementation("io.krabka:krabka-streams-test-utils")
}

Maven Central holds release 1.4.2 of all seven modules below, and earlier releases 1.0.0 and 1.1.0 of krabka-streams, krabka-streams-schema-serde, krabka-streams-columnar, and krabka-streams-test-utils. The bill of materials keeps the module versions aligned, so declare it once and leave the individual dependencies without a version.

Published Modules

Artifact Purpose
io.krabka:krabka-streamsApache Kafka Streams API and krabka defaults
io.krabka:krabka-streams-schema-serdeAvro, Protobuf, and JSON Schema serdes
io.krabka:krabka-streams-columnarApache Arrow batch processing and zero-copy operators
io.krabka:krabka-streams-columnar-schemaAvro and Protobuf Arrow memory bridges
io.krabka:krabka-streams-coordinationLeader election, leases, and fencing tokens
io.krabka:krabka-streams-test-utilsColumnarTestDriver and SchemaRegistryStub helpers
io.krabka:krabka-streams-bomBill of Materials dependency constraints

Direct Buffer JVM Requirement

Arrow 19 needs this JVM option when it uses direct buffers:

--add-opens=java.base/java.nio=ALL-UNNAMED

Version & Compatibility

Component Supported Version Notes
Java Bytecode Java 17 or later (CI: JDK 17, JDK 21) Compiled with a Java 17 toolchain; JDK 25 runs only the Javadoc tool
Kafka Wire Protocol Kafka 4.3.1 Transitive api dependency
Apache Arrow 19.0.0 Direct off-heap memory allocators
krabka Broker Apache Kafka 4.3.1 or a krabka broker The streams group protocol must be enabled and streams.version=1 finalized

Full version matrices and SLSA Level 3 build provenance records are tracked on the Versions page.