Package io.krabka.streams.columnar.schema

package io.krabka.streams.columnar.schema
Avro and Protobuf bridges into the Arrow columnar runtime.

This package connects the schema registry serdes to columnar topologies. The batch codecs, AvroBatchCodec and ProtobufBatchCodec, decode registry-framed records into Arrow batches whose columns follow the record schema — structs, lists, maps, decimals, and timestamps arrive as native Arrow types instead of the JSON text fallback of JsonRowBridge. The row bridges, AvroRowBridge and ProtobufRowBridge, expose the same conversion through the RowBridge interface for composition with your own serdes, and AvroArrowSchemas and ProtobufArrowSchemas translate schemas without touching data.

Every bridge derives its Arrow schema once, at construction, from the fixed reader schema or message descriptor. Records written with other registered writer schemas are resolved onto that reader view by the embedded serde, so every batch of a topology carries the same Arrow schema regardless of mid-stream schema evolution.

  • Classes
    Class
    Description
    AvroArrowSchemas
    Translates Avro schemas into Arrow schemas.
    AvroBatchCodec
    Decodes registry-framed Avro topics into Arrow batches and encodes them back.
    AvroRowBridge<T extends org.apache.avro.generic.IndexedRecord>
    Converts Avro records to and from Arrow batches with schema-derived columns.
    ProtobufArrowSchemas
    Translates Protobuf message descriptors into Arrow schemas.
    ProtobufBatchCodec
    Decodes registry-framed Protobuf topics into Arrow batches and encodes them back.
    ProtobufRowBridge<T extends com.google.protobuf.Message>
    Converts Protobuf messages to and from Arrow batches with descriptor-derived columns.