Class RowCodec<T>

java.lang.Object
io.krabka.streams.columnar.RowCodec<T>
Type Parameters:
T - the record value type
All Implemented Interfaces:
BatchCodec
public final class RowCodec<T> extends Object implements BatchCodec
Converts ordinary Kafka value records through a typed row bridge.

Use this codec when each record holds one value and you want a columnar view of a topic. Decoding deserializes every value with the supplied Serde, converts the values into Arrow columns through the RowBridge, and appends the reserved metadata columns. Encoding reverses it: payload columns become typed rows, each row becomes one record, and the key, timestamp, and headers are taken from the __key, __timestamp, and __headers columns. Row count is preserved in both directions.

The topic passed through decode(String, List) and encode(String, VectorSchemaRoot) is forwarded to the serde, so topic-derived schema registry subjects work the same way as in ordinary consumers and producers.

Example

record Transaction(String user, long amount) {}

var codec = new RowCodec<>(
    transactionSerde, new JsonRowBridge<>(Transaction.class), allocator);

try (var batch = codec.decode("transactions", records)) {
    // one row per record; columns "user" and "amount" plus metadata
}

  • Constructor Summary

    Constructors
    Constructor
    Description
    RowCodec(org.apache.kafka.common.serialization.Serde<T> valueSerde, RowBridge<T> rowBridge, org.apache.arrow.memory.BufferAllocator allocator)
    Creates a codec from a value serde and a row bridge.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.arrow.vector.VectorSchemaRoot
    decode(String topic, List<ConsumedRecord> records)
    Decodes records, forwarding the topic to the value serde.
    org.apache.arrow.vector.VectorSchemaRoot
    decode(List<ConsumedRecord> records)
    Decodes records with an empty topic name.
    List<ProduceRecord>
    encode(String topic, org.apache.arrow.vector.VectorSchemaRoot batch)
    Encodes a batch into one record per row, forwarding the topic to the serde.
    List<ProduceRecord>
    encode(org.apache.arrow.vector.VectorSchemaRoot batch)
    Encodes a batch with an empty topic name.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • RowCodec

      public RowCodec(org.apache.kafka.common.serialization.Serde<T> valueSerde, RowBridge<T> rowBridge, org.apache.arrow.memory.BufferAllocator allocator)
      Creates a codec from a value serde and a row bridge.
      Parameters:
      valueSerde - the serde that reads and writes record values
      rowBridge - the bridge between values and Arrow columns
      allocator - the allocator that owns decoded batches

  • Method Details

    • decode

      public org.apache.arrow.vector.VectorSchemaRoot decode(List<ConsumedRecord> records)
      Decodes records with an empty topic name.
      Specified by:
      decode in interface BatchCodec
      Parameters:
      records - the records of one topic partition batch
      Returns:
      the decoded batch; the caller must close it

    • decode

      public org.apache.arrow.vector.VectorSchemaRoot decode(String topic, List<ConsumedRecord> records)
      Decodes records, forwarding the topic to the value serde.
      Specified by:
      decode in interface BatchCodec
      Parameters:
      topic - the topic the records were fetched from
      records - the records of one topic partition batch
      Returns:
      the decoded batch, one row per record, with metadata columns appended; the caller must close it

    • encode

      public List<ProduceRecord> encode(org.apache.arrow.vector.VectorSchemaRoot batch)
      Encodes a batch with an empty topic name.
      Specified by:
      encode in interface BatchCodec
      Parameters:
      batch - the batch to encode; the codec reads it and leaves it open
      Returns:
      the encoded records in row order

    • encode

      public List<ProduceRecord> encode(String topic, org.apache.arrow.vector.VectorSchemaRoot batch)
      Encodes a batch into one record per row, forwarding the topic to the serde.

      A missing or null __key becomes a null record key, and a missing or null __timestamp becomes 0.

      Specified by:
      encode in interface BatchCodec
      Parameters:
      topic - the topic the records will be produced to
      batch - the batch to encode; the codec reads it and leaves it open
      Returns:
      the encoded records in row order