Class ProtobufRowBridge<T extends com.google.protobuf.Message>

java.lang.Object
io.krabka.streams.columnar.schema.ProtobufRowBridge<T>
Type Parameters:
T - the message type the bridge converts
All Implemented Interfaces:
RowBridge<T>
public final class ProtobufRowBridge<T extends com.google.protobuf.Message> extends Object implements RowBridge<T>
Converts Protobuf messages to and from Arrow batches with descriptor-derived columns.

The Arrow schema comes from ProtobufArrowSchemas.toArrowSchema(Descriptors.Descriptor) once, at construction, so every batch — including an empty one — carries the same columns. Nested messages become Struct columns, repeated fields List, map fields Map, and google.protobuf.Timestamp a UTC microsecond timestamp; sub-microsecond nanos are truncated on conversion. Fields without presence write their default value, so a proto3 implicit scalar that was never set and one set to its default are indistinguishable after a round trip.

Combine the bridge with a ProtobufSerde in a RowCodec to read a registry-framed topic, or use ProtobufBatchCodec, which packages exactly that composition.

Example

var bridge = ProtobufRowBridge.of(Order.getDefaultInstance());
try (var batch = bridge.rowsToBatch(List.of(order), allocator)) {
    List<Order> back = bridge.batchToRows(batch);
}

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.arrow.vector.types.pojo.Schema
    arrowSchema()
    Returns the Arrow schema every batch of this bridge uses.
    List<T>
    batchToRows(org.apache.arrow.vector.VectorSchemaRoot batch)
    Converts an Arrow payload batch back into one message per row.
    static <T extends com.google.protobuf.Message>
    ProtobufRowBridge<T>
    of(T defaultInstance)
    Creates a bridge for one message type.
    org.apache.arrow.vector.VectorSchemaRoot
    rowsToBatch(List<T> rows, org.apache.arrow.memory.BufferAllocator allocator)
    Builds one Arrow batch with one row per message.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • of

      public static <T extends com.google.protobuf.Message> ProtobufRowBridge<T> of(T defaultInstance)
      Creates a bridge for one message type.
      Type Parameters:
      T - the message type
      Parameters:
      defaultInstance - the default instance of the message type, for example Order.getDefaultInstance()
      Returns:
      a bridge that converts messages of the instance's type
      Throws:
      ColumnarException - if the descriptor uses an unsupported shape

    • arrowSchema

      public org.apache.arrow.vector.types.pojo.Schema arrowSchema()
      Returns the Arrow schema every batch of this bridge uses.
      Returns:
      the Arrow schema derived from the message descriptor

    • rowsToBatch

      public org.apache.arrow.vector.VectorSchemaRoot rowsToBatch(List<T> rows, org.apache.arrow.memory.BufferAllocator allocator)
      Builds one Arrow batch with one row per message.
      Specified by:
      rowsToBatch in interface RowBridge<T extends com.google.protobuf.Message>
      Parameters:
      rows - the messages to convert, in record order
      allocator - the allocator that owns the batch's buffers
      Returns:
      the payload batch; the caller must close it
      Throws:
      ColumnarException - if a value cannot be converted

    • batchToRows

      public List<T> batchToRows(org.apache.arrow.vector.VectorSchemaRoot batch)
      Converts an Arrow payload batch back into one message per row.
      Specified by:
      batchToRows in interface RowBridge<T extends com.google.protobuf.Message>
      Parameters:
      batch - the payload batch to convert; the bridge reads it and leaves it open
      Returns:
      the messages in row order
      Throws:
      ColumnarException - if a row cannot be converted back to the message type