Class GzipBatchCodec

java.lang.Object
io.krabka.streams.columnar.GzipBatchCodec
All Implemented Interfaces:
BatchCodec
public final class GzipBatchCodec extends Object implements BatchCodec
Adds per-record GZIP compression to any batch codec.

Encoding compresses each produced record's value after the delegate has encoded it; decoding decompresses each fetched record's value before the delegate sees it. Keys, timestamps, and headers pass through unchanged. Decompression is capped to guard against decompression bombs: a record that inflates past maxUncompressedBytes throws ColumnarException.

Example

var codec = new GzipBatchCodec(new BlobCodec(allocator));
topology.addSource("source", List.of("compressed-transactions"), codec);

  • Field Details

    • DEFAULT_MAX_UNCOMPRESSED_BYTES

      public static final int DEFAULT_MAX_UNCOMPRESSED_BYTES
      The default decompression ceiling: 16 MiB per record.
      See Also:

  • Constructor Details

    • GzipBatchCodec

      public GzipBatchCodec(BatchCodec delegate)
      Wraps a codec with the 16777216-byte ceiling.
      Parameters:
      delegate - the codec that handles the uncompressed record values

    • GzipBatchCodec

      public GzipBatchCodec(BatchCodec delegate, int maxUncompressedBytes)
      Wraps a codec with an explicit decompression ceiling.
      Parameters:
      delegate - the codec that handles the uncompressed record values
      maxUncompressedBytes - the largest value a record may decompress to
      Throws:
      IllegalArgumentException - if the ceiling is not positive or is Integer.MAX_VALUE

  • Method Details

    • decode

      public org.apache.arrow.vector.VectorSchemaRoot decode(List<ConsumedRecord> records)
      Decompresses record values, then decodes them with an empty topic name.
      Specified by:
      decode in interface BatchCodec
      Parameters:
      records - the records of one topic partition batch
      Returns:
      the delegate's decoded batch; the caller must close it

    • decode

      public org.apache.arrow.vector.VectorSchemaRoot decode(String topic, List<ConsumedRecord> records)
      Decompresses record values, then delegates decoding.
      Specified by:
      decode in interface BatchCodec
      Parameters:
      topic - the topic the records were fetched from
      records - the records of one topic partition batch
      Returns:
      the delegate's decoded batch; the caller must close it
      Throws:
      ColumnarException - if a value is not valid GZIP or inflates past the ceiling

    • encode

      public List<ProduceRecord> encode(org.apache.arrow.vector.VectorSchemaRoot batch)
      Encodes with an empty topic name, then compresses each record value.
      Specified by:
      encode in interface BatchCodec
      Parameters:
      batch - the batch to encode; the codec reads it and leaves it open
      Returns:
      the compressed records in the delegate's order

    • encode

      public List<ProduceRecord> encode(String topic, org.apache.arrow.vector.VectorSchemaRoot batch)
      Delegates encoding, then compresses each record value.
      Specified by:
      encode in interface BatchCodec
      Parameters:
      topic - the topic the records will be produced to
      batch - the batch to encode; the codec reads it and leaves it open
      Returns:
      the compressed records in the delegate's order