Skip to main content

Module kgrouped

Module kgrouped 

Source
Expand description

KGroupedStream<K,V>: the handle between a group op and an aggregation.

The group ops are groupByKey and groupBy. The terminal aggregations are count, reduce, and aggregate.

groupByKey and groupBy record no graph node. They capture four things: whether the upstream was key-changing, the optional Grouped name, the value serdes for the repartition topic, and a typed repartition-lowering thunk. The terminal aggregation then does three steps:

  1. It mints the store name at the JVM counter position. The name comes from Materialized, or from a fresh KSTREAM-AGGREGATE-STATE-STORE- or KSTREAM-REDUCE-STATE-STORE- counter.
  2. If the upstream is key-changing, it records a Repartition node. That node’s thunk lowers sink → add_repartition_topic → source, so the runtime re-groups records by key before the aggregation. This splits the topology into 2 subtopologies.
  3. It records an Aggregate node whose thunk adds the aggregation processor and its state store.

Repartition and changelog topic names follow the JVM naming rules, so the client can send the built topology through StreamsGroupHeartbeat.

Structs§

KGroupedStream
Handle produced by groupByKey and groupBy. A terminal aggregation consumes it.