Expand description
Cross-run aggregation, the “accurate averages” layer.
A benchmark cell is one CellKey: a scenario name with the whole
topology it ran at. The harness runs
each cell N times per stack. This module reduces those N RunOutputs
into a mean ± sample-stddev per metric, and averages the per-interval time
series across runs at each time offset. Both the Markdown summary and the
Plotly graphs read these aggregates, so one place computes “the average” and
no renderer re-derives it.
Structs§
- CellAgg
- One benchmark cell. It holds a
(scenario, topology)with the krabka and kafka aggregates side by side. - CellKey
- What makes two runs comparable: one scenario at one topology.
- Scalar
Metric - A headline scalar metric. It holds its key, a way to pull it from a run, and which direction counts as “better”, so a renderer can colour and ratio it correctly.
- Stack
Agg - Per-metric aggregates for one stack within a cell.
- Stat
- Mean and spread of a single scalar metric over the runs of one
(cell, stack). - TsPoint
- One averaged point of a time series. It holds the across-run mean of a metric at a fixed offset into the run, and how many runs gave that offset.
- TsSeries
- An across-run-averaged time series for one
(cell, stack, metric).
Functions§
- aggregate_
cells - Groups runs by
CellKeyand reduces each stack’s runs to a per-metricStat. This returns the cells in key order. - averaged_
timeseries - For each
(cell, stack, metric), averages the per-interval samples across all runs at each shared time offset. - scalar_
metrics - The headline scalar metrics rendered as krabka-vs-kafka bars.
- stat
- Mean, sample stddev, and count over a sample of run-level values.