Compare

Compare benchmark result files: deltas, speedups, optional stddev.

Structured as three stages so new comparison dimensions feed the same renderer:

1. Load (_load_sessions()): read a result file into per-session sample groups, discarding nothing.

2. Select (_select_latest(), _resolve_session()): resolve the groups to one sample set per file, either by path@selector or by defaulting to the latest session per name.

3. Render (_render()): compare a list of labelled columns, one per file or (under --by variant) one per variant of a single file.

class mew.compare.Sample(name, value, stddev, time_unit, session_date)

Bases: object

One benchmark’s reduced measurement, the unit every column compares.

Variables:

• name (str) – Canonical file.py::func[label] name, re-keyed per the match key.

• value (float) – Center across the benchmark’s per-repetition rows (median by default).

• stddev (float or None) – Sample stddev across repetitions; None for a single repetition.

• time_unit (str or None) – Unit value is expressed in; None for unitless metrics.

• session_date (str or None) – Date of the session this sample came from, for provenance display.

Parameters:

• name (str)

• value (float)

• stddev (float | None)

• time_unit (str | None)

• session_date (str | None)

name: str

value: float

stddev: float | None

time_unit: str | None

session_date: str | None

property cv: float | None

Coefficient of variation, or None without repetition data.

class mew.compare.SessionData(key, context, samples, session_tag=None)

Bases: object

One session’s worth of samples from a result file.

key is (date, host, session_id); the id component is empty for files written before sessions were persisted, where (date, host) is the best identity available.

Parameters:

• key (tuple[str, str, str])

• context (dict[str, Any])

• samples (dict[str, Sample])

• session_tag (str | None)

key: tuple[str, str, str]

context: dict[str, Any]

samples: dict[str, Sample]

session_tag: str | None

property date: str | None

property host: str | None

property session_id: str | None

mew.compare.compare(files, *, metric='real_time', key=None, pattern=None, literal=False, show_stddev=False, by=None, baseline=None, statistic=None, regressions=None, console=None)

Compare benchmark result files and render a comparison table.

The last file is the baseline; earlier files show their value plus percent delta and speedup against it.

Parameters:

• files (list[Path]) – Result files (JSON, JSONL, or JSONL.gz); the last is treated as the baseline (mew compare head.json baseline.json reads like “compare head against baseline”). A path@selector argument picks one session from a multi-session file; see docs/guide/regressions.md for the selector grammar.

• metric (str) – Metric to compare. One of "real_time", "cpu_time", "iterations", or (for files produced with --profile-memory) "memory.peak_bytes" or "memory.allocations_per_iteration" (the per-call allocation count, comparable across engines regardless of speed).

• key (str | None) – How benchmarks are matched across files: "name" uses the full registered name; "func" strips the file.py:: prefix so suites in different files with matching function names line up (A/B suites). Defaults to "func" with by="variant" (each variant’s rows keep their own file.py:: prefix, so the columns only line up on the function name) and "name" otherwise.

• pattern (str | None) – Regex (re.search) filter applied to benchmark names.

• literal (bool) – Match pattern as a literal string rather than a regex (e.g. to keep a name[label]’s brackets literal).

• show_stddev (bool) – Add per-file stddev columns when stddev data is present.

• by (str | None) – Pivot dimension. "variant" compares the variants within a single --variant result file (one column each) instead of comparing files.

• baseline (str | None) – With by="variant", which variant is the baseline (default: the first one written).

• statistic (Callable[[list[float]], float] | None) – Reducer over each benchmark’s per-repetition values, used as the displayed center and the regression-gate value (stddev is unaffected). Receives a list[float] and returns a float-castable scalar; defaults to statistics.median. The CLI resolves --statistic to one of these via mew._statistics.resolve_statistic().

• regressions (RegressionConfig | None) – If given, gate the second file against the baseline and append a regression panel.

• console (Terminal | None) – Output terminal; defaults to a fresh Terminal.

Return type:

int

Returns:

int – Exit code: 0 on success, 1 for no overlap, 2 if the regression gate fails.