Regressions

Regression gating for mew compare.

A regression is a benchmark whose delta against the baseline exceeds a threshold in the slower direction. For real_time / cpu_time larger is worse; for iterations smaller is worse. Per-benchmark allowlist rules can ignore a benchmark or raise its threshold, matched against the full name via fnmatch.fnmatchcase().

class mew.regressions.Verdict(*values)[source]

Bases: Enum

OK = 'ok'

Within the active threshold; does not contribute to gate failure.

REGRESSED = 'regressed'

Slower than the default threshold and not covered by any allow rule.

Causes the gate to fail (exit code 2).

ALLOWED_OVER = 'allowed_over'

Over the default threshold but inside a rule-supplied threshold.

Surfaced as a soft warning for periodic review.

IGNORED = 'ignored'

Out-of-scope per a matching ignore=true rule.

Listed in the panel so the allowlist stays visible.

class mew.regressions.AllowRule(pattern, reason, ignore=False, threshold_pct=None)[source]

Bases: object

Parameters:
pattern: str
reason: str
ignore: bool
threshold_pct: float | None
matches(name)[source]
Return type:

bool

Parameters:

name (str)

class mew.regressions.BenchmarkVerdict(name, delta_pct, verdict, rule)[source]

Bases: object

Per-benchmark gating outcome carried through the compare pipeline.

Parameters:
name: str
delta_pct: float
verdict: Verdict
rule: AllowRule | None
class mew.regressions.RegressionConfig(default_threshold_pct, rules=())[source]

Bases: object

Parameters:
default_threshold_pct: float
rules: tuple[AllowRule, ...]
find_rule(name)[source]
Return type:

AllowRule | None

Parameters:

name (str)

evaluate(name, delta_pct, *, higher_is_better=False)[source]

Classify a benchmark’s delta against this config.

Parameters:

  • name (str) – Full benchmark name (matched against rule.pattern).

  • delta_pct (float) – Signed percent change vs baseline. For real_time / cpu_time positive means slower; for iterations use higher_is_better=True.

  • higher_is_better (bool) – Invert the sign when computing the regression magnitude.

Return type:

BenchmarkVerdict

Returns:

BenchmarkVerdict – The verdict, with the matched rule attached (if any).

mew.regressions.load_config(*, default_threshold_pct, path=None, inline_allows=None)[source]

Build a RegressionConfig.

Reads [tool.mew.regressions] from path (or pyproject.toml in the cwd when path is None). Inline --allow strings from the CLI are appended after file rules; both lists are searched in order.

Return type:

RegressionConfig

Parameters:

  • default_threshold_pct (float)

  • path (Path | None)

  • inline_allows (list[str] | None)

mew.regressions.render_panel(verdicts, *, default_threshold_pct)[source]

Format the regression panel and compute the exit code.

Parameters:
Return type:

tuple[str, int]

Returns:

(text, exit_code) (tuple[str, int]) – Panel body (empty if nothing to report) and the exit code. exit_code is 2 if any verdict is Verdict.REGRESSED, else 0.

mew.regressions.report(verdicts, *, default_threshold_pct)[source]
Return type:

int

Parameters: