Skip to content
Impact Gate Impact Gate Impact Gate

How Impact Gate Works

Concept model

Impact Gate is easiest to understand as one pipeline: read a diff, map it to product areas, compare that to coverage, and only then decide whether AI should help extend the suite.

Pipeline

Diff in, evidence out, AI last

01 read a diff 02 map changed files to product areas 03 compare those areas to E2E coverage 04 produce a plan and a gate decision 05 optionally generate or heal with guardrails
Why this matters

The strongest story is deterministic first

The product is not “an AI test generator” first. It is a diff-aware evidence layer that becomes more valuable once a team trusts what it is telling them.

Core Mental Model

The product is not “an AI test generator” first. Its strongest path is deterministic:

  • read a git diff
  • determine what changed
  • determine what should run
  • determine what is missing
  • provide a test plan and mapping evidence for PR or release review

Use impact, plan --no-ai, and gate for that deterministic path. Spec mappings and heuristic scores do not establish measured behavior coverage or release readiness.

Step 1: Diff In

Input

Everything starts with a git comparison

  • origin/main for pull requests
  • a previous release tag for release readiness
  • a hotfix base for emergency verification
Example

Release-ready plan from one tag

npx impact-gate plan --no-ai --path . --since v2.1.0

Everything starts with a git comparison. In practice, teams use:

Step 2: Knowledge Layer

Project knowledge

The diff is interpreted through what the repo already knows

  • route families map code paths to product areas and flows
  • dependency graph catches transitive impacts
  • traceability imports file-to-test evidence from an explicit per-test source coverage map; a passing result or Git diff alone creates no coverage edges
  • historical failure data can raise confidence that an area needs attention
Interpretation

This is what turns files into flows and tests

Without the knowledge layer, you only know which files changed. With it, you can reason about which product areas and E2E checks actually matter.

The diff is interpreted through project knowledge:

Step 3: Coverage Planning

Plan output

Coverage planning answers what changed, what to run, and what is missing

  • what is impacted
  • what should run now
  • what confidence we have in that decision
  • what flows appear under-covered
  • whether new tests or more manual verification are needed

Ordinary plan --no-ai writes these artifacts under <testsRoot>/.e2e-ai-agents/. Its confidence is heuristic. The separate Mattermost advisory path emits JSON on stdout with unavailable confidence and retains the full suite for every nonempty diff.

Step 4: Gate Decision

Decision layer

Gate checks a spec-mapping threshold

  • fully mapped impacted features count toward the threshold
  • partial mappings do not count as fully covered
  • unassessed files fail the ordinary gate; invalid Git refs return errors

gate independently analyzes the diff and compares the percentage of fully mapped features to --threshold; it does not consume a saved plan or measure executed assertions. A valid empty Git diff can pass with no coverage percentage. gate --advisory requires the configured advisory suite and emits an advisory plan instead. Ordinary plan policy enforcement is a separate mechanism.

Step 5: Optional AI With Guardrails

Guarded AI

The AI path helps only after the deterministic picture exists

  • enrich flow understanding
  • generate specs for uncovered gaps
  • heal failing generated specs
  • power deeper exploratory or crew workflows

The generation path uses local API context, prompt constraints, quarantine, and explicit verification results. The current acceptance path requires direct local-source mutation evidence; browser application targets remain unverified. See the verification limits.

AI enters after the deterministic evidence is already established.

See AI Guardrails for the full safety model.

Why This Matters

Engineering evidence

Every layer answers a different confidence question

  • the diff explains why you are testing
  • the manifest explains what feature was affected
  • the plan explains spec mappings and gaps
  • the gate explains whether the spec-mapping threshold is met
  • the AI layer helps only after that foundation is already in place
Positioning

This keeps the product honest

The tool feels stronger because it can justify every recommendation with artifacts and repo knowledge, not just a prompt and a model opinion.