CI Integration

Operational guide

Add diff-aware spec-mapping checks to pull request and release workflows. Review missing mappings and execution evidence before making release decisions.

What CI gets you

Make impact analysis part of the release pipeline

Start with deterministic planning in CI, then add gating and optional AI only after the route-family manifest is reliable enough to trust.

Core CI loop

The same workflow works in pull requests and releases

npx impact-gate review --path . --since origin/main --ci-comment-path comment.md npx impact-gate gate --threshold 80 --path . --since origin/main

GitHub Actions Example

name: E2E Coverage Check
on:
  pull_request:
    branches: [main, master]

jobs:
  e2e-coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install impact-gate
        run: npm install -D @yasserkhanorg/impact-gate

      - name: PR Impact Review
        run: |
          npx impact-gate review \
            --path . \
            --since origin/${{ github.base_ref }} \
            --ci-comment-path comment.md

      - name: Coverage Gate
        run: npx impact-gate gate --path . --since origin/${{ github.base_ref }} --threshold 80

Gate Command

Thresholds

Use gate when you want an explicit pass/fail decision

The gate fails unassessed changes and missed spec-mapping thresholds. Partial mappings do not count as fully covered. This is spec presence, not measured behavior coverage or release approval.

Gate example

Thresholds use percentage-style values

npx impact-gate gate --threshold 80 --path . --since origin/main

--threshold accepts percentage-style values (0-100); legacy fractions in (0, 1] are converted (1 means 100). Invalid Git refs return nonzero. A valid empty diff is separate from a nonempty diff with zero matching features, which fails.

Mattermost shadow planning

Use the Mattermost advisory guide for plan --advisory --json. This isolated caller reads a pinned source checkout and explicit suite configuration, emits one report, and retains full-suite execution. gate --advisory also emits a nonblocking report; it does not evaluate the ordinary coverage threshold or declare a coverage pass. Keep the artifact job independent of the existing E2E runner and statuses.

CI Artifacts

Ordinary plan runs write the following files under <testsRoot>/.e2e-ai-agents/. Advisory runs bypass these writers; save their stdout explicitly:

Plan artifact

.e2e-ai-agents/plan.json

Structured plan with run sets, confidence, and decisions.

PR summary

.e2e-ai-agents/ci-summary.md

Markdown summary for pull requests and release reviews.

Metrics summary

.e2e-ai-agents/metrics-summary.json

Aggregated run metrics for dashboards and reporting.

Event log

.e2e-ai-agents/metrics.jsonl

Append-only metric log for later analysis and calibration.

JSON Output for Parsing

Machine-readable output

Use JSON when another job needs to consume the result

Structured JSON output is useful when CI needs to branch on impact results or feed the plan into another reporting step.

For plan --json, stdout contains the plan JSON and diagnostics go to stderr. An error returns nonzero and a JSON error object. Ordinary planning still writes its usual artifacts:

npx impact-gate plan --no-ai --json --path . --since origin/main

Free-Tier CI

No-key mode

Run the core planning loop in CI without any provider

Deterministic analysis only requires git, the repository, and the route manifest. Provider environment variables are optional.

For zero-cost CI runs, skip AI enrichment:

npx impact-gate impact --path . --since origin/${{ github.base_ref }}
npx impact-gate plan --no-ai --path . --since origin/${{ github.base_ref }}

These commands use deterministic analysis only and require no API key. Ordinary plan without --no-ai can use a configured provider. For the stricter no-write pilot contract, use --advisory with its suite configuration.