skywalkskills

CatalogueDelivery & Operations

E4Ready

Test Coverage & Confidence Audit

Says which behavior the tests actually lock, and which changes could ship without a failing test.

What it examines

  • Coverage of the paths a user can take
  • Tests that assert a behavior, not only that the code ran
  • Gaps a remediation would cross
  • Suites that are skipped, flaky, or unable to run

What you get back

A findings file. Each item has the evidence, how serious it is, and a fix.

Dependencies

  • An AI coding agent that reads SKILL.md files, such as Claude Code, Codex or Gemini CLI
  • Read access to the repository: source, tests and coverage tooling configuration
  • The ability to actually run the existing test suite, with its runtime, dependencies and any services it needs. If it cannot run, the report says so and the checks that depend on it become insufficient evidence.
  • Git history, used as the churn signal
  • Python 3.11 or newer with the jsonschema package, to check findings.json before fixing anything
  • Optional: a list of your business-critical paths. Without it, weighting falls back to churn alone and is labelled partial.
  • Optional: CI history, to date the last green run

How it works

The audit is read-only, apart from running your existing test suite. It runs seven checks: suite execution health, criticality-weighted coverage, tests with no assertions, flaky tests, tests tied to implementation, critical paths with zero coverage and suite staleness.

findings.md adds a suite verdict (Healthy, Gaps found or Cannot assess) and a remediation gate (Safe to modify, Modify with additional tests or Do not modify yet). The remediate halves of other packages honour that gate.

The remediate skill reads findings.json only. It adds characterization tests in small batches, one critical path at a time. Tests pin current behaviour, suspected defects are recorded rather than encoded as correct, and application code is not modified.

Setup

  1. Unzip the download, then copy the audit and remediate folders into your agent's skills folder (for Claude Code, .claude/skills/) in the repository you want to check.
  2. Install the test suite’s own dependencies and confirm it runs locally before you start.
  3. Ask your agent: Run a test coverage and confidence audit on [scope]. Business-critical paths: [list them, or say not sure]. Mode: audit only.
  4. Check the result with the validator: pip install jsonschema, then python3 schema/validate.py findings.json.
  5. Read findings.md. Once you have reviewed the findings, run the remediate skill using findings.json, or re-run with Mode: audit and remediate.

Run it with your agent

Copy the prompt, fill in the parts in square brackets, and paste it into your agent.

Audit prompt

Run the e4-test-coverage-confidence-audit skill on this repository. Business-critical paths: [list them, or say not sure]. You may run the existing test suite. Mode: audit only. Do not change any code. Write findings.json and findings.md in the repository root.

Fix prompt, after you have findings.json

Run the e4-test-coverage-confidence-remediate skill using findings.json. Work through the findings I choose, and leave the rest open.

What is inside

The files that make up this skill, as your agent reads them.

packages/e4-test-coverage-confidence

audit › SKILL.md

Sign in to download

Use the email and password you signed up with.