Guardrails / tests

Coverage is at or above the floor

tests/coverage-floor@v1

The coverage the build measured sits at or above the line and branch floors the team sets. Both floors default to zero, so the guardrail reports the number until a team chooses one.

Id	tests/coverage-floor
Version	v1
Category	tests
Default severity	warning
Interpreter	python3
Timeout	30 seconds
Violations tolerated	0
Collects	coverage

Collectors

This guardrail gathers nothing itself. It depends on the collectors below, which the CLI runs once per build before any check, and reads what they found out of GUARDRAIL_FACTS. A collector that collects nothing skips this guardrail rather than failing it.

Collector	Gathers	Inputs it is given
coverage	The coverage the build already measured, read from whichever report format it left behind and normalized into one shape: the line and branch totals across every report, and a summary per file. The covered and missed line numbers themselves are left out unless lineDetail asks for them, because a large repository's line map is bigger than the facts document is allowed to be. Reports that name the same file are merged line by line, so a per module report and a merged one agree, but a report carrying only totals adds to them.	none

The inputs above are this guardrail's own, passed straight through. Configuring one in buildnote.json changes what is collected, and two guardrails configured the same way share the one collection.

Configuration

{
  "guardrails": {
      "failOn": "error",
      "comment": true,
      "checks": [
          {
              "use": "tests/coverage-floor@v1",
              "severity": "warning",
              "with": {
                  "minLines": "0",
                  "minBranches": "0"
              },
              "exemptions": []
          }
      ]
  }
}

Inputs

Input	Description	Default	Environment variable
minLines	Lowest line coverage percentage that passes, 0 for no floor. Read only when the reports measured lines at all.	0	GUARDRAIL_INPUT_MINLINES
minBranches	Lowest branch coverage percentage that passes, 0 for no floor. Ignored when the reports carry no branches, which is a project without one as much as a format that omits them.	0	GUARDRAIL_INPUT_MINBRANCHES

How to fix

Cover the code the change added, then set the floor to the number the build already reaches, so it ratchets rather than aspires:

{
  "guardrails": {
    "checks": [
      { "use": "tests/coverage-floor@v1", "with": { "minLines": "72" } }
    ]
  }
}

The floor needs a coverage report in the workspace to read, which is a task of its own in most builds:

./gradlew test jacocoTestReport

Both floors default to 0, so a build that has not chosen a number is never failed by one. Raise them as the number comes up: a floor set above where the build stands today fails every change until somebody lowers it, which teaches the team to ignore it.

More in tests

• tests/no-failures. Every case the test reports name either passed or was skipped, so nothing red was carried past the gate.
• tests/no-skipped-growth. The reports name no more skipped cases than the budget allows. The budget is a fixed number this guardrail is configured with rather than a comparison against the previous build, so it catches growth only as far as the team lowers the number when a skip comes back.
• tests/results-published. The build left a test report that can be read, naming at least the cases the team expects, rather than reporting nothing at all.

All 4 tests guardrails