Guardrails / tests

Coverage is at or above the floor

tests/coverage-floor@v1

The coverage the build measured is at or above the line and branch floors the team sets. Both floors default to zero, so the guardrail just reports the number until the team chooses one.

Id	tests/coverage-floor
Version	v1
Category	tests
Default severity	warning
Interpreter	python3
Timeout	30 seconds
Violations tolerated	0
Collects	coverage

Collectors

This guardrail doesn't gather anything itself. It relies on the collectors below, which the CLI runs once per build before any check, and reads what they found from GUARDRAIL_FACTS. If a collector collects nothing, this guardrail is skipped, not failed.

Collector	Gathers	Inputs it is given
coverage	The coverage the build already measured, read from whichever report format it left behind and normalized into one shape: the line and branch totals across every report, and a summary per file. The covered and missed line numbers are left out unless you set lineDetail, because a large repository's line map is bigger than the facts document is allowed to be. Reports that name the same file are merged line by line, so a per module report and a merged report agree, but a report carrying only totals is added on top.	none

The inputs above are this guardrail's own inputs, passed straight through to the collector. Setting one in buildnote.json changes what is collected, and two guardrails configured the same way share a single collection.

Configuration

{
  "guardrails": {
      "failOn": "error",
      "checks": [
          {
              "use": "tests/coverage-floor@v1",
              "severity": "warning",
              "with": {
                  "minLines": "0",
                  "minBranches": "0"
              },
              "exemptions": []
          }
      ]
  }
}

Inputs

Input	Description	Default	Environment variable
minLines	Lowest line coverage percentage that passes, 0 for no floor. Only checked when the reports measured lines at all.	0	GUARDRAIL_INPUT_MINLINES
minBranches	Lowest branch coverage percentage that passes, 0 for no floor. Ignored when the reports contain no branch data, whether because the project has no branches or because the format omits them.	0	GUARDRAIL_INPUT_MINBRANCHES

How to fix

Add tests for the code the change introduced, then set the floor to the coverage the build already reaches, so it can only go up from there:

{
  "guardrails": {
    "checks": [
      { "use": "tests/coverage-floor@v1", "with": { "minLines": "72" } }
    ]
  }
}

The guardrail needs a coverage report in the workspace, which most builds produce with a separate task:

./gradlew test jacocoTestReport

Both floors default to 0, so a build that hasn't chosen a number is never failed by one. Raise them as coverage improves. A floor set above where the build is today fails every change until someone lowers it, and that teaches the team to ignore it.

More in tests

• tests/no-failures. Every test case in the reports passed or was skipped, so no failing test gets past the gate.
• tests/no-skipped-growth. The reports contain no more skipped cases than the budget allows. The budget is a fixed number in the configuration, not a comparison with the previous build, so it only catches growth if the team lowers the number as skipped tests come back.
• tests/results-published. The build left a readable test report naming at least as many cases as the team expects, instead of reporting nothing at all.

All 4 tests guardrails