Guardrails / tests

The build published its test results

tests/results-published@v1

The build left a test report that can be read, naming at least the cases the team expects, rather than reporting nothing at all.

Id	tests/results-published
Version	v1
Category	tests
Default severity	error
Interpreter	python3
Timeout	30 seconds
Violations tolerated	0
Collects	tests

Collectors

This guardrail gathers nothing itself. It depends on the collectors below, which the CLI runs once per build before any check, and reads what they found out of GUARDRAIL_FACTS. A collector that collects nothing skips this guardrail rather than failing it.

Collector	Gathers	Inputs it is given
tests	The test results the build already wrote, read from whichever report format it left behind and normalized into one shape: the totals across every report, and the cases that failed, errored or were skipped. The passing cases are left out, because a guardrail gates on the others and Buildnote already carries every case as a test event. Nothing identifies a case across reports, so a build writing both a report per class and a merged one counts every case in both.	none

The inputs above are this guardrail's own, passed straight through. Configuring one in buildnote.json changes what is collected, and two guardrails configured the same way share the one collection.

Configuration

{
  "guardrails": {
      "failOn": "error",
      "comment": true,
      "checks": [
          {
              "use": "tests/results-published@v1",
              "severity": "error",
              "with": {
                  "minTests": "1"
              },
              "exemptions": []
          }
      ]
  }
}

Inputs

Input	Description	Default	Environment variable
minTests	Fewest cases the reports must name between them. A report naming none is a file, not a test run.	1	GUARDRAIL_INPUT_MINTESTS

How to fix

Write the test reports the runner already produces into the workspace before buildnote guardrails runs, and keep them there. Gradle writes JUnit XML into build/test-results/, and most runners take a flag:

./gradlew test
pytest --junitxml=junit.xml
dotnet test --logger trx

The absence is the violation. A pipeline whose test step went green but left nothing behind cannot be shown to have run a single test, and a report naming no cases is that same absence in a file. Where a job genuinely has no tests, leave the guardrail off that job rather than lowering minTests to zero.

More in tests

• tests/coverage-floor. The coverage the build measured sits at or above the line and branch floors the team sets. Both floors default to zero, so the guardrail reports the number until a team chooses one.
• tests/no-failures. Every case the test reports name either passed or was skipped, so nothing red was carried past the gate.
• tests/no-skipped-growth. The reports name no more skipped cases than the budget allows. The budget is a fixed number this guardrail is configured with rather than a comparison against the previous build, so it catches growth only as far as the team lowers the number when a skip comes back.

All 4 tests guardrails