No test failed or errored
tests/no-failures@v1
Every test case in the reports passed or was skipped, so no failing test gets past the gate.
| Id | tests/no-failures |
| Version | v1 |
| Category | tests |
| Default severity | error |
| Interpreter | python3 |
| Timeout | 30 seconds |
| Violations tolerated | 0 |
| Collects | tests |
Collectors
This guardrail doesn't gather anything itself. It relies on the collectors below, which the CLI runs once per build before any check, and reads what they found from GUARDRAIL_FACTS. If a collector collects nothing, this guardrail is skipped, not failed.
| Collector | Gathers | Inputs it is given |
|---|---|---|
tests | The test results the build already wrote, read from whichever report format it left behind and normalized into one shape: the totals across every report, and the cases that failed, errored or were skipped. Passing cases are left out, because guardrails gate on the others and Buildnote already records every case as a test event. Nothing identifies a case across reports, so when a build writes both a report per class and a merged report, every case is counted in both. | none |
The inputs above are this guardrail's own inputs, passed straight through to the collector. Setting one in buildnote.json changes what is collected, and two guardrails configured the same way share a single collection.
Configuration
{
"guardrails": {
"failOn": "error",
"checks": [
{
"use": "tests/no-failures@v1",
"severity": "error",
"with": {},
"exemptions": []
}
]
}
}How to fix
Fix the failing tests, or delete the ones that no longer describe how the system behaves. A test that stays red stops being read, and the next real failure hides behind it.
Reproduce a single test case before pushing:
./gradlew test --tests 'com.company.widget.QueueTest'If a test fails because it's flaky, not because the code is wrong, quarantine it in the test framework so the quarantine can be reviewed next to the test. Turning off the guardrail makes the same decision somewhere nobody looks.
More in tests
tests/coverage-floor. The coverage the build measured is at or above the line and branch floors the team sets. Both floors default to zero, so the guardrail just reports the number until the team chooses one.tests/no-skipped-growth. The reports contain no more skipped cases than the budget allows. The budget is a fixed number in the configuration, not a comparison with the previous build, so it only catches growth if the team lowers the number as skipped tests come back.tests/results-published. The build left a readable test report naming at least as many cases as the team expects, instead of reporting nothing at all.