No test failed or errored
tests/no-failures@v1
Every case the test reports name either passed or was skipped, so nothing red was carried past the gate.
| Id | tests/no-failures |
| Version | v1 |
| Category | tests |
| Default severity | error |
| Interpreter | python3 |
| Timeout | 30 seconds |
| Violations tolerated | 0 |
| Collects | tests |
Collectors
This guardrail gathers nothing itself. It depends on the collectors below, which the CLI runs once per build before any check, and reads what they found out of GUARDRAIL_FACTS. A collector that collects nothing skips this guardrail rather than failing it.
| Collector | Gathers | Inputs it is given |
|---|---|---|
tests | The test results the build already wrote, read from whichever report format it left behind and normalized into one shape: the totals across every report, and the cases that failed, errored or were skipped. The passing cases are left out, because a guardrail gates on the others and Buildnote already carries every case as a test event. Nothing identifies a case across reports, so a build writing both a report per class and a merged one counts every case in both. | none |
The inputs above are this guardrail's own, passed straight through. Configuring one in buildnote.json changes what is collected, and two guardrails configured the same way share the one collection.
Configuration
{
"guardrails": {
"failOn": "error",
"comment": true,
"checks": [
{
"use": "tests/no-failures@v1",
"severity": "error",
"with": {},
"exemptions": []
}
]
}
}How to fix
Fix the failing tests, or delete the ones that no longer describe the system. A red test that stays red is a test nobody reads, and the next real failure hides behind it.
Reproduce one case before pushing:
./gradlew :buildnote-api:test --tests 'com.acme.widget.QueueTest'Where a case fails because it is flaky rather than because the code is wrong, quarantine it in the test framework, so the quarantine is reviewable beside the test. Turning the guardrail off moves the same decision somewhere nobody looks.
More in tests
tests/coverage-floor. The coverage the build measured sits at or above the line and branch floors the team sets. Both floors default to zero, so the guardrail reports the number until a team chooses one.tests/no-skipped-growth. The reports name no more skipped cases than the budget allows. The budget is a fixed number this guardrail is configured with rather than a comparison against the previous build, so it catches growth only as far as the team lowers the number when a skip comes back.tests/results-published. The build left a test report that can be read, naming at least the cases the team expects, rather than reporting nothing at all.