Writing a guardrail

A guardrail is a directory holding a JSON definition and the script that performs the check. The definition carries metadata and points at the script. The script does the work and prints its result as JSON, so there is no rules language to learn and no expression syntax to fight.

guardrails/<category>/<name>/guardrail.json
guardrails/<category>/<name>/check.sh

A library holds two sibling namespaces, guardrails/ and collectors/, so a guardrail id and a collector name can never collide. The path under guardrails/ is the guardrail id, so guardrails/git/conventional-commits/ is the guardrail git/conventional-commits.

Several guardrails in one directory

A directory can declare several guardrails at once in a guardrails.json bundle, with a script per guardrail beside it. This is how every guardrail Buildnote ships is declared, because it lets related checks share their inputs and their version:

guardrails/<category>/guardrails.json
guardrails/<category>/<name>.py
guardrails/<category>/<another-name>.py

The bundle carries what the whole directory shares and each entry carries what is its own, so a kotlin/guardrails.json declaring gradle-wrapper is the guardrail kotlin/gradle-wrapper, referenced and configured exactly like any other.

Field	Required	Description
version	yes	The version every guardrail in the bundle is at.
category	yes	The namespace, which is the directory holding the bundle.
guardrails	yes	Map of name to the fields that are not shared: name, description, severity, check, and optionally remediation, inputs and maxViolations. The key is the last segment of the id.
inputs	no	Inputs every guardrail in the bundle declares. An entry's own inputs override them by name.
icon	no	An SVG beside the bundle standing for the whole directory, shown against the category in these pages and at the head of every guardrail page in it. Draw it in currentColor so it follows the reader's theme, and use the technology's own mark where it has one.

{
  "version": "v1",
  "category": "kotlin",
  "inputs": {
    "projectDir": { "default": ".", "description": "Directory holding the project." }
  },
  "icon": "icon.svg",
  "guardrails": {
    "gradle-wrapper": {
      "name": "Gradle wrapper is committed",
      "description": "A Gradle project commits the wrapper.",
      "severity": "warning",
      "remediation": "Run `gradle wrapper` and commit every file it writes.",
      "check": { "run": "gradle-wrapper.py" }
    }
  }
}

The icon is documentation metadata: the CLI never fetches it, so it is not code that runs on your runner and it costs a guardrail run nothing.

A check.run in a bundle is relative to the bundle's directory rather than to a directory of the guardrail's own. A reference resolves against guardrails/<directory>/guardrails.json first and falls back to guardrails/<id>/guardrail.json, so both layouts sit side by side in one library and a guardrail can move between them without its id changing. The bundle is read first because it is the form most libraries use, which keeps a resolution to one read rather than a miss and a read.

A name that neither the bundle nor a directory of its own declares is a configuration error rather than a skip, the same as a malformed definition: a mistyped id must never pass quietly.

Referring to a guardrail

A check entry names a guardrail with its use reference, which carries the version:

<category>/<name>@<version>

Reference	Resolves to
git/conventional-commits@v1	the built in library first, then each guardrails.sources entry in order
github://acme/our-guardrails/git/example@v1	tag v1 of acme/our-guardrails, and nowhere else

The version is required. There is no default, so nothing ever resolves against a moving ref by accident.

The built in library is one version, so the path carries no version segment and the version is checked against the definition's own version field. A reference asking for a version other than the one your CLI ships is a configuration error, so upgrading a shipped guardrail is a CLI upgrade.

A reference with no prefix resolves against the built in library first and then against each guardrails.sources entry, so a library of your own adds guardrails beside the shipped ones rather than replacing them. A github://<owner>/<repository>/ prefix pins one reference to one repository instead, where the version is the git ref the path is built from.

Definition

Field	Required	Description
id	yes	Must match the directory path under the source root.
version	yes	The version a check entry must ask for. Asking for a different one fails the run with a message naming both.
name	yes	Human readable name, shown on the verdict and in the pull request comment.
category	yes	The namespace, which is the first path segment.
description	yes	One sentence describing what the check enforces.
severity	yes	error, warning or info. A check entry can override it.
check.run	yes	Script path relative to the guardrail directory. Its extension picks the interpreter, so there is nothing to declare.
check.timeoutSeconds	no	Wall clock limit for the script. Defaults to 60.
remediation	no	Markdown telling somebody how to fix a violation. It is shown under the violations in the pull request comment, so it can be a paragraph, a list or a fenced command block, not only a link.
inputs	no	Map of input name to { "default": "...", "description": "..." }. The description is markdown and is what the input's docs row shows.
maxViolations	no	Violations tolerated before the verdict is failed. Defaults to 0.

Inputs are plain string defaults, so a definition never interpolates anything into the script.

Interpreters

The check script's extension decides what runs it. There is no interpreter to declare and no way for a definition to name a command of its own.

Script	Runs as
check.py	python3
check.sh, check.bash	bash

Any other extension is rejected when the guardrail loads, the same as a malformed definition: it is reported and the command exits non zero whatever failOn says. The shebang is still required by the shared library's contract and is worth keeping accurate for anyone reading the file, but the CLI does not consult it.

Script contract

The CLI writes the script to a temporary file and runs it with that interpreter from your working directory, with these environment variables set:

• GUARDRAIL_INPUT_<NAME> for each input, uppercased, such as GUARDRAIL_INPUT_BASEREF. The value is the definition default overridden by the with values in your check entry.
• GUARDRAIL_ID, GUARDRAIL_VERSION, GUARDRAIL_SEVERITY.
• GUARDRAIL_LIB_DIR, the directory holding the Python helper described below.
• GUARDRAIL_FACTS, the file holding the facts of every collector the guardrail declared, empty when it declared none.
• BUILDNOTE_ORG, BUILDNOTE_PROJECT, BUILDNOTE_MODULE, BUILDNOTE_BUILD, BUILDNOTE_SHA, BUILDNOTE_REF.

The script prints one JSON object on stdout and nothing else. Everything it wants to log goes to stderr.

{
  "violations": [
    {
      "evidence": "a1b2c3d update stuff",
      "message": "Commit a1b2c3d does not follow Conventional Commits"
    }
  ]
}

Field	Description
violations	Array of { evidence, message }. Absent or empty means the check passed. evidence is what your exemptions are matched against, message is what the comment and the event show.
skip	Optional string. Its presence means the check could not gather its evidence, so the verdict is skipped and the string is the reason.
verdict	Optional explicit passed, failed or skipped, for a check that decides for itself.

When verdict is absent it is derived: a skip reason gives skipped, more violations than maxViolations gives failed, anything else gives passed.

The exit code is ignored when stdout parses as a result object, so a script is free to exit non zero when it finds something. Stdout that does not parse makes the verdict skipped, with the last stderr line as the reason, because a broken script must never fail somebody's build.

remediation is optional in the format, but every guardrail in the shared library is held to a stricter contract by a test: it must load, its category must match the first segment of its id, its script must start with a shebang, every declared input must have a default and a description, every collector it declares in collect must be one the library carries, it must ship a test beside its script, it must carry remediation written as markdown rather than a bare link, and running it over an empty commit range must pass while running it against a ref that does not resolve must skip. That same test generates these pages, so a guardrail cannot ship undocumented.

The Python helper

The CLI writes guardrail.py beside every check script it runs and names the directory in GUARDRAIL_LIB_DIR. The helper sits beside the script, which is already on sys.path, so a Python guardrail just imports it:

#!/usr/bin/env python3
from guardrail import Guardrail

with Guardrail() as guardrail:
    path = guardrail.input("path", "README.md")
    described = guardrail.facts("files")["files"][path]

    if not described["present"]:
        guardrail.violation(path, "No README at %s" % path)

The context manager prints the result and sets the exit code when the block ends, and turns a skip into a skip result, so a check never builds JSON itself.

Member	What it does
input(name, default="")	Reads GUARDRAIL_INPUT_<NAME>, falling back to the default when it is unset or empty.
number(name, default)	Reads an input as an integer, skipping when it is not a number.
facts(name, reason)	Reads one collector's document out of GUARDRAIL_FACTS, skipping with reason when it was not collected.
optional(name, default)	The same, returning default when it was not collected, for a check that treats a collector finding nothing as an answer rather than as missing evidence.
run(*command)	Runs a command and captures it, returning a subprocess.CompletedProcess. Skips when the executable is not on the runner.
violation(evidence, message)	Adds a violation to the result.
skip(reason)	Ends the check with a skip verdict and that reason.
log(message)	Writes a line to stderr.
id, version, severity	The guardrail's own metadata, read from the environment.

run is the escape hatch, not the default. Reach for it when no collector gathers what your check needs: it exists because an executable a runner does not have is missing evidence rather than a violation, so it skips rather than raising and a check never has to remember that. What it does not decide is what a non zero exit means. It hands back the CompletedProcess and the check judges it:

if guardrail.run("mytool", "--version").returncode != 0:
    guardrail.skip("mytool is not installed on this runner")

That is the whole helper. It carries the plumbing every guardrail needs and nothing about what any guardrail checks, so reading one guardrail never means reading a second. The guardrails Buildnote ships are each a single script, and each one gets its evidence from a collector rather than gathering it.

Collecting instead of shelling out

A guardrail that would otherwise run git or read build files declares a collector instead, and reads what it gathered:

{
  "collect": ["git"],
  "check": { "run": "conventional-commits.py" }
}

with Guardrail() as guardrail:
    git = guardrail.facts("git")

    if not git["resolved"]:
        guardrail.skip("base ref %s is not resolvable" % git["baseRef"])

    for commit in git["commits"]:
        if commit["merge"]:
            guardrail.violation(commit["short"], "Commit %s is a merge commit" % commit["short"])

The CLI runs every collector every configured guardrail declares before it runs the first check, and writes the document each check asked for to the file named by GUARDRAIL_FACTS, so two guardrails asking for the same data collect once between them and neither pays for what the other needed. Everything collected is attached to the run event as guardrail-facts.json, so a run in Buildnote carries the evidence its verdicts were decided from and you can see what the checks saw without rerunning the build. Every guardrail Buildnote ships is written this way: the check contains the policy and nothing else, which is also what makes it testable without a repository to point it at.

A collector is a script like a check, and a directory of its own like a guardrail. It sits in collectors/ beside them, holding a collector.json, the script, an icon.svg and its test:

collectors/<name>/collector.json
collectors/<name>/<name>.py
collectors/<name>/icon.svg
collectors/<name>/test_<name>.py

The directory name is the collector name and the key its facts arrive under, and the definition beside it must declare that same name as its id and the library version as its version. See Collectors for every field. It is written against the Collector helper rather than Guardrail:

#!/usr/bin/env python3
from guardrail import Collector

with Collector() as collector:
    base_ref = collector.input("baseRef", "origin/main")
    inside = collector.run("git", "rev-parse", "--git-dir")

    collector.facts["repository"] = inside is not None and inside.returncode == 0

    if not collector.facts["repository"]:
        collector.invalid("not a git repository")

Member	What it does
facts	The dictionary printed when the block ends.
input(name, default="") number(name, default)	The inputs the collector declares, the same as for a guardrail.
collected(name)	Reads the facts of a collector this one declares in collect, marking this one incomplete when it collected nothing.
optional(name, default)	The same, returning default when it collected nothing, so one collector finding nothing does not stop this one.
run(*command)	Runs a command, returning None when the executable is not on the runner rather than skipping.
invalid(reason)	Prints what was gathered so far and stops, for a fact set that cannot be completed.
nothing(reason)	Prints nothing and stops, for a project this collector has nothing to say about. Its key is then left out of the facts entirely, rather than carrying a document of empties.
log(message)	Writes a line to stderr.

A collector never has an opinion: it says what is there and lets the guardrail decide. {"repository": false} is a fact, not a skip, which is why the skip reasons in the shipped guardrails read the same as they did when each one shelled out for itself.

The definition declares every one of those facts, as a path into the document with what it means, and that is what a guardrail author reads instead of the script:

{
  "facts": {
    "repository": "Whether the CLI is running inside a git repository.",
    "commits[].subject": "First line of the commit message."
  }
}

Nothing keeps prose honest on its own, so the declaration is checked from both sides. The collector's own tests walk the facts every fixture collects and fail on a path the definition does not declare, nested paths included, and fail again when the definition declares a path no fixture ever collects. The contract test then runs every shipped collector for real as a backstop. A renamed fact is a definition change in the same commit, or it is neither.

The helper is a CLI resource rather than a library file, so a guardrail from any source gets it and a sha256 pin still covers every file that was fetched. It is versioned by the CLI release, which means a published guardrail keeps working and the helper only ever gains members. There is no equivalent for bash or sh: a shell guardrail builds its own result.

Building the result without the helper

A Python guardrail has json in the standard library, so building the result object is free even without the helper. A shell guardrail has no such luxury: build the JSON with string handling rather than calling jq, because a runner cannot be assumed to have it and a guardrail that skips for a missing tool reports nothing useful. If you do prefer jq, emit {"skip":"jq not available"} when command -v jq fails.

Whichever interpreter your extension picks, it has to be on the runner. A missing one is a skipped verdict with a reason, never a failure, so a .py guardrail degrades rather than breaking a build that has no python3.

Security

The CLI runs these scripts on your runner, so treat that as the thing to get right.

• Every script a run executes ships inside the CLI, so a guardrail cannot change under you without a CLI release, and a run downloads nothing.
• The resolved location and the script's SHA-256 are printed on every run, so each build records exactly what it executed.
• A check entry can carry a sha256 that is checked before the script runs. A mismatch skips the guardrail with a warning rather than executing the script.
• A definition cannot point the CLI at a location of its own, and a reference naming a repository is refused rather than fetched.

Where guardrails come from

Every guardrail and every collector a run uses comes from the library bundled into the CLI, so a run needs no network and there is nothing to publish or point at. A library of your own is not resolved for now; the way to ship a new guardrail is to add it to that library and release a CLI that carries it.