Writing a guardrail

A guardrail is a directory that holds a JSON definition and the script that performs the check. The definition holds the metadata and points to the script. The script does the work and prints its result as JSON, so there's no rules language or expression syntax to learn.

guardrails/<category>/<name>/guardrail.json
guardrails/<category>/<name>/check.sh

A library has two separate namespaces side by side, guardrails/ and collectors/, so a guardrail id and a collector name can never clash. The path under guardrails/ is the guardrail id: guardrails/git/conventional-commits/ is the guardrail git/conventional-commits.

Several guardrails in one directory

A directory can declare several guardrails at once in a guardrails.json bundle, with one script per guardrail next to it. Every guardrail Buildnote ships is declared this way, because it lets related checks share their inputs and version:

guardrails/<category>/guardrails.json
guardrails/<category>/<name>.py
guardrails/<category>/<another-name>.py

The bundle holds what the whole directory shares, and each entry holds what's specific to one guardrail. For example, a kotlin/guardrails.json that declares gradle-wrapper defines the guardrail kotlin/gradle-wrapper, which you reference and configure exactly like any other.

Field	Required	Description
version	yes	The version of every guardrail in the bundle.
category	yes	The namespace, which is the directory holding the bundle.
guardrails	yes	A map from name to the fields that aren't shared: name, description, severity, check, and optionally remediation, inputs and maxViolations. The key is the last segment of the id.
inputs	no	Inputs that every guardrail in the bundle declares. An entry's own inputs override them by name.
icon	no	An SVG file next to the bundle that represents the whole directory. It's shown beside the category on these pages and at the top of every guardrail page in it. Draw it in currentColor so it matches the reader's theme, and use the technology's own logo where it has one.

{
  "version": "v1",
  "category": "kotlin",
  "inputs": {
    "projectDir": { "default": ".", "description": "Directory holding the project." }
  },
  "icon": "icon.svg",
  "guardrails": {
    "gradle-wrapper": {
      "name": "Gradle wrapper is committed",
      "description": "A Gradle project commits the wrapper.",
      "severity": "warning",
      "remediation": "Run `gradle wrapper` and commit every file it writes.",
      "check": { "run": "gradle-wrapper.py" }
    }
  }
}

The icon is only used by the documentation. The CLI never fetches it, so it doesn't run on your runner and adds nothing to the cost of a guardrail run.

In a bundle, check.run is relative to the bundle's directory, not to a separate directory for the guardrail. A reference resolves against guardrails/<directory>/guardrails.json first and falls back to guardrails/<id>/guardrail.json. Both layouts can live in the same library, and a guardrail can move from one to the other without its id changing. The bundle is checked first because most libraries use it, so resolving a reference usually takes one read instead of a miss followed by a read.

If neither a bundle nor a directory of its own declares a name, that's a configuration error, not a skip, just like a malformed definition. A mistyped id should never pass quietly.

Referring to a guardrail

A check entry names a guardrail with its use reference, which includes the version:

<category>/<name>@<version>

Reference	Resolves to
git/conventional-commits@v1.2.0	tag v1.2.0 of buildnote/guardrails first, then each guardrails.sources entry in order
github://company/our-guardrails/git/example@v1	tag v1 of company/our-guardrails, and nowhere else

The version is required. There's no default, so a reference never resolves against a moving ref by accident.

The version is a git tag, and the guardrail is downloaded from GitHub at that tag. The CLI doesn't carry a copy of the library, so upgrading a guardrail means changing the tag in the reference, never upgrading the CLI. Any tag works; the library's releases are listed on GitHub.

A reference with no prefix resolves against buildnote/guardrails first, then against each guardrails.sources entry in turn. That way your own library adds guardrails next to the shipped ones without replacing them. A github://<owner>/<repository>/ prefix ties a reference to one repository instead. A source that is a local directory has a single version, so the tag is ignored there.

Definition

Field	Required	Description
id	yes	Must match the directory path under the source root.
version	yes	The library version the guardrail was written for. The tag in the reference decides what runs, and the two aren't compared.
name	yes	A human readable name, shown on the verdict and in the pull request comment.
category	yes	The namespace, which is the first path segment.
description	yes	One sentence describing what the check enforces.
severity	yes	error, warning or info. A check entry can override it.
check.run	yes	The script path, relative to the guardrail directory. Its extension picks the interpreter, so you don't declare one.
check.timeoutSeconds	no	The maximum time the script can run, in seconds. Defaults to 60.
remediation	no	Markdown that tells someone how to fix a violation. It's shown under the violations in the pull request comment, so it can be a paragraph, a list or a fenced command block, not just a link.
inputs	no	A map from input name to { "default": "...", "description": "..." }. The description is markdown and appears in the input's row in these docs.
maxViolations	no	How many violations are allowed before the verdict is failed. Defaults to 0.

Input defaults are plain strings, so a definition never substitutes values into the script.

Interpreters

The check script's extension decides what runs it. You don't declare an interpreter, and a definition can't name a command of its own.

Script	Runs as
check.py	python3
check.sh, check.bash	bash

Any other extension is rejected when the guardrail loads, just like a malformed definition. The CLI reports it and exits with a non zero code whatever failOn says. The shared library's contract still requires a shebang, and it's worth keeping accurate for anyone reading the file, but the CLI doesn't use it.

Script contract

The CLI writes the script to a temporary file and runs it with the matching interpreter from your working directory, with these environment variables set:

• GUARDRAIL_INPUT_<NAME> for each input, in upper case, such as GUARDRAIL_INPUT_BASEREF. The value is the definition's default, overridden by any with value in your check entry.
• GUARDRAIL_ID, GUARDRAIL_VERSION, GUARDRAIL_SEVERITY.
• GUARDRAIL_LIB_DIR, the directory holding the Python helper described below.
• GUARDRAIL_FACTS, the file holding the facts from every collector the guardrail declared. It's empty when the guardrail declared none.
• BUILDNOTE_ORG, BUILDNOTE_PROJECT, BUILDNOTE_MODULE, BUILDNOTE_BUILD, BUILDNOTE_SHA, BUILDNOTE_REF.

The script prints a single JSON object to stdout and nothing else. Anything it wants to log goes to stderr.

{
  "violations": [
    {
      "evidence": "a1b2c3d update stuff",
      "message": "Commit a1b2c3d does not follow Conventional Commits"
    }
  ]
}

Field	Description
violations	An array of { evidence, message }. If it's missing or empty, the check passed. Your exemptions are matched against evidence, and message is what the comment and the event show.
skip	An optional string. If it's present, the check couldn't gather its evidence, so the verdict is skipped and the string is the reason.
verdict	An optional explicit passed, failed or skipped, for a check that decides its own verdict.

If verdict is missing, the CLI works it out: a skip reason gives skipped, more violations than maxViolations gives failed, and anything else gives passed.

The exit code is ignored when stdout parses as a result object, so your script can exit with a non zero code when it finds something. If stdout doesn't parse, the verdict is skipped and the last stderr line is the reason, because a broken script should never fail someone's build.

remediation is optional in the format, but a test holds every guardrail in the shared library to a stricter contract. Each guardrail must:

• load successfully
• have a category that matches the first segment of its id
• have a script that starts with a shebang
• give every declared input a default and a description
• only declare collectors in collect that the library includes
• ship a test next to its script
• include remediation written as markdown, not just a link
• pass when run over an empty commit range, and skip when run against a ref that doesn't resolve

The same test generates these pages, so a guardrail can't ship without documentation.

The Python helper

The CLI writes guardrail.py next to every check script it runs and puts that directory in GUARDRAIL_LIB_DIR. Because the helper sits next to the script, it's already on sys.path, so a Python guardrail can simply import it:

#!/usr/bin/env python3
from guardrail import Guardrail

with Guardrail() as guardrail:
    path = guardrail.input("path", "README.md")
    described = guardrail.facts("files")["files"][path]

    if not described["present"]:
        guardrail.violation(path, "No README at %s" % path)

When the block ends, the context manager prints the result and sets the exit code. It also turns a skip into a skip result, so your check never has to build JSON itself.

Member	What it does
input(name, default="")	Reads GUARDRAIL_INPUT_<NAME>, falling back to the default when it's unset or empty.
number(name, default)	Reads an input as an integer, and skips if it isn't a number.
facts(name, reason)	Reads one collector's document from GUARDRAIL_FACTS, and skips with reason if it wasn't collected.
optional(name, default)	Like facts, but returns default if nothing was collected. Use it when a collector finding nothing is a valid answer for your check, not missing evidence.
run(*command)	Runs a command, captures its output and returns a subprocess.CompletedProcess. Skips if the executable isn't on the runner.
violation(evidence, message)	Adds a violation to the result.
skip(reason)	Ends the check with a skip verdict and that reason.
log(message)	Writes a line to stderr.
id, version, severity	The guardrail's own metadata, read from the environment.

Use run only when no collector gathers what your check needs. It exists because an executable missing from the runner means missing evidence, not a violation. run skips in that case instead of raising an error, so your check doesn't have to handle it. What run does not decide is what a non zero exit code means. It returns the CompletedProcess, and your check decides:

if guardrail.run("mytool", "--version").returncode != 0:
    guardrail.skip("mytool is not installed on this runner")

That's the whole helper. It handles the plumbing every guardrail needs and knows nothing about what any guardrail checks, so you can understand one guardrail without reading another. Each guardrail Buildnote ships is a single script that gets its evidence from a collector instead of gathering it.

Collecting instead of shelling out

Instead of running git or reading build files itself, a guardrail declares a collector and reads what it gathered:

{
  "collect": ["git"],
  "check": { "run": "conventional-commits.py" }
}

with Guardrail() as guardrail:
    git = guardrail.facts("git")

    if not git["resolved"]:
        guardrail.skip("base ref %s is not resolvable" % git["baseRef"])

    for commit in git["commits"]:
        if commit["merge"]:
            guardrail.violation(commit["short"], "Commit %s is a merge commit" % commit["short"])

Before running the first check, the CLI runs every collector that any configured guardrail declares. It writes the document each check asked for to the file named by GUARDRAIL_FACTS. Two guardrails that ask for the same data share a single collection, and neither pays for what only the other needed. Everything collected is attached to the run event as guardrail-facts.json, so each run in Buildnote includes the evidence behind its verdicts, and you can see what the checks saw without rerunning the build. Every guardrail Buildnote ships works this way. The check contains only the rule itself, which also means you can test it without a real repository.

A collector is a script, like a check, and has its own directory, like a guardrail. It lives in collectors/ and holds a collector.json, the script, an icon.svg and its test:

collectors/<name>/collector.json
collectors/<name>/<name>.py
collectors/<name>/icon.svg
collectors/<name>/test_<name>.py

The directory name is the collector name and the key its facts appear under. The definition next to it must declare that same name as its id. Collectors describes every field. A collector script uses the Collector helper instead of Guardrail:

#!/usr/bin/env python3
from guardrail import Collector

with Collector() as collector:
    base_ref = collector.input("baseRef", "origin/main")
    inside = collector.run("git", "rev-parse", "--git-dir")

    collector.facts["repository"] = inside is not None and inside.returncode == 0

    if not collector.facts["repository"]:
        collector.invalid("not a git repository")

Member	What it does
facts	The dictionary printed when the block ends.
input(name, default="") number(name, default)	Read the inputs the collector declares, the same as for a guardrail.
collected(name)	Reads the facts of a collector this one declares in collect. Marks this collector incomplete if that one collected nothing.
optional(name, default)	Like collected, but returns default if nothing was collected, so one collector finding nothing doesn't stop this one.
run(*command)	Runs a command. Returns None if the executable isn't on the runner, instead of skipping.
invalid(reason)	Prints what was gathered so far and stops. Use it when the set of facts can't be completed.
nothing(reason)	Prints nothing and stops. Use it for a project this collector has nothing to say about. Its key is then left out of the facts entirely, instead of holding a document full of empty values.
log(message)	Writes a line to stderr.

A collector never makes a judgement. It reports what's there and lets the guardrail decide. {"repository": false} is a fact, not a skip, which is why the skip reasons in the shipped guardrails read the same as they did when each guardrail ran its own commands.

The definition declares every one of those facts as a path into the document, with a description of what it means. Guardrail authors read this instead of the script:

{
  "facts": {
    "repository": "Whether the CLI is running inside a git repository.",
    "commits[].subject": "First line of the commit message."
  }
}

To keep the declaration accurate, it's checked from both sides. The collector's own tests walk the facts every fixture collects and fail on any path the definition doesn't declare, including nested paths. They also fail when the definition declares a path that no fixture ever collects. As a final safety net, the contract test runs every shipped collector for real. If you rename a fact, update the definition in the same commit.

The helper lives in the library's lib/ directory and is downloaded at the same tag as the guardrail using it. A source without a lib/ of its own gets the helper from buildnote/guardrails, so guardrails from any source can import it, and the helper only ever gains new members. A sha256 pin doesn't cover it. There's no equivalent for bash or sh: a shell guardrail builds its own result.

Building the result without the helper

A Python guardrail has json in the standard library, so building the result object is easy even without the helper. A shell guardrail doesn't. Build the JSON with string handling instead of calling jq, because you can't assume the runner has it, and a guardrail that skips because of a missing tool reports nothing useful. If you prefer jq, print {"skip":"jq not available"} when command -v jq fails.

Whichever interpreter your extension picks has to be installed on the runner. If it's missing, the verdict is skipped with a reason, never a failure, so a .py guardrail won't break a build on a runner without python3.

Security

Guardrail scripts run on your runner with the same access as the rest of your build, so treat a guardrail source like any other code your CI executes.

• Every guardrail is downloaded at the start of the run, together with the collector scripts and helper it declares, so the runner needs access to raw.githubusercontent.com. The reference's version is the git ref it's downloaded from, so point it at a tag you trust, never a branch.
• Run buildnote --verbose guardrails to print where each check and collector script was resolved from, along with its SHA-256.
• Add a sha256 to a check entry to pin its check script. If the script doesn't match, the guardrail is skipped and the script never runs. The pin covers the check script only, not the collector scripts it reads facts from.

Where guardrails come from

A reference with no prefix resolves against buildnote/guardrails on GitHub first, then against each guardrails.sources entry in the order you list them. A reference that names a repository, github://<owner>/<repository>/<category>/<name>@<version>, resolves against that repository only. The collectors a guardrail declares come from the same place as the guardrail, and fall back to buildnote/guardrails when that place doesn't ship them.

To share a guardrail across your own repositories, publish it in a repository or directory of your own and list it in sources. Guardrails added to buildnote/guardrails reach everyone once they are released under a new tag.