buildnote.json

The buildnote.json file configures the Buildnote CLI. It defines how the CLI collects and reports test results and build artifacts.

File Location

Place the configuration file in your project root, named either:

.buildnote.json (preferred)
buildnote.json (alternative)

You can also point the CLI at a different path with the --config argument.

buildnote --config path/to/config.json command <options>

Configuration

json

{
  "collect": {
    ...
  },
  "report": {
    ...
  },
  "guardrails": {
    ...
  }
}

collect: Configuration for data collection (see Collectors)
report: Configuration for reporting outputs (see Reporters)
guardrails: Policies run by buildnote guardrails (see the Guardrails reference)

Collectors

The collect section defines how the CLI gathers events and artifacts.

Test Results

Configures which test result formats the CLI collects.

json

{
  "collect": {
    "junit": {...},
    "qt": {...},
    "dart": {...},
    "nunit": {...},
    "ctrf": {...},
    "xunit": {...},
    "testng": {...},
    "robot": {...},
    "cucumber": {...},
    "trx": {...},
    "otr": {...}
  }
}

The following test collectors are available:

junit - JUnit XML (used by the JVM, the Mocha test reporter and the Jest test reporter)
qt - QT XML test reports (https://doc.qt.io/qt-6/qtest-overview.html#logging-options)
dart - Dart JSON test reporter (https://github.com/dart-lang/test/blob/master/pkgs/test/doc/json_reporter.md)
nunit - NUnit XML (v2.x and v3.x, used by .NET, Microsoft frameworks, https://nunit.org/)
ctrf - Common Test Report Format (a test report specification that gives test results the same structure across all tools and frameworks, https://ctrf.io/docs/intro)
xunit - XUnit XML (used by .NET, Microsoft frameworks, https://xunit.net/)
testng - TestNG XML format (used by JVM, https://testng.org/)
robot - Robot Framework XML format (used by Python, the JVM and others, https://robotframework.org/)
cucumber - Cucumber JSON (acceptance test framework for many languages, https://cucumber.io/docs/cucumber/reporting/#built-in-reporter-plugins)
trx - MSTest TRX format (used by Microsoft frameworks, https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-extensions-test-reports)
otr - Open Test Reporting XML (event-based and hierarchical formats, maintained by the JUnit team and used by the JUnit Platform, https://github.com/ota4j-team/open-test-reporting)

Example

json

{
  "enabled": true,
  "include": [
    "**/pattern.xml"
  ],
  "exclude": [
    "**/build/*"
  ],
  "metadata": {
    "some-constant": "value",
    "some-expression": "{{env.USER}}"
  }
}

Properties:

enabled (boolean): Enable/disable this collector
include (string[]): Glob patterns for files to include
exclude (string[]): Glob patterns for files to exclude
metadata (object): Additional key-value metadata. Supports expressions.

Files

Collects any files you choose as build artifacts:

json

{
  "collect": {
    "files": {
      ...
    }
  }
}

json

{
  "files": {
    "enabled": true,
    "include": [
      "./documentation/**/*",
      "**/*.png",
      "**/*.svg",
      "**/*.md"
    ],
    "exclude": [
      "**/node_modules",
      "**/.gradle"
    ],
    "metadata": {
      "foo.bar": "{{env.USER}}"
    }
  }
}

Events

Collects Buildnote event files.

json

{
  "collect": {
    "events": {
      ...
    }
  }
}

json

{
  "events": {
    "enabled": true,
    "include": [
      "**/buildnote-*-events.ndjson"
    ]
  }
}

Metadata

Global metadata that is added to all collected data:

json

{
  "collect": {
    "metadata": {
      "modified user": "User is {{env.USER}}"
    }
  }
}

Reporters

The report section configures where reports go and what format they use.

json

{
  "report": {
    "slack": {...},
    "discord": {...},
    "github": {...},
    "html": {...},
    "template": {...}
  }
}

Properties:

slack - Sends a Slack notification
discord - Sends a Discord notification
github - Creates a step summary and an issue comment in GitHub
html - Writes the report as an HTML file
template - The template used for the report

Slack

json

{
  "slack": {
    "enabled": true,
    "condition": "{{successful || failed}}",
    "url": "{{env.SLACK_WH}}",
    "title": "E2E test {{status}}",
    "titleUrl": "{{source.url}}",
    "template": {
      "components": [
        ...
      ]
    }
  }
}

Properties:

enabled (boolean): Enable/disable reporter
condition (string, optional): Boolean expression to conditionally enable (e.g., "{{successful || failed}}")
url (string): Slack webhook URL
title (string, optional): Notification title
titleUrl (string, optional): URL for the title link
template (object, optional): Template configuration with components

Discord

json

{
  "discord": {
    "enabled": true,
    "condition": "{{successful || failed}}",
    "url": "{{env.DISCORD_WH}}",
    "title": "E2E test {{status}}",
    "titleUrl": "{{source.url}}",
    "template": {
      "components": [
        ...
      ]
    }
  }
}

Properties:

enabled (boolean): Enable/disable reporter
condition (string, optional): Boolean expression to conditionally enable (e.g., "{{successful || failed}}")
url (string): Discord webhook URL
title (string, optional): Notification title
titleUrl (string, optional): URL for the title link
template (object, optional): Template configuration with components

GitHub

json

{
  "github": {
    "enabled": true,
    "condition": "{{successful}}",
    "token": "{{env.GITHUB_TOKEN}}",
    "commentEnabled": true,
    "commentUpdates": true,
    "commentTitle": "Test Results",
    "commentTitleUrl": "{{source.url}}",
    "template": {
      "components": [
        ...
      ]
    }
  }
}

Properties:

enabled (boolean): Enable/disable reporter
condition (string, optional): Boolean expression to conditionally enable (e.g., "{{successful || failed}}")
token (string, optional): GitHub token
commentEnabled (boolean, default: true): Enable PR/commit comments
commentUpdates (boolean, default: true): Update existing comments instead of creating new ones
commentTitle (string, optional): Comment title
commentTitleUrl (string, optional): URL for the comment title
template (object, optional): Template configuration with components

HTML

json

{
  "html": {
    "enabled": true,
    "outputFile": "build/htmlreps/report.html",
    "template": {
      "components": [
        ...
      ]
    }
  }
}

Properties:

enabled (boolean): Enable/disable reporter
condition (string, optional): Boolean expression to conditionally enable (e.g., "{{successful || failed}}")
outputFile (string): Path for the output HTML file
template (object, optional): Template configuration with components

Template

The template object defines the report structure:

json

{
  "template": {
    "components": [
      {
        "type": "testSummary"
      },
      {
        "type": "section",
        "title": "Details",
        "collapsible": true,
        "components": [
          {
            "type": "tagSummary"
          },
          {
            "type": "tests"
          }
        ]
      }
    ]
  }
}

If you don't specify a template, the CLI uses the default one.

Components

Components define the structure and content of a report. You list them in the template.components array.

markdown

Renders markdown content:

json

{
  "type": "markdown",
  "text": "# hello\n\n- one list\n- 2 list\n\n---\n\ntext content"
}

Properties:

text (string): Markdown content. Supports template variables.

alert

Displays an alert message with a severity level:

json

{
  "type": "alert",
  "severity": "warning",
  "text": "Some important message"
}

Properties:

severity (enum): note, tip, important, warning, caution
text (string): Alert message. Supports template variables.

testSummary

Displays a summary of test results:

json

{
  "type": "testSummary"
}

tagSummary

Displays a summary grouped by tags:

json

{
  "type": "tagSummary"
}

tests

Lists detailed test results:

json

{
  "type": "tests"
}

section

Groups components into a section that can be collapsed:

json

{
  "type": "section",
  "title": "More details...",
  "collapsible": true,
  "collapsed": true,
  "components": [
    {
      "type": "testSummary"
    },
    {
      "type": "tests"
    }
  ]
}

Properties:

title (string): Section title. Supports template variables.
collapsible (boolean, default: false): Whether the section can be collapsed
collapsed (boolean, default: true): Initial collapsed state
components (array): Child components

property

Displays a custom property, optionally compared against a baseline:

number

json

{
  "type": "property",
  "name": "Code Coverage",
  "valueType": "number",
  "format": "0.00'%'",
  "value": {
    "ref": "{{ref}}",
    "org": "my-org",
    "project": "my-project",
    "module": "my-module",
    "build": "build-id"
  },
  "baseline": {
    "ref": "main",
    "module": "my-module",
    "expectedTrend": "up",
    "tolerance": 0.05
  }
}

name (string): Property display name
valueType: "number"
format (string, optional): Number format pattern
value (object, optional): Query configuration for property value
- org (string, optional): Organization ID
- project (string, optional): Project ID
- module (string, optional): Module ID
- build (string, optional): Build ID
- ref (string, optional): Commit reference
baseline (object, optional): Query configuration for baseline comparison
- org (string, optional): Organization ID
- project (string, optional): Project ID
- module (string, optional): Module ID
- build (string, optional): Build ID
- ref (string, optional): Commit reference
- expectedTrend (enum, optional): up, down, same
- tolerance (number, optional): Acceptable variance threshold

Number Format Patterns

Symbol	Location	Meaning
`0`	Number	Digit - always shows a digit, using zero if necessary
`#`	Number	Digit - shows a digit if present, otherwise nothing
`.`	Number	Decimal separator or monetary decimal separator
`-`	Number	Minus sign
`,`	Number	Grouping separator or monetary grouping separator
`E`	Number	Separates mantissa and exponent in scientific notation
`;`	Subpattern boundary	Separates positive and negative subpatterns
`%`	Prefix or suffix	Multiply by 100 and show as percentage
`‰` (`\u2030`)	Prefix or suffix	Multiply by 1000 and show as per mille
`¤` (`\u00A4`)	Prefix or suffix	Currency sign, replaced by currency symbol
`'`	Prefix or suffix	Used to quote special characters in prefix or suffix

Common Format Examples

Format Pattern	Value	Output	Description
`"0.00"`	123.4	`123.40`	Always 2 decimal places
`"#.##"`	123.4	`123.4`	Up to 2 decimal places
`"#,##0"`	1234567	`1,234,567`	Grouping separator
`"#,##0.00"`	1234.5	`1,234.50`	Grouping + 2 decimals
`"0.00'%'"`	0.856	`0.86%`	Quoted percent symbol
`"0.00'%'"`	85.6	`85.60%`	Display as-is with %
`"#,##0.##'%'"`	85.678	`85.68%`	Grouping + decimals + %
`"0.00E0"`	1234	`1.23E3`	Scientific notation
`"#,##0.00;(#,##0.00)"`	-1234.5	`(1,234.50)`	Negative in parentheses
`"¤#,##0.00"`	1234.5	`$1,234.50`	Currency symbol
`"'$'#,##0.00"`	1234.5	`$1,234.50`	Literal dollar sign

Quoting Special Characters

Use single quotes ' to include literal text in the format pattern:

"0.00'%'" - Literal percent sign
"'USD '#,##0.00" - Literal "USD " prefix
"#,##0' units'" - Literal " units" suffix

string

json

{
  "type": "property",
  "name": "Another property",
  "valueType": "string",
  "value": {
    "ref": "{{ref}}"
  }
}

name (string): Property display name
valueType: "string"
value (object, optional): Query configuration for property value
- org (string, optional): Organization ID
- project (string, optional): Project ID
- module (string, optional): Module ID
- build (string, optional): Build ID
- ref (string, optional): Commit reference

Complete Example

The example below is a complete buildnote.json that uses all the available features.

Click to see full example

json

{
  "collect": {
    "junit": {
      "enabled": true,
      "include": [
        "**/TEST-*.xml"
      ],
      "metadata": {
        "test-user": "{{env.USER}}"
      }
    },
    "files": {
      "enabled": true,
      "include": [
        "./documentation/**/*",
        "**/*.png",
        "**/*.svg",
        "**/*.md"
      ],
      "exclude": [
        "**/node_modules",
        "**/.gradle"
      ]
    }
  },
  "report": {
    "discord": {
      "enabled": true,
      "url": "{{env.DISCORD_WH}}",
      "condition": "{{successful || failed}}",
      "title": "E2E test {{status}} - {{env.GITHUB_WORKFLOW}}"
    },
    "github": {
      "enabled": true
    },
    "html": {
      "enabled": true,
      "outputFile": "build/htmlreps/report.html"
    },
    "template": {
      "components": [
        {
          "type": "markdown",
          "text": "# Build Report\n\nStatus: {{status}}"
        },
        {
          "type": "testSummary"
        },
        {
          "type": "section",
          "title": "More details",
          "collapsible": true,
          "components": [
            {
              "type": "alert",
              "severity": "note",
              "text": "Important information"
            },
            {
              "type": "tagSummary"
            },
            {
              "type": "tests"
            }
          ]
        }
      ]
    }
  }
}

buildnote.json ​

File Location ​

Configuration ​

Collectors ​

Test Results ​

Files ​

Events ​

Metadata ​

Reporters ​

Slack ​

Discord ​

GitHub ​

HTML ​

Template ​

Components ​

markdown ​

alert ​

testSummary ​

tagSummary ​

tests ​

section ​

property ​

number ​

string ​

Complete Example ​

buildnote.json

File Location

Configuration

Collectors

Test Results

Files

Events

Metadata

Reporters

Slack

Discord

GitHub

HTML

Template

Components

markdown

alert

testSummary

tagSummary

tests

section

property

number

string

Complete Example