buildnote report
Collects test results and reports on them
Usage
buildnote report <options>Options
--source
Where the report data comes from: local or remote.
Local Mode
In local mode, reports are built from events collected on this machine, as configured by command line options or the buildnote.json config.
Remote Mode
In remote mode, reports are built from events already collected and stored by the Buildnote service. Remote mode requires the BUILDNOTE_API_KEY environment variable.
Values: (local|remote)
--org
Organisation identifier
Values: value
--project
Project identifier
Values: value
--module
Module identifier
Values: value
--build
Build identifier
Values: value
--sha
Commit SHA for the upload, such as 58e53b26ced5fffc72ba5427535e7b5867950b13
Values: value
--ref
Commit ref for the upload, such as refs/heads/main
Values: value
--template
Path to a template file that formats the report.
Example:
--template "template.json"This option overrides the report.template value in the buildnote.json config file.
Values: value
-h, --help
Show this message and exit
JUnit Collector
Options for the JUnit collector.
--junit-enabled
Turns the JUnit collector on or off.
Example:
--junit-enabled trueThis flag overrides the collect.junit.enabled value in the buildnote.json config file.
Values: [true|false]
--junit-include
Glob pattern for the files to include.
Example that includes nested files:
--junit-include "**/file-*.xml"Example with several patterns:
--junit-include "**/file-*.xml" --junit-include "another-file.txt"Patterns you pass here are added to the collect.junit.include values in the buildnote.json config file.
Values: text
--junit-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--junit-exclude "**/out"Example with several patterns:
--junit-exclude "**/out" --junit-exclude "build"Patterns you pass here are added to the collect.junit.exclude values in the buildnote.json config file.
Values: text
--junit-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--junit-metadata user="$USER" --junit-metadata host="$HOST"Metadata you pass here is added to the collect.junit.metadata values in the buildnote.json config file.
Values: value
TestNG Collector
Options for the TestNG collector.
--testng-enabled
Turns the TestNG collector on or off.
Example:
--testng-enabled trueThis flag overrides the collect.testng.enabled value in the buildnote.json config file.
Values: [true|false]
--testng-include
Glob pattern for the files to include.
Example that includes nested files:
--testng-include "**/file-*.xml"Example with several patterns:
--testng-include "**/file-*.xml" --testng-include "another-file.txt"Patterns you pass here are added to the collect.testng.include values in the buildnote.json config file.
Values: text
--testng-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--testng-exclude "**/out"Example with several patterns:
--testng-exclude "**/out" --testng-exclude "build"Patterns you pass here are added to the collect.testng.exclude values in the buildnote.json config file.
Values: text
--testng-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--testng-metadata user="$USER" --testng-metadata host="$HOST"Metadata you pass here is added to the collect.testng.metadata values in the buildnote.json config file.
Values: value
Cucumber Collector
Options for the Cucumber collector.
--cucumber-enabled
Turns the Cucumber collector on or off.
Example:
--cucumber-enabled trueThis flag overrides the collect.cucumber.enabled value in the buildnote.json config file.
Values: [true|false]
--cucumber-include
Glob pattern for the files to include.
Example that includes nested files:
--cucumber-include "**/file-*.xml"Example with several patterns:
--cucumber-include "**/file-*.xml" --cucumber-include "another-file.txt"Patterns you pass here are added to the collect.cucumber.include values in the buildnote.json config file.
Values: text
--cucumber-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--cucumber-exclude "**/out"Example with several patterns:
--cucumber-exclude "**/out" --cucumber-exclude "build"Patterns you pass here are added to the collect.cucumber.exclude values in the buildnote.json config file.
Values: text
--cucumber-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--cucumber-metadata user="$USER" --cucumber-metadata host="$HOST"Metadata you pass here is added to the collect.cucumber.metadata values in the buildnote.json config file.
Values: value
Dart/Flutter Collector
Options for the Dart/Flutter collector.
--dart-enabled
Turns the Dart/Flutter collector on or off.
Example:
--dart-enabled trueThis flag overrides the collect.dart.enabled value in the buildnote.json config file.
Values: [true|false]
--dart-include
Glob pattern for the files to include.
Example that includes nested files:
--dart-include "**/file-*.xml"Example with several patterns:
--dart-include "**/file-*.xml" --dart-include "another-file.txt"Patterns you pass here are added to the collect.dart.include values in the buildnote.json config file.
Values: text
--dart-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--dart-exclude "**/out"Example with several patterns:
--dart-exclude "**/out" --dart-exclude "build"Patterns you pass here are added to the collect.dart.exclude values in the buildnote.json config file.
Values: text
--dart-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--dart-metadata user="$USER" --dart-metadata host="$HOST"Metadata you pass here is added to the collect.dart.metadata values in the buildnote.json config file.
Values: value
NUnit Collector
Options for the NUnit collector.
--nunit-enabled
Turns the NUnit collector on or off.
Example:
--nunit-enabled trueThis flag overrides the collect.nunit.enabled value in the buildnote.json config file.
Values: [true|false]
--nunit-include
Glob pattern for the files to include.
Example that includes nested files:
--nunit-include "**/file-*.xml"Example with several patterns:
--nunit-include "**/file-*.xml" --nunit-include "another-file.txt"Patterns you pass here are added to the collect.nunit.include values in the buildnote.json config file.
Values: text
--nunit-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--nunit-exclude "**/out"Example with several patterns:
--nunit-exclude "**/out" --nunit-exclude "build"Patterns you pass here are added to the collect.nunit.exclude values in the buildnote.json config file.
Values: text
--nunit-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--nunit-metadata user="$USER" --nunit-metadata host="$HOST"Metadata you pass here is added to the collect.nunit.metadata values in the buildnote.json config file.
Values: value
XUnit Collector
Options for the XUnit collector.
--xunit-enabled
Turns the XUnit collector on or off.
Example:
--xunit-enabled trueThis flag overrides the collect.xunit.enabled value in the buildnote.json config file.
Values: [true|false]
--xunit-include
Glob pattern for the files to include.
Example that includes nested files:
--xunit-include "**/file-*.xml"Example with several patterns:
--xunit-include "**/file-*.xml" --xunit-include "another-file.txt"Patterns you pass here are added to the collect.xunit.include values in the buildnote.json config file.
Values: text
--xunit-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--xunit-exclude "**/out"Example with several patterns:
--xunit-exclude "**/out" --xunit-exclude "build"Patterns you pass here are added to the collect.xunit.exclude values in the buildnote.json config file.
Values: text
--xunit-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--xunit-metadata user="$USER" --xunit-metadata host="$HOST"Metadata you pass here is added to the collect.xunit.metadata values in the buildnote.json config file.
Values: value
CTRF Collector
Options for the CTRF collector.
--ctrf-enabled
Turns the CTRF collector on or off.
Example:
--ctrf-enabled trueThis flag overrides the collect.ctrf.enabled value in the buildnote.json config file.
Values: [true|false]
--ctrf-include
Glob pattern for the files to include.
Example that includes nested files:
--ctrf-include "**/file-*.xml"Example with several patterns:
--ctrf-include "**/file-*.xml" --ctrf-include "another-file.txt"Patterns you pass here are added to the collect.ctrf.include values in the buildnote.json config file.
Values: text
--ctrf-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--ctrf-exclude "**/out"Example with several patterns:
--ctrf-exclude "**/out" --ctrf-exclude "build"Patterns you pass here are added to the collect.ctrf.exclude values in the buildnote.json config file.
Values: text
--ctrf-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--ctrf-metadata user="$USER" --ctrf-metadata host="$HOST"Metadata you pass here is added to the collect.ctrf.metadata values in the buildnote.json config file.
Values: value
TRX Collector
Options for the TRX collector.
--trx-enabled
Turns the TRX collector on or off.
Example:
--trx-enabled trueThis flag overrides the collect.trx.enabled value in the buildnote.json config file.
Values: [true|false]
--trx-include
Glob pattern for the files to include.
Example that includes nested files:
--trx-include "**/file-*.xml"Example with several patterns:
--trx-include "**/file-*.xml" --trx-include "another-file.txt"Patterns you pass here are added to the collect.trx.include values in the buildnote.json config file.
Values: text
--trx-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--trx-exclude "**/out"Example with several patterns:
--trx-exclude "**/out" --trx-exclude "build"Patterns you pass here are added to the collect.trx.exclude values in the buildnote.json config file.
Values: text
--trx-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--trx-metadata user="$USER" --trx-metadata host="$HOST"Metadata you pass here is added to the collect.trx.metadata values in the buildnote.json config file.
Values: value
Robot Framework Collector
Options for the Robot Framework collector.
--robot-enabled
Turns the Robot Framework collector on or off.
Example:
--robot-enabled trueThis flag overrides the collect.robot.enabled value in the buildnote.json config file.
Values: [true|false]
--robot-include
Glob pattern for the files to include.
Example that includes nested files:
--robot-include "**/file-*.xml"Example with several patterns:
--robot-include "**/file-*.xml" --robot-include "another-file.txt"Patterns you pass here are added to the collect.robot.include values in the buildnote.json config file.
Values: text
--robot-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--robot-exclude "**/out"Example with several patterns:
--robot-exclude "**/out" --robot-exclude "build"Patterns you pass here are added to the collect.robot.exclude values in the buildnote.json config file.
Values: text
--robot-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--robot-metadata user="$USER" --robot-metadata host="$HOST"Metadata you pass here is added to the collect.robot.metadata values in the buildnote.json config file.
Values: value
QT Test Collector
Options for the QT Test collector.
--qt-enabled
Turns the QT Test collector on or off.
Example:
--qt-enabled trueThis flag overrides the collect.qt.enabled value in the buildnote.json config file.
Values: [true|false]
--qt-include
Glob pattern for the files to include.
Example that includes nested files:
--qt-include "**/file-*.xml"Example with several patterns:
--qt-include "**/file-*.xml" --qt-include "another-file.txt"Patterns you pass here are added to the collect.qt.include values in the buildnote.json config file.
Values: text
--qt-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--qt-exclude "**/out"Example with several patterns:
--qt-exclude "**/out" --qt-exclude "build"Patterns you pass here are added to the collect.qt.exclude values in the buildnote.json config file.
Values: text
--qt-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--qt-metadata user="$USER" --qt-metadata host="$HOST"Metadata you pass here is added to the collect.qt.metadata values in the buildnote.json config file.
Values: value
Open Test Reporting Collector
Options for the Open Test Reporting collector.
--otr-enabled
Turns the Open Test Reporting collector on or off.
Example:
--otr-enabled trueThis flag overrides the collect.otr.enabled value in the buildnote.json config file.
Values: [true|false]
--otr-include
Glob pattern for the files to include.
Example that includes nested files:
--otr-include "**/file-*.xml"Example with several patterns:
--otr-include "**/file-*.xml" --otr-include "another-file.txt"Patterns you pass here are added to the collect.otr.include values in the buildnote.json config file.
Values: text
--otr-exclude
Glob pattern for the files to exclude.
Example that excludes nested directories:
--otr-exclude "**/out"Example with several patterns:
--otr-exclude "**/out" --otr-exclude "build"Patterns you pass here are added to the collect.otr.exclude values in the buildnote.json config file.
Values: text
--otr-metadata
Extra metadata to attach to the collected events.
Example with several metadata items:
--otr-metadata user="$USER" --otr-metadata host="$HOST"Metadata you pass here is added to the collect.otr.metadata values in the buildnote.json config file.
Values: value
Slack Reporter
Options for the Slack reporter.
--slack-enabled
Turns the Slack reporter on or off.
Example:
--slack-enabled trueThis flag overrides the report.slack.enabled value in the buildnote.json config file.
Values: [true|false]
--slack-condition
Expression that decides whether the Slack webhook is sent.
Example that reports failing builds only:
--slack-condition "{{failed}}"You can also use environment variables and combine expressions.
Example that combines several conditions:
--slack-condition "{{failed || successful || env.USER == 'some-user'}}"This option overrides the report.slack.condition value in the buildnote.json config file.
Values: text
--slack-template
Path to a template file that formats the report.
Example:
--slack-template "template.json"This option overrides the report.slack.template value in the buildnote.json config file.
Values: value
--slack-url
Slack webhook URL that notifications are sent to.
Example:
--slack-url "https://hooks.slack.com/services/..."The URL is evaluated as an expression, so you can read it from an environment variable, for example.
Example with an environment variable:
--slack-url "{{env.SLACK_WEBHOOK_URL}}"This option overrides the report.slack.url value in the buildnote.json config file.
Values: text
--slack-title
Title of the Slack message, replacing the default title.
Example with a fixed title:
--slack-title "Some title"The title is evaluated as an expression, so you can also use environment and context variables.
Example that builds the title from variables:
--slack-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"This option overrides the report.slack.title value in the buildnote.json config file.
Values: text
--slack-title-url
Link for the Slack message title, replacing the default link.
The link is evaluated as an expression, so you can build it from environment and context variables.
Example with a variable:
--slack-title-url "https://www.example.com/status/{{status}}"This option overrides the report.slack.titleUrl value in the buildnote.json config file.
Values: text
Discord Reporter
Options for the Discord reporter.
--discord-enabled
Turns the Discord reporter on or off.
Example:
--discord-enabled trueThis flag overrides the report.discord.enabled value in the buildnote.json config file.
Values: [true|false]
--discord-condition
Expression that decides whether the Discord webhook is sent.
Example that reports failing builds only:
--discord-condition "{{failed}}"You can also use environment variables and combine expressions.
Example that combines several conditions:
--discord-condition "{{failed || successful || env.USER == 'some-user'}}"This option overrides the report.discord.condition value in the buildnote.json config file.
Values: text
--discord-template
Path to a template file that formats the report.
Example:
--discord-template "template.json"This option overrides the report.discord.template value in the buildnote.json config file.
Values: value
--discord-url
Discord webhook URL that notifications are sent to.
Example:
--discord-url "https://discord.com/api/webhooks/..."The URL is evaluated as an expression, so you can read it from an environment variable, for example.
Example with an environment variable:
--discord-url "{{env.DISCORD_WEBHOOK_URL}}"This option overrides the report.discord.url value in the buildnote.json config file.
Values: text
--discord-title
Title of the Discord message, replacing the default title.
Example with a fixed title:
--discord-title "Some title"The title is evaluated as an expression, so you can also use environment and context variables.
Example that builds the title from variables:
--discord-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"This option overrides the report.discord.title value in the buildnote.json config file.
Values: text
--discord-title-url
Link for the Discord message title, replacing the default link.
The link is evaluated as an expression, so you can build it from environment and context variables.
Example with a variable:
--discord-title-url "https://www.example.com/status/{{status}}"This option overrides the report.discord.titleUrl value in the buildnote.json config file.
Values: text
GitHub Reporter
Options for the GitHub reporter.
--github-enabled
Turns the GitHub reporter on or off.
Example:
--github-enabled trueThis flag overrides the report.github.enabled value in the buildnote.json config file.
Values: [true|false]
--github-condition
Expression that decides whether the GitHub report runs.
Example that reports failing builds only:
--github-condition "{{failed}}"You can also use environment variables and combine expressions.
Example that combines several conditions:
--github-condition "{{failed || successful || env.USER == 'some-user'}}"This option overrides the report.github.condition value in the buildnote.json config file.
Values: text
--github-template
Path to a template file that formats the report.
Example:
--github-template "template.json"This option overrides the report.github.template value in the buildnote.json config file.
Values: value
--github-token
GitHub token the reporter uses.
Example:
--github-token "gha_..."The token is evaluated as an expression, so you can read it from an environment variable, for example.
Example with an environment variable:
--github-token "{{env.GITHUB_TOKEN}}"To use the default GitHub Actions token (secrets.GITHUB_TOKEN), you need to adjust the workflow job permissions.
The permissions you need depend on the repository type:
Public repositories:
permissions:
checks: write
pull-requests: writePrivate repositories:
permissions:
contents: read
issues: read
checks: write
pull-requests: writeNOTE
If you pass --github-comment-enabled false, the issue comment is turned off and you don't need the pull-requests: write permission.
This option overrides the report.github.token value in the buildnote.json config file.
Values: text
--github-comment-enabled
Turns the GitHub comment on pull requests on or off.
Example:
--github-comment-enabled trueThis flag overrides the report.github.commentEnabled value in the buildnote.json config file.
Values: [true|false]
--github-comment-updates
Turns updates to the existing GitHub comment on or off.
With true, the reporter updates its existing comment on the issue. With false, it adds a new issue comment each time.
Example:
--github-comment-updates trueThis flag overrides the report.github.commentUpdates value in the buildnote.json config file.
Values: [true|false]
--github-comment-title
Title of the GitHub issue comment, replacing the default title.
IMPORTANT
The reporter finds the comment to update by its title. If you report from several stages of the build with the same title, a later report can overwrite the comment from an earlier one.
Example with a fixed title:
--github-comment-title "Some title"The title is evaluated as an expression, so you can also use environment and context variables.
Example that builds the title from variables:
--github-comment-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"This option overrides the report.github.commentTitle value in the buildnote.json config file.
Values: text
--github-comment-title-url
Link for the GitHub issue comment title, replacing the default link.
The link is evaluated as an expression, so you can build it from environment and context variables.
Example with a variable:
--github-comment-title-url "https://www.example.com/status/{{status}}"This option overrides the report.github.commentTitleUrl value in the buildnote.json config file.
Values: text
Html Reporter
Options for the Html reporter.
--html-enabled
Turns the Html reporter on or off.
Example:
--html-enabled trueThis flag overrides the report.html.enabled value in the buildnote.json config file.
Values: [true|false]
--html-condition
Expression that decides whether the Html reporter runs.
Example that reports failing builds only:
--html-condition "{{failed}}"You can also use environment variables and combine expressions.
Example that combines several conditions:
--html-condition "{{failed || successful || env.USER == 'some-user'}}"This option overrides the report.html.condition value in the buildnote.json config file.
Values: text
--html-template
Path to a template file that formats the report.
Example:
--html-template "template.json"This option overrides the report.html.template value in the buildnote.json config file.
Values: value
--html-output-file
File the Html reporter writes to.
Example with a fixed output file:
--html-output-file "test-report.html"You can also use environment variables and combine expressions.
Example that builds the file name from an expression:
--html-output-file "{{env.USER}}/test-report.html"This option overrides the report.html.outputFile value in the buildnote.json config file.
Values: text
Context Variables
You can use these context variables in expressions.
always
Always true.
build
Build identifier for this run.
failed
true when any event in the report failed.
failed.count
Number of failed events in the report.
failed.percentage
Formatted percentage of failed events in the report.
Example: 0.97%
module
Name of the module.
never
Always false.
org
Name of the organisation.
project
Name of the project.
ref
Commit ref for this report.
sha
Commit SHA for this report.
skipped
true when every event in the report was skipped.
skipped.count
Number of skipped events in the report.
skipped.percentage
Formatted percentage of skipped events in the report.
Example: 0.97%
status
Overall status of the report.
Example: successful, failed or skipped
successful
true when no events in the report failed and at least one succeeded.
successful.count
Number of successful events in the report.
successful.percentage
Formatted percentage of successful events in the report.
Example: 0.97%
total
Total number of events in the report.