buildnote report

Collect and report on test results

Usage

buildnote report <options>

Options

--source

Specifies source data mode (local or remote).

Local Mode

When local source data mode is selected, reports will be produced from locally collected events configured via command line options or buildnote.json config.

Remote Mode

When remote source data specified, reports will be produced from events collected and stored in buildnote service. Configuring BUILDNOTE_API_KEY environment variable is required for remote mode.

Values: (local|remote)

--template

Path to template file used to format report.

Example:

shell

--template "template.json"

This option will replace report.template value set in buildnote.json config file.

Values: value

--summary-enabled

Produces extended summary if possible.

If BUILDNOTE_API_KEY environment variable is configured and this option is enabled, the summary will be additionally fetched from the Buildnote service.

Values: [true|false]

--org

Organisation identifier

Values: value

--project

Project identifier

Values: value

--module

Module identifier

Values: value

--build

Build identifier

Values: value

--sha

Commit SHA associated with the upload (e.g: 58e53b26ced5fffc72ba5427535e7b5867950b13

Values: value

--ref

Commit Ref associated with the upload (e.g:refs/heads/main)

Values: value

-h, --help

Show this message and exit

JUnit Collector

Options for collecting junit events.

--junit-enabled

Enable or disable junit collector.

Example:

shell

--junit-enabled true

This flag will replace collect.junit.enabled value set in buildnote.json config file.

Values: [true|false]

--junit-include

Glob pattern for files to include.

Example of including nested files:

shell

--junit-include "**/file-*.xml"

Example of multiple includes:

shell

--junit-include "**/file-*.xml" --junit-include "another-file.txt"

This option will merge with collect.junit.include values set in buildnote.json config file.

Values: text

--junit-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--junit-exclude "**/out"

Example of multiple excludes:

shell

--junit-exclude "**/out" --junit-exclude "build"

This option will merge with collect.junit.exclude values set in buildnote.json config file.

Values: text

--junit-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--junit-metadata user="$USER" --junit-metadata host="$HOST"

This option will merge with collect.junit.metadata values set in buildnote.json config file.

Values: value

TestNG Collector

Options for collecting testng events.

--testng-enabled

Enable or disable testng collector.

Example:

shell

--testng-enabled true

This flag will replace collect.testng.enabled value set in buildnote.json config file.

Values: [true|false]

--testng-include

Glob pattern for files to include.

Example of including nested files:

shell

--testng-include "**/file-*.xml"

Example of multiple includes:

shell

--testng-include "**/file-*.xml" --testng-include "another-file.txt"

This option will merge with collect.testng.include values set in buildnote.json config file.

Values: text

--testng-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--testng-exclude "**/out"

Example of multiple excludes:

shell

--testng-exclude "**/out" --testng-exclude "build"

This option will merge with collect.testng.exclude values set in buildnote.json config file.

Values: text

--testng-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--testng-metadata user="$USER" --testng-metadata host="$HOST"

This option will merge with collect.testng.metadata values set in buildnote.json config file.

Values: value

Cucumber Collector

Options for collecting cucumber events.

--cucumber-enabled

Enable or disable cucumber collector.

Example:

shell

--cucumber-enabled true

This flag will replace collect.cucumber.enabled value set in buildnote.json config file.

Values: [true|false]

--cucumber-include

Glob pattern for files to include.

Example of including nested files:

shell

--cucumber-include "**/file-*.xml"

Example of multiple includes:

shell

--cucumber-include "**/file-*.xml" --cucumber-include "another-file.txt"

This option will merge with collect.cucumber.include values set in buildnote.json config file.

Values: text

--cucumber-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--cucumber-exclude "**/out"

Example of multiple excludes:

shell

--cucumber-exclude "**/out" --cucumber-exclude "build"

This option will merge with collect.cucumber.exclude values set in buildnote.json config file.

Values: text

--cucumber-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--cucumber-metadata user="$USER" --cucumber-metadata host="$HOST"

This option will merge with collect.cucumber.metadata values set in buildnote.json config file.

Values: value

Dart/Flutter Collector

Options for collecting dart/flutter events.

--dart-enabled

Enable or disable dart/flutter collector.

Example:

shell

--dart-enabled true

This flag will replace collect.dart.enabled value set in buildnote.json config file.

Values: [true|false]

--dart-include

Glob pattern for files to include.

Example of including nested files:

shell

--dart-include "**/file-*.xml"

Example of multiple includes:

shell

--dart-include "**/file-*.xml" --dart-include "another-file.txt"

This option will merge with collect.dart.include values set in buildnote.json config file.

Values: text

--dart-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--dart-exclude "**/out"

Example of multiple excludes:

shell

--dart-exclude "**/out" --dart-exclude "build"

This option will merge with collect.dart.exclude values set in buildnote.json config file.

Values: text

--dart-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--dart-metadata user="$USER" --dart-metadata host="$HOST"

This option will merge with collect.dart.metadata values set in buildnote.json config file.

Values: value

NUnit Collector

Options for collecting nunit events.

--nunit-enabled

Enable or disable nunit collector.

Example:

shell

--nunit-enabled true

This flag will replace collect.nunit.enabled value set in buildnote.json config file.

Values: [true|false]

--nunit-include

Glob pattern for files to include.

Example of including nested files:

shell

--nunit-include "**/file-*.xml"

Example of multiple includes:

shell

--nunit-include "**/file-*.xml" --nunit-include "another-file.txt"

This option will merge with collect.nunit.include values set in buildnote.json config file.

Values: text

--nunit-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--nunit-exclude "**/out"

Example of multiple excludes:

shell

--nunit-exclude "**/out" --nunit-exclude "build"

This option will merge with collect.nunit.exclude values set in buildnote.json config file.

Values: text

--nunit-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--nunit-metadata user="$USER" --nunit-metadata host="$HOST"

This option will merge with collect.nunit.metadata values set in buildnote.json config file.

Values: value

XUnit Collector

Options for collecting xunit events.

--xunit-enabled

Enable or disable xunit collector.

Example:

shell

--xunit-enabled true

This flag will replace collect.xunit.enabled value set in buildnote.json config file.

Values: [true|false]

--xunit-include

Glob pattern for files to include.

Example of including nested files:

shell

--xunit-include "**/file-*.xml"

Example of multiple includes:

shell

--xunit-include "**/file-*.xml" --xunit-include "another-file.txt"

This option will merge with collect.xunit.include values set in buildnote.json config file.

Values: text

--xunit-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--xunit-exclude "**/out"

Example of multiple excludes:

shell

--xunit-exclude "**/out" --xunit-exclude "build"

This option will merge with collect.xunit.exclude values set in buildnote.json config file.

Values: text

--xunit-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--xunit-metadata user="$USER" --xunit-metadata host="$HOST"

This option will merge with collect.xunit.metadata values set in buildnote.json config file.

Values: value

CTRF Collector

Options for collecting ctrf events.

--ctrf-enabled

Enable or disable ctrf collector.

Example:

shell

--ctrf-enabled true

This flag will replace collect.ctrf.enabled value set in buildnote.json config file.

Values: [true|false]

--ctrf-include

Glob pattern for files to include.

Example of including nested files:

shell

--ctrf-include "**/file-*.xml"

Example of multiple includes:

shell

--ctrf-include "**/file-*.xml" --ctrf-include "another-file.txt"

This option will merge with collect.ctrf.include values set in buildnote.json config file.

Values: text

--ctrf-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--ctrf-exclude "**/out"

Example of multiple excludes:

shell

--ctrf-exclude "**/out" --ctrf-exclude "build"

This option will merge with collect.ctrf.exclude values set in buildnote.json config file.

Values: text

--ctrf-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--ctrf-metadata user="$USER" --ctrf-metadata host="$HOST"

This option will merge with collect.ctrf.metadata values set in buildnote.json config file.

Values: value

TRX Collector

Options for collecting trx events.

--trx-enabled

Enable or disable trx collector.

Example:

shell

--trx-enabled true

This flag will replace collect.trx.enabled value set in buildnote.json config file.

Values: [true|false]

--trx-include

Glob pattern for files to include.

Example of including nested files:

shell

--trx-include "**/file-*.xml"

Example of multiple includes:

shell

--trx-include "**/file-*.xml" --trx-include "another-file.txt"

This option will merge with collect.trx.include values set in buildnote.json config file.

Values: text

--trx-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--trx-exclude "**/out"

Example of multiple excludes:

shell

--trx-exclude "**/out" --trx-exclude "build"

This option will merge with collect.trx.exclude values set in buildnote.json config file.

Values: text

--trx-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--trx-metadata user="$USER" --trx-metadata host="$HOST"

This option will merge with collect.trx.metadata values set in buildnote.json config file.

Values: value

Robot Framework Collector

Options for collecting robot framework events.

--robot-enabled

Enable or disable robot framework collector.

Example:

shell

--robot-enabled true

This flag will replace collect.robot.enabled value set in buildnote.json config file.

Values: [true|false]

--robot-include

Glob pattern for files to include.

Example of including nested files:

shell

--robot-include "**/file-*.xml"

Example of multiple includes:

shell

--robot-include "**/file-*.xml" --robot-include "another-file.txt"

This option will merge with collect.robot.include values set in buildnote.json config file.

Values: text

--robot-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--robot-exclude "**/out"

Example of multiple excludes:

shell

--robot-exclude "**/out" --robot-exclude "build"

This option will merge with collect.robot.exclude values set in buildnote.json config file.

Values: text

--robot-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--robot-metadata user="$USER" --robot-metadata host="$HOST"

This option will merge with collect.robot.metadata values set in buildnote.json config file.

Values: value

QT Test Collector

Options for collecting qt test events.

--qt-enabled

Enable or disable qt test collector.

Example:

shell

--qt-enabled true

This flag will replace collect.qt.enabled value set in buildnote.json config file.

Values: [true|false]

--qt-include

Glob pattern for files to include.

Example of including nested files:

shell

--qt-include "**/file-*.xml"

Example of multiple includes:

shell

--qt-include "**/file-*.xml" --qt-include "another-file.txt"

This option will merge with collect.qt.include values set in buildnote.json config file.

Values: text

--qt-exclude

Glob pattern for files to exclude.

Example of excluding nested directories:

shell

--qt-exclude "**/out"

Example of multiple excludes:

shell

--qt-exclude "**/out" --qt-exclude "build"

This option will merge with collect.qt.exclude values set in buildnote.json config file.

Values: text

--qt-metadata

Additional metadata associated with events.

Example of multiple metadata items:

shell

--qt-metadata user="$USER" --qt-metadata host="$HOST"

This option will merge with collect.qt.metadata values set in buildnote.json config file.

Values: value

Slack Reporter

Options for slack reporter.

--slack-enabled

Enable or disable slack reporter.

Example:

shell

--slack-enabled true

This flag will replace report.slack.enabled value set in buildnote.json config file.

Values: [true|false]

--slack-condition

Slack webhook condition evaluated as expression.

Example for failing builds only:

shell

--slack-condition "{{failed}}"

You can also access environment variables and combine expressions.

Example of complex condition:

shell

--slack-condition "{{failed || successful || env.USER == 'some-user'}}"

This option will replace report.slack.condition value set in buildnote.json config file.

Values: text

--slack-template

Path to template file used to format report.

Example:

shell

--slack-template "template.json"

This option will replace report.slack.template value set in buildnote.json config file.

Values: value

--slack-url

Slack webhook notification url.

Example:

shell

--slack-url "https://hooks.slack.com/services/..."

Urls are evaluated as expressions and can, for example, be sourced from environment variables.

Example with environment variable:

shell

--slack-url "{{env.SLACK_WEBHOOK_URL}}"

This option will replace report.slack.url value set in buildnote.json config file.

Values: text

--slack-title

Slack message title that replaces default one.

Example of fixed title:

shell

--slack-title "Some title"

Title is evaluated as expressions so you can also access environment and context variables.

Example of complex title:

shell

--slack-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"

This option will replace report.slack.title value set in buildnote.json config file.

Values: text

--slack-title-url

Slack message title url that replaces default one.

Title url is evaluated as expressions and can be templated with environment and context variables.

Example with variable:

shell

--slack-title-url "https://www.example.com/status/{{status}}"

This option will replace report.slack.titleUrl value set in buildnote.json config file.

Values: text

Discord Reporter

Options for discord reporter.

--discord-enabled

Enable or disable discord reporter.

Example:

shell

--discord-enabled true

This flag will replace report.discord.enabled value set in buildnote.json config file.

Values: [true|false]

--discord-condition

Discord webhook condition evaluated as expression.

Example for failing builds only:

shell

--discord-condition "{{failed}}"

You can also access environment variables and combine expressions.

Example of complex condition:

shell

--discord-condition "{{failed || successful || env.USER == 'some-user'}}"

This option will replace report.discord.condition value set in buildnote.json config file.

Values: text

--discord-template

Path to template file used to format report.

Example:

shell

--discord-template "template.json"

This option will replace report.discord.template value set in buildnote.json config file.

Values: value

--discord-url

Discord webhook notification url.

Example:

shell

--discord-url "https://discord.com/api/webhooks/..."

Urls are evaluated as expressions and can, for example, be sourced from environment variables.

Example with environment variable:

shell

--discord-url "{{env.DISCORD_WEBHOOK_URL}}"

This option will replace report.discord.url value set in buildnote.json config file.

Values: text

--discord-title

Discord message title that replaces default one.

Example of fixed title:

shell

--discord-title "Some title"

Title is evaluated as expressions so you can also access environment and context variables.

Example of complex title:

shell

--discord-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"

This option will replace report.discord.title value set in buildnote.json config file.

Values: text

--discord-title-url

Discord message title url that replaces default one.

Title url is evaluated as expressions and can be templated with environment and context variables.

Example with variable:

shell

--discord-title-url "https://www.example.com/status/{{status}}"

This option will replace report.discord.titleUrl value set in buildnote.json config file.

Values: text

GitHub Reporter

Options for GitHub reporter.

--github-enabled

Enable or disable GitHub reporter.

Example:

shell

--github-enabled true

This flag will replace report.github.enabled value set in buildnote.json config file.

Values: [true|false]

--github-condition

GitHub report condition evaluated as expression.

Example for failing builds only:

shell

--github-condition "{{failed}}"

You can also access environment variables and combine expressions.

Example of complex condition:

shell

--github-condition "{{failed || successful || env.USER == 'some-user'}}"

This option will replace report.github.condition value set in buildnote.json config file.

Values: text

--github-template

Path to template file used to format report.

Example:

shell

--github-template "template.json"

This option will replace report.github.template value set in buildnote.json config file.

Values: value

--github-token

GitHub token.

Example:

shell

--github-token "gha_..."

Token is evaluated as expressions and can, for example, be sourced from environment variables.

Example with environment variable:

shell

--github-token "{{env.GITHUB_TOKEN}}"

If you would like to use default GitHub Action token (secrets.GITHUB_TOKEN) adjusting the workflow job permissions is required.

Those permissions vary by repository type:

Public repositories:

yaml

permissions:
  checks: write
  pull-requests: write

Private repositories:

yaml

permissions:
  contents: read
  issues: read
  checks: write
  pull-requests: write

NOTE

When using --github-comment-enabled false the issue comment is disabled and the pull-requests: write permission is not required.

This option will replace report.github.token value set in buildnote.json config file.

Values: text

--github-comment-enabled

Enable or disable GitHub comment on PRs.

Example:

shell

--github-comment-enabled true

This flag will replace report.github.commentEnabled value set in buildnote.json config file.

Values: [true|false]

--github-comment-updates

Enable or disable GitHub comment updates.

If true it updates existing comment on issues, otherwise set to false it will create a new issue comment each time.

Example:

shell

--github-comment-updates true

This flag will replace report.github.commentUpdates value set in buildnote.json config file.

Values: [true|false]

--github-comment-title

GitHub issue comment title that replaces default one.

IMPORTANT

GitHub comments will be updated based on comment title. This may cause previous comments to be overridden if multiple reports are collected in different stages of the build.

Example of fixed title:

shell

--github-comment-title "Some title"

Title is evaluated as expressions so you can also access environment and context variables.

Example of complex title:

shell

--github-comment-title "E2E test {{status}} - {{env.GITHUB_WORKFLOW}} - {{env.GITHUB_RUN_ID}} (attempt: {{env.GITHUB_RUN_ATTEMPT}})"

This option will replace report.github.commentTitle value set in buildnote.json config file.

Values: text

--github-comment-title-url

GitHub issue comment title url that replaces default one.

Title url is evaluated as expressions and can be templated with environment and context variables.

Example with variable:

shell

--github-comment-title-url "https://www.example.com/status/{{status}}"

This option will replace report.github.commentTitleUrl value set in buildnote.json config file.

Values: text

Html Reporter

Options for html reporter.

--html-enabled

Enable or disable html reporter.

Example:

shell

--html-enabled true

This flag will replace report.html.enabled value set in buildnote.json config file.

Values: [true|false]

--html-condition

Html reporter condition evaluated as expression.

Example for failing builds only:

shell

--html-condition "{{failed}}"

You can also access environment variables and combine expressions.

Example of complex condition:

shell

--html-condition "{{failed || successful || env.USER == 'some-user'}}"

This option will replace report.html.condition value set in buildnote.json config file.

Values: text

--html-template

Path to template file used to format report.

Example:

shell

--html-template "template.json"

This option will replace report.html.template value set in buildnote.json config file.

Values: value

--html-output-file

Html reporter output file.

Example for fixed output file:

shell

--html-output-file "test-report.html"

You can also access environment variables and combine expressions.

Example of complex output file:

shell

--html-output-file "{{env.USER}}/test-report.html"

This option will replace report.html.outputFile value set in buildnote.json config file.

Values: text

Context Variables

This section contains the list of context variables available when evaluating expressions.

always

Boolean that is always true.

build

Build identifier for this run.

failed

Boolean that indicates if any events in the report had failed.

failed.count

Number of failed events in the report.

failed.percentage

Formatted percentage of failed events in the report.

Example: 0.97%

module

Name of the module.

never

Boolean that is always false.

org

Name of the org.

project

Name of the project.

ref

Ref associated with this report.

sha

Commit SHA associated with this report.

skipped

Boolean that indicates if all events in the report were skipped.

skipped.count

Number of skipped events in the report.

skipped.percentage

Formatted percentage of skipped events in the report.

Example: 0.97%

status

Overall status of the report.

Example: successful, failed or skipped

successful

Boolean that indicates no failures and that some events in the report were successful.

successful.count

Number of successful events in the report.

successful.percentage

Formatted percentage of successful events in the report.

Example: 0.97%

total

Total number events in the report.

buildnote report ​

Usage ​

Options ​

--source ​

--template ​

--summary-enabled ​

--org ​

--project ​

--module ​

--build ​

--sha ​

--ref ​

-h, --help ​

JUnit Collector ​

--junit-enabled ​

--junit-include ​

--junit-exclude ​

--junit-metadata ​

TestNG Collector ​

--testng-enabled ​

--testng-include ​

--testng-exclude ​

--testng-metadata ​

Cucumber Collector ​

--cucumber-enabled ​

--cucumber-include ​

--cucumber-exclude ​

--cucumber-metadata ​

Dart/Flutter Collector ​

--dart-enabled ​

--dart-include ​

--dart-exclude ​

--dart-metadata ​

NUnit Collector ​

--nunit-enabled ​

--nunit-include ​

--nunit-exclude ​

--nunit-metadata ​

XUnit Collector ​

--xunit-enabled ​

--xunit-include ​

--xunit-exclude ​

--xunit-metadata ​

CTRF Collector ​

--ctrf-enabled ​

--ctrf-include ​

--ctrf-exclude ​

--ctrf-metadata ​

TRX Collector ​

--trx-enabled ​

--trx-include ​

--trx-exclude ​

--trx-metadata ​

Robot Framework Collector ​

--robot-enabled ​

--robot-include ​

--robot-exclude ​

--robot-metadata ​

QT Test Collector ​

--qt-enabled ​

--qt-include ​

--qt-exclude ​

--qt-metadata ​

Slack Reporter ​

--slack-enabled ​

--slack-condition ​

--slack-template ​

--slack-url ​

--slack-title ​

--slack-title-url ​

Discord Reporter ​

--discord-enabled ​

--discord-condition ​

--discord-template ​

--discord-url ​

--discord-title ​

--discord-title-url ​

GitHub Reporter ​

--github-enabled ​

--github-condition ​

buildnote report

Usage

Options

--source

--template

--summary-enabled

--org

--project

--module

--build

--sha

--ref

-h, --help

JUnit Collector

--junit-enabled

--junit-include

--junit-exclude

--junit-metadata

TestNG Collector

--testng-enabled

--testng-include

--testng-exclude

--testng-metadata

Cucumber Collector

--cucumber-enabled

--cucumber-include

--cucumber-exclude

--cucumber-metadata

Dart/Flutter Collector

--dart-enabled

--dart-include

--dart-exclude

--dart-metadata

NUnit Collector

--nunit-enabled

--nunit-include

--nunit-exclude

--nunit-metadata

XUnit Collector

--xunit-enabled

--xunit-include

--xunit-exclude

--xunit-metadata

CTRF Collector

--ctrf-enabled

--ctrf-include

--ctrf-exclude

--ctrf-metadata

TRX Collector

--trx-enabled

--trx-include

--trx-exclude

--trx-metadata

Robot Framework Collector

--robot-enabled

--robot-include

--robot-exclude

--robot-metadata

QT Test Collector

--qt-enabled

--qt-include

--qt-exclude

--qt-metadata

Slack Reporter

--slack-enabled

--slack-condition

--slack-template

--slack-url

--slack-title

--slack-title-url

Discord Reporter

--discord-enabled

--discord-condition

--discord-template

--discord-url

--discord-title

--discord-title-url

GitHub Reporter

--github-enabled

--github-condition