GitHub Checks and Statuses from CDash

CDash can now attach checks and statuses to your commits on GitHub!

Here’s how to set this up for your project.

Step 1: Register a GitHub App for your CDash instance.

You can skip this step if you’re using a public, Kitware-hosted CDash instance like open.cdash.org or my.cdash.org.

In the example below I am setting up a GitHub app for my.cdash.org. Replace this URI with your own. Here’s what to enter for the various fields on the registration form.

• Homepage URL: the base URL of your CDash installation (http://my.cdash.org)
• User authorization callback URL: user.php on this CDash instance (https://my.cdash.org/user.php)
• Setup URL: leave blank
• Redirect on update: leave unchecked
• Webhook URL: /api/v1/GitHub/webhook.php (https://my.cdash.org/api/v1/GitHub/webhook.php).
• Webhook secret: optionally enter a high entropy string. This ensures that it’s really GitHub that’s hitting your webhook endpoint. If you do so, you’ll also have to save this value as $CDASH_WEBHOOK_SECRET in your config.local.php file.

Next, add the following Permissions to your app.

• Checks: read & write. This allows GitHub to post checks on your commits, and also react to other apps that create checks.
• Repository contents: read only. This allows CDash to determine what code changed between two builds.
• Commit statuses: read & write. This allows CDash to post statuses on commits.

Leave “User Permissions” blank.

In the Subscribe to events section, select “Status” and “Check run”. When either of these events occur, GitHub will tell CDash to create or update a check for the relevant commit.

Upload a logo. We recommend cdash.png.

For Where can this GitHub App be installed?, we recommend “any account” depending on who will be submitting to this CDash.

Now you can finally click the Create GitHub app button. Record your APP ID. this will go in your config.local.php file as $CDASH_GITHUB_APP_ID. Also generate a private key and upload this to config/github.pem.

Step 2: Install the app on your repository or organization

From https://github.com/settings/apps/<my-app-name> click the “Install App” link in the sidebar and follow the instructions. This will give you an installation ID. Here are some links to quickly install our public CDash instances: open.cdash.org, my.cdash.org.

Step 3: Add the installation ID to your CDash project

• Login to CDash as a project administrator.
• Navigate to Settings -> Project -> Repository
• Make sure your project’s GitHub URL is filled in under the Repository field.
• Input your installation ID under the username field.

Step 4: Start using CDash to report results to GitHub

Using CDash to post Checks to GitHub

If you made it this far, you’re all set! GitHub will now notify your CDash instance whenever a builder service (like CircleCI or TravisCI) posts a status or check to a commit of your project. This notification tells CDash to generate a check for this commit too. CDash’s check will contain the results for all the builds that were performed for this commit. It also provides links to specific details about anything that went wrong. Here’s an example of a check posted by CDash.

If you have an unreliable build that you would like to exclude from the check, you can do so by setting its skip checks property.

file(WRITE ${CTEST_BINARY_DIRECTORY}/props.json
  [=[ { "skip checks": 1 } ]=])

ctest_submit(
  CDASH_UPLOAD
    "${CTEST_BINARY_DIRECTORY}/props.json"
  CDASH_UPLOAD_TYPE BuildPropertiesJSON)

Using CDash to post Statuses to GitHub

This is the approach to take if you’re not using a builder service that posts statuses and/or checks to your commits on GitHub.

CDash will only post statuses for builds that explicitly ask for this feature. You tell CDash that your build should post its status to GitHub by setting the status context build property.

file(WRITE ${CTEST_BINARY_DIRECTORY}/props.json
  [=[ { "status context": "MySQL" } ]=])

ctest_submit(CDASH_UPLOAD
  "${CTEST_BINARY_DIRECTORY}/props.json"
  CDASH_UPLOAD_TYPE BuildPropertiesJSON)

You might not want every single build of a commit to be publishing its status to GitHub. That’s why we made this feature opt-in on a per-build basis. This also gives us the opportunity to specify a context for this build, so we can differentiate between multiple builds posting status on a single commit. Rather than adding another CTest variable to hold this information, we decided to use the “build properties” feature in CDash. This allows you to upload arbitrary key/value pairs (in JSON format) that will be associated with a build.
When testing CDash itself on CircleCI we do two builds, one for MySQL and one for Postgres. Here’s an example of what this looks like on GitHub. Click the green checkmark on the last commit to see the statuses posted by CDash.