To help you get started with GitHub Enterprise installs that use self-signed certificates, we've added the ability to disable TLS certificate verification in your GitHub Enterprise Buildkite settings 🔓

You can find the new setting in your GitHub Enterprise settings, and read more details in our GitHub Enterprise documentation.

In addition to running builds when code is pushed to GitHub forks, you can now trigger builds when forks are created 🍽🤖

You can use fork events to automatically provision staging environments for example, or to automate other aspects of your fork-based workflows.

Support for the fork event is supported for both GitHub and GitHub Enterprise, and can be configured from your pipeline’s GitHub or GitHub Enterprise Settings.

To improve API Access Token security we have removed the “All Organizations” scope from tokens. When managing your tokens you now explicitly select each organization 🔐

If you had tokens with “All Organizations” scopes, they have been automatically migrated and include all of your current organization memberships. When you join a new Buildkite organization, and wish to grant a token access to its data, you must edit the token's settings and select the new organization.

You can manage your tokens on the API Access Tokens Settings page.

The latest version of the stack, v4.3.1, introduces a new parameter: EnableExperimentalLambdaBasedAutoscaling. When set to true it will disable the default Amazon-AutoScaling-powered scaling behavior in favor of a Lambda that handles the scale-out.

The result is a much, much faster scale-out and a much simpler scale-in process that no longer requires lifecycled 🎉

For more information on this parameter, check out our Forum post about it: Experimental Lambda-based Scaler.

We've added a new soft_fail property to command steps, so you can ensure that some jobs never fail the build 🎈

1
2
3
4
5
steps:
  - label: '💨 Smoke Test'
    command: smoke-test.sh
    soft_fail:
      - exit_status: 1

To ensure you're not swallowing unintended errors, soft failures can be configured to handle specific exit statuses. And if a job soft fails, it'll be marked in the pipeline with a red warning icon, letting you keep an eye on it:

You can read more about the new soft_fail property in the Command Step documentation.

We've updated the way times are displayed, and they now respect your browser’s preferred language settings 🌏

For example, if your browser’s preferred language is set to Bokmål or Deutsch, Buildkite will now show 14:17 instead of 2:17 PM ⏰

The Environment Variables documentation has had a major update 🤩

🌈A new layout ⚡️A new section for deprecated variables 🍩Many previously undocumented variables added 💥Expanded descriptions

You can find the updated Environment Variables page under the Pipelines section of the documentation 🚀

To help debug your dynamic pipeline uploads you can now view pipeline uploads in the job Timeline tab 🆙

We've also made the job concurrency configuration clearer, by including the concurrency group and limit in the "Job created" timeline item:

You can now pipe updates from the Buildkite Changelog directly into your Slack channels, or other team chat tool, using the new Atom feed 🐶🗞

If you use Slack, see Slack’s guide to adding feeds. If you use other tools, or you need the full feed URL, you can access the new feed at https://buildkite.com/changelog.atom ✨

We’ve added a new Agent Tokens page to the documentation. It explains default organization tokens, creating and revoking tokens using GraphQL, session token exchange, and more 🕵🏻‍♀️

You can find the new Agent Tokens page under the Agent section of the documentation 🎊

We've been rolling out some big upgrades to Buildkite’s Single Sign-On (SSO) support based on your feedback 📣

The upgrades to Single Sign-On include:

• Admins can now invite users into an SSO organization prior to them joining, along with team membership.
• Team membership can now be passed through as a SAML user attribute.
• Contractors and bots can be added to an organization using any email domain.
• SSO can be made optional for some users, allowing you to have a "break glass" account in the case of your SSO provider being unavailable.
• People can participate in multiple Buildkite organizations with SSO enabled, and easily switch between them.

If your organization hasn't already made the switch, we'll be automatically moving your Buildkite organization to the new version of SSO over the next few weeks. There will be no interruption to your logins, and it won't require any changes to your SSO provider setup.

If you have any questions please email support@buildkite.com, and you can read more about SSO updates in our community forum post 😊

Block steps record the person who unblocks them, and this information is available to subsequent steps via $BUILDKITE_UNBLOCKER environment variables. 👮‍♂️

Due to a quirk in our job environment calculations, these variables were only available in steps that were pipeline uploaded after the block step. This has been fixed, so now all steps after a block step will have the correct unblocker environment variables, no matter how they’re defined or uploaded. 💯

So if you upload a deployment pipeline like this: 💁‍♀️

The $BUILDKITE_UNBLOCKER environment variables will be set to the person who presses the Deploy button: 🙋‍♀️

You can read more about block steps in our documentation. 📚

To make testing new pipelines easier, you no longer need to specify the "Message" of a build. Buildkite now automatically resolves the message, based on the git commit, when the build starts 🎉

We've also updated the Builds REST API and GraphQL API so your integrations can take advantage of this too.

We've added a new SSO section to the documentation, with instructions for setting up Single-Sign On with Google OAuth, Okta, OneLogin, and ADFS, and custom SAML providers ✨

You can now also find step-by-step instructions for managing SSO with GraphQL, allowing you to enable, disable and update your SSO settings via the Buildkite GraphQL API

Buildkite pipelines can now made publicly visible, allowing them to be opened up to the world for the first time! 🌏

If you’re an account admin, you can enable read-only public access for a pipeline from the pipeline settings page.

Many customers and projects are already using them, such as Angular.js and Bazel, and we'd love to read any feedback or ideas you might have on our new beta community forum.

Public pipelines are just the first in a series of improvements we have coming for open-source projects—we’re just getting started!

p.s. Did you know we offer free plans for open source projects? 😘

Back in October, Buildkite Agent v3.5.0 included a bug fix to ensure that plugins are always executed in the correct order. In the process of fixing that bug, we've updated our recommended syntax for using plugins to use an array instead of a map, to make it clearer that the plugins are executed in order.

For example, if your pipeline uses the following syntax, we recommend updating it:

1
2
3
4
5
6
7
steps:
  - label: '<img class="emoji not-prose size-[1em] inline align-[-0.1em]" title="shipit" alt=":shipit:" src="https://buildkiteassets.com/emojis/img-buildkite-64/shipit.png" draggable="false" />'
    plugins:
      docker-login#v2.0.1:
        username: ci
      docker-compose#v2.6.0:
        push: app

The new recommended syntax, which works on all versions of the Buildkite Agent, changes the plugin property to be an array:

1
2
3
4
5
6
7
steps:
  - label: '<img class="emoji not-prose size-[1em] inline align-[-0.1em]" title="shipit" alt=":shipit:" src="https://buildkiteassets.com/emojis/img-buildkite-64/shipit.png" draggable="false" />'
    plugins:
      - docker-login#v2.0.1:
          username: ci
      - docker-compose#v2.6.0:
          push: app

We've been working on a command-line tool for Buildkite for a while. Recently it's been upgraded with a local pipeline runner for testing out pipelines locally on your development machine.

We use this for testing plugins, and quickly iterating on pipelines in development. Secretly, it was just an experiment to use the image support in iTerm 2 for rendering inline custom emojis 😉💥

Download the latest release and let us know what you think.

You can now delete artifacts that you no longer want stored on Buildkite with the new delete button in the Artifacts tab:

When you delete an artifact, it's permanently deleted from our AWS S3 storage and removed from the job’s artifact list. If you’re using your own S3 bucket or Google Cloud Storage bucket for artifact storage, you'll need to manually remove them from your artifact store (don’t worry, we'll remind you to do this in the confirmation message).

If you’re performing deletions over a large number of builds, or wanting to automate deletion in general, we've updated the REST APIs to add support both artifact deletion and job log deletion.

We’ve updated the pipeline view of parallel jobs, so instead of showing every individual job they’re now shown as a single, expandable group 🎲

Each parallel job on the build page now also shows their number, and the parallel group size, alongside their label 🏷

Parallel jobs are a powerful way to reduce your build time by distributing your tests across agents 🌪 You can read about them in the parallel job documentation, or see them in action in the parallel testing screencast.

Happy 2019! To kick off the year, we've got some new agent changes fresh off the press!

Better error messages for cancelled jobs 🚨

When a job is cancelled, the agent receives the instruction and starts to shutdown the running job. The job is given 10 seconds to finish, after which it's terminated. Previously what had happened wasn't clear. We now clearly show that a command was interrupted by a signal:

Configurable grace period for cancellation ⏱

By default we give cancelled jobs 10 seconds to finish before they terminate. For some things like a Terraform tear-down, this can take a lot longer. We added a config of cancel-grace-period to customize that value.

1
2
# Wait 600 seconds (10 minutes) for cancellation
buildkite-agent start --cancel-grace-period 600

Future improvements will allow this to be set at a pipeline or step level.

Checkout directory is removed on error 🧹

Checkouts can fail for a variety of reasons which can sometimes leave the git repository in a corrupted state that prevents future checkouts.

The default behaviour for the agent is now to remove the checkout directory if there is a terminal checkout failure.

Windows agents run hooks after cancel 🎣

Windows unfortunately doesn't handle signals like posix systems do, so on Windows the cancellation behaviour was to simple terminate the job. This meant that artifacts weren't uploaded and hooks like pre-exit didn't run, which left things like docker-compose without a way to cleanup.

We now use Windows Terminal Process Groups and the CTRL_BREAK message to emulate signal handling in windows, which now means it should behave much like it's fairer cousins!

Windows gets terminal colors! 🖍

Ok this is a small improvement, but it represents our ongoing efforts to take advantage of the neat things that the Windows Terminal team is doing building for Terminal apps.

Spawn Multiple Agents 🐙

Currently we recommend that multiple agents on a host are run via your init system (launchd, systemd, etc). The new --spawn feature allows for multiple parallel agents to be spawned from the one installation.

We're still figuring out how this might fit into the bigger picture, but keen to see how folks use this feature!

Datadog metrics (Beta) 🐶

We are working with the Datadog team on becoming an official integration. First steps are getting some simple job metrics into Datadog.

You can use the new config metrics-datadog and metrics-datadog-host.

Let us know what you think at https://forum.buildkite.community/t/about-our-datadog-integration/216

Vendored Plugins 📦

We now support loading plugins from within your repository. This lets you vendor plugins to ensure that a third-party can't change them.

1
2
3
4
5
steps:
  - label: 'Test with vendored docker plugin'
    plugins:
      - './.buildkite/plugins/docker':
          image: 'hello-world'

Check out https://github.com/buildkite/agent/pull/878 for a bit more context.

Resolve $BUILDKITE_COMMIT correctly at pipeline upload ⚙️

We now resolve manually triggered builds with a commit of HEAD to the actual commit value in $BUILDKITE_COMMIT for your pipeline uploads.

See https://github.com/buildkite/agent/issues/456 for more context. I'm sorry it took us so long!

Better handling of submodules 🧤

We discovered a few gnarly bugs where we weren't ssh keyscanning submodules correctly and also a few cases where we weren't cleaning up lingering submodules in the checkout phase. These are now squashed!