Bitbucket branch protection, the hard way

20 May 2018 - New York

One of my favorite features on Microsoft GitHub 2018™ is the ability to prohibit merges from branches that aren’t up-to-date with the repository’s default branch.

Bitbucket doesn’t have that. Obviously. Because Bitbucket…

Turns out bits are pretty flammable

Bitbucket is a work in progress.

In any case, it’s a great feature because it protects you from those situations where

  1. You’ve just gotten a thumbs-up on your PR
  2. You proceed to smash that merge button
  3. That brief dopamine high triggered by clicking a large, brightly-colored button is harshly cut short by the fact that someone else’s change had some nth-order impact on your code or tests and now the build is broke on master and nice going you grifter how did you even get this job?

GitHub: So extra

Bitbucket: Basic af

Since we can’t always work with the tools we want, I recently had occasion to jury-rig something that approximates that feature on Bitbucket. Behold.

The Hack

Here’s a high-level sketch of how to ensure out-of-date branches are automatically prohibited from being merged into a protected branch on Bitbucket:

  • Use Git to ensure any builds fail if their branch is behind develop. Since the last build on a PR’s branch may have been run before a more recent merge to develop, we also have to trigger builds on topic branches whenever there’s a merge. Hence:
  • Use a webhook to notify an AWS Lambda function anytime a PR has been merged to develop.
  • Have this Lambda function use Bitbucket’s API to trigger CI builds on any pushed topic branches.

I’ll walk through the above in bottom-up fashion:

Git: Fail CI on out-of-date branches

First, make sure any CI builds fail if the commit being built is behind the default branch on the remote (in this case, that’s origin/develop):

# bitbucket-pipelines.yml

- step:
    name: up-to-date
    script:
      - git fetch origin "+refs/heads/*:refs/remotes/origin/*"
      - test "$(git rev-list --left-right --count origin/develop... | awk '{ print $1 }')" == 0

The first line of the script fetches from origin. This will require adding Bitbucket CI’s public key to your repo’s access key list:

Add an access key for Bitbucket CI

Add an access key for Bitbucket CI

The second line uses Git’s rev-list command to determine if the current commit is behind origin/develop. If it’s even with origin/develop, the left number returned by the rev-list invocation should be zero.

A quick demo:

Note in the following that after checking out head~4 from develop, origin/develop is 5 commits ahead of HEAD.

develop % git branch -vv
* develop 3a99d7c [origin/develop]

develop % git rev-list --left-right --count origin/develop...
0       0

develop % git checkout head~4
Note: checking out 'head~4'.

c1b55e5 % git rev-list --left-right --count origin/develop...
5       0

AWS Lambda: Trigger CI on your branches

Next, you’ll want a way to trigger CI builds in response to Bitbucket’s notifications that a PR has been merged. The most expedient way to do this in my case was to use Amazon’s API Gateway and Lambda services, which make getting an endpoint up and running insanely simple.

Now is the part of the blog post where we do some halfhearted literate programming.

the parser’s tale

The Lambda handler accepts an event parameter, and an incoming POST request’s body is nested under event.body.

Bitbucket’s PR-merge notification payload lists the repo name and target branch under

repository.full_name;

and

pullrequest.destination.branch.name;

respectively.

We parse this JSON and extract the name of the repository and target branch for the PR in question:

// index.js L4-6 (687bc02991)

const data = JSON.parse(event.body);
const repository = data.repository.full_name;
const targetBranch = data.pullrequest.destination.branch.name;
index.js L4-6 (687bc02991)

get some branches

Using the Bitbucket API, we then fetch a list of branches for the given repository, filter it to include only topic branches, and trigger CI pipelines for each (implementations to follow):

// index.js L13-28 (16b93cd88a)

getBranches(repository)
  .then((branches) =>
    Promise.all(
      branches
        .filter(
          (name) => name && !["develop", "master", targetBranch].includes(name)
        )
        .map((branch) => triggerPipeline(repository, branch))
    )
  )
  .then((branches) => {
    const message = `[${repository}] Triggered pipelines for ${branches.join(
      ", "
    )}`;
    callback(null, { statusCode: 200, body: message });
  })
  .catch((err, resp) => {
    callback(null, { statusCode: 500, body: `error: ${err}\ndata: ${resp}` });
  });
index.js L13-28 (16b93cd88a)

bitbucket api wrapper

The functions getBranches and triggerPipeline referenced above are straightforward wrappers for API calls. They’re exposed by a BitbucketApi module.

getBranches

Bitbucket API v2.0 exposes a branches endpoint you can query to return details on branches for a given repository:

// bitbucket-api.js L7-39 (e0a7d42a2b)

export const getBranches = (repository) => {
  const options = {
    method: "GET",
    hostname: "api.bitbucket.org",
    path: `/2.0/repositories/${repository}/refs/branches/`,
    headers: {
      "Content-Type": "application/json",
      Authorization: `Basic ${AUTH_STRING}`,
    },
  };

  const promise = new Promise((resolve, reject) => {
    const req = http.get(options, (res) => {
      let body = [];
      res.setEncoding("utf8");
      res.on("error", (err) => reject(err));
      res.on("data", (data) => {
        body.push(data);
      });
      res.on("end", () => {
        try {
          const data = JSON.parse(body.join(""));
          const branches = data.values.map((e) => e.name);
          resolve(branches);
        } catch (err) {
          reject(err, body);
        }
      });
    });

    req.end();
  });

  return promise;
};
bitbucket-api.js L7-39 (e0a7d42a2b)

triggerPipeline

A POST to the repository pipelines endpoint then triggers the CI pipeline for a given branch:

// bitbucket-api.js L41-77 (e0a7d42a2b)

export const triggerPipeline = (repository, branch) => {
  const postData = {
    target: {
      ref_type: "branch",
      type: "pipeline_ref_target",
      ref_name: branch,
    },
  };

  const options = {
    method: "POST",
    hostname: "api.bitbucket.org",
    path: `/2.0/repositories/${repository}/pipelines/`,
    headers: {
      "Content-Type": "application/json",
      "Content-Length": Buffer.byteLength(JSON.stringify(postData)),
      Authorization: `Basic ${AUTH_STRING}`,
    },
  };

  return new Promise((resolve, reject) => {
    const req = http.request(options, (res) => {
      res.setEncoding("utf8");
      res.on("data", (resp) => {
        console.log(`Triggering pipeline for branch: ${branch}`);
        resolve(branch);
      });
      res.on("error", (err) => {
        console.log(
          `Pipeline trigger failed for branch: ${branch}. Error: ${err}`
        );
        reject(err);
      });
    });

    req.write(JSON.stringify(postData));
    req.end();
  });
};
bitbucket-api.js L41-77 (e0a7d42a2b)

Bitbucket: Notify Lambda when PRs are merged

Lastly, we can add a webhook on the target repo to send our Lambda function a notification whenever a pull request is merged. This is very simple to set up by navigating to your repo’s webhooks config via Settings > Webhooks.

Note that simple here assumes you’ve already emailed the right people to request the email address of the appropriate DevOps gatekeeper who can give you admin permissions on the repository, and have actually emailed said gatekeeper and gotten a helpful response. If it’s even possible to get those permissions devolved to you. If not, there may be some additional legwork involving forms signed in triplicate.

And for chrissake don’t forget to CC the appropriate Vice President so the gatekeeper knows you’ve got your rubber stamps in order.

Set up a merge notification webhook

Set up a merge notification webhook

One more thing…

Actually preventing a merge with a failing build is a Premium feature on Bitbucket, so, if your corporate daddy is stingy, all this work only gets us 90% of the way toward a completely automated solution.

Enable the CI merge check

Enable the CI merge check

But: Even sans the premium plan, we at least get a friendly reminder when there’s a failing build on a PR…and what kind of a shameless monster merges a branch with a failing build?

PR merge checklist

PR merge checklist