This has been challenging in some ways. We were satisfied with the workflow we had using Gerrit and Zuul for testing so we decided to adapt GitHub to our requirements.
We know that Zuul now supports GitHub. However, that implies having your own testing infrastructure, something we can't afford. Instead, we rely on Travis, like most open-source projects hosted on GitHub.
The workflow we wanted to have was the following:
- A contributor creates a pull-request on GitHub.
- The pull-request is tested by Travis.
- The pull-request is reviewed by approved projects members.
- If the tests pass and two reviewers have approved the pull-request, then it can be merged.
This sounds simple, but it is actually not that simple.
First, when Travis tests the pull-request, it checks what has been sent by the contributor. If the contributor created a pull-request on top of an outdated version of the base branch, that's what will be tested by Travis during the initial pull-request creation.
Even if the pull-request has been created using the tip of the base branch, as time passes, the base branch will progress. However, the pull-request created by your contributor will not get those new commits – unless rebased manually.
That means the Travis tests result is now outdated invalid. Still, GitHub and Travis will both show you that this pull-request passed all tests – yes it did but with an old base branch from a while back!
If you added new tests in the meantime in your base branch, it's possible that this pull-request does not work anymore. Pressing the merge button might just break your project!
To help with that problem, GitHub recently added a button that allows you to base branch into the pull-request. That allows, in one click, to get the pull-requested updated with the base branch (e.g., master) and retested by Travis.
Still, this means that if you have ten pull-requests, you need to:
- Merge base branch into PR#1
- Wait for Travis to pass
- Wait for two reviewers to approve
- Merge PR#1
- All other nine pull-requests are not out of date. You need to do start back at operation 1. for each pull-request.
This is very tedious to do manually, especially when your projects has tons of pull-requests.
Pastamaker to the rescue
Pastamaker is a small Web application that implements the described workflow. Once connected to your GitHub project, it will set the proper permissions to protect it for accidental manual merge and force the workflow above to be followed.
Pastamaker listens for GitHub and Travis events to track the state of each pull-request. If it detects that a pull-request has been approved by two reviewers and that the initial Travis test run passed, it will merge the base branch if needed in it, wait for Travis to pass again, and then finally merge it.
If multiple pull-requests are approved at the same time and are candidates for a merge, it will order them, update once at a time, wait for Travis results and merge them if they pass. It essentially automates the workflow described above.
Pastamaker exposes its data via a simple dashboard, which allows seeing all the pull-requests for your project in a snap.
Pastamaker offers a lot of tiny other details that make the developers lives easier, such as posting the job result with direct links to the jobs logs in the pull-request – so you're informed as soon as they pass or fail and can fix them right away!
Pastamaker is obviously open-source, and we would love to see you give it a try!