GitHub Actions Isn't a CI

GitHub Actions runs your tests when you push. By the 2006 definition of continuous integration, that’s all it takes, so technically, yes, it’s CI. And yet nobody who has run a real pipeline on it believes that’s what they have.

Here’s the confusion I want to clear up: a runner is not a CI system. Actions is a superb runner. It is not a CI system, and the industry has spent a decade mistaking one for the other.

Say the awkward part first: Mergify sells into this layer, so I have a horse in this race. I’ll disclose it once and let the argument stand on its own.

The word is doing two jobs

“CI” means two different things now, and the whole mess comes from smashing them together.

There’s the practice: integrate often, run the build, catch breakage early. Martin Fowler wrote that down in 2006, and Actions does it fine. And there’s the system: the machine teams actually picture when they say “our CI,” the one that understands their tests, owns the build graph, keeps the pipeline fast, and decides what’s safe to merge. Buying a runner gets you the practice, and it feels like it should get you the system. It doesn’t. That gap is the entire post.

The 2006 definition predates everything hard about CI today: flaky tests at scale, monorepo build graphs, thousand-job pipelines, merge queues. The bar didn’t move. The problems did. Actions clears a bar that was set before the hard part existed.

What Actions actually is

Strip the branding and Actions is a general-purpose, event-driven job runner. Something happens (a push, a label, a schedule, a webhook) and it runs jobs in containers. That’s it, and that’s a lot. Deploys, releases, bots, cron, search reindexing at 3am, and yes, your tests, which sit in that list right next to “post to Slack.”

To the engine, your test suite is a command that exits zero or non-zero. It has no idea it’s a test suite. That isn’t a flaw in the runner. That’s the definition of a runner, and Actions is a good one. Good is generous (look closely and it’s a fairly basic runner too), but I’ll grant it anyway, because even a great runner still isn’t a CI system.

The four places the seam shows

Every daily pain with “Actions as CI” is the same thing underneath: a runner with no model of the work you handed it.

It has no model of your tests. It runs a script and reads an exit code, and that’s the whole relationship. It can’t flag a flaky test, because it doesn’t know a test ran. It can’t run only the tests your change affects, because it doesn’t know what a test is or what you touched. It can’t quarantine anything. A CI system treats tests as objects with history and behavior. Actions treats them as stdout and a number.

Its primitives are runner primitives, not CI primitives. Caching exists, as a step you wire by hand with keys you manage (actions/cache), not as a build graph the system understands, so it can’t reuse or invalidate intelligently. Secrets are a model everyone quietly works around. You can re-run a failed job, but not a single step inside it, so retrying one broken thing re-runs the whole job. And yes, there are matrices, reusable workflows, composite actions. They’re clever ways to script a runner. None of them make it understand what it’s running.

You author it blind. There’s no faithful local run. act approximates it, but it won’t match the runner images, the secrets, or the hosted context, so real debugging is still edit, commit, push, wait four minutes, read the logs, change one line, push again. The development loop for the thing that guards all your other development loops is the worst one you own. There’s no model to test against locally, only a remote runner to execute against.

The logic has nowhere to live but YAML. Expressions, if: conditionals, matrix expansions, bash templated inside a workflow key. Because the engine has no model of your process, every scrap of process logic gets crammed into config, and YAML quietly becomes a programming language. Every gnarly workflow file is a program in exile.

None of these are Actions failing at its job. They’re a runner being asked to do a job that needs a model it was never built to have.

Why the mislabel costs you

If this were only semantics I wouldn’t bother. It isn’t, because names set expectations.

Listen to how teams talk about CI. “Is CI green?” “CI is slow again.” “CI is flaky.” Nobody says “CI caught the flaky one” or “CI ran only the two hundred tests that mattered,” because their CI can’t do those things, so the words for them fell out of use. The vocabulary shrank to a status light, because the tool is a status light. A whole generation now believes that’s the ceiling of what CI is. It’s the floor. Teams mistake the floor for the building.

What to do instead

Keep Actions. It’s a great runner and you have a hundred legitimate uses for one: deploys, releases, automation, glue. Keep all of it.

A screwdriver is a fine tool. It’s a terrible hammer. We’ve spent ten years hammering with it and calling the bent nails “CI.”