Something's Rotten in the State of Jenkins

Automated, repeatable testing is a fairly widely-accepted cornerstone of -mature software development. Jenkins (and its predecessor, Hudson) has the -unique privilege of being both an early player in the niche and -free-as-in-beer. The blog space is littered with interesting articles about -continuous builds, automated testing, and continuous deployment, all of which -conclude on “how do we make Jenkins do it?”

This is unfortunate, because Jenkins has some serious problems, and I want it -to stop informing the discussion.

There's A Plugin For That

Almost everything in the following can be addressed using one or more plugins -from Jenkins' extensive plugin repository. That's good - a build system you -can't extend is kind of screwed - but it also means that the Jenkins team -haven't felt a lot of pressure to address key problems in Jenkins proper.

(Plus, the plugin ecosystem is its own kind of screwed. More on that later.)

To be clear: being able to fix it with plugins does not make Jenkins itself -good. Plugins are a non-response to fundamental problems with Jenkins.

No Granularity

Jenkins builds are atomic: they either pass en suite, or fail en suite. Jenkins has no built-in support for recording that basic compilation succeeded, unit tests failed, but linting also succeeded.

You can fix this by running more builds, but then you run into problems with -...

No Gating

... the inability to wait for multiple upstream jobs before continuing a -downstream job in a job chain. If your notional build pipeline is

Compile, then
Lint and unit test, then
Publish binaries for testers/users

then you need to combine the lint and unit test steps into a single build, or -tolerate occasionally publishing between zero and two copies of the same -original source tree.

No Pipeline

The above are actually symptomatic of a more fundamental design problem in -Jenkins: there's no build pipeline. Jenkins is a task runner: triggers cause -tasks to run, which can cause further triggers. (Without plugins, Jenkins -can't even ensure that chains of jobs all build the same revisioins from -source control.)

I haven't met many projects whose build process was so simple you could treat -it as a single, pass-fail task, whose results are only interesting if the -whole thing succeeds.

Plugin the Gap

To build a functional, non-trivial build process on top of Jenkins, you will -inevitably need plugins: plugins for source control, plugins for -notification, plugins for managing build steps, plugins for managing various -language runtimes, you name it.

The plugin ecosystem is run on an entirely volunteer basis, and anyone can -get a new plugin into the official plugin registry. This is good, in as much -as the barrier to entry should be low and people should be encouraged to -scratch itches, but it also means that the plugin registry is a swamp of -sporadically-maintained one-offs with inconsistent interfaces.

(Worse, even some core plugins have serious maintenance deficits: have a -look at how long -JENKINS-20767 was open. -How many Jenkins users use Git?)

The Plugin API

The plugin API also, critically, locks Jenkins into some internal design -problems. The sheer number of plugins, and the sheer number of maintainers, -effectively prevents any major refactoring of Jenkins from making progress. -Breaking poorly-maintained plugins inevitably pisses off the users who were, -quite happily, using whatever they'd cooked up, but with the maintainership -of plugins so spread out and so sporadic, there's no easy way for the Jenkins -team to, for example, break up the 4,000-line Jenkins class.

What Is To Be Done

Jenkins is great and I'm glad it exists. Jenkins moved the state of the art -for build servers forward very effectively, and successfully out-competed -more carefully-designed offerings that were not, in fact, better: -Continuum is more or less abandoned, and when -was the last time you saw a -CruiseControl (caution: SourceForge) -install?

It's interesting to compare the state of usability in, eg., Jenkins, to the -state of usability in some paid-product build systems -(Bamboo and -TeamCity for example) on the above -points, as well as looking at the growing number of hosted build systems -(TravisCI, MagnumCI) for -ideas. A number of folks have also written insightful musings on what they -want to see in the next CI tool: Susan Potter's -Carson includes an interesting -motivating metaphor (if you're going to use butlers, why not use the whole -butler mileu?) and some good observations on how Jenkins lets us all down, -for example.

I think it's time to put Jenkins to bed and write its successor.

- - - -

- - -comments powered by Disqus -

- - - - - -