Building Variations of Docker Images

This is no post on “best practices” on how to build or deploy customised versions of docker images. I don’t even know if this technique is advisable or not. Nevertheless, I came up with this when I was in dire need and out of time (so normal case).

The Problem

I faced the issue of having to set up an elasticsearch cluster along with all bells and whistles needed for an effective nolow-ops approach. I went with docker images which I built based on the officially provided ones. As I’m totally wild for automation, I made sure I had code for just about anything and my favourite tool to do so is gradle (of course).

At one point I decided to include extra “client” nodes (which hold no data and aren’t supposed to be ranked to master ever). As I didn’t want to spoil the nice gradle build with specialties while also abhorring anything unDRY I had to come up with some niftiness.

 

The Situation

The typical elasticsearch deployment comes also with related technologies to complement the use case. Normally there is logstash instances and a kibana host, monitoring stuff and backup, maybe an extra database or two. To ease the setup I put all resources for a specific docker image into dedicated sub folders, which would also determine the image tag.

So, having a folder like:

$base/awesome/elk/logstash/

would result in a docker image tagged like:

$registry/awesome/elk/logstash:$version

All those images would later be bound into a specific docker-compose based deployment.

All was well. Until I needed variations on that.

 

The Way Out

Build Configuration. That’s how I call it momentarily. I know.
Basically, its just a bunch of properties under an extra build (name) space. I’m bad at explaining. Maybe I come up with something better eventually.

Nevertheless, the existence of a specifically named property file in the current build context defines a variant of the very same build. So every resource is copied and filtered over using the key-value-pairs provided for the current variant.

Here is a build script dealing with that:

task build_elk(description: 'builds the ELK stack docker images') {
    doFirst {
        file("${projectDir}/etc/docker/elk").eachDir { File dir ->
            // iterate over build configs
            def cfgs = dir.listFiles().findAll {it.name =~ /build\.config/} ?: [null]

            cfgs.each { File cfg ->
                def cprops = new Properties()
                if (cfg) {
                    println ">>> Building config ${cfg.name - 'build.config.' - '.properties'}"
                    cfg?.withInputStream {cprops.load(it)}
                }

                def tag = "${registry}/${dir.name}${cprops['image_path'] ?: ''}:${AUX_IMG_VERSION}"

                copy {
                    from dir
                    into "${buildDir}/elk/$tag"
                    exclude 'build.config.**'
                    filter(ReplaceTokens, tokens: cprops)
                }

                exec {
                    workingDir "${buildDir}/elk/$tag"
                    commandLine 'docker', 'build', '--rm','--force-rm', '--no-cache', '-t', tag, '.'
                }
            }
        }
    }
}

The Way Forward

Lot’s of generification. Leveling this into the fabric of gradle itself. Maybe make a plugin. Maybe.

Advertisements

Development Bullshittery explained perfectly

perfectly analysed, couldn’t have said it better…
Awesome read!

[…] Sorry to say, but because of the short-sighted way in which management must think (again, likely due to the incentives provided by the larger economic environment), the integrity of your software system isn’t even on their radar. If they notice it at all, they’ll more likely see it as an obstacle to cut through rather than a constraint to respect. […]

read along behind the link