Building Variations of Docker Images

This is no post on “best practices” on how to build or deploy customised versions of docker images. I don’t even know if this technique is advisable or not. Nevertheless, I came up with this when I was in dire need and out of time (so normal case).

The Problem

I faced the issue of having to set up an elasticsearch cluster along with all bells and whistles needed for an effective nolow-ops approach. I went with docker images which I built based on the officially provided ones. As I’m totally wild for automation, I made sure I had code for just about anything and my favourite tool to do so is gradle (of course).

At one point I decided to include extra “client” nodes (which hold no data and aren’t supposed to be ranked to master ever). As I didn’t want to spoil the nice gradle build with specialties while also abhorring anything unDRY I had to come up with some niftiness.

 

The Situation

The typical elasticsearch deployment comes also with related technologies to complement the use case. Normally there is logstash instances and a kibana host, monitoring stuff and backup, maybe an extra database or two. To ease the setup I put all resources for a specific docker image into dedicated sub folders, which would also determine the image tag.

So, having a folder like:

$base/awesome/elk/logstash/

would result in a docker image tagged like:

$registry/awesome/elk/logstash:$version

All those images would later be bound into a specific docker-compose based deployment.

All was well. Until I needed variations on that.

 

The Way Out

Build Configuration. That’s how I call it momentarily. I know.
Basically, its just a bunch of properties under an extra build (name) space. I’m bad at explaining. Maybe I come up with something better eventually.

Nevertheless, the existence of a specifically named property file in the current build context defines a variant of the very same build. So every resource is copied and filtered over using the key-value-pairs provided for the current variant.

Here is a build script dealing with that:

task build_elk(description: 'builds the ELK stack docker images') {
    doFirst {
        file("${projectDir}/etc/docker/elk").eachDir { File dir ->
            // iterate over build configs
            def cfgs = dir.listFiles().findAll {it.name =~ /build\.config/} ?: [null]

            cfgs.each { File cfg ->
                def cprops = new Properties()
                if (cfg) {
                    println ">>> Building config ${cfg.name - 'build.config.' - '.properties'}"
                    cfg?.withInputStream {cprops.load(it)}
                }

                def tag = "${registry}/${dir.name}${cprops['image_path'] ?: ''}:${AUX_IMG_VERSION}"

                copy {
                    from dir
                    into "${buildDir}/elk/$tag"
                    exclude 'build.config.**'
                    filter(ReplaceTokens, tokens: cprops)
                }

                exec {
                    workingDir "${buildDir}/elk/$tag"
                    commandLine 'docker', 'build', '--rm','--force-rm', '--no-cache', '-t', tag, '.'
                }
            }
        }
    }
}

The Way Forward

Lot’s of generification. Leveling this into the fabric of gradle itself. Maybe make a plugin. Maybe.

Advertisements

orion dex backend | Trello

Todo-Lists:
For years, I used trackers for this. Bugzilla, Jira, Trac, etc. pp., you name it.

But no more. Modern agile methods use simple record cards to keep track of defined tasks. Trello (by FogCreek, Joel Spolsky et.al.) takes that to the next level.
Define card boards with custom lists, attach files, checklists, comments, assignees to cards. Share the thing or collaborate privately. The UI is slick and intiutive.
The thing is totally free-to-use (I would even pay for it, hell) so I use it for everything now (your data-paranoia-issues may vary). See an example (my current day job) behind the link.

Yer welcome.

https://trello.com/board/orion-dex-backend/4e711debf15968bffc04d44a

 

Vim Plugins You Should Know About, Part I: surround.vim – good coders code, great reuse

Vim Plugins, surround.vimHi all. I am starting a new article series called “Vim Plugins You Should Know About“. This series of articles is going to be about Vim plugins that you should know about and perhaps even be using. The first article in this series will be about one of my favorite plugins called “surround.vim“.