Posted by: John Bresnahan | September 2, 2015

Scrubbing Parent Images

This post is a reminder to scrub your images before making child images.  It is not sufficient to only think about doing this when making public images.  Today I ran into a situation where my child image did not properly boot due to stale data which was injected into the parent image at boot via cloud-init.

Configuring The Parent Instance For Child Image Creation

My goal was to launch a base VM, install custom software into it, and then create a child image from that VM.  I could then launch that child image and know the software I needed would be installed and running in it.

The easiest way for me to do this was to write a small bash script that installed the software and started the services which I needed.  I then launched a new VM with that script as user data.  From there cloud-init ran it and my software was installed.  Then I created a snapshot of that image for later use.  It was a good plan…

The Problem

To make a long a story short, I was using Ubuntu 10.04 which has cloud-init 0.5.10.  In its default configuration there the user data script was stored to the image in such a way that it would be automatically run every time that VM was booted.  Thus every time my child image was booted, the bash script I wrote to configure it strictly for its initial creation was run.  Sadly my script was not idempotent and when run a second time it removed software and configuration files but not until the services I wanted to run started.  Therefore I received some initial false positives that the run was successful only to later have everything fail.  This was all due to the fact that I did not properly sanitize my image before saving it.  Don’t live like me!

When making a snapshot of an image that was launched with user data and cloud-init please clean up the cloud-init logs and user data cache.  I did not and I lost almost an entire day trying to debug my image!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: