Saturday, October 10, 2009

Customizing an EC2 AMI

Setup the Amazon EC2 API Tools

note: this assumes you already have Java installed with the JAVA_HOME environment variable set.

Download and unzip
$ cd
$ wget
$ unzip
Set EC2 environment variables (assuming bash shell)

add the following to your ~/.bash_profile

export EC2_HOME=$HOME/ec2-api-tools-X.X-XXXXX
export PATH=$PATH:$EC2_HOME/bin
export EC2_CERT=$HOME/.ec2/cert-XXXXXXXXXXXXXXX.pem

the EC2_HOME variable will point to the directory created where was unzipped, replace the X's with the version numbers.

the EC2_PRIVATE_KEY and EC2_CERT are the paths to your AWS X.509 certificate files, again replace the X's to match your filenames.

if you have not generated/uploaded these files in your AWS account you can do so in the Security Credentials section of your AWS account. save the private key as well as the certificate file to the ~/.ec2 directory.

Reload your bash profile
$ source ~/.bash_profile
Start the base AMI

this can be done a number of ways but using Elasticfox or the AWS Management Console i feel are the easiest.

both will have you create a key pair in order to connect to the instance, remember where you saved this, you will need it later on.

Customize the AMI

connect to the running instance and customize it how you like.

Upload your private/public key to the instance

upload your X.509 private key and certificate files to the running instance's /mnt directory.

(local machine)

$ scp -i /path/to/ec2/ssh_key $EC2_PRIVATE_KEY $EC2_CERT root@public.dns.address:/mnt

the /path/to/ec2/ssh_key is the path to the ssh key you used to connect to the running EC2 instance (not the X.509 private key).

also replace public.dns.address with the public dns address of the running EC2 instance.

Bundle the Customized AMI

after you are done customizing the instance create a bundled Amazon Machine Image (AMI)

(EC2 instance)

$ ec2-bundle-vol -d /mnt -k /mnt/pk-XXXXXXXXXXXXXXX.pem -c /mnt/cert-XXXXXXXXXXXXXXX.pem -u <aws_account_id> -s 10240 -r i386

note: if you installed your own version of ruby and this command is failing, the easiest workaround is to just change into the /usr/lib/site_ruby directory and run the command.

your AWS account ID can be found in the Security Credentials section of your AWS account.

-dthe directory to create the bundle in (/mnt is ignored by default when bundling an AMI)
-kyour private key (uploaded in the previous step)
-cyour certificate file (uploaded in the previous step)
-uyour AWS account ID.
-sthe size (in MB) of the image file to create. this can be left out if you are not resizing the current AMI
-rthe architecture (i386 or x86_64)
Upload AMI bundle to S3

upload the AMI bundle to your S3 account

(EC2 instance)

$ ec2-upload-bundle -b bucket_name/key -m /mnt/image.manifest.xml -a <access_key_id> -s <secret_access_key>

your Access Key ID as well as yout Secret Access Key can be found in the Security Credentials section of your AWS account.

-bthe bucket name to upload the bundle to. you can use slashes to upload to a subdirectory (or key as Amazon calls them)
-mthe path to the bundle manifest file, this will be /mnt/image.manifest.xml if you did not change the destination directory or prefix in the previous step
-ayour aws access key ID
-syour aws secret access key
Register the AMI

the bundled AMI must be registered with Amazon before it can be used, by default the AMI will only be accessible from your account.

(local machine)

$ ec2-register bucket_name/key/image.manifest.xml

the bucket_name/key/image.manifest.xml is the bucket path on S3 the bundle was uploaded to in the previous step

if successful the AMI ID will be output and you will find it listed as an available AMI.

Saturday, February 14, 2009

Testing Rake Tasks with RSpec

Not sure if this is the correct way to go about testing rake tasks but it works pretty well. Hit up the comments if there is a better way.

So here is an example rake task I want to test:


namespace :app do
  namespace :options do
    desc "refreshes option values"
    task :refresh => :environment do
      options = YAML.load_file("config/options.yml")
      options.each do |attrs|
        option = Option.find_or_initailize_by_name(attrs["name"])
        option.update_attribute("value", attrs["value"])

This is just loading in a yml file and refreshing some default values. Simple enough but I want it tested.

Here is what I came up with:


require "/path/to/spec_helper"
require "rake"

describe "app rake tasks" do
  before do
    @rake =
    Rake.application = @rake
    Rake.application.rake_require "lib/tasks/app"

  describe "rake app:options:refresh" do
    before do
      @task_name = "app:options:refresh"
    it "should have 'environment' as a prereq" do
      @rake[@task_name].prerequisites.should include("environment")
    it "should load 'config/options.yml'" do
    it "should create or update all records in the config file" do
        { "name" => "option one", "value" => 10 },
        { "name" => "option two", "value" => 20 }
      option_one = mock(Option, :null_object => true)
      option_two = mock(Option, :null_object => true)
      Option.should_receive(:find_or_initialize_by_name).with("option one").and_return(option_one)
      Option.should_receive(:find_or_initialize_by_name).with("option two").and_return(option_two)

      option_one.should_receive(:update_attribute).with("value", 10)
      option_two.should_receive(:update_attribute).with("value", 20)

Let's walk through whats going on here...

@rake =
Rake.application = @rake

This is initializing a new rake application object and setting the current Rake.application to it. This is done so that each test is run with a fresh application object.

Rake.application.rake_require "lib/tasks/app"

This is requiring the rake file with the task I will be testing.


This is essentially stubbing the environment task which loads the rails environment. The rails env is already loaded when you are running your specs so this is unnecesary.


This is what actually runs the rake task.

The rest of the 'it' blocks are pretty self explanatory, testing that the task has the correct prereqs and then just spec'ing the functionality.