Posted on Fri 16 August 2013

Frontend deployments with Yeoman and Fabric

When we started the major 2013 technical upgrade of Stickyworld's consultation web app one of the first tasks we looked at was breaking up the backend from the frontend.

A lot of the frontend code in the previous version was written in a clean and organised manor but there was a lot of HTML, Handlebars templates, CSS and JavaScript tangled in with the backend's PHP code. The layers of technology were blurred together. We wanted to have one scripting language per file and to use more high-level, preprocessed languages to lessen the lines of code and verbosity of syntax.

We decided to go with yeoman.io as our starting base for tools and file layout organisation. Yeoman does a good job in three areas: (1) it installs a package management system for our external libraries, (2) a couple of fixture tests are created to help us get started with writing tests for our frontend and (3) a build system is installed that will minify and transform high-level markup into something most browsers can understand.

But what was missing was a system to deploy the code to a live server in a way we were happy with. Don't get me wrong, there are various grunt commands to do deployments but we didn't find one that matched each edge-case we were coming up against. I imagine that most organisations have unique deployment requirements so it's a piece of the puzzle which is probably left to the devops and security teams rather than one-size-fits-all grunt tasks.

Being a Python-shop we decided deployments would be done with a fabric script. It's a fairly small set of files that sits on our build server. Running the deploy task is straightforward:

(frontend)[build@ubuntu  frontend (master)] fab deploy

And this is what the task's code comprises of.

@task
def deploy():
    local('PATH=$PATH:`pwd`/node_modules/.bin; grunt build')
    exts = ('css', 'eot', 'gif', 'html', 'ico', 'jpg', 'js',
        'otf', 'png', 'svg', 'swf', 'ttf', 'woff')

    if isfile(env.local_zip_path):
        local('rm %s' % env.local_zip_path)

    with lcd(env.dist_path):
        cmd = "find . -type f -regex '.*\(%s\)$' -print | grep -v '\._' | " +\
            "xargs zip -q %s '{}' \;"
        local(cmd % ('\|'.join(exts), env.local_zip_path))
        local('ls -lh %s' % env.local_zip_path)

    run('touch %s/%s' % (env.home, env.zip_filename))
    run('rm %s/%s' % (env.home, env.zip_filename))
    put(env.local_zip_path, env.home)

    run('touch %s/rm_will_run_no_matter_what' % env.www_root)
    run('rm -fr %s/*' % env.www_root)

    with cd(env.www_root):
        run('unzip -q %s/%s' % (env.home, env.zip_filename))

    run('rm %s/%s' % (env.home, env.zip_filename))
    local('rm %s' % env.local_zip_path)
    push_config()

    utc_str = datetime.datetime.utcnow().strftime("%Y-%m-%d-%H-%M-%S")
    local('git tag deployment-%s' % utc_str)
    local('git push origin master --tags')

This task can be broken down into four major stages: (1) building and minifying, (2) packaging assets into a zip file, (3) deploying the zip file to production and (4) tagging the deployed commit with a timestamp.

Let me walk you through each of the stages:

Stage 1 of 4: Building and minifying

Here we're running a grunt task which will run several sub-tasks. The CoffeeScript-based code that configures which sub-tasks will run and in which order looks like the following:

grunt.registerTask "build", [
    "clean:dist",
    "coffee", "jade", "compass:dist",
    "useminPrepare",
    "imagemin", "cssmin", "htmlmin",
    "concat", "copy", "ngmin", "uglify", "rev",
    "usemin"]

clean:dist will clean out the folders and temporary locations where we'll be storing our files. coffee, jade and compass:dist will compile the relevant high-level markup files into JavaScript, HTML and CSS.

useminPrepare and usemin work as a team; the first detects all the code blocks where non-optimised assets are called from and the later will replace those references with optimised assets once they're created. Note that usemin is called at the very end of our build process.

In between useminPrepare and usemin you have the following tasks that will be called:

["imagemin", "cssmin", "htmlmin", "concat", "copy", "ngmin", "uglify", "rev"]

These will crush most assets down to as small as possible and rename the filenames in a way that describes the content of the asset. If an asset hasn't changed since the last build, it's filename won't change but if there has been a change to the contents of that file then the filename changes. This is a good way to break caches.

For example, in this build dist/scripts/room.js becomes c2423063.room.js. The c2423063 prefix is a SHA-1 hash of the contents of room.js. If I make a change to room.js the next time I run build the SHA-1 would compute to a different value and hence, the filename would change (e.g. 436a75a4.room.js).

Then certain javascript files will be concatenated together into a single file (scripts/8e1991c7.stickyworld.js for example).

Not all JavaScript files will work properly when concatenated together so the concat task needs to be picky. Things like use strict included at the top of JavaScript files tend to be called on a per-file basis. There could be libraries where some JavaScript will be written in a way that use strict would cause it not to work properly so these files have to be on their own.

You can wrap use strict in functions so perhaps this is an area that could be further optimised at some point.

usemin will then go around replacing references to the non-optimised assets with the optimised ones. So an HTML segment like the following:

<script src="scripts/world.js"></script>
<script src="scripts/room.js"></script>
<script src="scripts/profile.js"></script>

Could end up looking like this:

<script src="scripts/8e1991c7.stickyworld.js"></script>

Stage 2 of 4: Packaging assets into a zip file

We take all the optimised assets we want to host and compress them into a single ZIP file. There could be hundreds of files for our frontend and uploading each one-by-one would add a lot of overhead. By making it a single ZIP file and then only uploading that one file and decompressing it on the production server we're saving ourselves a lot of SFTP overhead.

There is a balancing act to be had. Most files are already compressed so ZIP compression won't do much to make them any smaller but the I/O, Memory and CPU overhead is so small I've decided to leave the default compression switches in place for the zip command.

But we need to pick which file types we're uploading carefully. We use bower to do our package management. It stores everything for each package in a components folder. We don't want to upload all of these components.

To give you an example, let's say I want to install moment.js:

➫ bower install --save-dev moment
bower cloning git://github.com/timrwood/moment.git
bower cached git://github.com/timrwood/moment.git
bower fetching moment
bower checking out moment#2.1.0
bower copying /home/mark/.bower/cache/moment/6beddf80703c9017efdf314e92cc18e8
bower installing moment#2.1.0

If I then look in it's folder I can see a few files I don't want to be serving:

➫ ls -l app/components/moment/
total 88
-rw-rw-r-- 1 mark mark  9925 Aug 16 01:10 component.json
drwxrwxr-x 2 mark mark  4096 Aug 16 01:09 lang
-rw-rw-r-- 1 mark mark  1056 Jul  5 05:53 LICENSE
drwxrwxr-x 3 mark mark  4096 Aug 16 01:09 min
-rw-rw-r-- 1 mark mark 53238 Aug 16 01:09 moment.js
-rw-rw-r-- 1 mark mark  9040 Aug 16 01:09 readme.md

I really only want to host the .js files, not component.json or readme.md.

To avoid including such files I've created a shortlist of file extensions that will be served and then only compress files in components/ with these extensions. My short list includes css, eot, gif, html, ico, jpg, js, otf, png, svg, swf, ttf and woff.

exts = ('css', 'eot', 'gif', 'html', 'ico', 'jpg', 'js',
    'otf', 'png', 'svg', 'swf', 'ttf', 'woff')
local("find . -type f -regex '.*\(%s\)$' -print | grep -v '\._' | " +\
    "xargs zip -q %s '{}' \;" % ('\|'.join(exts), env.local_zip_path))

This will look like the following when transformed into a bash command:

find . -type f -regex \
'.*\(css\|eot\|gif\|html\|ico\|jpg\|js\|otf\|png\|svg\|swf\|ttf\|woff\)$' \
-print | grep -v '\._' | xargs zip -q /tmp/package.zip '{}' \;

In the current version of our frontend the resulting zip file weighs 14MB.

Stage 3 of 4: Deploying the zip file to production

Once we have the ZIP file uploaded to the server we clear out any pre-existing frontend files and decompress our zip file into the root web directory.

run('touch %s/rm_will_run_no_matter_what' % env.www_root)
run('rm -fr %s/*' % env.www_root)

with cd(env.www_root):
    run('unzip -q %s/%s' % (env.home, env.zip_filename))

This is more primitive than I'm happy with. It would be good to create a new folder so we could do a sanity check that the files did arrive safe and sound and have a rollback option to each deployment we've ever made that would be quicker than building and deploying another committed revision of our code.

Stage 4 of 4: Tagging the deployed commit with a timestamp

When we've made a successful deployment we then tag the deployed commit with a timestamp. This lets us pinpoint what code was placed on the server and when. The tag contains the current time in UTC. If we were to move build servers around between different timezones we wouldn't want to be guessing which timezone a tag was done in.

utc_str = datetime.datetime.utcnow().strftime("%Y-%m-%d-%H-%M-%S")
local('git tag deployment-%s' % utc_str)
local('git push origin master --tags')

With a single git command we can see when there have been successful deployments:

➫ git tag
deployment-2013-06-28-13-44-46
deployment-2013-06-28-13-53-12
deployment-2013-07-03-15-35-24
deployment-2013-07-15-08-09-48

We can also see what changed in the codebase between deployments:

➫ git diff deployment-2013-06-28-13-44-46..deployment-2013-06-28-13-53-12
diff --git a/.gitignore b/.gitignore
index 25a3325..c8d681d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
+*.pyc
 app/*.html
 app/views/*.html
 app/views/[a-z]*/*.html

To make sure the timestamp is accurate we have the following cron task running on the build server:

➫ crontab -l
*/5 * * * * sudo /usr/sbin/ntpdate ntp.ubuntu.com pool.ntp.org 1>/dev/null

sudo restricts down the user, command and arguments to avoid having an attack vector on our build server.

I'm proud of where we've gotten our frontend processes to but there are still many areas we can research, build and improve upon. If you're interested in working in this area please do drop me a line.

Signup for our low-traffic newsletter:

Powered by MailChimp

© Giulio Fidente. Built using Pelican. You can fork the theme on github. .