Using a Github service hook and mod_wsgi for auto-deployment

Github has post-receive hooks that are called on each push to a repo.   And say on each push to a certain branch you'd like to automatically deploy or run other actions.  This describes how I did that with Apache, mod_wsgi and fabric.

The steps are something like this:

1. Configure the hook in Github to fire a POST to a given URL.
2. Configure Apache/mod_wsgi to catch the hook
3. Add a script that runs whenever the URL is POSTED to.
4. Check that it is a reasonable real hook from Github
5. Check the hook is for the correct branch
6. Run a deployment script or something

1. Configure the hook

Either follow the github help on the subject, or go to Admin on your github repo, select Service Hooks, then Webhook Urls, and add a URL that you control. Let's say I put http://myexample.com/myexamplewebhook

2. Configure Apache/mod_wsgi

I'm staging a Django app, so I already have Apache and mod_wsgi set up.  I need to make sure that I'm using a WSGI daemon process so I can set the user and group.  It ends up looking something like this in the appropriate myexample file in /etc/apache2/sites-available.

WSGIDaemonProcess myexample user=ubuntu group=ubuntu display-name=%{GROUP}
WSGIProcessGroup myexample

WSGIScriptAlias /myexamplewebhook "/home/ubuntu/myexampleapp/app/posthookwsgi.py" 

So, here we've made sure that the ubuntu user and group is used for the wsgi process, so our deployment actions are done with the user and group we expect, and we've added an alias for /myexamplewebhook to run our hooks script. You should have something like the last line already, that's calling the normal Django wsgi.py.

And we have to allow Apache to access our posthookwsgi.py:

<Directory /home/ubuntu/myexampleapp/app>
<Files "posthookwsgi.py">
Order deny,allow
Allow from all
</Files>
</Directory>

Reloading/restarting apache will make that available.  At this point, pushing  to the repo should cause an error to go into your error.log.

3-6. Add the Script

Now, I need a wsgi script to call that does what I want.  Here's one I prepared earlier:

This script is a wsgi application all of itself, so you need to have the application method as it shows.  There's an example debug printing line right under the function def.

So, what this script does:

1. Read the body of the post request, which is JSON, and convert it to a dict.
2. Match the conditions in the conditions array. Basically here you put in the conditions that you want the hook to meet, and they will be tested.  An error response means nothing else happens. In this example, I'm checking that the ref is the branch I expect.  If any other branch is pushed, we'll not deploy.
3. Work out the pathname for the base of our deploy
4. Run the auto_deploy fabric task
5. Respond OK

 Note that if anything exceptions during the script or the fabric run, we'll have a note in the error log.

Notes and thoughts:

• This is meant for refreshing a staging server, but you could use similar for all sorts of actions.  The hook POST tends to  arrive within a few seconds of the push, so that's perfect for the use case I need, where one or more html developers can push HTML/CSS changes and wanting to see them live in seconds.
• If you modify the script to do fancy things, beware you don't expose a security vulnerability by script injection.  The incoming JSON data "From Github" may actually come from an attacker trying to own your server.  Don't use that data in script paramters without sanitation.
• This isn't a replacement for serious production deployment.

Bringing the Blog Home

Posterous was acquired by Twitter. They keep telling me. As though that's going to make me happy.  Or is it a warning?

Actually, it does make me happy. Here's my newly migrated blog, made more of the raw materials I deal with daily than rarified startup hot-air OMG-you-can-blog-by-email-ness.

There will be a Colophon that talks about the details at some point. For now, a quick statement of what this is made of.  The stack is like this:

• Amazon Route53,
• Amazon EC2
• Ubuntu 
• Apache
• mod_wsgi
• Python
• Django 1.4 -- MySQL
• feincms
• feincms-elephantblog 
• Twitter Bootstrap

There's bit of HTML in there somewhere too, plus some Javascript tidbits.

-OO-

So, this is me bringing my blog back home to my own infrastructure. May it continue with as much delight and promise as I've experienced so far.

who.theskiff.org gives presence and profile info

As a result of the Skiff Hack, we now have Skiff member profiles and presence info available for the skiff at http://who.theskiff.org.  As in, you can see who is currently in the Skiff or who has been around recently, and a bit about them in the process.

Yes, that is another profile to maintain, but this helps.  We can find out who is around us at the Skiff on a day to day basis.  Lovely work Tom and Simon and I don't know who else made it.

My skiff profile is here.

Adding a sepia filter to django-imagekit

I'm rather fond of django-imagekit -- a useful django tool that organises a bunch of image processing on images saved in django models.  It comes with a bunch of useful scale, crop etc tools based on PIL. 

However, I was looking for a way to make a sepia images from my uploaded image.  I found Fredrik Lundh's PIL code for sepia in his post Sepia Toning with PIL and adapted that into the form needed for a django-imagekit image processor.  Here's how  that looks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

"""
Adds a sepia filter processor for django-imagekit which
can be used in an imagekit specification.

https://github.com/jdriscoll/django-imagekit

See http://effbot.org/zone/pil-sepia.htm for the original
code for the sepia filtering in PIL

"""
from PIL import ImageOps
from imagekit.processors import ImageProcessor


class Sepia(ImageProcessor):
    """
        Turn image sepia
    """
    @classmethod
    def process(cls, img, fmt, obj):

        def make_linear_ramp(white):
            # putpalette expects [r,g,b,r,g,b,...]
            ramp = []
            r, g, b = white
            for i in range(255):
                ramp.extend((r*i/255, g*i/255, b*i/255))
            return ramp

        img = img.convert('RGB')
        sepia = make_linear_ramp((255, 240, 192))
        if img.mode != "L":
            img = img.convert("L")
        img = ImageOps.autocontrast(img)
        img.putpalette(sepia)
        img = img.convert('RGB')
        fmt = "JPEG"
        return img, fmt

How to remap the # (hash) key on Apple UK keyboards

Given how useful the # key is, it has always annoyed me that on the standard UK Apple keyboard, getting a # key requires keying Alt-3 to get it to appear. 

If you are doing web development with jquery, or writing shell scripts or python, you end up using this key a lot and Alt-3 just doesn't work for me.  However, on the left of the recent Apple keyboard, there is the § key.  I've never used §, I guess somebody must.  

Anyway, so I went and found Ukelele, a keyboard layout editor, and modified the keyboard map to put the # where the § used to be, and the result is what I call the BritishLHash keyboard layout.

BritishLHash.keylayout Download this file

How to use this:

1. Download the file, and put it in your ~/Library/Keyboard Layouts folder.
2. Open Language and Text in System Preferences
3. Select the Input Sources tab
4. Tick next to the BritshLHash keyboard.  This makes it available as an input source for you.  And also Tick Show Input in menu bar
5. Find the little flag in the menu bar.  Pop it up and change it to BritishLHash.

That's all there is to it.  I've been using this mapped file for a year or so, and it works fine in OS X 10.5 and 10.6.

Hiding Promoted Tweets in Twitter Searches

I'm getting rather fond of a full mutiple-window experience when watch the Formula 1, and one of those windows is open on a search for f1 or #f1 to keep an eye on the discussion.

That works, except there is often a repeated promoted tweet that stays at the top of the search results. This confuses and annoys me, as I have to cast my eyes past this promoted tweet again and again and again to see the real search results.  Once, on the initial search, is enough, after that, these things are unnecessary noise. Like this:

So, I've made a Google Chrome Extension to take away promoted tweets after the first time you see them.  When there are more search items, and 'x new tweets'  shows, clicking on this renders the search results minus the promoted tweets.   You'll still see Top Tweets, though, I might take them on next.

Get the extension from here:  Hide Repeated Promoted Tweets.  Read more info here.  I hope it is useful.  And let me know what you think.

Unlogo.org -- excellent idea

Proper Javascript Documentation

Promote JS is all about referencing high-quality Javascript documentation, and I surely approve.

Why you don't need a new 'fridge.

For some years now, I've been, on occasions, having a good rave about wanting to start a (print) magazine that had neutral or balanced advertising.   I'd want a magazine that devoted equals space and effort to advertising to sell things to people, and advertisments to encourage people not to buy things.

So, on one page you'd have an ad for a new 'fridge, and on the other side, a similarly high-quality ad encouraging you to maintain and keep the old one or share the neighbours.  There are two sides of every buying decision, we just don't get to see them very often.  And when we are trying to save carbon, not buying appliances does help reduce the CO2 in the atmosphere.

So, you get the idea:  A nicely designed magazine with rules that mean that advertisers need to somehow ensure or allow or fund the for and against cases for their product.  Would anybody care to advertise? Would anybody dare to advertise?  It'd be an interesting experiment.

It would be mad to start a new print magazine right now.  So, what about something online -- and this is the new bit of my thinking.  Presumably, once can buy banner ads with the keyword 'fridge' or whatever and promote not buying a new fridge.  It's be easy enough to make ads like that, and they'd get nicely delivered onto pages where people were looking to buy fridges or were searching out a new 'fridge to buy.  Perhaps the click through rates wouldn't be that high, so it wouldn't cost too much, but who knows?

This has to be against ad placement terms and conditions, you think?  Well, I've just had a look at the google adwords policies, and I can't see anything in there that make it wrong to suggest not buying something might be a good option. Actually, under the 'Anti' and Violence section of the Content Guidelines, there is a specific allowance for "Consumer awareness ads (positive and negative) that don't violate the basics of the policy".   That sounds ok for now, but there's a lot of details to work on.

Okay, so, to make this thing fly, we need:

• funding for some ads
• destination website(s) with the reasons for all of this, or useful consumer information on 'sharing the neighbours fridge' or 'how to survive without a car' or similar.

And that probably means a charity or consumer organisation that wants to try it out. An initial experiment would be very interesting because it would yield click-through information about how consumers react to these messages, and could give a charity or similar a quite high profile.

Anybody interesting in trying this out?

Google Scribe. Can you uncover secrets?

Google Scribe provides text completion service. Using information from what you have already typed in a document, Google Scribe provides related word or phrase completion suggestions. In addition to saving keystrokes, Google Scribe's suggestions indicate correct or popular phrases to use.

I have another question for you is to become an editor of the newspaper and then at the end of their lifespan and regenerative capacity of these cells to their cognate receptors on the cell surface and then they will become more apparent from the following detailed description of the invention.