Beanstalk faster “releases” using web hooks

You need code changes deployed to the development and production servers faster? You’re using releases? Sometimes you need it faster? Then read on...

Recently I find myself more and more occupied debugging problems with Beanstalk releases functionality (Those of you using our support may have noticed). The main reason — FTP limitations. We have three daemons constantly uploading to customer servers, yet sometimes it takes an hour for the queue to reach the release. Even though our ftp client tries to act smart and do the so called “incremental” uploads, it still means checking each file in the repository.

We have marked sitecopy for evaluation, somewhere in the backlog.

“It is slow.” I feel your pain.

Not only this — we have a time limit of 20 minutes per release attempt. After 20 minutes the process gives up and leaves deployment in an unfinished state, which apparently is not the smartest thing to do (By the way, we have a discussion right now — do you think that it is a good idea to send mail to the account owner in this case?). This time limit often is not enough to upload a large changeset, if the connection between the servers is slow. Currently, I have doubled it to 40 minutes, hoping that this will resolve some cases of large repositories (especially imported ones) not passing.

OK, enough grim pictures. On to a solution.

But, first, let’s check if you have what it takes...

• A web server, accessible from the internet.
• Shell access to it.
• Subversion client installed on it.
• (sigh) PHP. Of course, any language will do.

We recently introduced a special integration called web hooks, very simple and versatile.

Ready? Set, go!

First of all, let’s checkout the project on the web server. Login using ssh, then:

cd /var/www
# Or wherever you keep your project.

svn co https://myspace.svn.beanstalkapp.com/myrepo
# Prompted for pass and username. I would suggest that you use a read-only account.

ls /var/www/myrepo
# The project should be here

Now, let’s install the web hook

Don’t expect something fancy, it is a 2-line script (initially it was one line, but I ran into troubles using system, and my PHP is very rusty.)

<?php
  exec('/usr/bin/svn up /var/www/myrepo --username petyo --password ***** --no-auth-cache', $out);
  print_r($out); // This is just to check if it works at all. And no, my password is not five asterisks.
?>

(You may double-check the path to the SVN binary, using which svn.)

Save this as PHP file, and put it somewhere visible from web. Try it by loading it in the browser — http://myserver.com/my-repo-hook.php. You should see a message like this:

Array ( [0] => At revision 9. )

Horray — you have set it up! There is a chance that exec and system are disabled through php.ini.

All that is left now is to specify the web hook. Go to Setup > Integration > Web Hooks and fill in the url of your web hook.

You’re done

This is it, in fact on every commit, Beanstalk will request the URL of your web hook, triggering svn update on the remote subversion copy.

Gotchas?

I can think of few, The most important thing is to check the permissions. The web hooks PHP file contains svn user and pass, so it is a good idea to set permissions just enough for the web server to execute it. Also, since the web server user (www-data, for example) will do the checkout, it is a good idea to chown the project root folder to it.

Disclaimer: This by no means comes to replace the releases functionality we already have, or serve as some sort of an excuse. Consider it more as a fun and alternative way to have your code changes propagated.