Design

With this problem in mind I designed a status page to help people understand what’s going on with Beanstalk at any time:

My goal was to make it really simple to get rid of any distraction. We use special headers for moments when we have issues or have a maintenance in progress:

Days without incidents

We thought that it would be interesting to count the days since the last incident, so this number is always there right below “All services are operating normally” message. We are really motivated to keep this number as high as we can!

It’s simple jQuery which calculates the days since the last update and replaces the default message:

// To keep code clean and semantic I store time in datetime attrribute of HTML5's time element
var lastUpdate = $('#updates article:first time').attr('datetime');

// By default this paragraph has universal message for those with disabled JS
$('header section.status-ok p').text(distanceOfTimeInWords());

function distanceOfTimeInWords(){
    var now      = new Date;
    var fromDate = new Date;
    var from     = fromDate.setTime(Date.parse(lastUpdate));

    // Convert milliseconds to days and round to the whole number
    var days     = Math.round(((now - from) / 1000) / 60 / 60 / 24);

    if (days < = 1) return 'Issues were reported in the last 24 hours. Read below for details.';
    else return 'It’s been ' + days + ' days without any incidents.';
}

Platform choice

We had internal debates on how to build this page — plain HTML is easy to build but hard to update and maintain, while simple Rails app hosted on Heroku takes some time to build, maintain and extend with extra features. Tumblr become a winner for a variety of reasons:

• It took only half an hour to create a new blog, convert HTML mockup to template and add a custom domain name.
• Everyone on our team already has Tumblr accounts, so we all can post updates.
• We can quickly post updates and even change headers with just an email.
• It already has an RSS feed.
• It doesn't share our servers and environment.

We use FeedBurner for all our RSS feeds and it has a nifty feature which lets our users subscribe to updates by email. This way subscribers can be notified of maintenance work or issues on our end within minutes.

Switching headers

Of course Tumblr wasn't intended for this use, so I had to come up with some workaround to show appropriate headers. Thankfully Tumblr has tags, which can be used even when posting by email. We agreed to use three tags - 'ok', 'problem' and 'maintenance'. The appropriate header is displayed with simple JS based on tag of the last post, while tags itself are hidden.

var state = $('#updates article:first .status');
var latestState = (state.length > 0) ? $(state).html().toLowerCase() : 'ok';

$('header section').hide();

if (latestState == 'problem') $('header section.status-problem').show();
else if (latestState == 'maintenance') $('header section.status-maintenance').show();
else $('header section.status-ok').show();

The status page is linked from many places on our landing site and within the application. Make sure to subscribe to RSS or email notifications or follow us on Twitter to get latest system updates in time.

Great Communication Earns Trust

One of the most common pieces of positive feedback we get from Beanstalk customers is how responsive we are to support and communication – but that doesn’t mean we can’t always be looking for ways to improve! We know that being clear communicators, not just when things are going wrong, helps people trust us and love our products even more.

This simple status page will help us build and keep the trust of our customers, hopefully you can use some of these techniques to do the same thing for your product status pages!

You might not have even noticed, but Gmail previews the first few lines of text in an email right after the subject. Normally, this is great, helping you peek inside an email to determine if you want to read the rest. But we realized that for our newsletters, the preview piece was always the same: “Follow us on Twitter. Read our Blog. View this Newsletter Online”. Not very helpful, is it? Wouldn’t it be great if those first few lines were more descriptive, or at least more enticing to the reader?

I poked Eugene about this problem, since he’d recently done a bunch of work to optimize our email templates for Beanstalk. He hadn’t heard of a good solution, so he set out to experiment and create his own.

The challenge was to create some text at the beginning of the email that would be shown as the preview, but not appear in the body of the email itself. It turns out, like many HTML email tricks, an old-school method ended up working best.

Remember spacer gifs?

Back in the days before beautiful CSS-powered layouts, we used tables. Sometimes a table cell needed to be propped open, and a common technique was to take a 1px by 1px transparent gif image file and set its height or width dimensions to stretch it as a “shim” to hold a table cell to a certain size.

Since HTML emails use tables for layout, Eugene was inspired to go back to some old tricks.

<img alt="Free tutorials, tips and best practices to help you make the most of your design and development workflow." src="spacer.gif" width="1" height="1" border="0">

By putting the text we want to show up in the Gmail preview in the alt tag of a 1px by 1px spacer gif, we were able to accomplish the desired effect without changing the visible content of the email. Success! Nice work Eugene!

What to put in your optimized email preview text?

The key to these previews being useful is to find information that supplements the subject line of an email.

For our newsletter announcing Beanstalk Guides, we made the preview more focused on the kind of person who might want to look at guides – in our case, people who wanted to make the most of their design and development workflows.

For your newsletter, you will want to write a preview that speaks to your ideal recipient. Let the subject focus on the specifics of a deal or offer – the preview should be a little more personal feeling!

Update: Looks like Campaign Monitor wrote about this technique…in 2006! We’re glad to see that we’re not the only ones who’ve found this solution, but we’re also sure that we’re not the only people who hadn’t seen Campaign monitor’s post, either. So we’re happy to give this topic a much needed “bump” and encourage people to improve their emails!

Mobile isn’t just in the browser

David Greiner from Campaign Monitor recently posted results from their report and it’s fascinating to see that mobile email clients grown from 4% to almost 20% of the market in just 2 years! From their historical chart, it seems fairly certain that mobile email usage will overgrow desktop email usage in a year or two. Thinking about my own iPhone usage supports these statistics — I check emails, feeds and tweets all the time, but go into Mobile Safari only when I have a good reason.

With this in mind, I began to think about Beanstalk’s emails — newsletters, digests and commit notifications. We’ve made them quite nice in desktop and web clients, but what about mobile clients? Can we keep them nice and readable here? Turns out we can! Mobile Mail on iPhone supports media queries, which makes possible to use the same responsive web design technique we applied to the rest of Beanstalk. How cool is that?

I’d like to show you how we applied screen-size specific media queries to target styles, and what design decisions were made to make the mobile email experience much better

Digests

I started with our Daily Digest emails, which provide an overview of the previous 24 hours activity for a given Beanstalk repository.

First was to add a media query for the iPhone’s small screen:

@media only screen and (max-device-width: 480px) { ... }

After improving font sizes and margins it was clear that mobile version was too cramped, so I decided to hide the commit time. It’s usually more important to see what’s changing than when exactly something was committed, and the time is still presented on changeset page for reference if needed. Next, I reformatted the changes summary — a vertical layout worked much better on a small screen. And the last change was to reformat header into two lines instead of just one. Here is how digests look now:

I noticed minor visual issues on iPad, but they were easy to fix by targeting with this media query for the iPad’s screen size:

@media only screen and (min-device-width: 768px) and (max-device-width: 1024px) { ... }

Newsletter

We also send a regular newsletter to Beanstalk customers, and with experience from digests, it was a piece of cake!

I improved font sizes and margins to use more of the screen space, resized the logo to be proportional to the screen, reorganized the navigation to avoid wrapping, and made images flexible. Now our newsletter doesn’t need any pinch-zooming to navigate and is much more readable on small screens!

What I have learned

• Testing in a mobile email client and not just in mobile browser or resized desktop browser is more important now than ever before. Mobile Safari and Mail on iPhone don’t work exactly the same.
• It’s not easy to customize layout based on table cells, but display: block; and negative margins work really well for for small changes.
• In my tests it seems like <meta name="viewport"> (?) doesn’t work in Apple Mail. This caught me off-guard, and may cause you problems as well.
• At first I tried to keep width of emails flexible but in a result every digest had different scaling based on content. That looks really weird if you receive a few digests every day. Sometimes it’s better to keep things fixed at 320px. This requires thinking about your use case!
• Yahoo! Mail removes media queries and applies all included mobile stylesheets to web client, but there is an easy fix for that.

Conclusion

It’s not hard to optimize emails for mobile email clients, and the result is worth it! Even while we still use table layouts and inline stylesheets for emails in 2011 it feels like a fresh breeze to drop in some media queries and advanced stylesheets for the clients that can support them.

We’ll continue to optimize our HTML emails here at Wildbit, and would like to see more optimized emails from the apps we use and love every day as well. Hopefully sharing our experience helps make that a reality!

We like to spread out our support hours and for that reason would love someone in North America or South America. We are a completely virtual team, so you’ll need to be self-motivated, very passionate, and organized. Our team is currently located in Philadelphia, Russia, Ukraine, Serbia, Germany and Canada.

If you’re interested, we’re going to be looking for the following information about you. Just email jobs@wildbit.com.

• Your experience working with / building web apps
• The city and country where you live
• Any open source projects you’ve contributed to or maintain
• Personal blog, site or articles
• Any blogs or people you follow (helps us understand your interests)
• Books that have inspired your work

Some things you should know about us:

• We plan and release in short iterations.
• We use Campfire for daily meetings
• We have team retreats in places like Greece, Turkey, Cyprus and Spain
• We have a dedicated QA Tester, but expect your work to be fully tested on your own as well.
• We have each person contribute to support in order to understand the needs of our customers.
• We schedule time for open source projects as part of Open Source Friday’s.

Interested?

Email jobs@wildbit.com and include the information above. If you’re not the right person but you know somebody who is, please share this post!

1. Semantic tagline and subtitles

<hgroup>
<h1>Beanstalk</h1>
<h2>Version Control with a Human Face</h2>
</hgroup>

The hgroup element saved me twice during this work – first for coding our logo and then for marking up subtitles on pages like Partners or Security. It was a pleasure to ditch <p class="tagline"> and other mess like that.

2. Using <h1> as section header

<section>
<h1>Mom, look at this header!</h1>
</section>

In HTML5 each section can have its own <h1> element, so it’s much easier to maintain the correct page outline.

3. Better forms

<label for="fname">Full name</label>
<input type="text" id="fname" placeholder="First">
<input type="text" id="lname" placeholder="Last">

The Placeholder attribute was a nice and painless way to add text placeholders without any JS.

<input type="email">
...
<input type="url">

On the signup and blog comments forms we used new input types – email and url. I especially love how they improve the user experience on the iPhone with customized keyboards.

<input type="text" required="required">

Finally there is a way to semantically mark required fields!

4. figure element in features tour

<figure>
<img src="../images/ss-repositories.png" alt="Beanstalk Repositories">
</figure>

This element may be used for much more than images, but I decided to use it in our feature tour. Sadly, optional <figcaption> child element didn’t allow titles and paragraphs, so I wasn’t able to use it for captions. The <figure> element may not suit the feature tour 100%, but I believe that so far this is the most appropriate element in HTML5.

5. Meaningful dates in blog

<time datetime="2010-03-18" pubdate="pubdate">Mar 18</time>

This speaks for itself. A great way to add semantic meaning to dates.

* * *

Overall, this project went really well. I had only 3 minor issues:

• Wrapping several block elements (like H1 and H2) in A element still doesn’t work well in most browsers.
• Textmate incorrectly highlights HTML5 elements in CSS.
• HTML5 elements code folding doesn’t work in Textmate, too.

After spending couple of weeks with HTML5 it’s painful to get back to old XHTML 1.0 projects – I miss all these funky features!

When we decided to partner with Engine Yard, I can say it was one of the best things that happened to Beanstalk. We let them take care of our infrastructure while we worked on strategy, design, usability, code, and improving the product overall. When it comes to Rails, no one does it better than Engine Yard. The problem is, Beanstalk is a lot more than just a Rails app, we’re a hosting service on our own. As we’ve grown, our need for a unique hosting environment became more apparent.

Making the decision

I have to admit, it was really freaking hard to move away from Engine Yard. They literally take care of everything and their support team is awesome (and a cool group of people). At the same time, we needed more control and a specific configuration for things like svn, storage, apache, virtualization, and so on. So, we decided to go with the leader, Rackspace. They are well known for their network, support, and are able to deliver the managed complex hosting environment that we needed.

Getting help where it is needed

Like I said, we know how to build web apps, but we are not server admins. Moving the amount of data and configurations we have to a new host is a scary process. To make it smoother, we reached out to the experts. For help with setting up our Rackspace environment, we contacted Tom Copeland at InfoEther. He’s been a big part of the process for setting up our new environment, tuning our servers, and understanding our current bottlenecks. It’s obviously a big help that he already runs RubyForge. When it comes to Subversion, we contacted Blair Zajac from Orcaware. Blair has provided some advice for us in the past. This time around, we had him review our Apache and Subversion setup to make sure we covered everything.

Better performance and stability

Beanstalk has had its share of performance problems in the past. The majority of these problems were due to our storage and IO performance. In our environment at Rackspace, we have invested heavily in the best hardware and storage options available. This should reduce most of the headaches when it comes to the speed of checkouts and commits in Subversion. I’m sure we’ll have some kinks to work out after the migration, but the new environment will benefit all of our customers (free and paid).

The migration

Planning the migration was tricky, but we’re confident we have it figured out. Over the next couple weeks we will be migrating groups of accounts over to the new Rackspace environment. We’ll have plenty of notices before it happens and we expect little downtime for each account. With the migration, we also have some really nice improvements like Subversion 1.6 support.

Want to know more about our setup?

This has been a really detailed and exhausting process, so I’d like to share more about the actual server architecture in a future post. Once we get through the migration process, I will post some more details about the new environment. We’re really excited about it.

I’m not kidding about eighty languages here, you can see a full list at the Pygments web-site. It includes stuff like a full family of C languages, Ruby, Python, Java, Scala, Haskell, Scheme, JavaScript, Io, Matlab, Erlang, HTML and many many more.

It also has lovely bits like “.html.erb”, that our fellow Rails-developers use every day in their projects, and will have both HTML and ERB highlighted properly. And much anticipated support for CSS for all the designers out there.

We also updated the diff viewer so that it will now display line numbers correctly for file comparisons.

So, jump into your Beanstalk account (or grab one for free) and check how nifty your code looks with the new Code Viewer. But please keep in mind that as with any other new cool feature there may be some raw edges, such as problems with various character encodings out there. Don’t hesitate to contact us and provide feedback that will help us make the Code Viewer even better.

Overview

When you log in to your account you will notice a new search field at the top right corner of the interface. It will always be there across all pages. If you are on the Dashboard, it will search for commits across all repositories. If you are in repository Activity page or Browser, or any other repository-related page, it will search only in that repository. When you enter something in that field you will be redirected to a Search Results page with a list of all revisions found for your search term.

Right now Search is only capable of finding your commits, searching for files contents is coming later. You can use Search to find by commit message, name of file or directory changed in commit, author name and revision number. Search is using stemming so it will treat words like “fishing”, “fish” and “fisher” as the same word.

And the search is blazing fast, because it’s using an industry standard Sphinx search engine.

Search Scopes

Ok, let’s do some advanced searching now. On the Search Results page you noticed a few options to adjust your search: select boxes for repository and user. Use these to search for commits only in some repository or made by a specific user. If you leave the search field empty and select some user, you will get a list of all commits made by him (or her). Very useful to get information about a user’s activity.

Tricks

There are two special triggers that you can use in the search field to adjust your search results: +by and +rev. +by is the same as the users select box on the Search Results page but it allows you to find by users that are not registered in your Beanstalk accounts. You can use that trigger like that (without quotes): “updating CSS +by gilbert”. Or you can use it without search term to get the list of all commits made by user: “+by ilya”.

The +rev trigger is my favorite. Using it you can quickly jump to a specific revisions in your repository. Say you are on Activity page and you want to see a changeset for revision 400 of your repository. Just enter this in the search field and you will get what you want: “+rev 400″ (again, without quotes). If you do that on the Dashboard you will get a list of revision 400 changesets for all repositories in your account.

How do you like it?

This implementation of Search is just iteration number one so it’s probably not ideal and won’t suit all your needs, but as we move forward we will continue improving it according to your feedback. So we would like to hear from you! Let us know how search works for you and what interesting ideas do you have about making it even better.

In addition to this, our OpenID support was never really complete. We lacked the proper verification and over time, it became a little difficult to maintain. This month we discussed internally whether we should finally improve OpenID implementation in Beanstalk. As we discussed the effort, we decided that it may be better to just remove it. This creates less maintenance for us and does not remove too much value for our customers since a user/pass is still required for SVN.

We’re hoping we can revisit this in the future as we improve integration options, especially if SVN gets OAuth support.

At Engine Yard, we have a group of slices that manage SVN, Web Servers, and our backend processes for things like deployments, integration tools, and so on. All of these slices use a shared storage system called GFS. This allows us to scale to many slices while still having access to essential data.

The issue

When the outage occurred, one of our slices had a severe memory problem. We attempted to fix it with Engine Yard, but due to the nature of GFS we had to first make sure there we no data corruptions. This forced us to shut down the entire environment while we checked the disks. It’s basically a precaution to make sure we don’t reboot with data corruption.

Since we have so much data, the disk checks took a really long time, which resulted in a full environment outage. It was the best approach to make sure we safely brought the environment back, but obviously not ideal for those who needed access to their code. After the disk checks and reboot all was back to normal and the data was verified.

Future avoidance

When things like this happen, we always need to reflect and figure out ways to avoid them from happening again. In this situation, we need to make sure we have enough resources to handle severe spikes that might cause full environment problems. I am confident we added the necessary resources and can control it in the future.

More updates coming

We recently got back from Railsconf. I have to say that hanging out with Engine Yard was the most valuable part of being there. We had a great opportunity to sit down with the team and discuss better ways for us to scale Beanstalk. There are many options, so choosing the right path can be tricky. We feel like we came up with a good plan to not only scale the service, but also isolate our shards (nodes) to avoid system wide issues when things go wrong. As of now, many of you should have already noticed SVN speed improvements.

We have tons of cool features to work on, but our first priority is performance. It’s amazing how challenging and rewarding a successful web product can be at the same time. We’re excited about our growth and ready to take on the challenge to continue scaling the service. Thanks for being patient and helping us grow.

So for me, RailsConf was all about people. Meeting Railsists and Rubyists as I am, sharing experiences, talking about our projects and just bragging about day-to-day shit. It was a very inspiring and interesting experience. RailsConf talks and tutorials were a good bonus to all this :)

I really enjoyed the talks by Obie Fernandez, DHH and Chris Wanstrath. Too bad I missed the talk about Smalltalk and Ruby by Robert Martin.

We also attended New Relic’s customers party. Free food and drinks, you can’t ignore something like that :) New Relic seriously rocks.

Probably the most useful thing we did at RailsConf is talk to the Engine Yard guys. We solved many many problems from our current Beanstalk setup. Hopefully this will make our scaling pain go away soon and make Beanstalk faster. We are working really hard on this one.

And of course, Vegas itself is a super-crazy place to visit! Probably the craziest city in the world. Super amazing shows, lot’s and lot’s of fun and beautiful stuff to see.

We also visited NYC and Philadelphia during the trip. It was also very interesting. I liked Philadelphia very much. Calm and beautiful place to live. I plan to move there for a couple of months to try living the US way.

So, RailsConf 09 was a blast, I really enjoyed it and will definitely attend next year. I hope we will make some small Beanstalk users meet-up also :)

Don’t forget to check out some photos that I took at the conference.

TeamSupport is a SaaS application that integrates a Help Desk with a Bug Tracking system. They feel that most software companies struggle with internal communication since they have different applications for their customer service group and their development/QA teams. Having one tool that both groups can use will facilitate internal communication and also lead to better customer satisfaction.

Of course, the Beanstalk integration is of most interest to the developers and designers using TeamSupport, but it is nice that the customer service group can also see that a particular ticket is addressed by a revision in Beanstalk.

When a new version is committed into Beanstalk, we send out a WebHook configured for TeamSupport. TeamSupport receives this and scans the description field for “version” and “tickets” tags. They then match up the tickets that the new version addresses and they add a note to the ticket with information about the commit, including the link back to their repository on Beanstalk so they can review the actual code.

Hope you like it. Be sure to give us feedback and let us know what you think.

In the beginning of the year we helped sponsor Space Cookies, An all-girls FIRST robotics team from the San Francisco Bay Area. They’ve been doing extremely well and were just interviewed on CNN for their competition in Atlanta. Really great work and best of luck!

View the CNN Video

Incidents

At Beanstalk we are always striving to make your experience as flawless as possible, to make things “just work”, but it’s not always possible. Sometimes errors happen, and up until now you had to contact support to do something about it.

Enter Incidents.

I’ve created a system that will report all kinds of different issues that may rise for your account, and give you a way to debug them or notify support.

For instance, in the case if you have a failing repository import – you will know exactly why it failed – maybe you uploaded the wrong file, and now you can resolve the issue by yourself, without waiting for an answer from support.

Another instance is failing Release deployment – these were hard to debug without support before.
Now if Release fails, you will see an error message, complete log information about what went wrong – it will help you setup and debug your Release servers correctly.

Third example where Incidents will come in handy is debugging integrations with third-party services that we offer to all of our customers. Sometimes API’s change, authentication tokens get changed or accounts get suspended – whatever the reason – you will know.

I want to add, that Incidents will not replace our support team. We’re always eager to help you with any issues, no matter if there are any Incidents involved. Incidents are for those cases when there is a big chance the issue is not on our side, and you want to take care of it yourself to continue with your work faster.

Releases

We’ve had Releases – automated and manual deployment of your repositories to your own servers – for a year already and yet it’s still in Beta. Here’s an explanation for this – we weren’t happy with the functionality or with the average time it took for Release to get deployed. Until now.

I’ve replaced a big chunk of Releases code with a simpler and more straightforward system, that works using a different principle. Let me explain. Before now, to deploy a Release we basically made an export of your repository on our server, then connected to your deployment server and just mirrored these two directories. It took a lot of time, because we had no way to know what had changed on your server, and even if you were deploying the change to a single file, we had to compare every single directory of your deployment directory with our local copy.

I must admit that it was my bad decision, back when I was developing the previous version of Releases. I won’t defend it now, let’s just say we all make bad judgements sometimes :)

Here’s how new Releases work. We figure out what is the last revision deployed on your server, and we know what revision you want to deploy. We take all the changesets in between these revisions, and extrapolate exactly what we should update, add and delete from the server. Now you see why it will be much faster than the previous way.

There is one catch though.

Subversion is not very bright. It thinks that when you move two directories, you actually delete old one and create new directory. It’s more complicated than that, but basically it is true. Because of this, when we collect changesets to deploy on your server, we have to delete directories that were removed/renamed/moved in Subversion repository from your server. This is an important change from the previous Releases, when nothing was ever removed from your server, even if you wanted it to.

We discussed an option where we would only delete files from your server if you enabled a checkbox in your Release configuration, but we decided that it won’t work. Just imagine what the deployment on your server would look like, if every source directory for any moved or renamed directory would just be left lying there forever. The tree would turn into a mess very fast. Anyway, we wanted to let everyone know about the change, so if anyone relies on the behavior that Releases do not delete files will have a chance to stop it.

If new Releases cause problems for you or you have some bright ideas on how to solve the situation with moved directories, please contact us. If everything goes fine, in the next few weeks hopefully Beta tag from Releases will go away forever.

Other small, but important stuff

While working on Incidents, I’ve also put a lot of effort in making daemons that manage background tasks in Beanstalk, such as Releases, Imports, Caching work faster and more reliable. One of the things that was improved is Amazon S3 backups – we had some issues with it, and now they are resolved and we have reliable constant running incremental backups to S3, in addition to our various other backups. Your data is now safer than ever.

We’ve also had some noticeable hardware and infrastructure improvements, thanks to EngineYard.

Petyo and Ilya also put a lot of hard work into fixing old and new bugs, that were bugging our users, and they have plenty new stuff up their sleeves. Igor Balos, testing superman, made this release stable, and our design gurus Gilbert and Eugene, added style to the UI.

We won’t be deploying all the upgrades at once though. Today we will deploy bugfixes, Incidents and Releases. Later this week we’ll deploy infrastructure changes such as new slice for our daemons and another GFS storage partition for your repositories.

This turned out to be one lengthy post, but so is this Beanstalk upgrade. We sincerely hope you’ll like it.

Please, let us know if you have any issues or comments.

Read the full post at JulianSchrader.de

Problem: Storage calculation

Each plan level in Beanstalk has a storage limit, which determines the plan you need or if you have exceeded your storage capacity across your repositories. In order to keep this information up to date, we need to constantly calculate storage. On the server, we would run a “du” on every commit to calculate the storage, then report back to the application. If the storage is too high, we can inform the customer that it is time to upgrade.

As you can imagine, this creates a huge problem as the application grows. Considering that we have thousands of commits each day, the process would have to calculate storage for literally thousands of files in each repository on every commit. The IO usage was absurd.

Solution: Adjust and compromise

The only solution was to reduce the number of times we calculate storage. Of course, this also means that some people could go over their storage limits and continue to use the system. For us this was an easy compromise. Having a stable system with the ability to grow compared to a short delay in exceeded storage is a no brainer.

We’ve had this fix ready to go since late last week, but it required some additional testing. After the problems in the past two days we decided enough is enough. We all got together today, tested the fix rigidly, and deployed the fix to our slices.

The results

It’s only been running for a short time, but so far load has gone down dramatically. We also increased the RAM on our the slices that handle our daemons. We’ll be watching this carefully to adjust and improve.

While we just recently released performance updates, it’s an ongoing process. This latest fix is another improvement that will help us grow. In the next several weeks we will be working with Engine Yard to improve the speed of our GFS drives as well as some tuning for Subversion specifically.

We really appreciate the patience while we improve and we welcome as much feedback as possible. Please don’t forget to follow us on Twitter for more frequent updates.

Right after we launched paid plans in November of 2007, we prepared a bunch of advertising campaigns to announce it. Some of the resources included The Deck, Daring Fireball, and Smashing Magazine. We were most excited about The Deck due to the number of high-profile sites and targeted audience.

When the ads were placed, we started seeing the traffic come in. We made sure to setup conversion tracking with Google Analytics and we also associated referrals to each new account that signed up. The results surprised us a bit. While The Deck has the most reach, we noticed much better results from Daring Fireball. Check out the chart below:

The first spike is from our ad on Daring Fireball. After that we noticed a steady increase in visitors from The Deck when the ads were posted in February. The second spike, however, is from a post on 37 Signals about our public site design.

What does this tell us?

We could probably conclude a bunch of things from this little bit of information. Overall, it made us realize two key distinctions:

• Banners, no matter what site they are on, are still banners and people tend to ignore them.
• Blog posts hit the readers directly, whether they are paid or not.

The Deck still brought lots of valuable traffic to our site over a longer timeframe and we can definitely recommend it for big announcements. What we learned is that instead of focusing on advertising, we should instead focus on making the product better and getting people excited about it. If people are excited about your product, they are more likely to post about it, giving you direct exposure to their readers. For example, check out this chart, which shows all of 2008:

You can see a series of spikes. The first two are the same from above, which shows Daring Fireball and the 37 Signals post. The gigantic one in the middle is when Versions announced their release (I’ll talk about partnerships in later post), the fourth larger spike is from TUAW in October.

There are lots of little spikes here and there, all from blog posts, integrations, or partnerships. The main point to take away from this is that we have not spent a penny on advertising since The Deck, but continue to see the traffic pour in.

While it is nothing new, we’ve learned that good promotion just depends on the quality of your product and whether or not people are excited. So put your head down, improve your product, and get people excited about it.

I’d love to hear what others have experienced when it comes to promotion and advertising for a web product. If you have some thoughts, post a comment.

• Free: From 20Mb to 100Mb
• Personal: From 1GB to 3GB
• Team: From 3GB to 6GB
• Business: From 8GB to 12GB
• Corporate: From 15GB to 24GB

We also introduced a new plan after some recent demand. The Giant Plan features 200 users, 150 repositories, and 48GB of space for $200/m.

We hope this helps as your repositories continue to grow and evolve.

Our team is located in Philadelphia, Russia, Ukraine, Serbia, Germany and Bulgaria, so location is not important. We’re accepting applicants from anywhere in the world. This is a pure telecommute job and we need an individual who is self-motivated, very passionate, and organized. (i.e. We don’t babysit at Wildbit:)

If you’re interested, please send the following to jobs@wildbit.com.

• Experience with Subversion and Beanstalk
• Past projects and your role / accomplishments
• Personal blog or site, Articles
• Any blogs or people you follow (helps us understand your interests)
• Favorite books

Some things about our team

• Our process is mostly Agile / Scrum
• We use Basecamp, talk in Campfire, and avoid IM as much as possible
• We have team retreats in places like Turkey and Cyprus
• We have a dedicated QA Tester

Upgrade to Subversion 1.5.5

We upgraded all Subversion servers to SVN 1.5.5. With this update you can now take advantage of Merge Tracking and many new features in the latest Subversion release. To make the conversion process painless, we added a simple tool to the Dashboard for account owners. Once the upgrade is activated, all repositories in your account will be upgraded to SVN 1.5.5. We also run svn-populate-node-origins-index, which is new as of Subversion 1.5, and behaves as a cache to vastly speed up certain history-querying operations. This process only takes a few minutes to complete. Read the full help article.

Major performance improvements

Beanstalk has been steadily growing since we launched. Along with the growth, performance has been a constant challenge. While the speed of our SVN servers have been solid in recent months, our web interface for browsing files, viewing activity, and reviewing changesets have been slow. All of this ended during this weekend’s release.

We identified the issue two weeks ago and narrowed it down to the Ruby SVN bindings. Along with updating the bindings and refactoring code, we also improved caching. The results have been fantastic! In some cases larger repositories that used to take 20 seconds to load (ouch!) now load in under a second.

You may notice pre-caching notices when going to a repository. We had to setup this process to cache your repository on the servers. This ensures your repositories load quickly in the future.

Known issues and troubleshooting

With an upgrade of this size, we have noticed a few issues. Here are some known problems and what you can do.

Receiving a 403 Forbidden error in SVN.

In some cases we have seen this happen. During the 1.5 upgrade some AUTH files were corrupted. While we think we fixed it, if you experience this issue just update your account password by going to “My profile” after logging in. Another thing that can help is to do a fresh checkout of the repository.

Pre-caching takes too long

A few cases of pre-caching taking too long have occurred. If you think the process is taking too long, please contact us and we will take a look.

Delay in the commit activity page

The cache is set to a short time frame. However, we are noticing longer delays in some accounts. We’re working on this and it should be resolved very soon.

We really hope these updates show our commitment to making Beanstalk the best hosted Subversion service available. Your feedback has been crucial in helping us improve and we greatly appreciate it.