Scalable Rails App with Nginx

If this is TL;DR just click here for the a look at the final Role cookbook on Github

Chances are that if you are working with Chef, you are using it to automate things not only for easy disaster recovery, but for handling how to work with things at scale. One of the problem requests that I had recently was how to have rails_apps setup at scale with Chef. These are some of the basic setup pieces that will comprise our new scalable rails server

• Ubuntu 14.04
• Nginx with reverse-proxy lookup
• Rails 5.0 and above
• Unicorn server

This sounded somewhat easy enough, but there are a few gotchas that were somewhat of a time sink before I figured them out. 

PROBLEM: Setting up Cookbook structure

SOLUTION: Use wrappers and roles/role cookbooks
This was the most obvious and simple solution. My organizing cookbook structure was as follows (diagram):

• org_webserver role_cookbook which was a wrapper for the following cookbooks
  • org_nginx
    • wrapper for nginx community cookbook
  • org_rails
    • Devopsjeff written code

This is a very common organizational structure for cookbooks (I.E. The Berkshelf Way). Here I use a role cookbook at the top level to run the list of app_cookbooks that build out the infrastructure. You can use ones written by the Chef community, or simply write your own. In this case I did one of each. Just make sure to include the dependencies in your Berksfile as well as your metadata.rb for them to be recognized and be included in the run_list.

PROBLEM: I am having trouble getting the community rails cookbook to work 

SOLUTION: Don't bother, and write your own.
This is straight from an experienced person who actually works at Chef. For every community cookbook that is well written and usable, there are probably half a dozen that are confusing, opinionated, or not well maintained. In fact, There was a recent kickstarter campaign to get a senior community member to spend A MONTH of time to help refactor the rails deployment cookbook. Depending on when you read this it might have been fixed, but I would bet on having to write your own. 

For this recipe, we just need to download and install a few packages and Rubygems to get things up and running. This first requires us to setup Ruby on our system. Rather than downloading external components with another cookbook, I usually elect to use the Ruby brought in with Chef as the system Ruby. As of later versions of Chef 11 and beyond it is pretty up to date with the latest stable ruby, so that's a good option. 

(setting the path to the installed ruby so the system has access)

Then, we would download and install the other required packages to get unicorn and rails running on your system. 

PROBLEM: Nginx community cookbook and it's default templates don't match what I want

SOLUTION: Write your own and turn off default linking
Nginx is a great open-source webserver, no complaints on that front. When being setup by Chef however, it defaults to a setup that doesn't match what I want. While trying to fix this I was running into an issue with conflicting symlinks for the default sites-enabled config file. This cost me over an hour of debug because I didn't know where to look. Luckily, someone already thought of this and created a simple solution.

In the default attributes of the community cookbook, there is an option to turn off the default sites_enabled config as to allow you to use your own config files. To set this, put an override attribute precedent in your wrapper cookbook to false, then add your own config in the run_list.

 

(In this setup, this bit would go in the default.rb of the org_nginx cookbook)

PROBLEM: How do I set up Nginx config to allow for scale?

SOLUTION: Nginx reverse Proxy setup to set up for scale

This is a situation where I won't go into a lot of detail on the setup, mostly because there is a huge amount of available material online already. Some good sites to visit in tandem with this one are as follows. 

https://www.digitalocean.com/community/tutorials/how-to-scale-ruby-on-ra...

https://gorails.com/setup/osx/10.10-yosemite

http://spin.atomicobject.com/2013/07/08/nginx-load-balancing-reverse-pro...

If you aren't intimately familiar with Nginx this may be a little confusing. In short though, you are telling Nginx to do the following. 

• Set the site root to the /home/app/mysite/current/public (or wherever you want to store your rails project depending on preferences)
• look for various index files in order until it reaches the @app location
• tell it to take all requests @app location and proxy pass it along to an upstream localhost socket
• display those contents on the browser default port 80.

(setup for linking your own nginx conf to the proper directory)

(contents of the nginx.conf template that we input

To me, this is a pretty cool parlor trick. Rather than just using Nginx as a server to display content, we are also setting it up as a load balancer in which we can add IP-addresses in a server block, and it will scale out our rails app horizontally. On top of that, you can set up extra location blocks so you can use other apps like a wordpress blog as a subsection of the site. There are lots of use cases where this feature would be useful. Even better, there is a chance that someone has already written a tutorial on how to do it! Rather than rehash every option here, I will stick with a bare bones approach and let you search for a solution to your specific use case.

PROBLEM: How would I set this up for automatically adding servers to the load balancer?

SOLUTION: It's outside of the scope of this how-to, but for a lot of you I would stick with manual entry.

The problem is that while you can use Chef for initial setup of the base webserver, adding extra ones into the nginx.conf is not something that is easily automated, at least not by Chef. This is something more associated with orchestration, and I think that  Chef has wisely tried to stay out of this realm. Automating this is something certainly worth exploring (and I plan to), but for many use-cases I could see adjusting it manually as the simplest solution. 

PROBLEM: How do you set up and Deploy the Rails App on the server?

SOLUTION: It's outside the scope of this how-to, but I'll show an important aspect. 

Ideally this is a problem that is solved as part of your code testing system and continuous delivery pipeline, not with Chef. For the purposes of this article I have included a basic sample setup so you can see it in action, but as a rule I would not include this logic in your cookbook. This is something better solved with rails deployment tools (Capistrano, Heroku, Mina, etc.) than with Chef. The one thing I wanted to note was that for this to work, you will need a unicorn.rb which listens on the described socket for requests. In my example I used the template resource to create the file, and put it in config directory of the rails app. 

(showing that I put the unicorn.rb in the APP_ROOT/config/ directory)

(whats in the unicorn.rb config file. Note where the unicorn socket is listening. It's the same socket that we described in the Nginx config)

So with all of the cookbooks written you would upload these cookbooks to your Chef server, run a bootstrap and converge, and check if it's running on your node. If so, congratulations! If not, then contact me and I will figure out what the issue is.