Hosting a NodeJs Express Application on Amazon Web Services (EC2)

Updated 2014-09-23 - NVM and Ubuntu 14.04 notes

For the past year or so I've been super intrigued by NodeJs. It's a very cool stack, and the thought of building an entire application (front and back) with Javascript certainly merits some attention. Add MongoDb (a javascript based, document database) to that, and you have a nice, cohesive, high performance application.

Hosting your application is also super easy, especially if you choose a service like Heroku. Their service is ultra simple to use, and they support git deployments out of the box. But you may have reasons to not use a service like Heroku. It turns out that setting your application up on Amazon EC2 is not all that difficult. You can have the same git deployment functionality that Heroku supports, and you will have full control over the server running your application. So the price will be the same if you run ten applications or one application on the same server. You'll also have local access to their whole array of AWS services, which is a plus.

Here's a walkthrough on how to get a simple Node Express application up and running on Amazon EC2 with git push deployments.

Build an App

For this walkthrough, you can use any regular node web application, or you can generate the default express website. Here's a quick example:

#get express if you don't have it already
npm install -g express-generator

#create the app
mkdir my-app  
cd my-app  
express  
npm install

#make it a git repository
echo "node_modules" > .gitignore  
git init  
git add .  
git commit -am "initial load"

#start it and browse to http://localhost:3000
npm start  

Getting Started

So again, this is going to be kind of a fly-by in terms of code because most of what I want to cover is related to operations on AWS. To get started, this is just another typical expressJS website. We first need a running EC2 instance, and I've always been partial to debian based linux, so I setup a micro (64bit) Ubuntu 14.04 LTS server. I'm going to assume you know how to get this done and get connected to your server via SSH. The default user account on EC2 Ubuntu is, "ubuntu" and it has sudo access, but you'll need to know your password. You can set it by changing your password "passwd", or you can create a whole new account. It's totally up to you. From there, you'll want to update your system before getting started. i.e. sudo apt-get update && sudo apt-get dist-upgrade -y Once all is current, we'll begin there.

Install Build Tools

We'll need to install some basic build tools and utilities to simplify future steps.

# install prerequisites
sudo apt-get install curl build-essential git-core  

Configure Port Redirection

We need to listen to port 80 and port 443 for web, but the only way we can do that directly with Node is to run it as root. That would be silly, so you have two options: reverse proxy or locally port redirect traffic. (Note, you can also use a Elastic Load Balancer, but that's beyond the scope of this post.

In the end, you're redirecting traffic from port 80/443 to the ports bound to your node application run by a non-privileged account. We'll only cover using iptables to port-forward traffic in this post, but here's a quick explanation for each.

Setting up a reverse proxy with a server like nginx or apache is great when you want to host multiple applications on the same server over port 80. You can use the request hostnames to filter and route traffic via the reverse proxy to different applications locally.

Alternatively, when you use iptables to just port-forward traffic, it really only enables you to run one node application because there's no way filter requests by hostname. This is the simplest solution and probably the most practical when you begin to scale out.

So, to get this going, run these few commands using a root bash.

#kick off a root bash
sudo bash

#creates the rules to the current system
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3000

iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8000

#persist the iptables changes (ubuntu)
apt-get install iptables-persistent

#exit the root bash
Ctrl-D  

Okay! This now enables port 80 traffic to forward to port 3000, and port 443 traffic will forward to 8000.

Application User

Now with everything we need installed, we just need to setup a new application account and get the app running. Create a new user account with a home directory.

sudo adduser app --system --disabled-password --group --shell /bin/bash  

You'll need shell access. Password authentication should be disabled on your EC2 instance, so you'll want to configure public key auth for the new user to connect to it.

sudo -u app bash  
cd /home/appuser  
mkdir -p .ssh  
cd .ssh

echo "--paste your public key here--" > authorized_keys

chmod 600 authorized_keys  

You can test this by disconnecting and reconnecting to your server as "appuser". It should log right in without prompting for credentials.

#from your local machine
ssh app@your-aws-host  

Install Node

I like to use NVM (node version manager) to manage my node installs. Their project page shows how to install it from a command line.

curl https://raw.githubusercontent.com/creationix/nvm/v0.16.1/install.sh | bash  

Update your .profile with NVM

echo "source ~/.nvm/nvm.sh" >> ~/.profile  

Log out and log back in to refresh your profile. nvm should work after you reconnect.

Then set your defaults and configure 0.10.

nvm install 0.10  
nvm alias default 0.10  

We'll use ForeverJS to host the node process. Install it globally.

npm install -g forever  

Keep in mind that nvm will install and run NodeJS and NVM locally in the app user account. If you update your NodeJS version (say 0.10.28 to 0.10.32), you'll need to re-install ForeverJS globally because the versions manage individual global packages.

Configure App Directories

Now to enable git push deployment, we're going to setup a prime/hub scenario. This is where you have a bare repository in your home directory that contains git hooks to automatically pull latest into a "prime" directory. The prime is a clone of the local repository that will run the application. Login as appuser, setup a bare git repository and clone it.

#starting from the home directory: ~/ 
mkdir app.git  
cd app.git  
git init --bare

cd ~/  
git clone file:///home/app/app.git  

The application will start from a bash script called "start.sh" placed in your app directory. The script will load the environment variables from your production shell script, then use forever to kick off the app. Here's an example of mine.

#!/bin/bash -e

#load NVM
source /$HOME/.nvm/nvm.sh 

#start the app
app=$HOME/app  
cd $app && source env.production.sh && forever start bin/www  

Also, in case it's not already, make it executable.

chmod 755 start.sh  

Make sure you've created/copied your production environment script env.production.sh containing your environment settings for your app. (In my case, this is not included in the git repository). If you don't need one, then just remove the && source env.production.sh from your startup script (start.sh). You should also only allow owner read/write other permissions to your configuration with chmod 600 env.production.sh.

Auto-start on Boot

To enable your application to automatically start when your system reboots; just use a local cron job to kick it off.

crontab -e  

Add this line to your cron file.

@reboot /home/app/app/start.sh

Git Hooks & Git Push Deploy

Now lets add a git hook to automatically stop, update and start the application in the prime directory. Create a script with the content below and save it to: ~/app.git/hooks/post-update.

#!/bin/bash

#load NVM
source $HOME/.nvm/nvm.sh

forever stopall  
unset 'GIT_DIR'  
cd $HOME/app && git fetch origin && git pull origin master && npm install && ./start.sh

exec git update-server-info  

Also make sure this one is executable.

chmod 755 ~/app.git/hooks/post-update  

This script will stop the app, pull the latest code, install any missing npm modules, and then restart the app. And this will happen every time you push a change to the git repository. That's it! Now you'll need to wire up your local git repository with a new remote pointed to this server. Do that with this command:

git remote add aws appuser@your-server-host:app.git

git push aws master  

In theory, at this point, the whole thing should light up. You'll push the full code-base up to the server. During the same push session, it'll pull latest into the prime directory and re-roll the application with the environment settings you have tucked away in the env.production.sh. The application should now be running and you should be able to hit it from a browser. http://your-aws-host/

To check your logs, identify the current log file and show its contents.

forever list  

forever list output

#show the contents of the log. 
forever logs 0  

Your log file (if you repeat the log inspection we did earlier) should contain a nice clean response showing your app is running on port 3000 and/or 8000.

Your workflow now is super simple. Update code locally; and when you're ready to deploy,

git push aws master  

Done!