Create Software Load-Balancer using nginx

nginx-logo

I wrote an article a few years ago with instructions on how to build a software load-balancer with nginx, haproxy and stunnel. Needless to say, it was a pain in the butt. Those technologies were not super mature and it took a lot of work to get things going. Luckily nginx is a lot further along than it was back in 2011 when I created that setup. Now everything that I needed back-then can be done with nginx!!

The important part to take into consideration is this configuration supports end-to-end SSL which combines SSL termination (front end) with SSL initiation (back end), virtual directory (uri) matching which points URI matches to specific serverfarms or pools and http to https redirection. That’s a mouthful, but to me that type of configuration is pretty standard these days.

These steps are based off a configuration that I did using a CentOS 6.6 image running in Azure cloud. The steps assume you have a similar RedHat based distribution with a static public IP (NAT’ed hopefully) and an internal IP that can communicate to the internal server network.

 Update the server

# yum update
# reboot

 Add the nginx Repository

# vi /etc/yum.repos.d/nginx.repo

Add the Following

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

 Validate Repository

The nginx repository will be listed.

# yum repolist

 Install nginx

# yum install nginx

Request Certificate

Enter the information as OpenSSL prompts you, then submit the csr to the CA that you prefer.

# openssl req -nodes -newkey rsa:2048 -keyout /etc/nginx/myserver.key -out /etc/nginx/server.csr

Once you get your cert bundle back from the CA, [lace the certificate file and intermediate certificate in the directory of your choosing (I’m using /etc/nginx to simplify things).

Merge the intermediate cert with your server cert.

# cat myserver.crt intermediate.crt >> mycertificate.crt

Modify your nginx.conf file

Here is where things are going to depend on your specific application. The example I will paste below will show a basic example that covers the functions above. Add or modify to suit your needs.

user nginx;

#This server has two processors
worker_processes 2;

#Assigning each worker to a processor
worker_cpu_affinity 01 10;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
 worker_connections 1024;
}

http {
 include /etc/nginx/mime.types;
 default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
 '$status $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on; 
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;

#Define default server pool
upstream DEFAULT {
 server 192.168.10.5:443;
}

#Define a secondary application server
upstream APP1 {
 server 192.168.10.6:443;
}

#Define a third application server
upstream APP2 {
 server 192.168.10.7:443;
}

#Create an http to https redirect
server {
 listen 192.168.10.10:80;
 return 301 https://$host$request_uri;
}

server {

 listen 192.168.10.10:443;
 server_name mydomain.com;

 ssl_certificate /etc/nginx/mycertificate.crt;
 ssl_certificate_key /etc/nginx/myserver.key;

 ssl on;

 ssl_session_cache builtin:1000 shared:SSL:10m;
 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
 ssl_prefer_server_ciphers on;

#Define application 1
 location ~* ^/application1 {
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_redirect off;
  proxy_max_temp_file_size 0;

 proxy_pass https://APP1;
 }

#Define application 2
 location ~* ^/application2 {
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_redirect off;
  proxy_max_temp_file_size 0;

 proxy_pass https://APP2;
 }

#Default catch-all
location ~* ^/(.*) {
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Host $http_host;
 proxy_redirect off;
 proxy_max_temp_file_size 0;

 proxy_pass https://DEFAULT;
 }
}
}

Restart Service & Test Application

If your server doesnt start properly, check log files in /var/log/nginx/error.log

# service nginx restart

If nginx starts properly, check your public DNS name via HTTP to validate the redirect works, then the virtual directories to make sure pattern-matching is working.

Create Software Load-Balancer using nginx