Professional Documents
Culture Documents
Nginx
Nginx
Stephen Corona
Boston
nginx
by Stephen Corona
Copyright 2016 Stephen Corona. All rights reserved.
Printed in the United States of America.
Published by OReilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
OReilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or [email protected].
May 2016:
First Edition
978-1-491-92477-8
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
1. Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Installing nginx
Installing from source
Modules in nginx
Installing from a package
11
12
14
16
2. Basic Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
The nginx.conf File
Configuring and running nginx
Filling in the blanks
Reloading and Stopping nginx
Serving Static Files
The Location Block
Basic Location Blocks
Regular Expression Location Blocks
Named Location Blocks
Location Block Inheritance
Virtualhosts
Default Server Block
Configuring SSL
Sharing a wildcard certificate
SNI and the future of SSL
19
20
21
27
29
30
31
33
36
37
37
39
40
41
42
43
46
iii
FastCGI Basics
FastCGI Basic Config
46
47
4. Reverse Proxy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Forward Proxy vs Reverse Proxy
Configuring a basic Rails Application
A more robust reverse proxy
Custom Error Pages
Adding headers to the upstream
Reverse Proxying Node.js & Websockets
Reverse Proxy with WebSockets
Future Sections in this Chapter
56
59
62
51
53
65
67
71
5. Load Balancing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Your first load balancer
Load Balancing vs Reverse Proxy?
Handling Failure
Configuring the Upstream Directive
Weighted Servers
Health Checks
Removing a server from the pool
Backup Servers
Slow Start
Load Balancing Methods
C10K with nginx
Scalable Load Balancer Configuration
Tuning Linux for a Network Heavy Load
nginx vs ELB vs HAProxy
HTTP and TCP Load Balancing
Future Sections
74
74
75
76
77
78
80
80
81
82
84
85
85
86
87
89
6. Content Caching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Thinking about caching
Reverse Proxy Caching
Building a CDN with nginx
Advanced Caching
Purging from the Cache
Varnish or nginx for Caching?
FastCGI Caching
91
92
95
99
99
99
100
iv
Table of Contents
101
Setting variables
Dynamically setting variables with map
If, and why it is EVIL!
103
103
105
Table of Contents
Preface
Used for program listings, as well as within paragraphs to refer to program ele
ments such as variable or function names, databases, data types, environment
variables, statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values deter
mined by context.
This element signifies a tip or suggestion.
vii
viii
Preface
McGraw-Hill, Jones & Bartlett, Course Technology, and hundreds more. For more
information about Safari Books Online, please visit us online.
How to Contact Us
Please address comments and questions concerning this book to the publisher:
OReilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at http://www.oreilly.com/catalog/<catalog
page>.
To comment or ask technical questions about this book, send email to bookques
[email protected].
For more information about our books, courses, conferences, and news, see our web
site at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia
Acknowledgments
Preface
ix
CHAPTER 1
Getting Started
nginx, pronounced engine x is a web server thats gained incredible popularity among
the most highly visited websites on the internet.
It isnt fair to think of nginx as just a web server. It can do so much more it can serve
HTTP and HTTPS traffic, proxy to Rails, PHP or Node applications, run as a software
load balancer, http cache, off-load ssl or even act as a SMTP, POP3, and IMAP mail server.
Most people think of nginx as a replacement to Apache. Thats true too, when looking at
what the biggest websites on the internet are using today. According to Netcraft, as of
March 2015, nginx is used by 21% of the top 1 million busiest websites on the internet.
As youll learn in this book, nginx is all of these things, but at the very core its an HTTP
router, a living, breathing part of the web stack that gives developers the flexibility to
build and deploy web apps easier than ever before.
Installing nginx
Lets get the basics out of the way and get started with installing nginx. While its pos
sible to run nginx on Windows, its not recommended, and the book operates under
the assumption that youre planning to install and run nginx on a common distribu
tion of Linux.
You have two installation options, directly from source or a pre-built package for
your operating system. While installing the package is much more convenient, its
typically an outdated version. Fortunately, nginx.org publishes their own set of up-todate packages for a variety of operating systems.
11
From here, building the source code is relatively simple if you have the basic build
tools already installed.
12
$ ./configure
$ make
$ sudo make install
By default, make install will install nginx and all related files into the folder /usr/
local/nginx/.
The executable binary for nginx will be located at /usr/local/nginx/sbin/nginx.
You can test that everything went correctly and nginx works by running the com
mand nginx -V. The -V flag will make nginx print the version number and compiletime build options. The output should look similar to this:
$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.9.9
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
configure arguments:
13
For instance, the most common nginx installation would have the nginx binary loca
ted in /usr/sbin and the configuration files in /etc/nginx. In order to make this
happen, wed run the configure script with the following flags:
$ ./configure --prefix=/usr conf-path=/etc/nginx
Modules in nginx
nginx is an extremely modular piece of software. Even some of seemingly built-in pieces
of the software, such as gzip or SSL, are actually built as modules that can be enabled and
disabled during build time.
One of the benefits of building nginx from source is that youre able to specify exactly the
modules you want, and which ones you dont. The downside is that if you dont include
something that you later need, youll have to rebuild nginx from source.
By default, nginx comes out of the box with the following modules some enabled by
default and some disabled by default.
Module Name
Flag
Description
Installed by
Default?
Charset
http_charset
Yes
Gzip
http_gzip
Yes
SSI
http_ssi
Yes
Userid
http_userid
Yes
Access
http_access
Yes
Basic Auth
http_auth_basic
Yes
Auto Index
http_autoindex
Yes
Geo
http_geo
Yes
Map
http_map
Yes
Split Clients
http_split_clients
Yes
Referer
http_referer
Yes
Rewrite
http_rewrite
Yes
Proxy
http_proxy
Yes
Fastcgi
http_fastcgi
Yes
14
uWSGI
http_uwsgi
Yes
SCGI
http_scgi
Yes
Memcached
http_memcached
Yes
Limit Conn
http_limit_conn
Yes
Limit Req
http_limit_req
Yes
Empty GIF
http_empty_gif
Yes
Browser
http_browser
Yes
Perl
http_perl
No
SSL
http_ssl
No
SPDY
http_spdy
No
RealIP
http_realip
No
Addition
http_addition
No
XSLT
http_xslt
No
Image Filter
http_image_filter
No
GeoIP
http_geoip
No
Substitution
http_sub
No
WebDAV
http_dav
No
FLV
http_flv
No
MP4
http_mp4
No
Gunzip
http_gunzip
No
No
Auth Request
http_auth_request
No
Random Index
No
Secure Link
http_secure_link
No
Degradation
http_degradation
No
Stub Status
http_stub_status
No
Enabling modules in nginx is as simple as passing an extra flag to the configuration script.
For instance, if you wanted to enable the SSL module, youd add the
--with-http_ssl_module flag to enable it.
15
While the barebones build of nginx will get you through most of this book, I recommend
that you re-build nginx with a few extra modules, specifically SSL, SPDY, RealIP, and Stub
Status.
To accomplish this, re-run the configuration script with the extra flags listed below, as
well as make and make install.
Remember, in order to build the SSL module, youll need to have OpenSSL installed on
your machine.
$ ./configure --with-http_ssl_module \
--with-http_spdy_module \
--with-http_realip_module \
--with-http_stub_status_modul
$ make
$ sudo make install
After re-building nginx with the new modules, you may re-run nginx -V to verify that
the new version of nginx, with the modules, have been properly installed.
You will notice in the output that TLS SNI has been enabled, as well as the configuration
arguments we used above.
$ /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.9.9
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
TLS SNI support enabled
configure arguments: --with-http_ssl_module --with-http_spdy_module
--with-http_realip_module --with-http_stub_status_module
That being said, nginx provides its own up-to-date pre-built packages for many of the
most popular Linux distributions, including RedHat, CentOS, Debian, Ubuntu, and SLES.
Another advantage of installing nginx from your package manager is that the packages
come with the necessary init or upstart scripts to easily run nginx as a daemon or auto
matically start after a reboot.
If youre just getting started, I highly recommend you take this path the packages are
built with sane default modules and generally involve less fuss.
Installing the RedHat/CentOS Package
$ sudo yum install nginx
Once thats done, the next step is to add the key you just downloaded from nginx.org to
apt, using the apt-key command.
$ sudo apt-key add nginx_signing.key
Now, we need to find the codename of the Ubuntu/Debian release youre running. The
easiest way to do this is to run lsb_release -a and look for the row labeled Codename.
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty
In this case, the codename the Ubuntu release Im using is trusty. Well use this informa
tion for the next step.
17
For Debian, open the file /etc/apt/sources.list and add the entries below to the bot
tom of the file, replacing codename, with the one from running the lsb_release com
mand.
deb http://nginx.org/packages/debian/ codename nginx
deb-src http://nginx.org/packages/debian/ codename nginx
For Ubuntu, open the file /etc/apt/sources.list and add the entries below to the bot
tom of the file, replacing codename, with the one from running the lsb_release com
mand.
deb http://nginx.org/packages/ubuntu/ codename nginx
deb-src http://nginx.org/packages/ubuntu/ codename nginx
Great, almost there. The last step is to update apt and install the latest version of nginx.
$ apt-get update
$ apt-get install nginx
In all cases, no matter which route you chose, you can test the success by attempting to
running nginx -V. You should see output similar to the output below:
$ nginx -V
nginx version: nginx/1.9.9 (Ubuntu)
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
TLS SNI support
enabled configure arguments:
18
CHAPTER 2
Basic Configuration
In this chapter, well cover the basics of configuring nginx from getting it up and
running for the first time, to configuring and running it in production.
local/nginx/conf.
19
Here is a table for reference (this can be accessed by running nginx -h)
Option
Description
-?, -h
-v
-V
-t
-q
-s signal
-p prefix
-c filename
-g directives
-T
If you dont want to use the default path of the nginx configuration file, you can pass
in your own location with nginx -c filename.
For instance, if you decided you wanted to use a nginx.conf file that was located in
your home directory, the command would look something like this:
$ nginx -c /home/user/nginx.conf
You can paste this configuration into your nginx.conf file, but before anything hap
pens, you need to run the nginx server. There are a few different ways to do that,
depending on if you built nginx from source or installed it from a package.
20
If you used a package manager to install nginx, it was probably packaged with an init
script that can be used service nginx start will start the server.
If you installed nginx from source, you can simply run nginx to start the server.
You may not see any output when you start nginx, thats completely normal. To verify
that nginx is running, you can use ps to check if the process is running or try to
access the web server with curl.
Using ps, you grep the output for nginx, and should see a master and worker process
running.
$ ps aux | grep nginx
root 24323 0.0 0.0 85772 1213 ? Ss 01:24 0:00
nginx: master process
nobody 24324 0.0 0.0 85992 2004 ? S 01:24 0:00 nginx: worker process
Using curl, you should see some HTML output if you try to access port 80 on local
host.
$ curl 127.0.0.1
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
Wait, thats right with the simple 7-line configuration file nginx is actually serving
up static files to the internet. By default (with this empty configuration), nginx will
share the contents of the /usr/share/nginx/html directory on port 80.
If you add a file to this directory and try to access it from curl, the contents would be
returned to you.
21
http {
server {
listen *:80;
server_name "";
root /usr/share/nginx/html;
}
}
With this filled out configuration file, you can start to get a sense for what the nginx
configuration syntax looks like and how it works.
The configuration file is made up of directives. There are two types of directives that
youll encounter.
Simple Directives
The most basic is called a simple directive. A simple directive is just a simple state
ment, like listen *:80;.
The simple directive is made up of a name (listen), the parameters (*:80), and a clos
ing semicolon (;).
Simple directives often can take multiple parameters, some of which can be optional.
When a directive has optional parameters, they are typically named parameters and
passed into the directive as a key-value pair OR just the key name for true values.
For instance, the listen directive has 14 optional parameters, which can be specified
in any order. Two of the optional parameters for the listen directive are ssl and
backlog, which are turned on the example below.
listen *:80 ssl backlog=511;
Note that for ssl, the parameter was passed without a corresponding value. This indi
cates a true value and turns SSL on.
Context Directives
The context directive has a similar syntax as the simple directive except instead of
ending in a semicolon, it wraps a group of directives inside of curly braces.
For example, the server block wraps the listen, server_name, and root directives.
Context directives can be nested, and provide somewhat of a configuration inheri
tance model within the configuration file.
Examples of context directives are events, http, and server. There are a handful of
other contexts that youll see presented in the next few chapters.
22
Typically, simple directives can only be included in specific context directives. For
example, the listen directive can only be present in the server context. The list of con
texts supported is shown in the documentation for each directive.
The last thing to note there is an implied main context wrapping the configuration
file. Putting something in the main context simply means that its at the top-level of
the configuration file and isnt included under any other context.
Directive Inheritance
nginx has a very lightweight inheritance model. Directives are always inherited down
wards.
For example, you may overwrite previously defined directives in nested contexts.
server {
root /usr/share/nginx/html;
location /foo {
root /usr/share;
}
}
While we havent covered the location context yet, you can see in the example that
were overwriting the root directive.
If the root directive was not explicitly specified inside of the location block, the value
would be inherited from the server context.
The more complex inheritance rules will be covered in the Advanced Configuration
chapter.
So lets dig into the configuration file and talk about whats going on with each line,
starting with the first two.
user nobody nogroup;
worker_processes 1;
Lines 1 and 2 configure the user directive and the worker_processes directive. Since
they arent wrapped in a context directive, they are implicitly part of the main con
text.
The user directive sets the unix user and group that the nginx worker processes will
run as. By default, the user is nobody and the group is nogroup.
23
Remember, in the unix world, every process needs to be run as an explicit user. Gen
erally, with network facing services, you want the daemon to run with the least
amount of privileges are possible.
This user will potentially need to have permission to read and write to the log files,
store temporary data, and read data from static files.
You may notice that if you run ps, there are two nginx processes, a master running as
root and a worker running as nobody.
$ ps aux | grep nginx
root
24484 0.0 0.0 85772 1312 ? Ss 1:38 0:00 nginx: master process
nobody 24485 0.0 0.0 85992 2004 ? S 1:38 0:00 nginx: worker process
On a Linux, only the root user is able to bind to port numbers below port 1024.
Because of this, nginx is typically started as root to allow it to bind to port 80 and port
443.
The master process reads and executes the nginx configuration, binds the necessary
ports, and runs the worker processes.
The worker process, running as the user specified in the configuration file, is where
incoming HTTP requests are served from.
The number of worker processes is configured with the worker_processes directive. By
default the value is 1, meaning that nginx will run 1 worker process.
Its common practice to set the number of worker processes to the same number of
CPU cores that you have on your machine. The value of auto can be set on
worker_processes to have nginx auto-detect the number of logical CPU cores. For
example, a single Intel CPU core with Hyperthreading enabled will appear as 2 logical
CPU cores.
Events Context
The events context is the next section of the configuration file. This context is used to
configure parts of the connection processing part of nginx.
There can only ever be one events context and it must be in the main context.
events {
worker_connections 512;
}
24
nginx is an event driven server. Each worker process is single-threaded and runs a
non-blocking event loop to process requests very quickly.
The events context allows you to tweak and tune some of the settings that drive event
loop.
The worker_connections directive sets the maximum number of simultaneous con
nections that can be opened by each worker process. The default is 512 connections
per worker process.
This limit doesnt just include connections from clients (e.g, browsers making
requests to nginx), but also any subsequent connections that nginx has to make to
talk to your PHP or Rails application.
On Linux, each connection is counts as an open file, and there is an operating system
limit on the maximum number of open files allowed per process. If you decide
increase this value, youll need to also increase the maximum number of open files
allowed. More on tuning this in the future chapters.
Lets look at whats going on here very simply, were creating an HTTP server to lis
ten on port 80 of every network interface, with no HTTP Host specified, and a root
path of /usr/share/nginx/html.
Listen Directive.
The listen directive tells the server which network interface and port to listen on, and
defaults to *:80.
Configuring and running nginx
25
A listen directive may be specified multiple times in a single server block. For exam
ple, if you wanted a server to listen on two different ports
server {
listen *:80;
listen *:81;
}
Typically this is used when you turn on SSL on, for example, to listen for non-SSL
traffic on port 80 and SSL traffic on port 443.
server {
listen *:80;
listen *:443 ssl;
}
The Host header (example.org) is used to match against the value in the server_name
directive.
The server_name can be an exact domain name (including subdomains), a wildcard,
or regular expression.
A value of means that no Host header is necessary.
Root Directive.
The root directive specifies the path of the directory to serve static files out of. Any
and all files in the root directory will become publicly available over HTTP.
server {
listen *:80;
server_name "";
root /usr/share/nginx/html;
}
If you were to add the file test.html to this directory, it could be accessed over
HTTP by hitting http://127.0.0.1/test.html.
Likewise, any directory in the URI is just appended to the root path, so a request to
the URL http://127.0.0.1/foo/test.html would resolve to the filesystem
path /usr/share/nginx/html/foo/test.html.
When you reload nginx, it will actually verify the syntax of your configuration file
first, so you dont have to worry about accidentally killing nginx if you try to reload it
with an invalid configuration file.
Reloading nginx actually starts new worker processes and kills off the old ones. Its
able to do this completely gracefully, without dropping or killing any web requests. It
does this by starting the new workers, discontinuing to send traffic to the old work
ers, and waiting for the old workers to complete all in-flight requests before killing
them.
Likewise, you can also shutdown nginx gracefully without dropping any in-flight or
processing requests by sending it the QUIT signal:
kill -QUIT `cat /var/run/nginx.pid`
The list of unix signals that nginx responds to (and the behavior) is listed below:
Signal
Description
TERM, INT
Quick shutdown
QUIT
Graceful shutdown
27
Signal
Description
KILL
HUP
Configuration reload
USR1
USR2
WINCH
In this example, the master PID is 24484. We need to send it the USR2 signal, which
starts the new master process and new workers.
$ kill -USR2 24484
After sending USR2 to the old master process, nginx will start up a new master pro
cess and workers. These two processes will serve incoming requests together.
Next, we need to kill the children of the old nginx master. Since we dont want to drop
any in-flight requests, we use the WINCH signal to tell the old nginx master to stop
taking new requests and shutdown down the children once all in-flight requests are
finished.
$ kill -WINCH 24484
28
After a short time, when all in-flight requests are complete, the old nginx children will
shutdown and the upgraded version of nginx will be running and serving request.
Theres still one last thing to do. The old nginx master is still running and hanging
around.
If you need to bail out of the upgrade, you can actually ask the old master to spin up
children and serve requests again by sending it the HUP signal.
Otherwise, you can go ahead and kill the old master by sending it the QUIT signal:
$ kill -QUIT 24484
What happens when you access a URL without specifying a file, for example, http://
example.org/? By default, nginx blesses a special file named index.html to be shown
when a filename isnt specified in the URL.
Its possible to override this behavior and specify your own default index with
the index directive.
29
You may also pass multiple index files to the index directive. When you do this, nginx
will check for the existence of each file and serve the first one that exists. The index
file name can include nginx variables. The last file name in the list may also be an
absolute path.
Example 2-4. Multiple index files
server {
listen *:80;
root /usr/share/nginx/html;
index maintenance.html index.html;
}
In the example above, were implicitly checking for the existence of the index file
maintenance.html. If it exists, that file is served as the index. If it doesnt, we proceed
as normal and serve the regular index.html page.
This mechanism is a common way to handle deployments or other scheduled down
time before deploying or making changes, you create a maintenance.html page and
have nginx serve that. When youre done, just delete the file and itll go back to serv
ing index.html.
30
The example above defines two prefix location blocks, one for the path / and the
other for the path /foobar/.
The first location block simply serves request URIs matching the prefix / out of the
directory /var/www/html. For example, http://example.org/file.html will be
resolved to /var/www/html/file.html. This is the exact same behavior as completely
skipping the location block and specifying the root directive within the server block.
The second location block does something different. It matches against the prefix /
foobar/ and serves it out of a different directory entirely. When a request URI begins
with the prefix /foobar/, it will use the root directory /data.
So, for example, a request to http://example.org/foobar/test.html will resolve to
the filesystem path /data/foobar/test.html.
31
To accomplish this, we can use the alias directive instead. Its similar to root, except it
replaces the specified location.
images/gif/business_cat.gif instead.
The trailing slash is important with alias, as without it, the path would end up being /
data/imagesbusiness_cat.gif
When there are overlapping prefix location blocks, the most specific match is used.
Lets look at the following example:
Example 2-8. Overlapping prefix location blocks
location /foobar/images {
...
}
location /foobar/ {
...
}
32
In this example, the location block will only match the exact request URI for the
path /foobar/images/business_cat.gif. The fact that it matches a filename is not
important, it can match any arbitrary request URI.
Example 2-10. Matching an exact request URI without a filename
location = /foobar/ {
...
}
This location block will only match an exact request URI to /foobar/. It wont
match /foobar/index.html or anything besides an exact match.
Location blocks with an exact match modifier are the first blocks to be checked for a
match and will terminate the search and immediately be selected for use.
Example 2-11. Example of = block being selected
location = /foobar/ {
...
}
location /foobar/ {
...
}
In the example above, a request URI for /foobar/ will always use the first location
block, even though the prefix block also matches the same request URI.
Because the exact match modifier will immediately terminate the location block
search, you can improve performance by creating exact match blocks for highly
accessed URIs when you have a configuration with many location blocks (this
becomes even more useful with regular expression blocks, discussed in the next sec
tion, as they are more intensive to process).
33
This location block uses the regular expression \.(gif|jpg)$ to match any URIs
ending in the extension .gif or .jpg.
Notice how the location block also uses a modifier, ~, before the regular expression.
The ~ is the modifier to create a case-sensitive regular expression location block.
Because this example uses a case-sensitive matcher, it will match the URI images/
picture.gif but not images/picture.GIF.
Creating a case-insensitive regular expression location block is almost the same, but
uses the ~* modifier instead.
Example 2-13. Example of a case-insensitive regular expression
location ~* \.(gif|jpg)$ {
...
}
Because regular expression are more processor intensive to process and match
against, use them only when necessary! It will always be faster to process a location
block that uses prefixes than one that uses regular expressions.
Because regular expressions are matched after the prefix locations, if there is a
request URI for /foobar/images/dog.gif, the regular expression block will always
be used. The two blocks are competing for the same paths.
34
One way to solve this would be to rewrite the regular expression to exclude the /
foobar/images path, but this would make it more complicated and confusing.
Instead, we can change the /foobar/images location block to use the carat-tilde
modifier.
Example 2-15. Example using the carat-tilde modifier
location ^~ /foobar/images {
root /var/www/foobar;
}
location ~* \.(gif|jpg)$ {
root /var/www/images;
}
When the carat-tilde modifier is chosen as the best prefix for a URI, it is immediately
selected and skips matching against the regular expression blocks.
Therefore, a URI for /foobar/images/dog.gif will use the first location block
while /images/cat.gif will match the regular expression and use the second location
block.
Description
(none)
Prefix
Exact Match
Case-Sensitive Regular
Exression
~*
Case-Insensitive Regular
Expression
^~
Non-Regular Expression
Prefix
35
The exact search algorithm that nginx uses to select the location block is a described
as follows:
1. The exact match location blocks are checked. If an exact match is found, the
search is terminated and the location block is used.
2. All of the prefix location blocks are checked for the most specific (longest)
matching prefix.
a. If the best match has the ^~ modifier, the search is terminated and the block is
used.
3. Each regular expression block is checked in sequential order. If a regular expres
sion match occurs, the search is terminated and block is used.
4. If no regular expression block is matched, the best prefix location block deter
mined in step #2 is used.
In the above example, we declare a named location block. In its current form it does
nothing! It will never be used unless its called explicitly within the nginx configura
tion file. So how do we call it?
Example 2-17. Named locations for try_files
location / {
try_files maintenance.html index.html @foobar;
}
location @foobar {
...
}
The example above introduces a new directive, try_files, which checks for the exis
tence of the file names provided, in order from left to right, and uses the first one
found to service the request.
36
The last parameter to try_files may be either an internal redirect using a named
location block (in this case, @foobar) or an explicit error code (shown in the next
example).
When try_files cant find any of the files in its parameter list, it will redirect to the
@foobar location block and allow the block the handle the request.
Well dig into try_files more in Chapter 3 (FastCGI) and Chapter 4 (Reverse
Proxy), as its commonly used when talking to dynamic backends, such as PHP and
Ruby applications.
Example 2-18. try_files with an error code
location / {
try_files maintenance.html index.html =404
}
If neither maintenance.html nor index.html are found, try_files will return the HTTP
error code 404.
Virtualhosts
So far, weve only talked about configuring a single server block, which limits us to
running a single website within nginx.
Fortunately, nginx allows us to define as many server blocks to host as many websites
as we want! This is called virtual hosting.
Example 2-19. Basic virtual hosts example
server {
listen 80;
server_name example.com;
}
server {
listen 80;
server_name foobar.com;
}
Virtualhosts
37
In the above example, we define two server blocks, both listening on port 80, for the
domain names example.com and foobar.com. These two blocks are completely inde
pendent and can define their own location directives.
Remember that server_name can contain subdomains, wildcards, and regular expres
sions. The first domain parameter to server_name is considered the primary server
name.
All of these are valid server_name directives:
server_name example.com www.example.com;
Using wildcards:
server_name example.com *.example.com www.example.*;
Regular expressions may also contain captures, allowing you to set dynamic variables
based on the match.
Example 2-20. Example using capture group
server {
listen 80;
server_name ~^(www\.)?(?<domain>.+)$;
location / {
root /sites/$domain;
}
}
This example sets the root path dynamically based on the name of the domain.
Notice how the server_name regex contains the syntax (?<domain>.+)? This is the
syntax for a named capturing group, the name being everything between the brackets.
When you use a named capturing group, youre basically saying remember whatever
data matches the regular expression between these two parenthesis. In this case, that
regular expression is matching everything after the www. in the domain name.
Because youve asked nginx to remember the data matched in the regular expres
sion and you used a named capture group to name it domain, the variable $domain
is set with that data and subsequently used a few lines lower to dynamically set the
root path.
38
Well dig into variables later in the chapter, but for now all you need to know is that
they allow you pass dynamic data into various directives.
If there no server_name directives that match the Host header (or the Host header is
not present), nginx chooses the default server block. Out of the box, the default server
block is the first one defined in the configuration file.
You may explicitly choose the default
the default_server parameter on the listen directive.
server
block
by
setting
In the above example, the second server block will match requests to foo
bar.com and any requests on port 80 that arent to the domain example.com.
39
The way you do this is by defining a server block with an empty server_name that
returns a special nginx HTTP error code.
The return directive stops processing the request and returns the specified HTTP
error code. In this case, we return a non-standard HTTP code, 444, which is special
code that tells nginx to immediately close the connection.
Additionally, if youd like to drop requests made to your server without an explicitly
defined domain name, you can mark this server block as the default_server.
Configuring SSL
SSL (Secure Sockets Layer) is a way to create a secure connection between an HTTP
Client and an HTTP(S) Server. nginx offers a very robust and high performance SSL
implementation.
To get started with setting up SSL, youll need to have an SSL certificate and key.
Example 2-23. Most basic SSL Configuration
server {
listen 80;
listen 443 ssl;
server_name www.foobar.com;
ssl_certificate www.foobar.com.crt;
ssl_certificate_key www.foobar.com.key;
}
Youll notice a few new things here first, there are two listen directives specified,
one listening on port 80 and other on 443. This setup allows us to create a compact
server block that server both non-SSL and SSL requests. We use the ssl parameter to
tell nginx that we expect SSL traffic on this port.
The other two new directives are ssl_certificate and ssl_certificate_key. ssl_certifi
cate tells nginx the path to find the signed certificate (in PEM format) for this
domain. Likewise, ssl_certificate_key specifies the path to the key for the certifi
cate
40
If you specify just a filename or relative path (i.e, ssl/certificate.crt), nginx will
prefix the path with the configuration directory, /etc/nginx.
You may also specify an absolute path, i.e /usr/share/ssl/foobar.com.crt.
This also has the benefit of having a lighter memory footprint as the certificate data
will only be loaded into nginx one time.
Configuring SSL
41
42
CHAPTER 3
Until the mid-1990s, there wasnt an easy way to create dynamic pages for the web. If
you wanted to do more than serve static HTML files or images, youd have to write
your own web server.
Writing your own web server for every application is clearly a case of reinventing the
wheel, so it was obvious that something better was needed a way to extend the web
server.
To serve that need, CGI (Common Gateway Interface) was officially born in 1997.
CGI is an interface for web servers to communicate with programs (or scripts) on the
web server. CGI gave programmers the ability to write their own dynamic web appli
cation by extending the web server software.
Because its a Common Gateway Interface, CGI is a programming language independ
ent.
Although plain CGI isnt used much in recent years (its not even supported out-ofthe box by nginx), its worth understanding because the more modern replacements
that well cover later in the chapter are built on it.
43
For example, for this hypothetical example, lets pretend we have the following PHP
script that we want to run as a CGI script. The script will be saved to the file system
as script.php.
#!/usr/bin/php
<?php
echo "Content-Type: text/plain\n";
echo "PHP-CGI Script! Woohoo<br>";
echo time();
If you wanted to run this script from the command line on the server yourself, youd
just type the name of the script, press enter, and it would execute.
$ ./script.php
Content-Type: text/plain
PHP-CGI Script! Woohoo<br>
1429629271
Well, what does that have to do with CGI? Remember, CGI is just a fancy way of say
ing the web server executes the script directly. When you access script.php from the
web server, its just running ./script.php and returning the output to the browser.
You might be thinking, that seems insecure. What if you tried to run dangerous com
mands like rm to delete files? If the web server is just running the commands,
couldnt you run anything? And youd be right.
Thats where the cgi-bin directory comes in to play. Typically, with CGI, theres a spe
cial directory thats used to designate which programs are safe to run. This directory
can be named anything, but its commonly named and referred to as the cgi-bin
directory.
The other missing piece of CGI is the context, or environment, of the HTTP request.
The CGI script needs a way to retrieve things like the Request URI, Client IP Address,
or Server Host from the HTTP Request.
With CGI, these are all passed into the script as environment variables. For example,
if we wanted our CGI script to output the Client IP Address, we might write the fol
lowing:
#!/usr/bin/php
<?php
echo "Content-Type: text/plain\n";
echo "PHP-CGI Script! Woohoo<br>";
echo $_SERVER["REMOTE_IP"];
44
If we were executing this script from the command line, wed have to pass in the
REMOTE_IP environment variable, shown below.
$ REMOTE_IP=127.0.0.1 ./script.php
Content-Type: text/plain
PHP-CGI Script! Woohoo<br>
127.0.0.1
When accessed via CGI through the web server, the same thing is happening. There
are a standard group of environment variables that are passed from the web server to
the CGI script to provide context about the web request. These environment variables
are listed in the table below.
Variable
Definition
Example
GATEWAY_INTERFACE
The version of the CGI protocol being used by the web server.
CGI/1.1
SERVER_SOFTWARE
The name of the web server software making the CGI call.
nginx
QUERY_STRING
foo=bar&foobar=1
REQUEST_METHOD
POST
CONTENT_TYPE
text/plain
CONTENT_LENGTH
100
SCRIPT_FILENAME
/var/www/cgi-bin/test.php
SCRIPT_NAME
/cgi-bin/test.php
REQUEST_URI
/cgi-bin/test.php?foo=bar
DOCUMENT_URI
/cgi-bin/test.php
DOCUMENT_ROOT
/var/www/html
SERVER_PROTOCOL
HTTP/1.0
REMOTE_ADDR
108.22.12.23
REMOTE_PORT
9583
SERVER_ADDR
192.168.10.1
SERVER_PORT
80
SERVER_NAME
app01
While serving up static html is easy, theres only so much that can be done with html
and javascript. Its impossible to create rich applications without a backend or serverside component.
Theres generally two different ways to use nginx with a backend web application
FastCGI is one of the ways. The other method, using a reverse proxy, will be covered
in the next chapter.
45
Typically, FastCGI is only relevant if youre deploying PHP applications. Most other
languages (Ruby, Node, Go, etc) have moved past CGI and its best practice to use a
Reverse Proxy Server.
What is FastCGI?
nginx does not support regular, old-school CGI. In the traditional CGI model, when
the script is accessed, the web server actually has to fork and execute the script.
This has all of the disadvantages that you might imagine. The performance is rela
tively poor, since forking is a slow and an expensive system call to make.
The security model is also risky. If an attacker finds a way to upload a script to that
blessed cgi-bin directory, they can run any script that they desire. You dont want the
webserver to be able to execute any arbitrary script on your server.
However, CGI has one advantage: its extremely simple. Plop a perl or php script into
a special directory and youre good to go. No special servers, no deployment.
FastCGI Basics
FastCGI is a direct replacement for CGI. It solves the performance and security con
cerns but adds a little bit of complexity to the way that applications are run.
With CGI, the web server is executing the scripts directly. There is no layer between
the two- they talk directly.
46
Figure 3-2. Web Server executing CGI program through FastCGI Server
When a request comes in, the webserver makes a TCP or Unix Socket request to the
FastCGI server, which runs the script and returns the result to the web server.
The FastCGI server is typically long running, so it doesnt need to fork itself for each
request, and can typically handle many connections.
This all seems really complex. Its not. In almost all cases, when youre writing or con
figuring php apps to work with nginx, youll almost never have to deal with FastCGI
directly, because PHP has PHP-FPM (PHP FastCGI Process Manager) built in. It does
almost all of the work for you.
FastCGI Basics
47
If youre on Ubuntu, you can install PHP-FPM with the command apt-get install
php5-fpm.
To configure php-fpm, you need to edit the configuration file. On Ubuntu, this file is
located in /etc/php5/fpm/pool.d/www.conf
To start php-fpm on Ubuntu, you run the command service php5-fpm start.
location ~ script.php {
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php
include fastcgi_params;
}
This location block matches a single PHP script, script.php. When a request comes in
that matches this block (http://example.org/script.php), it will pass, or proxy, the
request to the upstream FastCGI server.
The FastCGI server will use the SCRIPT_FILENAME FastCGI Parameter to deter
mine which PHP script to execute.
By default, the SCRIPT_FILENAME parameter is set to the value below
$document_root$fastcgi_script_name
This value just concatenates the $document_root, which is the document root of the
current request (specified by the root directive) and another variable called
$fastcgi_script_name.
$fastcgi_script_name is a variable thats set automatically by nginx based on the
path of the request URI.
48
The first is when you have many individual PHP files in a single directory and want
them to be all individually executable.
FastCGI Basics
49
CHAPTER 4
Reverse Proxy
One of the most popular reasons for running nginx is to serve a dynamic web appli
cation written in Ruby, PHP, Node.Js, Python or one of the many web languages
available today.
Unlike Apache, nginx does not have the ability to embed the programming language
interpreter into the webserver, like Apache does with mod_php.
Instead, nginx takes a much lighter weight approach. Its just a webserver and gener
ally tries to keep its footprint very small, running a web application is delegated to a
separate server and proxies upstream.
51
52
The only difference in the case of load balancing is that typically a load balancer is
communicating upstream to another nginx or web server, located on a different phys
ical machine, while the reverse proxy is often talking to a service on the same
machine.
The terms are different but the underlying concepts are nearly identical. In fact,
because reverse proxy just proxies HTTP, and most dynamic web applications speak
HTTP, you can think of it as just a local load balancer.
53
54
This configuration has nginx receiving every incoming request on port 80 and for
wards it to our WEBrick server running on port 3000.
As simple as it is, you already get massive benefits from putting nginx in front of
WEBrick.
First, its much more scalable WEBrick is single-threaded, so it can only process a
single request at a time. With nginx in front, it can multiplex the incoming connec
tions and accept them while its waiting for earlier WEBrick to processes to finish.
Second, the error handling is much better. If the upstream server times out or goes
down, nginx will gracefully return 503 Service Unavailable and 502 Bad Gateway
respectively. While throwing a 500 error isnt exactly ideal, without the reverse proxy
in place, there would be no response at all.
Our configuration still has a problem, however. Notice, that because of the single
catch-all location block, were still proxying all of the requests through to WEBrick,
What wed actually prefer to serve all of the static assets (images, javascript, css)
through nginx, since its much better at doing that, and can reduce the load put on
WEBrick.
To do that, all we need to do is add a new location block to the configuration for the /
assets directory (which is where all assets are stored in a Rails application).
server {
listen *:80;
root /path/to/application/public;
location / {
proxy_pass http://rails_app;
}
55
location /assets {
}
}
Weve added a new location block to catch anything starting with the prefix /assets.
Because theres no proxy_pass directive specified in the location block, the default
behavior is to fallback serve files statically from the root path, which is inherited from
the server context, /path/to/application/public/assets.
One last tip its often common to set long lived HTTP cache headers for static
assets. You can accomplish this by adding the expires and add_header directives.
location /assets {
expires max;
add_header Cache-Control public;
}
The expires directive sets the Expires and Cache-Control HTTP headers. The value
can be a specified date, relative time (e.g, 42d), or the constant value max.
When set to max, nginx sets the Expires header to "Thu, 31 Dec 2037 23:55:55
GMT" and Cache-Control header to 10 years.
Even though expires sets a Cache-Control header for us, we still need to tweak it a
bit. By default, the Cache-Control header marks the content as private, which means
that intermediate HTTP caches will not cache the content. Since assets are typically
public and shared across all users, we need to explicitly mark this header as public so
that they have maximum cache-ability.
If you make a web request to a file in the /assets directory, you will see HTTP headers
that look something like this.
$ curl -I 127.0.0.1/assets/test.jpg
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Cache-Control: public
56
This is a convenient way to gracefully take your application down for maintenance.
For example, if you wanted to deploy a new version of your code but had to do a
length database migrate, you might want to display a temporary come back soon
page.
Example 4-2. Serving a dynamic application using try_files
server {
listen *:80;
root /path/to/application/public;
location / {
try_files maintenance.html @rails;
}
location @rails {
proxy_pass http://rails_app;
}
}
With this configuration, weve moved the proxy_pass directive to its own named
location block and check for the existence of maintenance.html, which will shortcircuit forwarding the request to the upstream if it exists.
A simplistic catch-all proxy_pass location block, like I demonstrated in the original
example, also has the disadvantage of forwarding everything onward to the upstream
application unless explicitly defined, like we did for the /assets folder.
What if there was a videos directory inside of the nginx root path that we wanted to
have nginx serve directly? One way to would be to add yet-another location block to
configuration, but thats kind of a pain. We can use try_files to solve this for us.
Example 4-3. Dynamically checking for file existence with try_files
server {
listen *:80;
root /path/to/application/public;
location / {
try_files $uri $uri/index.html @rails;
}
location @rails {
proxy_pass http://rails_app;
}
}
57
Notice that we use try_files, but this time instead of using a static filename, were
using the nginx variable $uri, which contains the normalized (decoded and sani
tized) URI of the web request.
This will allow any files and folders in the root path to be served up statically by nginx
and not be forwarded to the upstream backend.
If youre coming from the Apache world, this is similar to checking for the existence
of a file or directory with RewriteCond before forwarding to the upstream.
An equivalent Apache configuration would look something like this:
Example 4-4. Equivalent Apache configuration
<VirtualHost *:80>
DocumentRoot /path/to/application/public
RewriteEngine On
# This is roughly the same as the behavior our try_files
# configuration above
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://rails [P,QSA,L]
# This is equivalent to upstream in nginx
<Proxy balancer://rails>
BalancerMember http://127.0.0.1:3000
</Proxy>
</VirtualHost>
While using try_files to serve static files before falling back on the backend is best
practice, its important to notice that everything in the root path will be publicly acces
sible so you need to make sure that you dont put anything sensitive or private in
there.
Its for this reason that we always make sure the nginx root is our public directory. If
we were set the root one level lower, for example, to /path/to/application, anyone
would be able to read source code files directly from nginx by accessing the paths
directly, by going to http://example.org/app/config/secrets.yml.
58
In the case of a Reverse Proxy configuration where your upstream server is on the
same machine, you can get better performance by skipping the TCP/Network stack
and using Unix Domain Sockets.
A Unix Domain Socket is communication mechanism to do inter-process messaging
within the operating system kernel, using the file system as the name space.
To enable Unix Domain Sockets, you need to first tell your upstream server to listen
on a domain socket instead of TCP.
Unfortunately, WEBrick doesnt support Unix Domain Sockets, but most other
servers (for Rails and otherwise), do. For this example, well have to use thin.
First, install thin with Rubygems by running the command gem install thin.
Next, instead of running rails s to start WEBrick, run thin start --socket /tmp/
rails.sock. The path that we pass to the --socket flag is the location of the unix
socket.
Once you do that, its just a matter of changing your nginx upstream server to talk
over the domain socket instead of TCP.
upstream rails_app {
server unix:/tmp/rails.sock;
}
59
Likewise, you will see a 500 Internal Server Error if your code doesnt compile or
hits some fatal error usually this is returned by your upstream server (thin, uni
corn, etc) as a last resort.
Typically, if you see a 502 Bad Gateway , its actually coming from nginx. When nginx
returns this status code, it means that it cant reach the server inside of the upstream
block.
Instead of having to use the (very unattractive) default error pages that come out of
the box with nginx, we can define our own HTML error pages to use with the
error_page directive.
60
The error_page directive takes a list of one or more error codes and a file to return as
the body of the error. This is how Twitter made the fail whale page.
Example 4-5. Overriding default nginx error pages
server {
listen *:80;
root /path/to/application/public;
location / {
error_page 404
error_page 500 502 503 504
/404.html;
/50x.html;
You probably shouldnt do this unless recovering with a static file is appropriate
behavior many applications and API clients depend on the status code to determine
if there was an error.
The example below will change a 404 Not Found into a 200 and return the static file,
index.html.
error_page 500 =200 /index.html;
It can redirect to a completely different URL, sending back a 302 Found with a redi
rect location that can be on any server.
The example below will cause any 500 error pages to actually return a 302 Found with
a redirect to http://www.google.com.
error_page 500 http://www.google.com;
It can internally redirect to a named location block, this is useful if you have a dif
ferent upstream backend that you want to dynamically generate the error page from.
Note If youre thinking about using this for redundancy or high-availability (i.e,
A more robust reverse proxy
61
trying to re-run the request), you want the proxy_next_upstream directive, which is
described in Chapter 7).
location / {
error_page 404 = @fallback;
}
location @fallback {
proxy_pass http://different_backend;
}
62
By default, nginx will use proxy_set_header to dynamically set the Host header to
the variable $proxy_host, which contains the name and port of the value passed to
proxy_pass.
In the example above, nginx would proxy the request with the Host header set as
rails_app.
This behavior might be undesirable if you want to know the actual host/domain that
was used for the request, for example, if use use the subdomain to identify the
account as is common in many SaaS applications.
You just need to override the Host header with proxy_set_header and pass it the
value $host instead of $proxy_host.
proxy_set_header Host $host;
Real IP Module
A common problem that people often run into when reverse proxying is the inability
to retrieve the client IP Address. Lets explore this problem for a second and under
stand whats happening under the hood when a request from a client is made to the
load balancer.
A client, 203.0.113.1, makes a web request to an nginx server at example.com, which
resolves to the IP Address 192.0.2.9.
Next, the load balancer at 192.0.2.9 reads the HTTP request from 203.0.113.1 and
proxies the request to one of the app servers inside of its upstream directive.
The million dollar question: What client IP address does the backend think that the
request came from?
The answer is 192.0.2.9 because nginx is proxying the request to the backend, from
the perspective of the backend, its technically the originator of the HTTP request.
The snag you run into is, what if, in your application, youd like to access or log the IP
Address of the client making the request? Its seemingly impossible because, to the
app server, all of the requests appear to be coming from the load balancer.
The Real IP module in nginx solves this problem. This module is packaged with the
nginx source code, but needs to manually enabled by passing the --withhttp_realip_module flag to the build configuration script. If youre using a pre-build
nginx package, it probably includes this module already.
How Real IP Works
Typically, with HTTP, the client IP Address is injected as the REMOTE_ADDR HTTP
Header.
Real IP Module
63
On the reverse proxys, you need to add the real_ip_header directive again, as well
as specify the set_real_ip_from directive.
The set_real_ip_from directive takes a list of IP Addresses or CIDR blocks of trusted
sources for the X-REAL-IP HTTP Header. Without this, anyone would be able to
spoof their IP Address by making HTTP Requests with bogus values in the X-REALIP HTTP Header.
The value of set_real_ip_from should be the IP Address(es) or CIDR block of your
load balancer, in the case of our example, 203.0.113.1.
http {
real_ip_header X-Real-IP;
set_real_ip_from 203.0.113.1;
server {
...
}
}
If there were multiple load balancers, youd just have multiple set_real_ip_from state
ments. It can also take a CIDR block to cover an entire range of addresses.
http {
real_ip_header X-Real-IP;
64
set_real_ip_from 203.0.113.1;
set_real_ip_from 203.0.113.2;
set_real_ip_from 203.0.113.0/24;
server {
...
}
}
This code starts up an HTTP server listening on port 3000 and will send back the
current unix timestamp to any web request that it receives.
65
You can run this application directly by running node app.js from the command
line and visiting http://127.0.0.1:3000 in your browser.
Figure 4-5.
An example of the pm2 user interface
You can install pm2 with npm by running npm install -g pm2
Now, just like before, we want to put nginx in front of our node.js server. The config
uration will look shockingly similar to how we configured the reverse proxy for rails.
Example 4-8. Basic Reverse Proxy for Node
http {
upstream node_app {
server 127.0.0.1:3000;
}
server {
listen *:80;
root /path/to/application/public;
location / {
try_files $uri $uri/index.html @node;
}
66
location @node {
proxy_pass http://node_app;
}
}
}
If the server supports Websockets, it will respond with the HTTP status code 101
Switching Protocols instead of the normal 200 Ok, and begin speaking WebSock
ets.
The response would look something like this after which, the client and server
would begin communicating using the WebSocket protocol over the same TCP con
nection.
67
You might think of WebSockets as switching to something similar to a 2-way chat ses
sion between the client and server, where both sides are able to send and receive mes
sages for a long period of time.
Okay, so whats the point? Well, out of the box nginx supports WebSockets. Awesome.
But, WebSockets gets a bit more complex with a Reverse Proxy because the upgrade
handshake needs to be forwarded to the backend (not all backends support WebSock
ets so even though nginx does, it cant assume the backend can too).
In addition to this, the default reverse proxy configuration uses very simple and state
less connections to the backend and every new request to nginx creates a new request
to the backend server, 1 for 1 that is, it uses HTTP 1.0 and not HTTP 1.1, which is
required for WebSockets.
Luckily, its easy to enable this out of the box by switching the reverse proxy to use
HTTP 1.1 and passing along the Upgrade and Connection headers (these types of
HTTP headers are called hop-by-hop headers and explicitly forbidden by the HTTP
1.1 spec from being passed along by proxies or caches. Normally, headers are forwar
ded along by nginx automatically)
Example 4-9. Basic WebSocket Example
location /chat {
proxy_pass http://node_app;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
We first use proxy_http_version to explicitly set the HTTP version that nginx uses
for communicating with with the backend to be HTTP 1.1. Note that this has no
impact on the HTTP version that the client/browser can use to communicate with
nginx even if this is not set, nginx will use HTTP 1.1 for communicating with the
end client.
The next thing that we do is use proxy_set_header to set the two previously men
tioned headers Upgrade and Connection.
In the case of Upgrade, were setting it to the variable $http_upgrade which is just the
value of the Upgrade header from the original client request.
For the Connection header, we just hard code it as the string upgrade, asking the
backend to upgrade to whatever protocol was passed along.
68
The problem with this configuration is that it will attempt to upgrade every HTTP
connection that is made to it, which means that youd need to have a specific location
block for the WebSocket endpoint if you wanted to share it with a backend that also
supports regular HTTP requests.
We can solve for this by dynamically setting the value of Connection based on the
value of Upgrade, using a new nginx directive called map.
Example 4-10. Dynamically mapping Connection based on Upgrade
map $http_upgrade $connection_upgrade {
'websocket' upgrade;
default close;
}
location @node {
proxy_pass http://node_app;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
Lets dissect the map bit, because I always think that these can be confusing at first
glance if you havent used it before.
If you were to write it in psuedo-programming language code, the logic would
actually look something like this:
switch ($http_upgrade) {
case "websocket":
$connection_upgrade = "upgrade";
break;
default:
$connection_upgrade = "close";
}
The map block maps between the value of the first variable passed, $http_upgrade,
and uses it to set the values for any subsequent variables passed as parameters, in this
case, $connection_upgrade.
So, when $http_upgrade equals websocket, we set $connection_upgrade to
upgrade.
The default case sets the value of $connection_upgrade to close if it doesnt match
anything.
Okay, so weve got nginx configured to use WebSockets, lets test this thing out by
changing out Node.JS dummy code to use WebSockets instead of plain old HTTP.
69
We need to install the websocket library for Node with npm. To do that, just run the
command npm install websocket.
Now, change the app.js file to create a websocket server:
Example 4-11. Example Websocket Node.js Server
var WebSocketServer = require('websocket').server;
var http = require('http');
var server = http.createServer(function (request, response) {
time = (new Date).getTime().toString();
response.writeHeader(200, {"Content-Type": "text/plain"});
response.end(time);
});
server.listen(3000, "127.0.0.1");
console.log("Starting server running at http://127.0.0.1:3000/");
websocketServer = new WebSocketServer({
httpServer: server,
autoAcceptConnections: false
});
websocketServer.on('request', function(request) {
console.log("Received connection");
var connection = request.accept('echo-protocol', request.origin);
time = (new Date).getTime().toString();
connection.SendUTF(time);
});
70
71
CHAPTER 5
Load Balancing
One of the most common, and easiest, ways to scale a busy web application is to
use load balancing. The idea behind load balancing is to distribute load, or web traf
fic, across multiple application servers.
Lets take an example youve just created a Ruby on Rails application and deployed
it to a single server. In order to show the world your Rails application, you modify the
DNS zone of your website to point to the Rails app.
So far, all is good. Visitors type in your domain name and get routed straight to your
single application server where nginx serves up your Ruby on Rails application.
That is, until, one of your biggest fans (mom) is so proud of your application that she
submits it to the popular link sharing website, Reddit. Others love your app too, and
they upvote it to the front-page.
Now, there are thousands of people visiting your website, and a single server cant
possibly handle the load. You know you need to scale out, maybe add some more
servers, but how?
Thats when the load balancer comes in to play the load balancer receives all of the
incoming traffic and distributes it across a pool of application servers.
There are many types of load balancers, both software (implemented in software) and
hardware (implemented in hardware, typically installed as a device).
nginx has the ability to run as a software load balancer using the http_proxy module,
and its one of the most common and useful ways to use nginx in a web stack.
73
When run with the configuration above, nginx will take all traffic on port 80 and bal
ance it across two servers, 192.0.2.10 and 192.0.2.11.
The key directives that make this work are the proxy_pass and the upstream direc
tives.
The default configuration for an upstream is to balance requests using a round-robin
method for example, the first request will be sent to 192.0.2.10, the second to
192.0.2.11, the third to 192.0.2.10, and so on.
Round-robin load balancing is completely ignorant, so factors like the actual load of
the server or number of connections is not considered.
The proxy_pass directive takes a host and causes nginx to proxy the request to that
host. Since the example is passing in a named upstream (i.e., http://backend), nginx
will pick which host to use depending on the load balancing strategy, round-robin in
this case.
74
2. Load Balancers usually operate at either Layer 7 (Application Layer, i.e, HTTP) or
Layer 4 (Transport Layer, i.e, TCP) while typically you will only operate at the
Application Layer when reverse proxying a modern web application.
3. Scale is critical for load balancers on busy sites, they will see 20x the amount of
traffic that a single application server receives. For instance, a powerful applica
tion server will only need to handle 100-200 requests per second, while a load
balancer may very well receive over 10,000 requests per second.
Handling Failure
What happens when a server goes away?
One important use case of load balancing is to handle failure, when a server crashes,
someone trips over the power cord, or it just seems to go away.
By default, when a single request to a server inside of the upstream directive fails, the
server will be removed from the pool for 10 seconds. These settings are configurable
and will be discussed later in the chapter.
At the most basic, load balancers solve two problems for us:
Load balancers help us scale. They make horizontally scaling easy, by spreading the
load across multiple backends. Instead of having a single very-powerful, veryexpensive server, we can use load balancing to share the load across 5 cheap mediumpowered instead.
Load balancers help us deal with failure. Even if you have plenty of power or only
need a single server to serve your web application, that server will eventually go
down. It will crash. The CPU will get fried. In the age of cloud computing, servers are
ephemeral and not long-lived. Load balancers allow us to be highly available by pro
viding redundant backends.
75
The DNS lookup would return something like the result below:
IN
287 IN
287 IN
287 IN
A
A
A
208.0.113.36
208.0.113.34
208.0.113.38
In the example above, there are 3 load balancers for the website example.com, each
with a different IP.
The drawback here is that although the clients are supposed to chose a random IP
from the list, poorly implemented ones will always use first IP presented. Modern
DNS providers (AWS Route 53 and Dynect come to mind) will randomize the order
ing for each DNS request to help mitigate hotspots.
The other drawback is that DNS records are often cached for long periods of time. So,
if a load balancer goes away, the IP Address may still be cached for some users, even if
you remove it from your DNS entry.
Worst still, users get no indication that something is wrong, just a hard error that they
could not connect. Even with DNS TTLs as low as 60s, this is unacceptable.
Therefore, a combination of DNS Load Balancing and a mechanism to re-assign IP
Addresses is necessary to be highly-available. Depending on your provider, you can
use something like keepalived to send heartbeats between servers and takeover the IP
Address of a downed server by sending an ARP broadcast.
Traditionally, keepalived did not work on cloud providers like AWS because they usu
ally filter broadcast traffic from the network for security. However, since version
1.2.8, keepalived supports unicast and makes a great way to build high availability
nginx configurations even on AWS.
We learned about the basic use of upstream in Chapter 4, where we used to connect
to a single Rails or Node backend, but with load balancing, well use it to represent a
group of servers.
For reference, the most basic multi-server upstream block looks something like this:
upstream backend {
server 192.0.2.10;
server 192.0.2.11;
}
The servers in an upstream block can be IP addresses, hostnames, unix domain sock
ets, or a mix any of the above.
upstream backend {
# IP Address with Port
server 192.0.2.10:443;
# Hostname
server app1.example.com;
# Unix Socket
server unix:/u/apps/my_app/current/tmp/unicorn.sock
}
Weighted Servers
Weighting alters the proportion of traffic that will be routed to a particular server in
an upstream.
Weights are useful if you want to send more traffic to particular server because it has
faster hardware, or, if you want to send less traffic to a particular server to test a
change on it.
Example 5-2. Example upstream where one server receives 3x more traffic
upstream
server
server
server
}
backend {
app1.example.com weight=3;
app2.example.com;
app3.example.com;
In the example above, the weight for app1.example.com is set as 3. When the weight
is not specified, as is the case for app2 & app3, it is implied to be 1.
Since app1 has a weight of 3 and app2/app3 have a weight of 1, in the example above,
app1 will have 3x more requests routed to it.
77
Example 5-3. Example upstream where one server receives half the traffic
upstream
server
server
server
}
backend {
app1.example.com weight=2;
app2.example.com weight=2;
app3.example.com weight=1;
In this example, we set the weight as 2 for both app1 & app2. The weight of app3 is set
to 1. Because app3 has a lower weight, it will receive only 20% of the total traffic. This
can be used as a great way to validate configuration changes or new deployments.
Health Checks
In the open-source version of nginx, health checks dont really exist. What you get
out of the box are passive checks checks that will remove a server from the pool if it
causes an error a certain number of times.
The default handling is, if an incoming request to an upstream server errors out or
times out once, the server will be removed from the pool for 10s.
The pitfalls with this type of health checking is
1. It is passively activated, it does not actively check the health of a server except
during the lifecycle of load balancing an incoming request.
2. It does not validate the response of the server. The server might be spewing out
500 errors, but as long as it responds to HTTP, nginx considers it healthy.
You can tweak the behavior of the health checks with three directives: max_fails,
max_fails
Max fails controls the number of times that the server can be marked as unhealthy
before it is removed from the pool.
The conditions of what
proxy_next_upstream.
is
considered
unhealthy
is
determined
by
The default setting is max_fails=1 and is set per server directive within the upstream
directive.
upstream backend {
server app1.example.com max_fails=2;
server app2.example.com max_fails=100;
}
fail_timeout
This directive controls the behavior of two different settings.
78
1. The amount of time to remove the server from the upstream pool when it is
marked as unhealthy.
2. The amount of time that the max_fails count is valid for. That is to say, if
max_fails=2 and fail_timeout=10, the server needs to fail 2 times in 10 seconds
for it to be marked unhealthy.
The default setting is fail_timeout=10 (implied as seconds) and, similar to
max_fails, is set per server directive within the upstream.
upstream backend {
server app1.example.com max_fails=2 fail_timeout=5;
server app2.example.com max_fails=100 fail_timeout=50;
}
proxy_next_upstream
This directive controls the error conditions that will cause an unsuccessful request
and increment the max_fails counter.
As mentioned earlier, the default setup is for nginx to only count complete failures
(an error connecting to the server or a timeout) as a failed request. That behavior is
controlled by this setting.
Below is a table all of the possible values (multiple may be set) for
proxy_next_upstream.
Its worth noting that there are multiple uses (and settings) for this directive, but in
this section, well only discuss the ones applicable to health checks.
Name
Meaning
Default?
error
Yes
timeout
Yes
invalid_header
Yes
http_500
No
http_502
No
http_503
No
http_504
No
The values error, timeout, and invalid_header are set as the default for
proxy_next_upstream. In fact, these three settings are not configurable and will
always be set even if they arent explicitly set.
For most production setups, I find it useful to also turn on http_502, http_503, and
http_504, because they tend to be signs of an unhealthy server (typically meaning that
the web application is unresponsive or out of workers).
Configuring the Upstream Directive
79
Typically, I leave http_500 off, as this isnt always a sign of an unhealthy server and
can be tripped by a broken code path in your web application.
The proxy_next_upstream directive is specified in the location block of the load bal
ancer configuration.
In the example below, I turn on health checks for HTTP 502, 503, and 504 status
codes. Note that even though I dont specify error, timeout, or invalid_header that
they are enabled regardless.
location / {
proxy_next_upstream http_502 http_503 http_504;
proxy_pass http://backend;
}
backend {
app1.example.com;
app2.example.com down;
app3.example.com;
In the example upstream block above, app2 is marked as permanently down and will
receive no traffic from the load balancer.
The reason that youd use down to remove a server instead of commenting it out is to
preserve hashes when using a load balancing algorithm such as ip_hash (described
later in the chapter).
Backup Servers
Its possible to keep a set of backup servers within your pool that will only be used
if all of the other servers in the pool go away.
To do so, simply mark the server with the backup parameter.
upstream
server
server
server
80
backend {
app1.example.com;
app2.example.com;
app3.example.com;
In the example above, Ive added a new server (app4) to be used as a backup server.
Because it will only receive traffic if all of the hosts in the pool are marked as unavail
able, its usefulness is limited to smaller workloads, as youd need enough backup
servers to handle the traffic for your entire pool.
Slow Start
The server directive supports a slow_start parameter, that tells nginx to dynamically
adjust the weight over time, allowing it to ramp up and slowly begin receiving traffic
after it has become recovered or became available.
This is incredibly useful if youre running a JIT (Just In Time) compiled language,
such as the JVM, which can take several minutes to warm up and run at optimal
speeds.
The slow_start parameter takes a time, in seconds, that it will use to ramp up the
weight of the server.
Example 5-4. Example demonstrating the slow_start directive
upstream backend {
server app1.example.com slow_start=60s;
server app2.example.com;
}
81
Ive seen this problem manifest several times, with people configuring nginx to be a
reverse proxy in-front of Amazon ELB (Elastic Load Balancer).
ELB re-maps IPs between clients very frequently, so its incredibly unreliable to cache
the ELB IP Address much longer than the DNS TTL. Anyways, nginx will be config
ured to act as a reverse proxy in-front of ELB and everything will work great, until the
IP Address of the ELB is re-mapped to another client and suddenly nginx starts
reverse proxying to another website! Oops!
Luckily, the fix is simple we just need to tell nginx to obey the DNS TTL and reresolve the hostname.
This can be done by setting the resolve parameter on the server directive and config
uring the resolver directive.
For the resolver, you want to pass it the IP Address of a DNS server that it can use to
resolve domains. In the example, I use 8.8.8.8, which is Googles public DNS resolver,
but its preferable to use a DNS resolver or cache thats local to your network I rec
ommend dnsmasq.
82
Least Connections
The least connections load balancing algorithm picks the server that has the least
number of active connections from nginx.
Many often choose this thinking that its the fairest or most balanced option, however,
often times the number of active connections is not a reliable indicator of server load.
In my experience, this method tends to cause overwhelming and spiky floods of traf
fic to particular servers, especially during times of failure when servers are marked
unavailable by the health check and come back online with 0 active connections.
Turn on this algorithm by specifying the least_conn directive within the upstream
context.
upstream {
least_conn;
server app1.example.com;
server app2.example.com;
}
IP Hash
With round robin and least connection load balancing, when a user makes an HTTP
request and is load balanced to a particular server, they are not guaranteed to have
their request served by the same server for the future requests.
Often times this is desirable, but in some cases, you want the same users to always hit
a particular server, perhaps because of session or file locality.
This method of load balancing is often called sticky sessions meaning that, once a
user is routed to a server, they are stuck to it and subsequent requests will be served
by the same server.
Honestly, I dont recommend using sticky sessions and often it is a symptom of an
unscalable application.
The open-source version of nginx implements sticky sessions by hashing based on
the IP Address of the user. If the IP of the user changes, they will unstick, and the
server theyre routed to may change.
Turn on IP hashing by specifying the ip_hash directive within the upstream context.
upstream {
ip_hash;
server app1.example.com;
server app2.example.com;
}
Because this load balancing algorithm uses a hashing function to determine the
server for a user, all sticky sessions will be remapped if you add or remove any servers
Configuring the Upstream Directive
83
from the pool. The down parameter should be used when removing servers to pre
serve the session hashing.
In other words, dont bet the farm on always having uses routed to the same server, all
of the time.
Hash
The Hash algorithm is very similar to ip_hash, except it allows to hash to a particular
server based on any arbitrary variable in nginx. For example, this can be used to hash
based on the URL or Host.
Hash was originally part of the commercial nginx plus version, but is now available in
the open-source version of nginx.
Example 5-6. Hashing based on the URI
upstream {
hash $uri consistent;
server app1.example.com;
server app2.example.com;
}
The consistent parameter tells nginx to use the ketama hashing algorithm, which is
almost always desirable, as it prevents a full hash remap when servers are added or
removed.
84
With load balancers, latency and scalability is much more important and you want to
make sure that you get it right.
The C10K Problem is the idea of serving 10,000 simultaneous, concurrent connec
tions through a web server.
Since nginx is incredibly scalable, with a bit of tuning (both nginx and the linux ker
nel), it can scale to 10,000 connections without much difficulty. In fact, its not
unheard of to even scale it to 100,000 or more connections.
The ability for your application to be able to handle it, though, is an entirely different
problem!
85
This is not optional, its essential for high performance. The linux kernel offers some
crazy tunables that are hard for even experts to fully understand. While you can find
handfuls of sysctl settings online, its usually not a good idea to copy them onto your
machine blindly without fully understanding the (often complicated) implications.
The handful of settings that Im going to share below are well understood and nonrisky. They also make up over 80% of the performance gains. Ill also share some set
tings to do your own research on.
These kernel flags can be set with the sysctl tool, but will be lost upon reboot. In
order to persistent the changes across reboot, you must add them to /etc/
sysctl.conf or /etc/sysctl.d/.
Whats a good value to set? Well, it totally depends on the level of traffic that youre
load balancing will be receiving, but I recommend setting it high enough to handle
the highest amount of burst traffic that youll be able to handle.
That is, if you have enough capacity to handle 10,000 requests per second, set it to
10,000.
You can set the value with the sysctl tool, like so:
sysctl -w net.core.somaxconn=10000
Afterwards, make sure to edit /etc/sysctl.conf and add the following line to the
file:
net.core.somaxconn=10000
ancer, and http cache, so it naturally competes with many other specialized pieces of
software.
The list is long, but is nginx a better choice than AWS ELB, HAProxy, Varnish, or
Apache?
The answer in my mind is a resounding yes.
In my first book on scaling, I even recommended using HAProxy as the load balancer
instead of nginx, but Ive changed my position.
The reason is that nginx is so incredibly powerful and fast. It doesnt come with much
bloat and will be easiest part of your infrastructure stack to scale.
Technically, on paper, HAProxy and Varnish may be 5-10% faster than nginx for load
balancing and http caching, respectively. But, you can accomplish the same things
with a much simpler nginx stack and only have to become an expert in one piece of
software. Id trade that over a marginal gain in speed.
Many people on AWS choose to use ELB out of convenience and because its so cheap.
While ELB scales automagically, its not very robust in terms of features compared to
nginx. For example, it does not support URI-based routing, HTTP caching, or Vir
tualHosts.
Outside of basic setups, if doing anything serious on AWS, I highly recommend using
nginx as your http routing layer.
87
ences on HTTP, and SSL can pass-thru to the backend application servers transpar
ently.
Up until April 2015, the only way to do TCP load balancing with nginx was to pay for
the commercial license or use an open-source competitor like HAProxy.
Fortunately, nginx has made TCP Load Balancing part of their open-source offering
as of nginx 1.9.0 and now everyone has access to it for free!
Performance aside, this opens up many new uses cases for nginx, such as using it to
load balance database servers or really anything that speaks TCP.
Example 5-9. Example showing TCP Load Balancing
stream {
server {
listen *:80;
proxy_pass backend;
}
upstream
server
server
server
}
backend {
app01;
app02;
app03;
Two things worth noting TCP load balancing introduces the stream block thats
used instead of the http block to define TCP-based servers.
The second thing is that the proxy_pass directive is not contained inside of a location
block, its at the top level server block. Location blocks (and many of the other http
specific features) no longer make sense with TCP load balancing.
For instance, you cant use proxy_set_header to inject an HTTP header. With sim
plicity comes faster performance but less robust features.
88
Fortunately, HAProxy solved this problem by adding the Proxy Protocol to TCP,
which allows reverse proxies to inject the originating source address into the TCP
connection.
To enable this, youll have to make a change on both your nginx load balancer and
your reverse proxy servers.
On your load balancer:
stream {
server {
listen *:80;
proxy_pass backend;
proxy_protocol on;
}
}
Future Sections
1. Caching static content at the Load balancer
2. Keep-Alive for Load balancers
Future Sections
89
CHAPTER 6
Content Caching
In earlier chapters, we covered how nginx can work as a reverse proxy, either as a
proxy for a dynamic web application or as a load balancer, proxying to other nginx
servers.
In both types configurations, there is a 1 to 1 relationship between incoming requests
to the nginx server and the outgoing requests proxied to the backend. No caching is
done. If you were to make 1,000,000 requests for the same file through a location
block that used an upstream, nginx would forward 1,000,000 request to the upstream.
The same proxy module that provides the proxy_pass functionality also includes the
ability to cache responses from the upstream, so after an initial request is
made, future requests can be saved locally and re-used, without making subsequent
calls to backend.
This behavior is called a caching proxy and can be used for advanced applications. For
instance, you could use it to build a static file CDN, reduce load on your backend
servers, or cache assets (images, css, javascript) from your Rails app (for instance,
from the Rails Asset Pipeline). Additionally, you can use the nginx configuration syn
tax that youre already familiar with to create complex rules to only cache content
based on URI, Cookies, HTTP Headers, and more.
91
Web caching is damn hard. Modern apps embody words like real-time, "dynamic,
streaming, and personalized, none of which lend themselves to being easily cache
able.
The way that the proxy caching module in nginx works is that it takes the entire page
payload, everything that is returned from the backend, stores it on disk and re-uses it
for subsequent requests. Notice I said the entire page pageload. That means HTTP
Headers, Cookies, everything!
Right off the bat, that means that any dynamic content is immediately off the table.
You can also scratch off any session-protected content or pages that are customized
for the logged in user.
There is also the dreaded nav bar. Often times, websites display a navigation bar on
the page with information about the currently logged in user, even when displaying
public content. Even though the data contained in the page may be easily cacheable,
tracking the logged in state and showing a small piece of data like name of the cur
rently logged in user or avatar can ruin the entire cacheablity of the page.
Needless to say, it turns out for all but the truly static assets, proxy caching has a
pretty limited scope of use. But, when used properly, can be incredibly valuable as
nginx is significantly faster than serving static content from Ruby, PHP, or really any
dynamic backend.
Content that is easily cacheable is usually:
Static (HTML files, Images, CSS, Javascript, WebFonts)
Preferably Immutable (The file is never modified)
Has a unique cache-buster identifier in the URI
This is just a normal reverse proxy, no magic here.. yet. Any request we make to this
location will proxy the entire request to Amazon S3 and return the result to the client.
92
Now, lets get our feet wet and add in caching. To start, we only need to add two more
configuration directives, proxy_cache and proxy_cache_path.
With caching, the configuration changes to this:
Example 6-1. Amazon S3 Reverse Proxy with Caching
proxy_cache_path /cache levels=1:2 keys_zone=s3_cache:8m
max_size=1000m inactive=60s;
location / {
proxy_pass http://bucket.s3.amazonaws.com;
proxy_cache s3_cache;
}
If you were to run this example, when an incoming request is received by nginx, the
following steps would happen:
1. Generate a cache key
The cache key is used do a lookup to determine if the file has already been cached. By
default, it will generate an MD5 hash of the URI Scheme (i.e, HTTP or HTTPS), the
proxy host (http://bucket.s3.amazonaws.com) and the request URI.
You can change this cache key by using the proxy_cache_key directive. Based on the
formula described above, the actual default value is the concatenation of the following
variables:
$scheme$proxy_host$request_uri
Because the cache key is deviated using a hashing function, even minor changes to
the proxy_cache_key directive will cause it to generate wildly different keys and you
will invalidate 100% of the existing cache.
2. Check if the key is in the cache
In the proxy_cache_path directive, there are a few different configuration parameters.
The third parameter, keys_zone, sets the name and size of the shared memory to use
for storing the cache keys. In this case, weve created a 8MB of shared memory named
s3_cache.
Its important to size your key cache appropriately, depending on how many files you
anticipate caching. 8MB will hold about 8,000 keys.
If the key is in the cache, it will read the file from the directory specifies as the first
parameter of proxy_cache_path. In this case, we told it to use the /cache directory.
93
If the key has not been accessed for longer than the time period specified by the inac
tive parameter (in our example, we set it to inactive=60s), it will be expired from
the cache.
Inactive Tracking
By the way many people turn off access times (i.e, atime) from their filesystems for
performance reasons. Luckily, nginx does not depend on this to determine when an
item in the cache was last accessed. Instead, the expiration time is stored within the
key metadata.
If the key is NOT in the cache, nginx will continue to the next step and actually proxy
the request to the upstream.
3. Proxy request to upstream and store result
If the key is not in the cache, nginx will proxy the request upstream just like normal.
When nginx receives the result of the request from the upstream, it will store the
entire payload, the body AND headers to a file on the filesystem.
Which file will it store that data in? Its based on a few of the parameters that we
passed to proxy_cache_key.
The base directory of the file will be the first parameter, in this case /cache. Easy
enough.
nginx creates a hierarchy of folders within the cache folder in order to optimize per
formance. Most filesystems are not optimized for handling single directories with
hundreds of thousands or millions of files.
This folder hierarchy is set by the levels parameter. In our example, we set the levels
parameter to have the value 1:2. Lets walk through exactly what that means.
Lets assume that the hash of our proxy_cache_key was calculated to be the
hash: 826fe37bf12bdf928ef703ebd7e62fc7.
With the parameter levels=1:2, nginx will store the file containing the payload at the
path /cache/7/fc/826fe37bf12bdf928ef703ebd7e62fc7.
See how the 1st nested folder is 7, corresponding to the last character in the hash?
And that the 2nd nested folder is fc, corresponding to the next two characters in the
hash?
With levels, the colon is the directory delimiter and the value is the number of char
acters, starting from the end of the previous level (or the end of the hash if its the first
level).
94
Keep in mind that two levels of hierarchy (levels=1:2) is the most common and rec
ommended setting.
After storing the payload on the filesystem, nginx will return the result of the request
to the client.
proxy_cache_temp
Actually, if were being 100% accurate, nginx will actually write the
payload to a temporarily file first and move it to the location deter
mined above. You can change the temporary location with the
proxy_cache_temp directive.
You can disable nginx from writing to a temporary file and have it
directly write to the cache directory by setting use_temp_path=off
in the proxy_cache_path directive. For high performance (espe
cially when you have a dedicated cache drive or partition), you
should set use_temp_path=off.
95
location / {
proxy_cache cdn;
proxy_pass origin.example.org;
}
}
Thats pretty simple, isnt it? This config will create the server cdn.example.org which
acts as a caching proxy for origin.example.org. Spread a few of these around the
globe and youve basically built your own CDN.
Because weve set the shared memory for the keys to be 100MB (with keys_zone),
nginx will be able to cache about 800,000 files.
The use_temp_path parameter is also set to off for high performance. This is impor
tant if you have a dedicated volume for caching, as copying between filesystems (i.e,
from the temporary directory to the cache directory) is a slow operation.
96
this problem is RAID0, but that kind of sucks if you lose a single drive, the entire
array dies.
The alternative solution to RAID0 is to manually provision symlinks in the cache
directory yourself. Lets take a hypothetical situation where our machine is configured
with 8 SSDs, /dev/sdb through /dev/sdi. Also, assume each drive is formatted and
mounted in the directory /volumes/sdX.
Because of how nginx uses levels to shard the data across multiple directories within
the main cache directory, we could symlink those internal directories to various
drives within /volumes. Remember that the file and directory naming scheme nginx
uses is a based off of an MD5 hash, which is a hexadecimal value, meaning that there
are only 16 possible directories in the first level (if you have set levels=1:2).
Sharding the directories across multiple drives is as simple as making a symlink for
each of the directories, for instance:
ln
ln
ln
ln
-s
-s
-s
-s
/volumes/sdb
/volumes/sdb
/volumes/sdc
/volumes/sdc
/cache/0
/cache/1
/cache/2
/cache/3
How are they generating that header? Well, it turns out to be pretty easy with the
add_header directive and the $upstream_cache_status variable.
Example 6-3. Adding the cache status
add_header Cache-Status $upstream_cache_status
97
With this directive, nginx will return an HTTP Header named Cache-Status with the
status of accessing the response cache.
$upstream_cache_status has a few different values that can be returned: MISS,
BYPASS, EXPIRED, STALE, UPDATING, REVALIDATED, and HIT.
Name
Description
MISS
BYPASS
EXPIRED
STALE
UPDATING
REVALIDATED Response was fetched from the cache after nginx verified that expired response was valid by querying
upstream with If-Modified-Since header. Depends on proxy_cache_revalidate being
enabled.
HIT
Secure Link
One of the downsides of caching is that security becomes much harder. You no longer
have application logic to protect items that may be in the cache. If someone asks for
an item, nginx will return it (unless you have username/password authentication
turned on within nginx).
Sometimes, there may be assets within the cache that you want to protect from prying
eyes or hot linking. Things like paid product downloads, for instance, come to mind.
nginx has the Secure Link module just for this use case. Secure Link checks
the
98
Advanced Caching
proxy_cache_methods
proxy_cache_valid
proxy_no_cache
proxy_cache_use_stale
proxy_ignore_headers
proxy_cache_revalidate
99
FastCGI Caching
Similar to HTTP proxy caching, the fastcgi module in nginx also supports caching
content from the fastcgi backend and re-using it for subsequent requests. Since fastcgi
applications typically imply extremely dynamic pages (PHP, Ruby, Perl), its common
to for it to not be used very often.
However, if youre one of the few with highly cachable pages coming from a fastcgi
backend, youre in luck! The syntax and rules are very similar to the proxy caching
discussed earlier in the chapter.
Lets take our most basic PHP fastcgi configuration:
Example 6-4. FastCGI in nginx without caching
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass 127.0.0.1:8080;
}
To cache every single response, its as easy as adding the fastcgi_cache directive as
well as defining a zone and cache key with fastcgi_cache_path and fastcgi_cache_key.
Example 6-5. FastCGI in nginx with caching
fastcgi_cache_path /var/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass 127.0.0.1:8080;
fastcgi_cache ZONE_NAME;
}
fastcgi_cache_bypass
fastcgi_cache_lock
fastcgi_cache_min_uses
fastcgi_cache_purge
fastcgi_cache_revalidate
fastcgi_cache_use_stale
fastcgi_cache_valid
100
CHAPTER 7
In Chapter 2, we dove into some of the basic ways of configuring nginx the differ
ent types of directives, how to create a basic server, and even creating virtual hosts
and setting up SSL.
It turns out, thats not all that the nginx configuration can do. In fact, the nginx con
figuration is so powerful, its almost like a programming language it includes some
basic logic, variables, and content manipulation functionality.
Even more, its possible to embed full-fledged programming languages like Lua and
nginScript into nginx so that you can actually write code that gets run by nginx. I
doubt that theres a problem in the web app space that cant be solved by nginx
whether or not its terrible idea and all of your coworkers will hate you, is another
thing.
Taking it to the complete extreme, the open-source project OpenResty has built a
framework that somewhat resembled an MVC-like web framework.. running inside
of nginx.
Variables
nginx has variables that can be get, set statically, and dynamically defined at runtime.
Variables in nginx are evaluated at runtime when a request is processed and are
meant to be a tool to make your configuration more dynamic. They are not designed
to be used as template macros or just to hold static strings.
Most nginx directives will allow variables as arguments. However, some directives,
such as location blocks, will not allow you to use variables to define location routes
dynamically.
101
Out of the box, nginx has many pre-defined variables that you can use freely. The full
alphabetic list of variables is pretty long, so Im not going to cover them all here, but
you can find it in the nginx documentation.
The variables youre most likely to use (and why) is included in the table below.
Variable
Description
$scheme
$http_name
$remote_addr
$server_protocol
$request_time
$request_uri
$uri
$upstream_addr
$upstream_response_time
$upstream_status
Getting the value of a variable is as simple as using the variable in place of static text
for a configuration directive that will be evaluated at runtime for instance, maybe
youd like to let the upstream server know whether or not the request was over SSL.
Example 7-1. Pass $scheme to upstream
location / {
proxy_pass http://upstream;
proxy_set_header X-Forwarded-Proto $scheme;
}
found this standard to be well adapted and most frameworks and apps still use the old
notation.
The new forwarded header is pretty simple and looks like this:
Forwarded: for=192.51.100.0; proto=http; by=192.0.2.10
You can be ahead of the curve and implement the Forwarded header in nginx with
this syntax:
proxy_set_header Forwarded "for=$remote_addr; proto=$scheme; by=$server_addr";
Setting variables
Setting variables can be done two ways the first is using the set directive.
set $someVariable fooBar;
You can also mix variables within the value using string interpolation. For example:
set $someVariable "Current time is $time_local";
Overall, this is a pretty static method of setting variables. While youre do string inter
polation, using set tends be a convenient way to DRY up your configuration file not
add dynamic behavior.
A few things to note in the above example We are setting the value of the variable
$chrome_header dynamically based on the contents of the $http_user_agent variable.
Variables
103
There are two source values in the map the first is a special default case. This defines
the fallback value to use if nothing is matched in the map.
The second source value is the matcher Im using to detect Google Chrome. Since the
actual HTTP User Agent will be filled with garbage like Mozilla/4.0; incompatible
MSIE 14.0 Lunar Module Ready; Google Chrome, I needed to use a regular expres
sion.
In this case, the source value ~*chrome will be a case insensitive regular expression
for the string chrome. The syntax for regular expressions within maps is similar to
the syntax used for location blocks: if you start the string with ~, it will use casesensitive matching while ~* will result in case-insensitive matching.
You are also able to match regular strings, in which case it will always be caseinsensitive.
In the case where you might mix regular expressions and strings, and the regular
expression and string both match, the string will take precedence.
Mapping hostnames
Maps also include a special mode for matching hostnames. When you use a host
name map, it allows you to match using a prefix or suffix wildcard mask.
In order to indicate a hostname mask, you simple add the keyword hostnames inside
of the mask.
An example of a hostname map would look like this:
Example 7-3. Example hostname map
map $http_host $name {
hostnames;
default 0;
example.com 1;
*.example.com 2;
www.example.* 3;
}
The source values without masks will have higher precedence, so example.com will
match against the string, not *.example.com.
Likewise, the prefix mask *.example.com will have a higher precedence than the suf
fix mask, www.example.*, so www.example.com will match *.example.com.
Why would you use this? Well, heres one example many people are running web
applications that have multiple components in a single application, like a low-latency,
highly visited front-end and a slower, seldom visited control panel. An application of
mapping on the hostname would be to change to a different upstream for the control
104
The if directive is almost always a sign of a poorly thought out or outdated nginx
configuration. I say almost because there are a few legitimate use cases, which Ill
cover later.
Variables
105
Lets dig in. Most articles (and even the official documentation) dont make it super
clear why it is so bad. Everyone is a little hand-wavy but hardly anyone explains why.
Because if is a huge hack!
When you use if inside of a location block, under the hood its actually creating a nes
ted location block that inherits the current location block.
The original intention was for it to only support rewrite-level directives, rewrite and
return. Unfortunately, the ability to use any directives inside of the if directive was
So, to go back on the my original statement, if is actually fine and 100% safe when the
only directives used inside of it are rewrite or return.
However, if you are using other directives inside of if, it will not always do the obvi
ous thing that you expect. If youre interested in more specifics, I recommend reading
"If is Evil" from the official NGINX website and "How nginx location if works" by
Yichun Zhang.
Example
if ($fooBar) {
if ($fooBar = 1) {
if ($fooBar ~* foo) {
if ($fooBar !~ foo) {
if (-f index.html) {
if (-d /public/) {
if (!-x rm) {
In most cases, when youd reach for if to test for the existence of a file or directory,
you actually want to use try_files. For instance, take the following example:
# Don't actually do this =]
if (!-e $request_filename) {
rewrite index.php?q=$uri&$args;
}
If you search the web, youll find many old examples that use this syntax for running
PHP applications in nginx. Its checking if the file being requested is a static asset, and
106
if its not, rewriting the request to index.php, which presumably is also being handled
by nginx.
Guess what this is a terrible use of if and try_files actually solves this exact prob
lem. Instead, you could rewrite it as:
# Do this instead!
try_files $uri $uri/ index.php?q=$uri&$args;
Rewriting
Variables
Basic Lua Scripting
Headers
Cookies
3rd Party Extensions
Variables
107