Varnish is a HTTP reverse proxy that can be installed in front of the web server to provide caching. If the VCL’s are properly configured for your site, Varnish can greatly offset the backend server load many times over.
In this guide, it is assumed you already have a running LAMP stack, and Apache’s vhost configurations are stored in /etc/httpd/vhost.d/. At the end of this guide if all goes well, you will be running Varnish 3, with Varnish listening for inbound connections on port 80, and passing any backend connections that cannot be served via cache to Apache on port 8080.
CentOS 6 – Installation and initial configuration
Install the varnish-release package repository, then install:
[root@web01 ~]# rpm --nosignature -ivh https://repo.varnish-cache.org/redhat/varnish-3.0.el6.rpm [root@web01 ~]# yum -y install varnish
Now update your Apache ports and vhosts to 8080 since Varnish will be listening on port 80:
[root@web01 ~]# sed -i "s/Listen 80\$/Listen 8080/g" /etc/httpd/ports.conf [root@web01 ~]# sed -i "s/NameVirtualHost \*:80\$/NameVirtualHost \*:8080/g" /etc/httpd/ports.conf [root@web01 ~]# sed -i "s/:80>/:8080>/g" /etc/httpd/vhost.d/*
Configure Varnish to pass connections back to Apache on port 8080:
[root@web01 ~]# sed -i 's/port = "80"/port = "8080"/g' /etc/varnish/default.vcl
Then update Varnish so it listens on port 80:
[root@web01 ~]# sed -i 's/VARNISH_LISTEN_PORT=6081$/VARNISH_LISTEN_PORT=80/g' /etc/sysconfig/varnish
Finally, restart Apache and Varnish:
[root@web01 ~]# service httpd restart [root@web01 ~]# service varnish start [root@web01 ~]# chkconfig varnish on
Ubuntu 12.04 / 14.04 – Installation and initial configuration
First, setup the Varnish repos:
# Ubuntu 12.04 [root@web01 ~]# curl -sL http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add - [root@web01 ~]# echo "deb http://repo.varnish-cache.org/ubuntu/ precise varnish-3.0" > /etc/apt/sources.list.d/varnish.list # Ubuntu 14.04 [root@web01 ~]# curl -sL http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add - [root@web01 ~]# echo "deb http://repo.varnish-cache.org/ubuntu/ trusty varnish-3.0" > /etc/apt/sources.list.d/varnish.list
Now install Varnish:
[root@web01 ~]# apt-get update [root@web01 ~]# apt-get install varnish
Next update your Apache ports and vhosts to 8080 since Varnish will be listening on port 80:
[root@web01 ~]# sed -i "s/Listen 80\$/Listen 8080/g" /etc/apache2/ports.conf [root@web01 ~]# sed -i "s/NameVirtualHost \*:80\$/NameVirtualHost \*:8080/g" /etc/apache2/ports.conf [root@web01 ~]# sed -i "s/:80>/:8080>/g" /etc/apache2/sites-available/*
Configure Varnish to pass connections back to Apache on port 8080:
[root@web01 ~]# sed -i 's/port = "80"/port = "8080"/g' /etc/varnish/default.vcl
Then update Varnish so it listens on port 80:
[root@web01 ~]# sed -i 's/^DAEMON_OPTS="-a :6081/DAEMON_OPTS="-a :80/g' /etc/default/varnish [root@web01 ~]# sed -i 's/START=no/START=yes/' /etc/default/varnish [root@web01 ~]# service apache2 restart [root@web01 ~]# service varnish restart
Varnish VCL configuration examples
All of the tunings take place within the vcl’s. For the purpose of this guide, we are going to just use the default varnish configuration file in /etc/varnish/default.vcl for our examples.
How to enable basic caching of static resources:
sub vcl_recv {
...
if (req.url ~ "\.(html|gif|jpg|jpeg|png|js|css)$") {
unset req.http.cookie;
return(lookup);
}
return(pass);
}
...
If the request is coming in from CloudFlare or a load balancer, here is how to set the real IP of the client:
sub vcl_recv {
...
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
...
Here is an example of how to exclude things like phpmyadmin, apc.php, and server-status from being cached:
sub vcl_recv {
...
if (req.url ~ "(?i)/(phpmyadmin|apc.php|server-status)") {
return(pass);
}
...
Here is how you can exclude a specific URL from being cached:
sub vcl_recv {
...
if (req.url ~ "^/example") {
return (pass);
}
...
Perhaps you have 30 domains on the server, and you need one of them to be excluded from the cache. Or maybe your actively working on the site. Here is how you can prevent the domain from being served through varnish:
sub vcl_recv {
...
if (req.http.host ~ "^(www.)?domain.com") {
return (pass);
}
...
If you find a script running via your browser, and suspect it is timing out due to varnish, you can adjust the timeout on that specific script by:
sub vcl_recv {
...
if (req.url == "^/bigscript.php") {
set bereq.first_byte_timeout = 10m;
}
...
Here is an example of how to never cache PUT and DELETE requests for a domain:
sub vcl_recv {
...
if ( req.http.host == "subdomain.domain.com" ) {
if (req.method == "PUT" || req.method == "POST" || req.method == "DELETE")
{
return(pass);
}
}
...
Varnish Troubleshooting
One of the most common errors I see on sites utilizing Varnish is a error message:
Error 503 Service Unavailable Guru Meditation: XID: 1234567
Typically Varnish is not the problem, but instead its something else such as Apache or PHP-FPM (aka the backend) not being available. If you can replicate the error in your browser, then run the following command so you can see if you can catch the issue as its happening in the logs:
[root@web01 ~]# varnishlog -d -c -m TxStatus:503
This will return a bunch of output. You are most interesting in the lines surrounding ‘FetchError’ as shown below:
11 SessionOpen c 192.168.1.56 60015 :80 11 ReqStart c 192.168.1.56 60015 311889525 11 RxRequest c GET 11 RxURL c / 11 RxProtocol c HTTP/1.1 11 RxHeader c Host: example.com 11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 11 RxHeader c Connection: keep-alive 11 RxHeader c Cookie: wordpress_test_cookie=WP+Cookie+check; wp-settings-time-1=1455921695 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/601.4.4 11 RxHeader c Accept-Language: en-us 11 RxHeader c DNT: 1 11 RxHeader c Accept-Encoding: gzip, deflate 11 VCL_call c recv pass 11 VCL_call c hash 11 Hash c / 11 Hash c example.com 11 VCL_return c hash 11 VCL_call c pass pass 11 FetchError c no backend connection 11 VCL_call c error deliver 11 VCL_call c deliver deliver 11 TxProtocol c HTTP/1.1 11 TxStatus c 503 11 TxResponse c Service Unavailable 11 TxHeader c Server: Varnish 11 TxHeader c Content-Type: text/html; charset=utf-8 11 TxHeader c Retry-After: 5 11 TxHeader c Content-Length: 418 11 TxHeader c Accept-Ranges: bytes 11 TxHeader c Date: Fri, 19 Feb 2016 23:04:03 GMT 11 TxHeader c X-Varnish: 311889525 11 TxHeader c Age: 0 11 TxHeader c Via: 1.1 varnish 11 TxHeader c Connection: close 11 Length c 418 11 ReqEnd c 311889525 1455923043.127803802 1455923043.128304243 0.000658751 0.000423908 0.000076532
And in the example above, where is has ‘FetchError’, it gave the 503 as Apache was not running. Which was why is says: “no backend connection’.