Setting up a load balanced Magento setup can be a bit daunting. Anyone that has worked with Magento in the past knows that getting the architecture right the first time around is key. The right architecture will vary from solution to solution depending on the needs of the site.
The scalable solution outlined in this document will be build on the Rackspace Cloud, and will have the following server components:
- Domain: www.example.com - Load Balancer: lb01-http.example.com (With SSL termination) - Servers db01.example.com 64.123.123.1 / 192.168.1.1 (Master DB Server) web01.example.com 64.123.123.3 / 192.168.1.3 (Master Web Server) web02.example.com 64.123.123.4 / 192.168.1.4 (Slave Web Server) web03.example.com 64.123.123.5 / 192.168.1.5 (Slave Web Server)
And our setup will utilize the following software to create a scalable and high performing solution:
- Apache 2.2.x with PHP 5.5 - NFS installed on web01 to export the directory /media to the slave web servers - Lsyncd installed on web01 to sync the documentroot to the slave, and exclude /media, /var, /.git - Set the 'Admin Base URL' in Magento to point to http://admin.example.com to the master web server, web01 - MySQL 5.6 installed on db01 - Redis 3.x installed on db01 to handle both sessions and provide a centralized cache for all web servers
A special note about web servers: Don’t drive yourself nuts trying to determine which is faster, nginx or Apache. The real performance bottleneck is PHP, and it can be mitigated with a properly configured solution, and using a Full Page Cache (FPC) like Turpentine. I prefer Apache as it is the least complicated one to support in my opinion.
Requirements
Magento CE can be very CPU intensive, even for small to mid size sites. Therefore, you need fast servers with many CPU available. When using the Rackspace Cloud, the minimum server size for lower traffic sites would be 4G General Purpose servers. However as Magento is very CPU intensive, I strongly recommend using 8G General Purpose servers.
The following hard requirements as posted in Magento’s documentation is below for Magento CE 1.9:
Apache 2.x MySQL 5.6 (Oracle or Percona) PHP 5.4 or PHP 5.5 Redis or Memcached (For session or cache storage)
The MySQL versions should be noted as Magento does not appear to explicitly state support for MariaDB at this time. They also do not explicitly state support for PHP 5.6. So deviate from these requirements at your own risk!
As per the Magento documentation, if you use MySQL database replication, Magento does not support MySQL statement-based replication. Make sure you use only row-based replication.
[root@db01 ~]# vim /etc/my.cnf ... binlog-format = ROW ...
Web server prep
Servers involved: All web servers
The prerequisites outlined in here can be found in Magento’s documentation. This guide will assume that you already have Apache and PHP installed on your web servers.
First, apply any needed updates to CentOS 6:
yum update
Now install the required PHP modules for Magento.
# php 5.6 (Unsupported PHP version by Magento!) yum -y install php56u-xml php56u-mcrypt php56u-gd php56u-soap php56u-devel php56u-mysql php56u-mbstring # php 5.5 yum -y install php55u-xml php55u-soap php55u-mcrypt php55u-gd php55u-devel php55u-mysql php55u-mbstring # php 5.4 yum -y install php-mcrypt gd gd-devel php-gd php-mysql mbstring
Increase the PHP memory limit:
vim /etc/php.ini ... memory_limit = 512M ...
Apache Setup
Servers involved: All web servers
Setting up the Apache vhost for the Magento site is pretty straight forward. Below is a verbose version of the Apache vhost file needed.
First setup the documentroot:
mkdir -p /var/www/vhosts/example.com
Now setup the Apache vhost:
[root@web01 ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
ServerName example.com
ServerAlias www.example.com
#### This is where you put your files for that domain
DocumentRoot /var/www/vhosts/example.com
### Enable this if you are using a SSL terminated Load Balancer
SetEnvIf X-Forwarded-Proto https HTTPS=on
#RewriteEngine On
#RewriteCond %{HTTP_HOST} ^example.com
#RewriteRule ^(.*)$ http://www.example.com [R=301,L]
<Directory /var/www/vhosts/example.com>
Options -Indexes +FollowSymLinks -MultiViews
AllowOverride All
Order deny,allow
Allow from all
</Directory>
CustomLog /var/log/httpd/example.com-access.log combined
ErrorLog /var/log/httpd/example.com-error.log
# New Relic PHP override
<IfModule php5_module>
php_value newrelic.appname example.com
</IfModule>
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
</VirtualHost>
##
# To install the SSL certificate, please place the certificates in the following files:
# >> SSLCertificateFile /etc/pki/tls/certs/example.com.crt
# >> SSLCertificateKeyFile /etc/pki/tls/private/example.com.key
# >> SSLCACertificateFile /etc/pki/tls/certs/example.com.ca.crt
#
# After these files have been created, and ONLY AFTER, then run this and restart Apache:
#
# To remove these comments and use the virtual host, use the following:
# VI - :39,$ s/^#//g
# RedHat Bash - sed -i '39,$ s/^#//g' /etc/httpd/vhost.d/example.com.conf && service httpd reload
# Debian Bash - sed -i '39,$ s/^#//g' /etc/apache2/sites-available/example.com && service apache2 reload
##
# <VirtualHost _default_:443>
# ServerName example.com
# ServerAlias www.example.com
# DocumentRoot /var/www/vhosts/example.com
# <Directory /var/www/vhosts/example.com>
# Options -Indexes +FollowSymLinks -MultiViews
# AllowOverride All
# </Directory>
#
# CustomLog /var/log/httpd/example.com-ssl-access.log combined
# ErrorLog /var/log/httpd/example.com-ssl-error.log
#
# # Possible values include: debug, info, notice, warn, error, crit,
# # alert, emerg.
# LogLevel warn
#
# SSLEngine on
# SSLCertificateFile /etc/pki/tls/certs/2016-example.com.crt
# SSLCertificateKeyFile /etc/pki/tls/private/2016-example.com.key
# SSLCACertificateFile /etc/pki/tls/certs/2016-example.com.ca.crt
#
# <IfModule php5_module>
# php_value newrelic.appname example.com
# </IfModule>
# <FilesMatch \"\.(cgi|shtml|phtml|php)$\">
# SSLOptions +StdEnvVars
# </FilesMatch>
#
# BrowserMatch \"MSIE [2-6]\" \
# nokeepalive ssl-unclean-shutdown \
# downgrade-1.0 force-response-1.0
# BrowserMatch \"MSIE [17-9]\" ssl-unclean-shutdown
#</VirtualHost>
Then restart Apache to apply the changes:
[root@web01 ~]# service httpd restart
Magento Installation
Servers involved: web01 only
Download a copy of Magento from their site, and upload it to the /root directory. Once done, move it into place by:
[root@web01 ~]# cd /root [root@web01 ~]# tar -xvf magento-1*.tar [root@web01 ~]# cd /root/magento [root@web01 ~]# cp -a ./* /var/www/vhosts/example.com [root@web01 ~]# cp -a ./.htaccess /var/www/vhosts/example.com [root@web01 ~]# chown -R apache:apache /var/www/vhosts/example.com [root@web01 ~]# crontab -e */5 * * * * /bin/bash /var/www/vhosts/example.com/cron.sh
A special note: The cron.sh script only needs to run on the master (admin) web server.
Now browse to your site’s URL, and complete the post installation wizard. When it asks for where you want to store sessions, be sure to specify ‘database’.
Magento admin separation
Servers involved: web01 only
Specifying a master web server for all admin operations is critical for an application like Magento. This allows you to create a subdomain such as ‘http://admin.example.com’, from which all your administrative or backend functions can be run. This helps prevent the age old issue of your images and other work through Magento being uploaded to a slave web server by accident.
Some prefer to do this through Varnish. However in my experience, while Varnish is great for caching, and it is a complete nightmare for admin redirection. So this guide will not be using Varnish. Instead, we’ll use the functionality already provided to us in Magento.
Setting up an admin base URL in Magento CE 1.9 is very simple. First, you need to create an “A” record in DNS to point ‘admin.example.com’ to your master web server. If your using bind, the entry would look like this:
admin.example.com. IN A 64.123.123.3
On web01 only, update Apache’s vhost configuration for the site to include a server alias for the new subdomain, admin.example.com:
[root@web01 ~]# vim /etc/httpd/vhost.d/example.com.conf <VirtualHost *:80> ... ServerAlias www.example.com admin.example.com ... </VirtualHost> <VirtualHost _default_:443> ... ServerAlias www.example.com admin.example.com ... </VirtualHost>
Then restart Apache to apply the change:
[root@web01 ~]# service httpd restart
Finally, log into Magento’s backend control panel, and update the admin base url by:
System -> Configuration -> Admin -> Admin Base URL Use Custom Admin URL: Yes Custom Admin URL: http://admin.example.com/ Use Custom Admin Path: No
Lsyncd Setup
Servers involved: web01 only
To ensure that any code changes made on web01 get pushed down to the slave web servers, we are going to install Lsyncd on web01.
On web01 only, install Lsyncd by:
[root@web01 ~]# yum -y install lsyncd [root@web01 ~]# chkconfig lsyncd on
Now setup the lsyncd configuration by:
[root@web01 ~]# vim /etc/lsyncd.conf
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 20
}
servers = {
"192.168.1.4",
"192.168.1.5"
}
for _, server in ipairs(servers) do
sync {
default.rsyncssh,
source="/var/www/",
host=server,
targetdir="/var/www/",
excludeFrom="/etc/lsyncd-excludes.txt",
rsync = {
compress = true,
archive = true,
verbose = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}
end
Setup the required excludes for Lsyncd:
[root@web01 ~]# vim /etc/lsyncd-excludes.txt vhosts/example.com/media vhosts/example.com/var vhosts/example.com/.git
Finally, start the service
[root@web01 ~]# chkconfig lsyncd on [root@web01 ~]# service lsyncd start
NFS Setup
Servers involved: NFS server (web01) / NFS Client (all slave web servers)
For load balanced Magento installations, Magento recommends that the /media directory is NFS mounted. On web01, setup NFS by:
[root@web01 ~]# yum install rpcbind nfs-utils -y
Now perform some basic tuning for NFS since the defaults are a bit outdated. Uncomment or add the following variables in /etc/sysconfig/nfs
[root@web01 ~]# vim /etc/sysconfig/nfs ... RPCNFSDCOUNT=64 RQUOTAD_PORT=875 LOCKD_TCPPORT=32803 LOCKD_UDPPORT=32769 MOUNTD_PORT=892 STATD_PORT=662 STATD_OUTGOING_PORT=2020 ...
Open the firewall to allow your private network access to the NFS services. You may have to adjust your rules as my private network resides on eth2. Do not allow this on the public interface without adjusting the source IP’s accordingly!
[root@nfs01 ~]# vim /etc/sysconfig/iptables ... -A INPUT -i eth2 -s 192.168.1.0/24 -j ACCEPT ... root@web01 ~]# service iptables restart
Export the directory to be shared, along with its permissions, in /etc/exports:
[root@web01 ~]# vim /etc/exports ... /var/www/vhosts/example.com/media 192.168.1.0/24(rw,no_root_squash) ...
Now start the services, and enable them to start at boot time:
[root@web01 ~]# service rpcbind start; chkconfig rpcbind on [root@web01 ~]# service nfslock start; chkconfig nfslock on [root@web01 ~]# service nfs start; chkconfig nfs on
Now that the NFS server is ready, the NFS clients now need to be setup to connect. This MUST be performed on each slave server. Install the required packages on the NFS clients by:
[root@web02 ~]# yum install rpcbind nfs-utils -y
Now start the services, and enable them to start at boot time.
[root@web02 ~]# service rpcbind start; chkconfig rpcbind on [root@web02 ~]# service nfslock start; chkconfig nfslock on [root@web02 ~]# chkconfig netfs on
Configure the mount point in /etc/fstab:
[root@web02 ~]# vim /etc/fstab 192.168.1.3:/var/www/vhosts/example.com/media /var/www/vhosts/example.com/media nfs vers=3,proto=tcp,hard,intr,rsize=32768,wsize=32768,noatime 0 0
Now create the placeholder directory on the client, mount, and verify it works:
[root@web02 ~]# mkdir /var/www/vhosts/example.com/media
[root@web02 ~]# mount -a
[root@web02 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
14G 1.8G 11G 15% /
tmpfs 939M 0 939M 0% /dev/shm
/dev/sda1 477M 74M 378M 17% /boot
192.168.1.3:/data 14G 1.9G 11G 15% /var/www/vhosts/example.com/media
[root@web02 ~]#
[root@web02 ~]# grep /data /proc/mounts
192.168.1.3:/var/www/vhosts/example.com/media /var/www/vhosts/example.com/media nfs rw,noatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.3,mountvers=3,mountport=892,mountproto=tcp,local_lock=none,addr=192.168.1.3 0 0
[root@web02 ~]#
[root@web02 ~]# touch /var/www/vhosts/example.com/media/test-file
[root@web02 ~]# ls -al /var/www/vhosts/example.com/media/test-file
-rw-r--r-- 1 root root 0 May 5 20:23 /var/www/vhosts/example.com/media/test-file
Be sure to setup the NFS client’s on each slave web server.
Redis Setup
Managing Magento’s cache on a load balancer setup can be a bit of a pain since you would have to log into each server to flush the contents of var/cache whenever you want to empty the cache. This is where a centralized Redis server can come into play. According to Magento’s documentation, they recommend using Redis for session management, and caching. As Magento CE 1.9 supports Redis out of the box, its pretty simple to setup:
On db01, install Redis:
yum install redis30u chkconfig redis on
Now setup redis to listen on our local network, setup the memory limits, and disable disk caching since we want it to be served out of memory:
vim /etc/redis.conf ... bind 192.168.1.1 maxmemory 1500mb maxmemory-policy allkeys-lru # save 900 1 # save 300 10 # save 60 10000 ...
Then start the service:
service redis restart
Now on each web server, install the Redis PHP module:
# PHP 5.4 yum install php54-pecl-redis # PHP 5.5 yum install php55u-pecl-redis # PHP 5.6 yum install php56u-pecl-redis
Then restart Apache:
service httpd restart
Finally, update Magento’s configuration to make use of Redis. The redis section is in bold. This only needs to be performed on web01:
cd /var/www/vhosts/example.com/app/etc
cp local.xml local.xml.orig
vim local.xml
...
<config>
<global>
<install>
<date>
</install>
<crypt>
<key>
</crypt>
<disable_local_modules>false</disable_local_modules>
<resources>
<db>
<table_prefix><![CDATA[]]></table_prefix>
</db>
<default_setup>
<connection>
<host><![CDATA[192.168.1.1]]></host>
<username><![CDATA[example]]></username>
<password><![CDATA[YOUR_PASSWORD]]></password>
<dbname><![CDATA[example]]></dbname>
<initStatements><![CDATA[SET NAMES utf8]]></initStatements>
<model><![CDATA[mysql4]]></model>
<type><![CDATA[pdo_mysql]]></type>
<pdoType><![CDATA[]]></pdoType>
<active>1</active>
</connection>
</default_setup>
</resources>
<session_save><![CDATA[files]]></session_save>
<redis_session>
<host>192.168.1.1</host>
<port>6379</port>
<password></password>
<timeout>2.5</timeout>
<persistent></persistent>
<db>2</db>
<compression_threshold>2048</compression_threshold>
<compression_lib>gzip</compression_lib>
<log_level>1</log_level>
<max_concurrency>6</max_concurrency>
<break_after_frontend>5</break_after_frontend>
<break_after_adminhtml>30</break_after_adminhtml>
<bot_lifetime>7200</bot_lifetime>
</redis_session>
<cache>
<backend>Mage_Cache_Backend_Redis</backend>
<backend_options>
<server>192.168.1.1</server>
<port>6379</port>
<persistent></persistent>
<database>1</database>
<password></password>
<force_standalone>0</force_standalone>
<connect_retries>3</connect_retries>
<read_timeout>10</read_timeout>
<automatic_cleaning_factor>0</automatic_cleaning_factor>
<compress_data>1</compress_data>
<compress_tags>1</compress_tags>
<compress_threshold>20480</compress_threshold>
<compression_lib>gzip</compression_lib>
<use_lua>0</use_lua>
</backend_options>
</cache>
</global>
<admin>
<routers>
<adminhtml>
<args>
<frontName><![CDATA[admin]]></frontName>
</args>
</adminhtml>
</routers>
</admin>
</config>