Memcached is commonly used to alleviate backend database contention by temporarily storing recently requested database records in memory. As a result, the database traffic is reduced as Memcached is able to pull the records from cache.
Installing Memcached is quick. However there are a number of steps that need to be taken to secure Memcached. Memcached has the potential to store a lot of sensitive information, so its critical that the service is locked down properly to prevent outside access or data leakage.
Installation
# CentOS 6 [root@memcached01 ~]# yum install memcached [root@memcached01 ~]# chkconfig memcached on [root@memcached01 ~]# service memcached start # CentOS 7 [root@memcached01 ~]# yum install memcached [root@memcached01 ~]# systemctl enable memcached [root@memcached01 ~]# systemctl start memcached # Ubuntu 14.04 and 16.04 [root@memcached01 ~]# apt-get update [root@memcached01 ~]# apt-get install memcached
Configuration
Memcached listens on port 11211 by default and does not have any type of restrictions built in to prevent it from being queried from the public internet. If you do not protect Memcached with a firewall, there is a extremely high risk of leaking sensitive data. In recent years, unprotected memcached servers have also been exploited to launch DDoS amplification attacks.
If you do not have a dedicated firewall, use your OS’s built in firewall to only allow in connections for trusted web servers using their internal IP’s. Some quick examples are below:
# iptables [root@memcached01 ~]# vim /etc/sysconfig/iptables ... -A INPUT -p tcp -m tcp --dport 11211 -s client_server_private_IP -m comment --comment "memcached" -j ACCEPT service iptables restart # firewalld [root@memcached01 ~]# firewall-cmd --permanent --new-zone=memcached [root@memcached01 ~]# firewall-cmd --permanent --zone=memcached --add-port=11211/tcp [root@memcached01 ~]# firewall-cmd --permanent --zone=memcached --add-source=client_server_private_IP [root@memcached01 ~]# firewall-cmd --reload # ufw [root@memcached01 ~]# ufw allow from client_server_private_IP/32 to any port 11211
When configuring memcached itself, there are a few options to set. In my case, to help limit exposure to the service, I want to set Memcached to only listen on my private network interface and disable UDP. I’ll also set max connections to be 16384 and set the max cachesize to be 1.5G (1536M). Adjust the connections and cachesize as needed for your situation. Below is the example using the options above:
# CentOS 6 and 7 [root@memcached01 ~]# vim /etc/sysconfig/memcached # Ubuntu 14.04 and 16.04 [root@memcached01 ~]# vim /etc/memcached.conf
The configuration options to use to reflect the requirements above:
PORT="11211" USER="memcached" MAXCONN="16384" CACHESIZE="1536" OPTIONS="-l memcached_servers_private_IP -U 0"
Then restart memcached
[root@memcached01 ~]# service memcached restart
Client setup
The typical use cases I run into on a day to day basis are clients using memcached for their PHP application. Memcached can be used to store cached content, or it can be used to centrally store sessions. Therefore these examples will be PHP focused.
Client setup – General data caching
For storing data, there is nothing that needs to be configured on the client side. The application code itself is what controls storing content within Memcached. To ensure memcached can be reached to store content, create the following test script:
[root@web01 ~]# vim /var/www/html/test-memcached.php
if (class_exists('Memcache')) {
$meminstance = new Memcache();
} else {
$meminstance = new Memcached();
}
$meminstance->addServer("memcached_servers_private_IP", 11211);
$result = $meminstance->get("test");
if ($result) {
echo $result;
} else {
echo "No matching key found. Refresh the browser to add it!";
$meminstance->set("test", "Successfully retrieved the data!") or die("Couldn't save anything to memcached...");
}
Then run it in your browser. You can further confirm it works by running it on the command line and confirming ‘cmd_set’ increments by one indicating it was able to store the object:
[root@web01 ~]# echo stats | nc localhost 11211 | grep cmd_set; curl localhost/test-memcached.php; echo stats | nc localhost 11211 |grep cmd_set STAT cmd_set 2 No matching key found. Refresh the browser to add it!STAT cmd_set 3 [root@web01 ~]# echo stats | nc localhost 11211 | grep cmd_set; curl localhost/test-memcached.php; echo stats | nc localhost 11211 |grep cmd_set STAT cmd_set 3 Successfully retrieved the data!STAT cmd_set 3
Digital Ocean has a good article that goes into far more detail on various ways to test/use memcached:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-memcache-on-ubuntu-14-04
Client setup – Storing sessions in memcached
To have memcached act as a central server for sessions, some additional configuration is needed on each client web server. Install php-memcache for your version of PHP. Assuming the default PHP version is installed from the package manager, you can install it by:
# Red Hat / CentOS: [root@web01 ~]# yum install php-pecl-memcached [root@web01 ~]# service httpd graceful [root@web01 ~]# php -m |grep memcached # Ubuntu [root@web01 ~]# apt-get update [root@web01 ~]# apt-get install php-pecl-memcached [root@web01 ~]# service apache2 graceful [root@web01 ~]# php -m |grep memcached
Then update php.ini on both server as follows:
session.save_handler = memcached session.save_path="memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15"
On CentOS and Red Hat servers, depending on what version of PHP was installed and how, you may have to update another file as it will override the php.ini. Only change this if the values exist and are configured for files:
[root@web01 ~]# vim /etc/httpd/conf.d/php.conf php_value session.save_handler "memcached" php_value session.save_path "memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15"
Test to ensure sessions are now being stored in memcached:
[root@web01 ~]# vim /var/www/html/test-sessions.php <?php session_start(); ?> Created a session
Perform the test by running the following on the command line and confirming ‘cmd_set’ increases, indicating it was able to store the session:
[root@web01 ~]# echo stats | nc localhost 11211 | grep cmd_set; curl localhost/session.php; echo stats | nc localhost 11211 |grep cmd_set STAT cmd_set 17 Created a session STAT cmd_set 19
Client setup – Distributed sessions across multiple Memcached instances
Building solutions that can withstand failure is always recommended. Having a Memcached server go offline that is storing sessions will most likely result in unhappy customers. Memcached does not have a built in mechanism to replicate data between multiple memcached servers. This functionality is instead done on the client side.
To allow another Memcached server to take over connections without replicating the session data, update the php.ini on the web servers with the snippet below and restart Apache:
memcache.hash_strategy = consistent session.save_handler = memcache memcache.allow_failover = 1 session.save_path="memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15,memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15"
If you wanted to have automatic failure and also ensure that the sessions are replicated to each Memcached server, update the php.ini on the web servers with the snippet below and restart Apache:
memcache.hash_strategy = consistent memcache.session_redundancy=3 memcache.allow_failover = 1 session.save_handler = memcache session.save_path="memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15,memcached_servers_private_IP:11211?persistent=1&weight=1&timeout=1&retry_interval=15"
Important note: To determine what memcache.session_redundacy should be set to, simply total up all the Memcached servers and add 1 to that total. So in the example above, I have 2 Memcached servers. Therefore, the memcache.session_redundacy should be set to 3.
Troubleshooting
Confirm the web server can reach the memcached server:
[root@web01 ~]# telnet memcached_servers_private_IP 11211
Verify traffic is being sent from the web servers to the memcached server:
[root@web01 ~]# tcpdump -i any port 11211
Checking memcached stats:
[root@memcached01 ~]# echo stats | nc localhost 11211
To see some of the more commonly used stats, use:
[root@memcached01 ~]# echo stats | nc localhost 11211 |grep -E 'total_connections|curr_connections|limit_maxbytes|bytes'
Check to see if you may need to increase the size of the cache. If bytes is reaching the total memory allocation defined by ‘limit_maxbytes’, you may need to increase the cachesize setting:
[root@memcached01 ~]# echo stats | nc localhost 11211 |grep -E 'bytes|limit_maxbytes' |grep -v bytes_ |grep -v _bytes
To flush all the data within memcache:
[root@memcached01 ~]# echo flush_all | nc localhost 11211
To retrieve the version of memcached:
[root@memcached01 ~]# echo version | nc localhost 11211