What is an I/O scheduler? The I/O scheduler is a kernel level tunable whose purpose is to optimize disk access requests. Traditionally this is critical for spinning disks as I/O requests can be grouped together to avoid “seeking”.
Different I/O schedulers have their pro’s and con’s, so choosing which one to use depends on the type of environment and workload. There is no one right I/O scheduler to use, it all simply ‘depends’. Benchmarking your application before and after the I/O scheduler change is usually your best indicator. The good news is, the I/O scheduler can be changed at run time and can be configured to persist after reboots.
The three common I/O schedulers are:
– noop
– deadline
– cfq
noop
The noop I/O scheduler is optimized for systems that don’t need an I/O scheduler such as VMware, AWS EC2, Google Cloud, Rackspace public cloud, etc. Since the hypervisor already controls the I/O scheduling, it doesn’t make sense for the VM to waste CPU cycles on it. The noop I/O scheduler simply works as a FIFO (First In First Out) queue.
You can update the I/O scheduler to noop by:
## CentOS 6 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] [root@db01 ~]# echo 'noop' > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler [noop] anticipatory deadline cfq # Change at boot time by appending 'elevator=noop' to end of kernel line: [root@db01 ~]# vim /boot/grub/grub.conf kernel /vmlinuz-2.6.9-67.EL ro root=/dev/vg0/lv0 elevator=noop ## CentOS 7 # Change at run time [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] [root@db01 ~]# echo 'noop' > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler [noop] anticipatory deadline cfq # Change at boot time by appending 'elevator=noop' end of the following line, then rebuild the grub config: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel00/root rd.lvm.lv=rhel00/swap elevator=noop" ... [root@db01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg ## Ubuntu 14.04 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq [root@db01 ~]# echo noop > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler [noop] deadline cfq # Change at boot time by appending 'elevator=noop' end of the following line, then rebuild the grub config: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=noop" ... [root@db01 ~]# grub-mkconfig -o /boot/grub/grub.cfg ## Ubuntu 16.04 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq [root@db01 ~]# echo noop > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler [noop] deadline cfq # Change at boot time by appending 'elevator=noop' end of the following line, then rebuild the grub config: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=noop" ... [root@db01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
deadline
The deadline I/O scheduler is optimized by default for read heavy workloads like MySQL. It attempts to optimize I/O request by putting it in a read queue or write queue and assigning a timestamp to the request. For requests in the read queue, they have 500ms (by default) to execute before they are given the highest priority to run. Requests entering the write queue have 5000ms to execute before they are given the highest priority to run.
This deadline assigned to each I/O request is what makes deadline I/O scheduler optimal for read heavy workloads like MySQL.
You can update the I/O scheduler to deadline by:
## CentOS 6 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] [root@db01 ~]# echo 'deadline' > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory [deadline] cfq # Change at boot time by appending 'elevator=deadline' to end of kernel line apply the changes to grub: [root@db01 ~]# vim /boot/grub/grub.conf kernel /vmlinuz-2.6.9-67.EL ro root=/dev/vg0/lv0 elevator=deadline ## CentOS 7 # Change at run time [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] [root@db01 ~]# echo 'deadline' > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory [deadline] cfq # Change at boot time by appending 'elevator=deadline' end of the following line and apply the changes to grub: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel00/root rd.lvm.lv=rhel00/swap elevator=deadline" ... [root@db01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg # Ubuntu 14.04 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop deadline [cfq] [root@db01 ~]# echo deadline > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq # Change at boot time by appending 'elevator=deadline' end of the following line apply the changes to grub: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=deadline" ... [root@db01 ~]# grub-mkconfig -o /boot/grub/grub.cfg # Ubuntu 16.04 # Change at runtime [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop deadline [cfq] [root@db01 ~]# echo deadline > /sys/block/sda/queue/scheduler [root@db01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq # Change at boot time by appending 'elevator=deadline' end of the following line apply the changes to grub: [root@db01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=deadline" ... [root@db01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
cfg
The cfg I/O scheduler is probably best geared towards things running GUIs (like a desktop) where each process needs a fast response. The goal of the cfq I/O scheduler (Complete Fairness Queueing) is to give a fair allocation of disk I/O bandwidth for all the processes which requests an I/O operation.
You can update the I/O scheduler to cfq by:
## CentOS 6 # Change at runtime [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory [deadline] cfq [root@server01 ~]# echo 'cfq' > /sys/block/sda/queue/scheduler [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] # Change at boot time by appending 'elevator=cfq' to end of kernel line apply the changes to grub: [root@server01 ~]# vim /boot/grub/grub.conf kernel /vmlinuz-2.6.9-67.EL ro root=/dev/vg0/lv0 elevator=cfq ## CentOS 7 # Change at run time [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory [deadline] cfq [root@server01 ~]# echo 'cfg' > /sys/block/sda/queue/scheduler [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq] # Change at boot time by appending 'elevator=cfq' end of the following line and apply the changes to grub: [root@server01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel00/root rd.lvm.lv=rhel00/swap elevator=cfq" ... [root@server01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg # Ubuntu 14.04 # Change at runtime [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq [root@server01 ~]# echo cfq > /sys/block/sda/queue/scheduler [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop deadline [cfq] # Change at boot time by appending 'elevator=cfq' end of the following line apply the changes to grub: [root@server01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=cfq" ... [root@server01 ~]# grub-mkconfig -o /boot/grub/grub.cfg # Ubuntu 16.04 # Change at runtime [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop [deadline] cfq [root@server01 ~]# echo cfq > /sys/block/sda/queue/scheduler [root@server01 ~]# cat /sys/block/sda/queue/scheduler noop deadline [cfq] # Change at boot time by appending 'elevator=cfq' end of the following line apply the changes to grub: [root@server01 ~]# vim /etc/default/grub ... GRUB_CMDLINE_LINUX="elevator=cfq" ... [root@server01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
As with any performance tuning recommendations, there is never a one size fits all solution! Always benchmark your application to establish a baseline before you make the change. After the performance changes have been made, run the same benchmark and compare the results to ensure that they had the desired outcomes.