There are a number of ways to add more disk space to a VM on VMware. This guide will discuss 5 different ways to handle expanding the existing disk in VMware, which are:
- Expand existing disk to LVM (Not previously expanded) - Expand existing disk to LVM (Previously expanded) - Expand existing disk with LVM not in use (Dangerous) - Add a new disk into an existing LVM Volume Group - Add a new disk as a separate mount point
Many VMware solutions set their disk labels to MBR, so for this guide, we’ll be making extensive use of fdisk. If your disk label is set to GPT, please use caution when following this guide!
As with any disk expansion operations, always be sure you have solid backups in place in case something goes wrong!
Expand existing disk to LVM (Not previously expanded)
Assuming the VM’s disk has already been expanded within VMware, you have to rescan the specific SD device to detect the new properties. You can do this by:
[root@web01 ~]# echo 1 > /sys/block/sdX/device/rescan or [root@web01 ~]# echo 1 > /sys/bus/scsi/drivers/sd/SCSI-ID/block/device/rescan or [root@web01 ~]# echo 1 > /sys/bus/scsi/drivers/sd/SCSI-ID/block\:sdX/device/rescan
Whether the added a new disk, or expanded an existing one, you can usually detect the change by:
[root@web01 ~]# dmesg|tail ... sd 2:0:0:0: [sda] 67108864 512-byte logical blocks: (34.3 GB/32.0 GiB) sd 2:0:0:0: [sda] Cache data unavailable sd 2:0:0:0: [sda] Assuming drive cache: write through sda: detected capacity change from 17179869184 to 34359738368
Now you need to determine if the volume has actually be expanded. Check for the ‘resize_inode’ flag by:
[root@web01 ~]# tune2fs -l /dev/vglocal00/lvroot | grep -i "^filesystem features"
Check to see if storage has increased in size yet first by:
[root@web01 ~]# fdisk -cul /dev/sda [root@web01 ~]# pvs [root@web01 ~]# vgs [root@web01 ~]# lvs [root@web01 ~]# df -h
Once the root disk has been expanded in VMware, rescan the disk which should now show additional sectors have been added:
[root@web01 ~]# echo 1 > /sys/block/sda/device/rescan [root@web01 ~]# fdisk -cul /dev/sda
Now we need to add a partition for the new space. As fdisk only allows 4 primary partitions, we are going to use extended partitions so we can create logical partitions to hold the new space:
[root@web01 ~]# fdisk -cu /dev/sda p n e (extended) 3 enter enter n l (logical) enter enter p w
Now rescan the partitions so the system can detect the new one without rebooting:
[root@web01 ~]# ls /dev/sda* [root@web01 ~]# partx -l /dev/sda [root@web01 ~]# partx -v -a /dev/sda # There may be some errors here, ignore. [root@web01 ~]# partx -l /dev/sda [root@web01 ~]# ls /dev/sda*
Now setup LVM on that new partition, and add it to the existing volume group and expand the logical volume:
[root@web01 ~]# pvcreate /dev/sda5 [root@web01 ~]# vgs [root@web01 ~]# vgextend YOUR_VG_NAME /dev/sda5 [root@web01 ~]# pvdisplay /dev/sda5 | grep Free Free PE 4095 [root@web01 ~]# lvextend --extents +4095 -n /dev/YOUR_VG_NAME/lv_root
Finally, expand the filesystem doing an online resize:
[root@web01 ~]# resize2fs /dev/YOUR_VG_NAME/lv_root [root@web01 ~]# df -h |grep root
Expand existing disk to LVM (Previously expanded)
If there is a VM where a previous expansion already took place, or otherwise is already on an extended partition with the first (only) logical partition taking up all the room, then this is the section you want.
Once the root disk has been expanded in VMware, rescan the disk which should now show additional sectors have been added:
# Print out disk information [root@web01 ~]# fdisk -cul /dev/sda # Then rescan the device [root@web01 ~]# echo 1 > /sys/block/sdX/device/rescan or [root@web01 ~]# echo 1 > /sys/bus/scsi/drivers/sd/SCSI-ID/block/device/rescan or [root@web01 ~]# echo 1 > /sys/bus/scsi/drivers/sd/SCSI-ID/block\:sdX/device/rescan # Print out disk information to confirm it detected the additional space [root@web01 ~]# fdisk -cul /dev/sda
Expand the existing extended partition:
[root@web01 ~]# parted /dev/sda unit s pri Number Start End Size Type File system Flags 1 2048s 41431039s 41428992s primary lvm 2 41431040s 41943039s 512000s primary ext3 boot 3 41943040s 52428799s 10485760s extended 5 41945088s 52428799s 10483712s logical resize 3 41943040s -1 (Take whatever the extended start value is, and the number) pri quit
Now partition the new space, setup LVM, expand and resize the filesystem:
[root@web01 ~]# fdisk -cu /dev/sda p n l (logical) enter enter p w [root@web01 ~]# ls -hal /dev/sda* [root@web01 ~]# partx -l /dev/sda [root@web01 ~]# partx -v -a /dev/sda # There may be some errors here, ignore. [root@web01 ~]# partx -l /dev/sda [root@web01 ~]# ls -hal /dev/sda* [root@web01 ~]# pvcreate /dev/sda6 # Or whatever the new partition was [root@web01 ~]# vgs [root@web01 ~]# vgextend YOUR_VG_NAME /dev/sda6 [root@web01 ~]# pvdisplay /dev/sda6 | grep Free Free PE 4607 [root@web01 ~]# lvextend --extents +4607 -n /dev/YOUR_VG_NAME/lv_root [root@web01 ~]# df -h [root@web01 ~]# resize2fs /dev/YOUR_VG_NAME/lv_root [root@web01 ~]# df -h
Expand existing disk to without LVM (Dangerous)
This section assumes that LVM was never setup for the disk. Therefore you would need to recreate the partitions to use the new space.
Re-creating partitions is a high risk operation as there is there is the potential for data loss, so make sure you have known good backups you can restore to. And at the very least, snapshot your VM! It also requires a reboot to occur on the VM. Ideally, you should first check to see if an additional disk can simply be mounted to a different mount point instead.
First, list the current partitions:
[root@web01 ~]# fdisk -l
Now within VMware or on the SAN presenting the disk, expand the disk. Once that is done, we need to rescan the volume and confirm the new space:
[root@web01 ~]# echo 1 > /sys/block/sda/device/rescan
[root@web01 ~]# fdisk -l
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 274 2096482+ 83 Linux
/dev/sda3 275 796 4192965 82 Linux swap / Solaris
/dev/sda4 797 2610 14570955 5 Extended
/dev/sda5 797 2610 14570923+ 83 Linux
Using the example above, you will notice that the new partitions (4 and 5) end on the same cylinder (2610). So the extended and logical partitions need to be set to use the new space. So to attempt to help you in the event everything goes wrong, list out the following information and store it somewhere safe so you can refer to it later:
[root@web01 ~]# fdisk -l /dev/sda [root@web01 ~]# df -h [root@web01 ~]# cat /etc/fstab
Now hold on to your butts and expand the disks by deleting the partitions (which *shouldn’t* affect the underlining data), then recreate the partitions with the new sizes:
[root@web01 ~]# fdisk /dev/sda d 5 d 4 n e (Pick original extended position (Should be default, just hit enter) (Pick the new, much larger cylinder ending position (for default all space until end, hit enter) n 5 (or it should just assume right now here) (Pick original logical partition started point (should be default next cylinder, hit enter) (Pick new, much larger cylinder ending position (just default all space until end, hit enter) p (Double check everything, ensure starting cylinders to extended partition 4 and logical partition 5 have the SAME starting cylinder number w
Now reboot the system so it can use the new space:
[root@web01 ~]# shutdown -r now
Then expand the filesystem:
[root@web01 ~]# df -h | grep sda5 /dev/sda5 14G 2.6G 11G 21% / [root@web01 ~]# resize2fs /dev/sda5 [root@web01 ~]# df -h | grep sda5 /dev/sda5 19G 2.6G 15G 15% /
Add a new disk or into an existing LVM Volume Group
This section assumes the new disk is the second disk on the VM, and is enumerated as /dev/sdb. The disk will be added to an existing Volume Group, and we’ll use all the new space on the disk for the volume group and logical volume.
[root@web01 ~]# parted -s -- /dev/sdb mklabel gpt [root@web01 ~]# parted -s -a optimal -- /dev/sdb mkpart primary 2048s -1 [root@web01 ~]# parted -s -- /dev/sdb align-check optimal 1 [root@web01 ~]# parted /dev/sdb set 1 lvm on [root@web01 ~]# parted /dev/sdb unit s print [root@web01 ~]# pvcreate --metadatasize 250k /dev/sdb1 [root@web01 ~]# vgs [root@web01 ~]# vgextend YOUR_VG_NAME /dev/sdb1 [root@web01 ~]# pvdisplay /dev/sdb1 | grep Free Free PE 4095 [root@web01 ~]# lvextend --extents +4095 -n /dev/YOUR_VG_NAME/lv_root [root@web01 ~]# df -h [root@web01 ~]# resize2fs /dev/YOUR_VG_NAME/lv_root [root@web01 ~]# df -h
Add a new disk as a separate mount point
This section assumes the new disk is the second disk on the VM, and it enumerated as /dev/sdb. We are going to use GPT and LVM as a best practice (even if the root disk/partition has the disk label set to MBR or is non-LVM). This example also uses the whole disk in one partition.
# RHEL/CentOS 5: Scan for new disk, check for existing partitions
# setup gpt, align, and partition:
[root@web01 ~]# for x in /sys/class/scsi_host/host*/scan; do echo "- - -" > ${x}; done
[root@web01 ~]# parted /dev/sdb unit s print
[root@web01 ~]# fdisk -l /dev/sdb
[root@web01 ~]# parted /dev/sdb
mktable gpt
quit
[root@web01 ~]# parted -s -- /dev/sdb mkpart primary 2048s -1
[root@web01 ~]# parted /dev/sdb set 1 lvm on
[root@web01 ~]# parted /dev/sdb unit s print
# RHEL/CentOS 6: Scan for new disk, check for existing partitions
# setup gpt, align, and partition:
[root@web01 ~]# for x in /sys/class/scsi_host/host*/scan; do echo "- - -" > ${x}; done
[root@web01 ~]# parted /dev/sdb unit s print
[root@web01 ~]# fdisk -l /dev/sdb
[root@web01 ~]# parted -s -- /dev/sdb mklabel gpt
[root@web01 ~]# parted -s -a optimal -- /dev/sdb mkpart primary 2048s -1
[root@web01 ~]# parted -s -- /dev/sdb align-check optimal 1
[root@web01 ~]# parted /dev/sdb set 1 lvm on
[root@web01 ~]# parted /dev/sdb unit s print
Now on both OS’s, setup LVM, format, and mount the volume to /mnt/data:
[root@web01 ~]# VGNAME=vglocal$(date +%Y%m%d)
[root@web01 ~]# LVNAME=lvdata01
[root@web01 ~]# MOUNTPOINT="/mnt/data"
[root@web01 ~]# FILESYSTEM=`mount | egrep "\ \/\ " | awk '{print $5}'`
[root@web01 ~]# pvcreate --metadatasize 250k /dev/sdb1
[root@web01 ~]# vgcreate ${VGNAME} /dev/sdb1
[root@web01 ~]# lvcreate --extents 100%VG -n ${LVNAME} ${VGNAME}
[root@web01 ~]# mkfs.${FILESYSTEM} /dev/mapper/${VGNAME}-${LVNAME}
[root@web01 ~]# mkdir ${MOUNTPOINT}
[root@web01 ~]# echo -e "/dev/mapper/${VGNAME}-${LVNAME}\t${MOUNTPOINT}\t${FILESYSTEM}\tdefaults\t0 0" >> /etc/fstab
[root@web01 ~]# mount -a
[root@web01 ~]# df -hP | grep "${VGNAME}-${LVNAME}"
/dev/mapper/vglocal20160830-lvdata01 16G 44M 15G 1% /mnt/data