Edit 2009-11-16: Use these instructions instead – much cleaner way of doing it

In my day job, I help look after a pretty big VMware cluster (20 blades running 350+ servers, mix of Linux and Windows). On Linux, we use LVM for flexibility.

Conventional wisdom on the subject of growing seems to have said that after resizing your virtual disk, you had to reboot to see the extra space. This is probably not a bad idea, but sometimes you’re just a bit too impatient for that. Or I am anyway.

Let’s follow this through our layers of misdirection between the actual filesystem people can store files on, and the physical disks underneath it.

At the lowest level, we run a NetApp filer pair (well, several) with spinning disks, assembled into aggregates, then split into raid sets, then presented as virtual file systems over NFS to our VMware ESX boxes. Virtual Center stores the vmdks associated with VMs on there. These virtual file systems automatically grow, waffle, backup, etc, transparent to the VMs. It’s very cool.

Ok, growing the file system. The first step is to grow the size of the virtual disk in vCenter (Edit Settings -> Hard Drive 1 -> Resize). Wait for that to finish.

On your Linux VM, rescan the SCSI device:

# echo 1 > /sys/class/scsi_device/0\:0\:0\:0/device/rescan


Run fdisk, and verify that the disk is now seen as its new size:

# fdisk /dev/sda

The number of cylinders for this disk is set to 9137.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 75.1 GB, 75161927680 bytes
255 heads, 63 sectors/track, 9137 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 14 104391 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 14 4178 33450007+ 8e Linux LVM
/dev/sda3 4178 8354 33549073+ 8e Linux LVM


Create your new LVM partition in the free space:


Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Selected partition 4
First cylinder (8355-9137, default 8355):
Using default value 8355
Last cylinder or +size or +sizeM or +sizeK (8355-9137, default 9137):
Using default value 9137

Command (m for help): t
Partition number (1-4): 4
Hex code (type L to list codes): 8e

Print out the partition table (p) then write to disk (w) and exit (q).

FDISK will ominously warn:

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.


And if you try to use this new partition, it won’t be seen.


# pvcreate /dev/sda4
Device /dev/sda4 not found (or ignored by filtering).


The step that every one of the otherwise fine sets of instructions I found online missed out was to rescan the partition table again:


# partprobe -s
/dev/sda: msdos partitions 1 2 3 4


And it works!


# pvcreate /dev/sda4
Physical volume "/dev/sda4" successfully created


And we can extend onto it..


# vgextend -v VGroot /dev/sda4
Checking for volume group "VGroot"
Archiving volume group "VGroot" metadata (seqno 6).
Wiping cache of LVM-capable devices
Adding physical volume '/dev/sda4' to volume group 'VGroot'
Volume group "VGroot" will be extended by 1 new physical volumes
Creating volume group backup "/etc/lvm/backup/VGroot" (seqno 7).
Volume group "VGroot" successfully extended


And create new filesystems, resize filesystems into it, etc. So there you go.