Expanding LVM on Boot disk under VMWare 3.5 without rebooting

Edit 2009-11-16: Use these instructions instead – much cleaner way of doing it

In my day job, I help look after a pretty big VMware cluster (20 blades running 350+ servers, mix of Linux and Windows). On Linux, we use LVM for flexibility.

Conventional wisdom on the subject of growing seems to have said that after resizing your virtual disk, you had to reboot to see the extra space. This is probably not a bad idea, but sometimes you’re just a bit too impatient for that. Or I am anyway.

Let’s follow this through our layers of misdirection between the actual filesystem people can store files on, and the physical disks underneath it.

At the lowest level, we run a NetApp filer pair (well, several) with spinning disks, assembled into aggregates, then split into raid sets, then presented as virtual file systems over NFS to our VMware ESX boxes. Virtual Center stores the vmdks associated with VMs on there. These virtual file systems automatically grow, waffle, backup, etc, transparent to the VMs. It’s very cool.

Ok, growing the file system. The first step is to grow the size of the virtual disk in vCenter (Edit Settings -> Hard Drive 1 -> Resize). Wait for that to finish.

On your Linux VM, rescan the SCSI device:

# echo 1 > /sys/class/scsi_device/0\:0\:0\:0/device/rescan

Run fdisk, and verify that the disk is now seen as its new size:

# fdisk /dev/sda

The number of cylinders for this disk is set to 9137.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 75.1 GB, 75161927680 bytes
255 heads, 63 sectors/track, 9137 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 14 104391 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 14 4178 33450007+ 8e Linux LVM
/dev/sda3 4178 8354 33549073+ 8e Linux LVM

Create your new LVM partition in the free space:


Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Selected partition 4
First cylinder (8355-9137, default 8355):
Using default value 8355
Last cylinder or +size or +sizeM or +sizeK (8355-9137, default 9137):
Using default value 9137

Command (m for help): t
Partition number (1-4): 4
Hex code (type L to list codes): 8e

Print out the partition table (p) then write to disk (w) and exit (q).

FDISK will ominously warn:

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

And if you try to use this new partition, it won’t be seen.


# pvcreate /dev/sda4
Device /dev/sda4 not found (or ignored by filtering).

The step that every one of the otherwise fine sets of instructions I found online missed out was to rescan the partition table again:


# partprobe -s
/dev/sda: msdos partitions 1 2 3 4

And it works!


# pvcreate /dev/sda4
Physical volume "/dev/sda4" successfully created

And we can extend onto it..


# vgextend -v VGroot /dev/sda4
Checking for volume group "VGroot"
Archiving volume group "VGroot" metadata (seqno 6).
Wiping cache of LVM-capable devices
Adding physical volume '/dev/sda4' to volume group 'VGroot'
Volume group "VGroot" will be extended by 1 new physical volumes
Creating volume group backup "/etc/lvm/backup/VGroot" (seqno 7).
Volume group "VGroot" successfully extended

And create new filesystems, resize filesystems into it, etc. So there you go.

The secret life of ships

Following the armistice of 1918, the German Navy was moved to Scapa Flow, and moored there with their German crews while the greedy allies decided who would get what ships.

Ze Germans however, wanted nothing of this, so despite the armistice, scuttled the ships, an act of war in and of its own. Some of the ships were saved and scuttled as target practice in the 1920s, but most went to the bottom. Some were refloated, but again, most are still there. There’s a wikipedia article, but don’t read it right now..

Fast forward to the 1940s. The somewhat unfair settlement against the germans has lead (in part) to another world war, and the americans develop a massive weapon, the likes of which have never been seen before. The war in Europe finishes, and this new weapon brings about the end to the war in Asia. More of these mighty weapons are tested, and from the fallout, the world’s background radiation increases just a bit. So much so that most of the world’s steel refineries produce steel that contains trace levels of radiation (in some cases, not so trace..)

Anyway, this produces a problem. How do you compare radiation levels when everything is now radioactive, to a degree? Well, the easy way is to compare it to something you know is mostly non radioactive. So they grab steel that’s been under water since the 1920s, and thus shielded from most radiation. Remember the Kaiser’s ships? Yup, them.

The circle of life continues.

upgraded pc

A few weeks ago, I bought a Acer Aspire X1301 + LCD, since it was a great deal.. C$399 + tax + enviro fees.

I then sold the LCD for $50, leaving me with a C$349 computer.

The plucky little Acer had some problems.. its power for gaming was.. limited, to say the least, and it only took half height cards, which meant it would be difficult to actually get a good card in there. Then I opened it to check something and the power switch stopped working..

So being me, I bought a new case, PSU and a Radeon 4890 1GB graphics card, and .. erm.. upgraded it. I swapped in the motherboard + drives from the Acer, which meant Windows 7 didn’t need to reactivate, and then plugged in this grunty ATI giant of a card. And it works, I get 100+ fps in everything I’ve tried so far ;)

The only problem is that the graphic card doesn’t have HDMI (despite the box saying in big letters that it does have HDMI, and then having a small weasel statement about requiring an adapter), which means right now.. no sound or use VGA. But I have the adapter on its way, courtesy of eBay.

Mike Neville

I received news that my former boss at UWA Arts, Mike Neville, passed away sometime last night.

There’s so much that could be said. I know he pretty seriously pissed off some people doing his job at UWA, but in the end he was a force for doing things right, and he was still there after they left. He was also a geek at heart, and we often chatted about the esoterics of computer and electronics hardware.

On a more personal level, I thought he was a great guy, both in and out of work. My heart goes out thinking of his wife, delightfully nutty daughter and son. He was also a great friend and mentor to so many of us, but I think none more so than my former co-worker Dan.

I’ll really miss the guy when I am next in Perth.

roadtrips

We had a fun roadtrip down to Seattle and Portland last weekend. About 1200kms round trip. No tickets, but we did have something fall off a truck and scratch our car. Will wait to see if Avis wants my kidneys for that.

US Customs were nice and quick. Seattle was great, but a short visit (but it’s a quick drive, so maybe we’ll head down another weekend). We arrived late, had breakfast at Pike Place markets. Lots of character. Went to the Museum of Flight and saw the first 727, 737, 747 and 757, as well as a Concorde. Very nice.

Portland was also great. Caught up with Perth-friend Jen S-B, and long time online friend Kristal. Had many yummy foods, before doing the drive back to Vancouver directly in one afternoon. All good.

Work has been fun. Really getting my teeth into it.

Updates

It seems I never posted about this.. we both have jobs now, and an awesome 17th floor apartment in Richmond BC. I’m working for a Big University, looking after operations for their VMWare environment (299 VMs and counting). Elizabeth is working for a small software company that does something like cpanel, but for IIS. We both passed our Canadian driving tests, so can legally drive here after the 24th of this month.

Which is coming up quite rapidly, much like the 28th of this month. Also an important date, for those following at home.

Truth is not universal

Working at a University, as I for some reason continue to do so, you are exposed to a large amount of political material from various student (and staff) groups. I also know some people who are pretty interested in Canadian politics.

I remember that at one point, I’d see this political speech and think “wow, the ruling party suck, how could they do these things!”. But as I’ve started thinking more critically, I’ve started to notice how sometimes these hot topics are mutually exclusive.

One example:
– BC Liberals have cut millions from student support loans etc!
– BC Liberals are predicting a deficit of over $400 million next year!
– BC Liberals introducing HST – tax increase!
– BC Liberals won’t raise minimum wage!

.. so which would you like guys? you can’t spend money you don’t have without going into deficit. You can’t have the money to spend without cutting programs or raising taxes. You can’t provide services without taxing, etc.

Another example, Federal Liberal blogger friend made a tweet about how the Conservative Party were spending into deficit. But the thing is, stimulating economies during depressions/recessions is best done by government spending.. so spending money in and of itself isn’t bad. At the same time, the Conservative Party (the ones in government) are running attack ads against the head of the federal Liberal party, talking about how terrible it is that he used to live in the US..

Which brings me to the conclusion that as a politician, EVERYTHING you do is going to piss SOMEONE off, and pissed off people ranting sells newspapers and gets viewers on TV. Which is another factor.. newspapers in particular are in a lot of financial trouble these days, and so are fanning the flames of discontent.

I think the trick in being a politician is to make sure you don’t do things that will piss off too many people at once.

People I admire…

Some of you may have noticed I like to make stuff. Or at least talk about making stuff. Regardless, the ideas are there.

Being a sharing person, let me show you the sites of some people I admire for their skills and abilities, and why I admire them:

  • Mike Harrison – http://www.electricstuff.co.uk/
  • Kenneth Maxon – http://www.users.uswest.net/~kmaxon/
  • Tatja Van Vark – http://www.tatjavanvark.nl/index.html

Let’s start with Mike. Mike is the kind of guy who picks up a discarded high speed camera from the late 1980s and tries to get it working again. Having not received the control panel, he finds a picture of it and tries to guess how the control panel would talk to the main unit. He has some success, but is thwarted by the fact it records onto an esoteric data tape format, and the tape drive it came with is broken. Not to be put off easily, Mike builds his own controller and power supply, complete with direct driven LCD panel, and puts it all in the original box. He also has a bunch of other pages on his electronics projects..

Next up, Ken, or K’Max as he seems to call himself. His page is nominally about his robots. But if you look closer, there’s a huge series of writeups on all of his tools, including his home-built automated injection moulder, and some pages about things that he actually gets paid to do, which seems to be designing avionics systems. There are hundreds and hundreds of pages on tools, machining, electronics and more on his site.

Finally, Tatja Van Vark. She doesn’t actually maintain this site, but it’s pretty amazing. She’s built an improved clone of the Enigma Machine, reconstructed the navigation system from a cold-war era bomber and lots of other cool stuff. Some of the things she’s built, like the Harmonium and Antikythera Mechanism, are so complex I don’t even know what they do, let alone how they work.

So there you go.