How to remove a physcal volume/drive from a Volumegroup or How to move a root volume group to new drive

Scenario

The root VG needs to be moved to a different physical disk (e.g. to convert from ATA to SATA drives).

Assumptions

1. You know how to partition disks using fdisk and can edit /etc/fstab with a text editor (such as vim).
2. Your /boot filesystem is on a separate partition, and your root filesystem is part of an LVM VG.
3. Your root VG contains only one PV.

Environment

This procedure was developed on a system with an Intel Core i5 3.2 GHz CPU with 4 GB RAM. The source disk is a Seagate ST3802110A (80 GB ATA), and the destination is a Maxtor 7Y250M0 (250 GB SATA). The Linux distribution used was CentOS 5.4, but these commands should apply to nearly all Linux distributions which support LVM.

Special note

Because LVM is below the filesystem layer (see filesystem layers overview for more information), all of the following operations (with the exceptions of the reboots in the first and last steps) may be done online, without unmounting any filesystems. However, this work should be scheduled for non-peak times, because there is no way (to my knowledge) to control the rate at which pvmove copies data. Therefore, it will usually utilise almost all of your disk's I/O bandwidth.

Procedure

1. Connect the new disk to the system. Depending on your hardware and kernel version, you may need to reboot your system to detect it.
2. Check the partition structure on the source drive. In this case, it has one partition for /boot and the rest of the drive is LVM (output from fdisk -l):
   

   Disk /dev/hde: 80.0 GB, 80026361856 bytes
   255 heads, 63 sectors/track, 9729 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes
      Device Boot      Start         End      Blocks   Id  System
   /dev/hde1   *           1          13      104391   83  Linux
   /dev/hde2              14        9729    78043770   8e  Linux LVM

3. Create partitions on the destination drive to match the source drive. It's OK if they are larger than their equivalents on the source drive. In this scenario, i've increased /boot to 1 GB, and used the remainder of the 250 GB drive forLVM. Here's the final partition setup:
   

   Disk /dev/sdb: 250.0 GB, 250058268160 bytes
   255 heads, 63 sectors/track, 30401 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes
      Device Boot      Start         End      Blocks   Id  System
   /dev/sdb1   *           1         123      987966   83  Linux
   /dev/sdb2             124       30401   243208035   8e  Linux LVM

4. Create the boot filesystem on the new drive:
   

   mkfs -t ext3 -L /boot /dev/sdb1

5. Mount it on /mnt:
   

   mount /dev/sdb1 /mnt

6. Change /etc/fstab to use label rather than path to mount the disk. The line should look something like this when you're done:
   

   LABEL=/boot  /boot  ext3  defaults  1 2

7. Copy files from the existing /boot to the new /boot (mounted on /mnt):
   

   cd /mnt
   dump -0 -b 1024 -f - /boot/ | restore -r -f - -b 1024

8. Unmount the new /boot filesystem:
   

   umount /mnt

9. Install GRUB to the new drive:
   

   grub
   grub> device (hd1) /dev/sdb
   device (hd1) /dev/sdb
   grub> root (hd1,0)
   root (hd1,0)
    Filesystem type is ext2fs, partition type 0x83
   grub> setup (hd1)
   setup (hd1)
    Checking if "/boot/grub/stage1" exists... no
    Checking if "/grub/stage1" exists... yes
    Checking if "/grub/stage2" exists... yes
    Checking if "/grub/e2fs_stage1_5" exists... yes
    Running "embed /grub/e2fs_stage1_5 (hd1)"...  15 sectors are embedded.
   succeeded
    Running "install /grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/grub/stage2 /grub/grub.conf"... succeeded
   Done.
   grub> quit                                                                                             

10. Set up the other partition on your new root disk as an LVM PV:
    

    pvcreate /dev/sdb2

11. Find your current root VG and PV names:
    

    vgdisplay -v

12. Add the new PV to your root VG (substitute vg00 for the name of your VG):
    

    vgextend /dev/vg00 /dev/sdb2

13. Now comes the LVM magic to move the LVs on your current root PV to the new PV. First, test the operation we're about to commence:
    

    pvmove --test --verbose /dev/hde2 /dev/sdb2

    
    Then run it:
    

    pvmove --verbose /dev/hde2 /dev/sdb2

14. This operation will take some time, depending on the size of your data and the speed of your disks. (The system with which i tested this article (see above for specifications) completed the move in approximately 45 minutes.) Pvmove will keep you up-to-date with its progress, but i like to also see how it's performing by running iostat in another terminal:
    

    iostat -dkx 30

    
    Iostat is part of the sysstat package on most Linux distributions. It is not installed by default.
15. When the pvmove is complete, remove the old root disk from the VG (once again, substitute the name of your VGfor vg00):
    

    vgreduce /dev/vg00 /dev/hde2

16. Shut down your system, disconnect the old drive, and reboot using your new drive.

Miscellaneous notes

• If something has gone wrong with your /boot copy or GRUB install, you can boot your system by installing the old disk again, because the old disk contains a fully functional /boot partition, and the LVM VG will be found regardless of which drive it resides upon.
• It is possible to interrupt pvmove with Ctrl-C and resume it by re-running the same command.
• I have never seen pvmove fail if its test run succeeds, so i have no idea what to do if this happens. :-)