You may come into a situation where you have to remove a PV from a VG that has failed or not even physically connected but still recorded in the lvmtab. Such a PV is sometimes called a “ghost disk” or “phantom disk”. You can get a ghost disk if the disk has failed before VG activation, maybe because the system has been rebooted after the failure.
If you cannot use vgcfgrestore to write the original LVM header back to the new disk because a valid LVM configuration backup file (/etc/lvmconf/vgXX.conf[.old]) is missing or corrupted you have to remove that PV from the VG (vgreduce) to get a clean configuration.
NOTE: In such situations the vgcfgrestore command may fail to restore the LVM header, complaining about a ‘Mismatch between the backup file and the running kernel’. If you are 100% sure that your backup is valid you may override this check using the –R option.
In order to remove a PV from a VG you have to free it first, i.e. remove all logical extents from it. If the LVs on such a disk is not mirrored data is lost anyway. If it is mirrored you need to reduce the mirror before removing the PV.
A ghost disk is usually indicated by vgdisplay reporting more current PVs than active ones. Additionally LVM commands may complain about the missing PVs:
# vgdisplay vg01
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t11d0":
The specified path does not correspond to physical volume attached to this volume group
vgdisplay: Couldn't query the list of physical volumes.
--- Volume groups ---
VG Name /dev/vg01
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 3
Open LV 3
Max PV 16
Cur PV 2 (number of PVs recorded in the lvmtab)
Act PV 1 (number of PVs recorded in the kernel)
Max PE per PV 1016
VGDA 2
PE Size (Mbytes) 4
Total PE 511
Alloc PE 38
Free PE 473
Total PVG 0
Running vgreduce with the -f option would remove all PVs that are “free”, i.e there is no LV having extents on that PV. Otherwise - if the PV is not free - vgreduce -f reports an extent map to identify the associated LVs:
# vgreduce -f vg01
skip alternate link /dev/dsk/c1t2d2 vgreduce: Couldn't query physical volume "/dev/dsk/c0t11d0": The specified path does not correspond to physical volume attached to this volume group Not all extents are free. i.e. Out of 508 PEs, only 500 are free.
You must free all PEs using lvreduce/lvremove before the PV can be removed.
Example:
lvreduce -A n -m 0 /dev/vg01/lvol1. lvremove -A n /dev/vg01/lvol1.
Here's the map of used Pes
--- Logical extents ---
LE LV PE Status
1 0000 lvol1 0000 ??? 0001 lvol1 0001 ??? 0002 lvol1 0002 ??? ...
In this case lvol1 is having extents on device c0t11d0. You have to remove these extents from the PV before you are allowed to actually remove the PV from the VG. If the LV is mirrored use the command lvreduce to remove its mirrored extents. If the LV is unmirrored, data is lost anyway and you have to use lvremove to delete the LV.
Check the LV state:
# lvdisplay -v /dev/vg01/lvol1
lvdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t11d0":The specified path does not correspond to physical volume attached to this volume group lvdisplay: Couldn't query the list of physical volumes.
--- Logical volumes ---
LV Name /dev/vg01/lvol1
VG Name /dev/vg01
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 32
Current LE 8
Allocated PE 16
HP-UX Handbook – Rev 13.00 Page 64 (of 110)
Chapter 13 LVM
October 29, 2013
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict IO Timeout (Seconds) default
--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c0t0d2 8 8
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 ??? 00000 stale /dev/dsk/c0t0d2 00000 current
00001 ??? 00001 stale /dev/dsk/c0t0d2 00001 current
00002 ??? 00002 stale /dev/dsk/c0t0d2 00002 current
00003 ??? 00003 stale /dev/dsk/c0t0d2 00003 current
00004 ??? 00004 stale /dev/dsk/c0t0d2 00004 current
00005 ??? 00005 stale /dev/dsk/c0t0d2 00005 current
00006 ??? 00006 stale /dev/dsk/c0t0d2 00006 current
00007 ??? 00007 stale /dev/dsk/c0t0d2 00007 current
The PV key of a disk indicates its order in the VG. The first PV has the key 0, the second has the key 1, etc. This does not necessarily have to be the order of appearance in lvmtab altough it is usually like that, at least when a VG is initially created.
The PV key can be used to address a PV that is not attached to the VG. This usually happens if it was not accessible during activation, e.g. due to a hardware or configuration problem.
NOTE: The PV may be unattached due to some temporary problem during VG activation which is no longer present. In this case you should try to re-activate the VG to force LVM to re-scan the devices listed in lvmtab: In this example you can see, that the LV in question is mirrored. One of its PVs is not attached to the VG, so its device file is unknown to LVM and displayed as “???”. Addressing this PV is no longer possible using the device file name. Removing a PV using its PV key
# vgchange -a y vgXX or # vgchange -a e vgXX (for exclusively activated Cluster VGs)
If the problem persists follow these steps to clear the situation:
1. Obtain the PV key using the -k option of lvdisplay:
# lvdisplay –v –k /dev/vg01/lvol1 ... ...
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 00000 stale 1 00000 current
00001 0 00001 stale 1 00001 current
00002 0 00002 stale 1 00002 current
00003 0 00003 stale 1 00003 current
00004 0 00004 stale 1 00004 current
00005 0 00005 stale 1 00005 current
00006 0 00006 stale 1 00006 current
00007 0 00007 stale 1 00007 current
Compared to the output above the ??? have been replaced with the PV key (= 0).
NOTE: You can use the xd(1) command to display the PV key because it is stored at a fixed position in the LVM header, exactly 8222 bytes from the beginning of the disk:
# xd –j8222 -N2 /dev/rdsk/c1t6d0
2. Reduce the mirror with the obtained key as argument: # lvreduce –k –m 0 /dev/vg01/lvol1 0
3. After that the PV can be removed from the VG:
# vgreduce -f vg01
vgreduce: Couldn't query physical volume "/dev/dsk/c0t11d0":
The specified path does not correspond to physical volume attached to this volume group PV with key 0 sucessfully deleted from vg vg01 Repair done, please do the following steps.....:
1. save /etc/lvmtab to another file
2. remove /etc/lvmtab
3. use vgscan(1m) -v to re-create /etc/lvmtab
4. Perform the above steps indicated above in order to remove the PV from the lvmtab:
# mv /etc/lvmtab /etc/lvmtab.org
# vgscan –v
...
...
Scan of Physical Volumes Complete.
*** LVMTAB has been created successfully.
*** If PV links are configured in the system.
*** Do the following to resync information on disk.
*** #1. vgchange -a y
*** #2. lvlnboot -R
5. Check the results: # strings /etc/lvmtab /dev/vg01 /dev/dsk/c0t0d2 /dev/dsk/c1t2d2
6. Re-activate the VG and backup the LVM config: # vgchange -a y vg01 # vgcfgbackup vg01
No comments:
Post a Comment