This appendix contains examples of possible recovery techniques for various potential failures. The examples are presented as is, with no guarantee, and should be used only if the problem is fully understood.
Unlike the examples in Practical Examples that were executed in an AIX Version 4 environment, the examples here are presented in an AIX V3.2 environment. Hence, although the recovery principles are similar for AIX Version 4 and AIX V3.2, some modifications of the following procedures may be required.
Scenario: System has two volume groups, rootvg and vg00. Volume group vg00 has an external disk drive, hdisk4. The power supply fails on hdisk4, the disk media itself is not harmed.
# chpv -vr hdisk4
# rmdev -l hdisk4
# mkdev -l hdisk4
# chpv -v a hdisk4
# lsvg -p vg00
vg00:
PV_NAME PV STATE TOTAL PPs FREE PPs ...
hdisk2 active 75 60
hdisk3 active 75 55
hdisk4 missing 75 50
# varyonvg vg00
# syncvg -p hdisk4
Scenario: System has two volume groups, rootvg and vg00. A disk in vg00 fails and must be replaced. The disk name is hdisk5. The LVs on hdisk5 are: /dev/lvpat, /home/pat, /dev/lvcad, and /cad (mirror copy).
# chpv -v r hdisk5
# umount /home/pat
# rmfs /home/pat
# rmlvcopy lvcad 1 hdisk5
# reducevg -df vg00 hdisk5
# rmdev -d -l hdisk5
# cfgmgr (or an IPL)
# extendvg vg00 hdisk5
# mklv -t jfs -y lvpat vg00 5 hdisk5
# crfs -v jfs -d lvpat -m /home/pat
# mklvcopy lvcad 2 hdisk5
# syncvg -p hdisk5
Scenario: System has two volume groups, rootvg and vg00. Volume group vg00 contains three drives, hdisk1, hdisk2 and hdisk3. hdisk3 failed and has been replaced prior to any clean up. The LVs on hdisk3 are:
/dev/lvpat /home/pat /dev/lvca /cad (mirror copy)
# lsvg -p vg00
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIB.
hdisk1 active 75 59 15..01..13..15..15
hdisk2 active 84 65 17..06..08..17..17
0516-304 lsvg: Unable to find device id 000045af344545ef in the
Device Configuration Database
000045af344545ef missing 95 4 03..01..00..00..00
# reducevg vg00 hdisk3
0516-022 ldeletepv: Illegal parameter or structure value.
0516-884 reducevg: Unable to remove physical volume hdisk3.
# lspv
hdisk0 0000457a839d9efe rootvg
hdisk1 00004224dce3930a vg00
hdisk2 0000175005450a7f vg00
hdisk3 00000601c3a717a4 none
# lqueryvg -p hdisk1 -At
Max LVs: 256
PP Size: 22
Free PPs: 192
LV count: 6
PV count: 3
Total VGDAs: 3
Logical: 00001750ed06a88b.1 paging00 1
00001750ed06a88b.2 lvpat 1
00001750ed06a88b.3 loglv00 1
00001750ed06a88b.4 lvcad 1
Physical: 0000175005450a7f 1 0
00004224dce3930a 1 0
000045af244545ef 1 0
# rmfs /home/pat
# rmlvcopy lvcad 1
# ldeletepv -g 00001750ed06a88b -p 000045af244545ef
# odmdelete -q "value like '000045af244545ef*'" -o CuAt
# savebase
Scenario: System has two volume groups, rootvg and vg00. Volume group vg00 has three disk drives in it, hdisk1, hdisk2 and hdisk3. hdisk3 failed and has been replaced prior to any clean up. Don't know what LVs were on hdisk3.
# lsvg -p vg00
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIB.
hdisk1 active 75 59 15..01..13..15..15
hdisk2 active 84 65 17..06..08..17..17
0516-304 lsvg: Unable to find device id 000045af344545ef in the
Device Configuration Database
000045af344545ef missing 95 4 03..03..00..00
# reducevg vg00 hdisk3
0516-022 ldeletepv: Illegal parameter or structure value.
0516-884 reducevg: Unable to remove physical volume hdisk3.
# lspv
hdisk0 0000457a839d9efe rootvg
hdisk1 00004224dce3930a vg00
hdisk2 0000175005450a7f vg00
hdisk3 00000601c3a717a4 none
# lqueryvg -p hdisk1 -At
Max LVs: 256
PP Size: 22
Free PPs: 192
LV count: 6
PV count: 3
Total VGDAs: 3
Logical: 00001750ed06a88b.1 paging00 1
00001750ed06a88b.2 lvpat 1
00001750ed06a88b.3 loglv00 1
00001750ed06a88b.4 lvcad 1
Physical: 0000175005450a7f 1 0
00004224dce3930a 1 0
000045af244545ef 1 0
# lsps -a
Page Space Physical Vol Vol Group Size %Used Active Auto
0516-304 : Unable to find device id 000045af244545ef in the Device
Configuration Database.
hd6 hdisk0 rootvg 64B 2 yes yes yes
# lspv -l hdisk3
0516-320 : Physical volume 000045ab34dd34ab is not assigned to a
volume group.
# ldeletepv -g 00001750ed06a88b -p 000045af244545ef
0516-016 ldeletepv: Cannot delete physical volume with allocated
partitions. Use either migratepv to move the partitions or
reducevg with the -d option to delete the partitions.
# lspv -l hdisk3
0516-320 : Physical volume 00000601c3a717a4 is not assigned to a
volume group.
# lquerypv -p 000045af244545ef -g 00001750ed06a88b -dt | pg
:
:
PVMAP: 000045af244545ef:1 0 ODMTYPE 000000000000000.0 0
0000000000000000:0 000000000000000:0
:
:
PVMAP: 000045af244545ef :21 0 ODMTYPE 00001750ed06a88b.1 0
000000000000000:0 000000000000000:0
PVMAP: 000045af244545ef :22 0 ODMTYPE 00001750ed06a88b.4 0
000000000000000:0 000000000000000:0
PVMAP: 000045af244545ef :23 0 ODMTYPE 00001750ed06a88b.4 0
000000000000000:0 000000000000000:0
PVMAP: 000045af244545ef :24 0 ODMTYPE 00001750ed06a88b.2 0
000000000000000:0 000000000000000:0
:
:
# rmlvcopy lvcad 1
# rmfs /home/pat
# rmps paging00
0517-062 rmps: Paging space paging00 is active
0517-061 rmps: Cannot remove paging space paging00
Edit /etc/swapspaces and remove the paging00 stanza:
paging00:
dev = /dev/paging00
Reboot.
# rmps paging00
rmlv: Logical volume paging00 is removed
# ldeletepv -g 00001750ed06a88b -p 000045af244545ef
0516-010 ldeletepv: Volume group must be varied on: use varyon
command
The volume group is varied on????
# lchangepv -g 00001750ed06a88b -p 000045af244545ef -r2
# ldeletepv -g 00001750ed06a88b -p 000045af244545ef
# odmdelete -q "value like '000045af244545ef*'" -o CuAt
# extendvg vg00 hdisk3
# mklvcopy lvcad 2 hdisk3
# mklv -t jfs -y lvpat vg00 5 hdisk3
# crfs -v jfs -d lvpat -m /home/pat
# mkps -s 10 -n -a vg00 hdisk3
# syncvg -p hdisk3
Scenario: System has two disks in rootvg, hdisk0 and hdisk1. Most of the operating system is on hdisk0. hdisk0 fails and there is data on hdisk1 that needs be recovered. The logical volumes on hdisk1 are:
/dev/lv0 /home/cad1 /dev/lv01 /home/wordper /dev/loglv00 log logical volume
A jfslog logical volume must be on hdisk1:
# importvg -y rootvg hdisk1
# varyonvg -f -n rootvg
# fsck -y -V jfs /dev/lv00
# fsck -y -V jfs /dev/lv00
# mount -o log=/dev/loglv00 /dev/lv00 /mnt
# cd /mnt
# for i in ./* ./*/* ./*/*/*
> do
> echo $i
> done | pax -wvf/dev/rmt0
Note: This requires that you know the number of levels of subdirectories
# umount /mnt
# mount -o log=/dev/loglv00 /dev/lv01 /mnt
# cd /mnt
# for i in ./* ./*/* ./*/*/*
> do
> echo $i
> done | pax -wvf/dev/rmt0
Note: This requires that you know the number of levels of subdirectories.
Scenario: System has four disks in the rootvg. LVs hd1, hd2, hd3, hd4 hd6 and hd6 are on hdisk0. The LV hd9var is on hdisk1. Other user LVs are 1, on hdisk1, hdisk2 and hdisk3. hdisk1 and hdisk2 fail, system now boots ot to 552 (cannot varyon the rootvg due to lack of quorum).
# getrootfs hdisk0
0516-052 varyonvg: Volume group cannot be varied on without a
quorum. More physical volumes in the group must be active
Run diagnostics on inactive PVs.
0516-780 importvg: Unable to import volume group from hdisk0.
# importvg -fy rootvg hdisk0
PV Status: hdisk0 000005960941e8c2 PVACTIVE
hdisk1 0000175005450a7f NONAME
hdisk2 00000330ecb0948f NONAME
hdisk4 0000188edb0944dd PVACTIVE
varyonvg: Volume group rootvg is varied on
0516-510 updatevg: Physical volume not found for physical volume
identifier 0000175005450a7f.
0516-510 updatevg: Physical volume not found for physical volume
identifier 00000330ecb0948f.
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
0516-782 importvg Partially successful importing of hdisk0 and hdisk4.
# varyonvg -fn rootvg
PV Status: hdisk0 000005960941e8c2 PVACTIVE
hdisk1 0000175005450a7f NONAME
hdisk2 00000330ecb0948f NONAME
hdisk4 0000188edb0944dd PVACTIVE
varyonvg: Volume group rootvg is varied on.
# fsck /dev/hd2
# fsck /dev/hd4
# mount /dev/hd4 /mnt
# mount /dev/hd2 /mnt/usr
# cd /mnt/usr/bin
# cp sed /mnt/mysed
# cp chmod /mnt/mychmod (Don't copy to /mnt/sed & /mnt/chmod)
# sync; sync
# fsck /dev/hd4
# mount /dev/hd4 /mnt
# cp /mnt/mysed /usr/sbin/sed
# cp /mnt/mychmod /usr/sbin/chmod
# cd /usr/sbin
# cat getrootfs | sed "s/importvg -y rootvg/importvg -fy rootvg/" >myfs
# chmod 777 myfs
# umount /mnt
# myfs hdisk0 (whole bunch of messages)
# mount
node mounted mounted over vfs date options
---- ------- ------------ --- ---- -------
/dev/ram0 / jfs Oct 14 rw
/dev/hd4 / jfs Oct 14 rw...
/dev/hd2 /usr jfs Oct 14 rw...
/dev/hd3 /tmp jfs Oct 14 rw...
# mkdir /var/tmp
# TERM="whatever type terminal you are using"
# export TERM
# vi /etc/filesystems
On 3.2.4 or later also comment out lines:
# bosboot -a -d/dev/hdisk0
# shutdown -F
Scenario: System had a volume group vg00. Volume group vg00 included hdisk2 and hdisk3. hdisk2 failed and VGDA is broken on hdisk3.
# mkvg -y newvg hdisk3
# mklv -m /home/mapfile3.a -t jfs -y lv00
# mklv -m /home/mapfile3.b -t jfs -y lv01
# mklv -m /home/mapfile3.c -t jfs -y lv02
# mklv -t jfslog -y loglv00 newvg 1 hdisk3
# logform /dev/loglv00
# for i in `lspv | cut -f1 -d" "`
> do
> lspv -M $i > /home/map.$i
> done
Format: PVname:ppnum1[2]
# cat map.file
hdisk1:1-10
hdisk1:23
hdisk1:33
# cat /etc/filesystems
:
/home/newfs1:
dev = /dev/lv00
vfs = jfs
log = /dev/loglv00
mount = true
options = rw
:
# mount /home/newfs1
# mount /home/newfs2
# mount /home/newfs3