憤Ⓘ鬥Ⓣ屎: 回收 Veritas Storage Disk on RHEL 6.4

先 umount 掛載點
再執行 deport
刪除 path
徹底清空 path

先確認是否有 Veritas Cluster

# vxdctl -c mode
mode: enabled: cluster inactive

出現這個 cluster inactive 代表沒有啟用 Cluster，所以可以接續做下去

顯示準備要下線的 vxdg group 的掛載點

# df -h | grep -A 1 testdbdg
/dev/vx/dsk/testdbdg/testdb-archive01
                      100G  7.2G   93G   8% /DB_TEST/archive01
/dev/vx/dsk/testdbdg/testdb-archive02
                      750G  596G  153G  80% /DB_TEST/archive02
/dev/vx/dsk/testdbdg/testdb-archive03
                      750G  591G  158G  79% /DB_TEST/archive03
/dev/vx/dsk/testdbdg/AP_TEST
                      100G  299M   94G   1% /AP_TEST

確認目前 lsscsi 狀態

# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
[2:0:1:0]    disk    VMware   Virtual disk     1.0   /dev/sdb
[2:0:2:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdc
[2:0:3:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdd
[2:0:4:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sde
[2:0:5:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdf
[2:0:6:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdg
[2:0:8:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdh
[2:0:9:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdi
[2:0:10:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdj
[2:0:11:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdk
[2:0:12:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdl
[2:0:13:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdm
[2:0:14:0]   disk    VMware   Virtual disk     1.0   /dev/sdn
[2:0:15:0]   disk    VMware   Virtual disk     1.0   /dev/sdo
[3:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sdp
[3:0:1:0]    disk    VMware   Virtual disk     1.0   /dev/sdq

這時候會看到有 LUN C-Mode 是從 NETAPP 來的，你也可以使用 lsscsi -c or cat /proc/scsi/scsi 也可以獲得類似相同的訊息

# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 01 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 02 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 03 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 04 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 05 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 06 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 08 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 09 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 10 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 11 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 12 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 13 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 14 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 15 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi3 Channel: 00 Id: 01 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02

依據上面的輸出顯示，可以發現都有固定的格式，參考如下:

[H:C:T:L]

          H == hostadapter id (first one being 0)
          C == SCSI channel on hostadapter (first one being 0)
          T == ID
          L == LUN (first one being 0)

依據上面的說明，可以得到下面這樣的資訊

Host: scsi2 Channel: 00 Id: 02 Lun: 00
這個就等於 [2:0:2:0]

為了要驗証是否相同，我們可以執行 lsscsi 2:00:02:00 or lsscsi 2:0:2:0，都會獲得上面相同的輸出訊息

# lsscsi 2:00:02:00
[2:0:2:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdc
# lsscsi 2:0:2:0
[2:0:2:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdc

Umount Mount Point

# df -h | grep -A 1 testdbdg | grep '/DB_TEST' | awk '{print $NF}' | head -n 1 | xargs umount

Remove Veritas Volume

請先使用 vxdisk -o alldgs list | grep testdbdg 觀察資訊

# vxdisk -o alldgs list | grep testdbdg
cl8080b_0_11 auto:cdsdisk    testdbdg13    testdbdg      online clone_disk
cl8080b_0_12 auto:cdsdisk    testdbdg01    testdbdg      online clone_disk
cl8080b_0_13 auto:cdsdisk    testdbdg03    testdbdg      online clone_disk
cl8080b_0_14 auto:cdsdisk    testdbdg12    testdbdg      online clone_disk
cl8080b_0_15 auto:cdsdisk    testdbdg10    testdbdg      online clone_disk
cl8080b_0_16 auto:cdsdisk    testdbdg11    testdbdg      online clone_disk
cl8080b_0_18 auto:cdsdisk    testdbdg15    testdbdg      online clone_disk

接著使用 vxprint -htg testdbdg 觀察資訊

# vxprint -htg testdbdg
DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
ST NAME         STATE        DM_CNT   SPARE_CNT         APPVOL_CNT
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
CO NAME         CACHEVOL     KSTATE   STATE
VT NAME         RVG          KSTATE   STATE    NVOLUME
V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO
EX NAME         ASSOC        VC                       PERMS    MODE     STATE
SR NAME         KSTATE

dg testdbdg      default      default  46000    1383821912.75.luffy01p

dm testdbdg01    cl8080b_0_12 auto     65536    524103024 -
dm testdbdg03    cl8080b_0_13 auto     65536    524103024 -
dm testdbdg10    cl8080b_0_15 auto     65536    209534256 -
dm testdbdg11    cl8080b_0_16 auto     65536    209534256 -
dm testdbdg12    cl8080b_0_14 auto     65536    209534256 -
dm testdbdg13    cl8080b_0_11 auto     65536    524103024 -
dm testdbdg15    cl8080b_0_18 auto     65536    209534256 -

v  AP_TEST       -            ENABLED  ACTIVE   209532928 SELECT   -        fsgen
pl AP_TEST-02    AP_TEST       ENABLED  ACTIVE   209532928 CONCAT   -        RW
sd testdbdg10-01 AP_TEST-02    testdbdg10 0       209532928 0        cl8080b_0_15 ENA

v  testdb-archive-node2 -     ENABLED  ACTIVE   209532928 SELECT   -        fsgen
pl testdb-archive-node2-02 testdb-archive-node2 ENABLED ACTIVE 209532928 CONCAT - RW
sd testdbdg11-01 testdb-archive-node2-02 testdbdg11 0 209532928 0    cl8080b_0_16 ENA

v  testdb-archive-node3 -     ENABLED  ACTIVE   209532928 SELECT   -        fsgen
pl testdb-archive-node3-02 testdb-archive-node3 ENABLED ACTIVE 209532928 CONCAT - RW
sd testdbdg15-01 testdb-archive-node3-02 testdbdg15 0 209532928 0    cl8080b_0_18 ENA

v  testdb-archive-node4 -     ENABLED  ACTIVE   209532928 SELECT   -        fsgen
pl testdb-archive-node4-02 testdb-archive-node4 ENABLED ACTIVE 209532928 CONCAT - RW
sd testdbdg12-01 testdb-archive-node4-02 testdbdg12 0 209532928 0    cl8080b_0_14 ENA

v  testdb-archive03 -         ENABLED  ACTIVE   1572141056 SELECT  -        fsgen
pl testdb-archive03-02 testdb-archive03 ENABLED ACTIVE 1572141056 CONCAT -   RW
sd testdbdg13-02 testdb-archive03-02 testdbdg13 75136 523937792 0    cl8080b_0_11 ENA
sd testdbdg01-01 testdb-archive03-02 testdbdg01 0 524101632 523937792 cl8080b_0_12 ENA
sd testdbdg03-01 testdb-archive03-02 testdbdg03 0 524101632 1048039424 cl8080b_0_13 ENA

過濾出準備要回收的 Volume，並觀察資訊

# vxprint -htg testdbdg | grep testdb-archive03
v  testdb-archive03 -         ENABLED  ACTIVE   1572141056 SELECT  -        fsgen
pl testdb-archive03-02 testdb-archive03 ENABLED ACTIVE 1572141056 CONCAT -   RW
sd testdbdg13-02 testdb-archive03-02 testdbdg13 75136 523937792 0    cl8080b_0_11 ENA
sd testdbdg01-01 testdb-archive03-02 testdbdg01 0 524101632 523937792 cl8080b_0_12 ENA
sd testdbdg03-01 testdb-archive03-02 testdbdg03 0 524101632 1048039424 cl8080b_0_13 ENA

對應欄位的意義

V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
v  testdb-archive03 -         ENABLED  ACTIVE   1572141056 SELECT  -        fsgen

PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
pl testdb-archive03-02 testdb-archive03 ENABLED ACTIVE 1572141056 CONCAT -   RW

SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
sd testdbdg13-02 testdb-archive03-02 testdbdg13 75136 523937792 0    cl8080b_0_11 ENA
sd testdbdg01-01 testdb-archive03-02 testdbdg01 0 524101632 523937792 cl8080b_0_12 ENA
sd testdbdg03-01 testdb-archive03-02 testdbdg03 0 524101632 1048039424 cl8080b_0_13 ENA

停止 VxVM 啟用中的 Volume

指令: vxvol [-g diskgroup] stop volume_name

# vxvol -g testdbdg stop testdb-archive03

執行 vxprint -htg testdbdg | grep testdb-archive03 再次觀察資訊

# vxprint -htg testdbdg | grep testdb-archive03
v  testdb-archive03 -         DISABLED CLEAN    1572141056 SELECT  -        fsgen
pl testdb-archive03-02 testdb-archive03 DISABLED CLEAN 1572141056 CONCAT -   RW
sd testdbdg13-02 testdb-archive03-02 testdbdg13 75136 523937792 0    cl8080b_0_11 ENA
sd testdbdg01-01 testdb-archive03-02 testdbdg01 0 524101632 523937792 cl8080b_0_12 ENA
sd testdbdg03-01 testdb-archive03-02 testdbdg03 0 524101632 1048039424 cl8080b_0_13 ENA

會發現開頭 v 行中的 KSTATE & STATE 被改成 DISABLED & CLEAN，這代表 volume 已被停止了，接下來就是要刪除 Volume 了

刪除 VxVM 已停止的 Volume

指令: vxassist [-g diskgroup] remove volume volume_name

# vxassist -g testdbdg remove volume testdb-archive03

再次執行 vxprint -htg testdbdg | grep testdb-archive03，並觀察資訊

這樣就已完成 VxVM Remove Volume 了，接下來就是刪除 Disk

Remove Disk From DiskGroup

請執行 vxdisk -o alldgs list | grep -Ev '^sd'，並觀察資訊

# vxdisk -o alldgs list | grep -Ev '^sd'
DEVICE       TYPE            DISK         GROUP        STATUS
cl8080b_0_11 auto:cdsdisk    testdbdg13    testdbdg      online clone_disk
cl8080b_0_12 auto:cdsdisk    testdbdg01    testdbdg      online clone_disk
cl8080b_0_13 auto:cdsdisk    testdbdg03    testdbdg      online clone_disk
cl8080b_0_14 auto:cdsdisk    testdbdg12    testdbdg      online clone_disk
cl8080b_0_15 auto:cdsdisk    testdbdg10    testdbdg      online clone_disk
cl8080b_0_16 auto:cdsdisk    testdbdg11    testdbdg      online clone_disk
cl8080b_0_17 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_18 auto:cdsdisk    testdbdg15    testdbdg      online clone_disk
cl8080b_0_19 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_20 auto:cdsdisk    -            -            online clone_disk

會發現有幾個 Disk 出現 - 這樣的符號，代表還沒加入過任何的 VxVM 中的 DiskGroup Volume，接下來要進行刪除 Disk 了，因為先前我們有紀錄下 Disk，所以可以參考下面的資訊來進行刪除。

# vxprint -htg testdbdg | grep testdb-archive03
v  testdb-archive03 -         DISABLED CLEAN    1572141056 SELECT  -        fsgen
pl testdb-archive03-02 testdb-archive03 DISABLED CLEAN 1572141056 CONCAT -   RW
sd testdbdg13-02 testdb-archive03-02 testdbdg13 75136 523937792 0    cl8080b_0_11 ENA
sd testdbdg01-01 testdb-archive03-02 testdbdg01 0 524101632 523937792 cl8080b_0_12 ENA
sd testdbdg03-01 testdb-archive03-02 testdbdg03 0 524101632 1048039424 cl8080b_0_13 ENA

刪除 VxVM 中的 Disk

指令: # vxdg [-g diskgroup] rmdisk diskname

# vxdg -g testdbdg rmdisk testdbdg13
# vxdg -g testdbdg rmdisk testdbdg01
# vxdg -g testdbdg rmdisk testdbdg03

再次執行 vxdisk -eo alldgs list | grep -Ev '^sd*'，請觀察資訊

# vxdisk -o alldgs list | grep -Ev '^sd*'
DEVICE       TYPE            DISK         GROUP        STATUS
cl8080b_0_11 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_12 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_13 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_14 auto:cdsdisk    testdbdg12    testdbdg      online clone_disk
cl8080b_0_15 auto:cdsdisk    testdbdg10    testdbdg      online clone_disk
cl8080b_0_16 auto:cdsdisk    testdbdg11    testdbdg      online clone_disk
cl8080b_0_17 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_18 auto:cdsdisk    testdbdg15    testdbdg      online clone_disk
cl8080b_0_19 auto:cdsdisk    -            -            online clone_disk
cl8080b_0_20 auto:cdsdisk    -            -            online clone_disk

徹底清空 PATH

請先執行 vxdisk -eo alldgs list | grep -Ev '^sd*'，請觀察資訊

# vxdisk -eo alldgs list | grep -Ev '^sd*'
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
cl8080b_0_11 auto:cdsdisk   -            -           online clone_disk    sdi              tprclm
cl8080b_0_12 auto:cdsdisk   -            -           online clone_disk    sdk              tprclm
cl8080b_0_13 auto:cdsdisk   -            -           online clone_disk    sdm              tprclm
cl8080b_0_14 auto:cdsdisk   testdbdg12    testdbdg     online clone_disk    sdg              tprclm
cl8080b_0_15 auto:cdsdisk   testdbdg10    testdbdg     online clone_disk    sdc              tprclm
cl8080b_0_16 auto:cdsdisk   testdbdg11    testdbdg     online clone_disk    sde              tprclm
cl8080b_0_17 auto:cdsdisk   -            -           online clone_disk    sdl              tprclm
cl8080b_0_18 auto:cdsdisk   testdbdg15    testdbdg     online clone_disk    sdf              tprclm
cl8080b_0_19 auto:cdsdisk   -            -           online clone_disk    sdj              tprclm
cl8080b_0_20 auto:cdsdisk   -            -           online clone_disk    sdh              tprclm

再執行 lsscsi，請觀察資訊

# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
[2:0:1:0]    disk    VMware   Virtual disk     1.0   /dev/sdb
[2:0:2:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdc
[2:0:3:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdd
[2:0:4:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sde
[2:0:5:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdf
[2:0:6:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdg
[2:0:8:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdh
[2:0:9:0]    disk    NETAPP   LUN C-Mode       9000  /dev/sdi
[2:0:10:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdj
[2:0:11:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdk
[2:0:12:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdl
[2:0:13:0]   disk    NETAPP   LUN C-Mode       9000  /dev/sdm
[2:0:14:0]   disk    VMware   Virtual disk     1.0   /dev/sdn
[2:0:15:0]   disk    VMware   Virtual disk     1.0   /dev/sdo
[3:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sdp
[3:0:1:0]    disk    VMware   Virtual disk     1.0   /dev/sdq

可以透過觀察上面二個資訊後，可以得到 OS_NATIVE_NAME 會對應到 lsscsi 中的 /dev/sd* 這樣就可以來做徹底清空 Disk Path

執行 echo 1 > /sys/class/scsi_device/2:0:9:0/device/delete，下面三種可以選一個自已喜歡的方式

echo 1 >  /sys/class/scsi_device/2:0:9:0/device/delete

or

echo 1 > /sys/block/sdi/device/delete

or 

echo "scsi remove-single-device 2 0 9 0" > /proc/scsi/scsi

請再次執行 lsscsi | grep sdi，請觀察資訊

請再次執行 $ cat /proc/scsi/scsi，請觀察資訊

$ cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 01 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 02 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 03 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 04 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 05 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 06 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 08 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 10 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 11 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 12 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 13 Lun: 00
  Vendor: NETAPP   Model: LUN C-Mode       Rev: 9000
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi2 Channel: 00 Id: 14 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi2 Channel: 00 Id: 15 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02
Host: scsi3 Channel: 00 Id: 01 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0
  Type:   Direct-Access                    ANSI  SCSI revision: 02

請再次執行 vxdisk -eo alldgs list | grep -Ev '^sd*'，請觀察資訊

$ vxdisk -eo alldgs list | grep -Ev '^sd*'
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
cl8080b_0_11 auto           -            -           error                sdi              tprclm
cl8080b_0_12 auto:cdsdisk   -            -           online clone_disk    sdk              tprclm
cl8080b_0_13 auto:cdsdisk   -            -           online clone_disk    sdm              tprclm
cl8080b_0_14 auto:cdsdisk   testdbdg12    testdbdg     online clone_disk    sdg              tprclm
cl8080b_0_15 auto:cdsdisk   testdbdg10    testdbdg     online clone_disk    sdc              tprclm
cl8080b_0_16 auto:cdsdisk   testdbdg11    testdbdg     online clone_disk    sde              tprclm
cl8080b_0_17 auto:cdsdisk   -            -           online clone_disk    sdl              tprclm
cl8080b_0_18 auto:cdsdisk   testdbdg15    testdbdg     online clone_disk    sdf              tprclm
cl8080b_0_19 auto:cdsdisk   -            -           online clone_disk    sdj              tprclm
cl8080b_0_20 auto:cdsdisk   -            -           online clone_disk    sdh              tprclm

依據上面的觀察得知，確認 /dev/sdi 已經被清除了，但執行 vxdisk -eo alldgs list | grep -Ev '^sd*' 還是會有 error，這個等等再來修正先把要刪除的 Disk Path 都給執行完，我會用不同的刪除方法執行

# echo 1 > /sys/block/sdk/device/delete
# lsscsi | grep sdk

# echo "scsi remove-single-device 2:0:13:0" > /proc/scsi/scsi
# lsscsi | grep sdm

請再次執行 lsscsi | grep sdi，請觀察資訊

請再次執行 $ cat /proc/scsi/scsi，請觀察資訊

以上執行完後，就可以了，最後就是要來修正執行 vxdisk -eo alldgs list | grep -Ev '^sd*' 還是會有 error的問題其實很簡單，只要執行 vxdisk scandisks 這樣就行了。

Referenece:

憤Ⓘ鬥Ⓣ屎

2019年1月28日星期一

回收 Veritas Storage Disk on RHEL 6.4

沒有留言:

張貼留言

2019年1月28日 星期一

回收 Veritas Storage Disk on RHEL 6.4

沒有留言:

張貼留言

2019年1月28日星期一