2017年7月17日 星期一

Cisco ASA - paramiko is required but does not appear to be installed

最近在試 Cisco ASA 5512X 這個型號,就想說能不能用 Ansible 來管設定,所以就先從簡單的 show version 的指令開始測試。
OS / ENVIRONMENT
macOS Sierra
Version 10.12.5
ANSIBLE VERSION
ansible 2.3.1.0
  config file =
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Apr  4 2017, 08:46:44) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
ANSIBLE CONFIGURATION
ansible inventory - hosts
asa           ansible_host=192.168.1.1   
---
- hosts: asa
  connection: local
  gather_facts: yes
  tasks:
    - name: run show version on remoter devices
      asa_command:
        provider:
          host: IP
          username: USERNAME
          password: PASSWORD
          authorize: yes
          auth_pass: PASSWORD
          timeout: 30
        commands:
          - show version
      register: version
    - name: show version
      debug:
        var: version.stdout_lines
      with_items: version.results

STEPS TO REPRODUCE

ansible-playbook -i hosts asa.yml -vvv
No config file found; using defaults

PLAYBOOK: asa.yml *************************************************************************************************
1 plays in asa.yml

PLAY [cisco] ******************************************************************************************************
META: ran handlers

TASK [Show Version] ***********************************************************************************************
task path: /Users/daniel/work/tenmax/lab/asa.yml:6
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/network/asa/asa_command.py
 ESTABLISH LOCAL CONNECTION FOR USER: daniel
 EXEC /bin/sh -c 'echo ~ && sleep 0'
 EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700 `" && echo ansible-tmp-1500340664.8-207253128376700="` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700 `" ) && sleep 0'
 PUT /var/folders/5w/tkr8l6gn7zs76_8n1z6jg_r80000gn/T/tmpMpIKV3 TO /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700/asa_command.py
 EXEC /bin/sh -c 'chmod u+x /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700/ /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700/asa_command.py && sleep 0'
 EXEC /bin/sh -c '/usr/bin/python /Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700/asa_command.py; rm -rf "/Users/daniel/.ansible/tmp/ansible-tmp-1500340664.8-207253128376700/" > /dev/null 2>&1 && sleep 0'
fatal: [cisco]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "auth_pass": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "authorize": true,
            "commands": [
                "show version"
            ],
            "context": null,
            "host": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "interval": 1,
            "match": "all",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": null,
            "provider": {
                "auth_pass": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "authorize": true,
                "host": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER"
            },
            "retries": 10,
            "ssh_keyfile": null,
            "timeout": 10,
            "transport": null,
            "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "wait_for": null
        }
    },
    "msg": "paramiko is required but does not appear to be installed.  It can be installed using  `pip install paramiko`"
}
    to retry, use: --limit @/Users/daniel/work/tenmax/lab/asa.retry

PLAY RECAP ********************************************************************************************************
cisco                      : ok=0    changed=0    unreachable=0    failed=1
執行後,會看到有錯誤 
"msg": "paramiko is required but does not appear to be installed.  It can be installed using  `pip install paramiko`"
實際上,我已經裝了,後來詢問谷歌大神後,發現會出現這樣的問題原因是,有可能你的作業系統裡,有安裝多個不同的版本的 Python ,所以要解決這個問題的話,就是去指定你的Pythonn解釋器,目前ansible都還是以Python2.X版的為主,接下來在 inventory file裡面去新增 ansible_python_interpreter,這樣就解決了。
asa           ansible_host=192.168.1.1     ansible_python_interpreter=/usr/bin/python 
Reference:

Cisco ASA timeout trying to send command: enable

Same issue here with asa_command

SUMMARY

簡單地說,就是目前還不能用,我已經有發issue給ansible了,詳請可以參考下面的鏈結
ISSUE TYPE
  • Bug Report
COMPONENT NAME
  • asa_command
ANSIBLE VERSION
ansible 2.3.1.0
  config file =
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Apr  4 2017, 08:46:44) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
CONFIGURATION
hostsasa ansible_host=192.168.1.1 ansible_python_interpreter=/usr/bin/python 
---
- hosts: asa
  connection: local
  gather_facts: yes
  tasks:
    - name: run show version on remoter devices
      asa_command:
        provider:
          host: IP
          username: USERNAME
          password: PASSWORD
          authorize: yes
          auth_pass: PASSWORD
          timeout: 30
        commands:
          - show version
      register: version
    - name: show version
      debug:
        var: version.stdout_lines
      with_items: version.results
OS / ENVIRONMENT
Darwin Danielde-MacBook-Pro.local 15.6.0 Darwin Kernel Version 15.6.0: Tue Apr 11 16:00:51 PDT 2017; root:xnu-3248.60.11.5.3~1/RELEASE_X86_64 x86_64
STEPS TO REPRODUCE
$ ansible-playbook -i hosts asa.yml -vvv
No config file found; using defaults

PLAYBOOK: asa.yml *************************************************************************************************************
1 plays in asa.yml

PLAY [asa] ********************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/system/setup.py
<192 .168.1.1=""> ESTABLISH LOCAL CONNECTION FOR USER: daniel
<192 .168.1.1=""> EXEC /bin/sh -c 'echo ~ && sleep 0'
<192 .168.1.1=""> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260 `" && echo ansible-tmp-1500304102.91-122764280426260="` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260 `" ) && sleep 0'
<192 .168.1.1=""> PUT /var/folders/03/ns9kvh215g12x_h6gyq7f7tr0000gn/T/tmpGAqiRu TO /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260/setup.py
<192 .168.1.1=""> EXEC /bin/sh -c 'chmod u+x /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260/ /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260/setup.py && sleep 0'
<192 .168.1.1=""> EXEC /bin/sh -c '/usr/local/bin/python /Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260/setup.py; rm -rf "/Users/daniel/.ansible/tmp/ansible-tmp-1500304102.91-122764280426260/" > /dev/null 2>&1 && sleep 0'
ok: [asa]
META: ran handlers

TASK [run show version on remoter devices] ************************************************************************************
task path: /Users/daniel/work/lab/asa.yml:6
Using module file /usr/local/lib/python2.7/site-packages/ansible/modules/network/asa/asa_command.py
<192 .168.1.1=""> ESTABLISH LOCAL CONNECTION FOR USER: daniel
<192 .168.1.1=""> EXEC /bin/sh -c 'echo ~ && sleep 0'
<192 .168.1.1=""> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732 `" && echo ansible-tmp-1500304104.0-136124011056732="` echo /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732 `" ) && sleep 0'
<192 .168.1.1=""> PUT /var/folders/03/ns9kvh215g12x_h6gyq7f7tr0000gn/T/tmpdXRCx3 TO /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732/asa_command.py
<192 .168.1.1=""> EXEC /bin/sh -c 'chmod u+x /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732/ /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732/asa_command.py && sleep 0'
<192 .168.1.1=""> EXEC /bin/sh -c '/usr/local/bin/python /Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732/asa_command.py; rm -rf "/Users/daniel/.ansible/tmp/ansible-tmp-1500304104.0-136124011056732/" > /dev/null 2>&1 && sleep 0'
fatal: [asa]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "auth_pass": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "authorize": true,
            "commands": [
                "show version"
            ],
            "context": null,
            "host": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "interval": 1,
            "match": "all",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": null,
            "provider": {
                "auth_pass": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "authorize": true,
                "host": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "timeout": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER"
            },
            "retries": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "ssh_keyfile": null,
            "timeout": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "transport": null,
            "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "wait_for": null
        }
    },
    "msg": "timeout trying to send command: enable"
}
    to retry, use: --limit @/Users/daniel/work/lab/asa.retry

PLAY RECAP ********************************************************************************************************************
asa                        : ok=1    changed=0    unreachable=0    failed=1
How to fix this issue ?
Reference:

Cisco ASA 5512 設定 DHCP & NAT

顯示目前的interface狀態
ciscoasa# conf t
ciscoasa(config)# show int ip bri
找一個interface當做vlan接口
ciscoasa(config)# int gigabitEthernet 0/5
ciscoasa(config-if)# no shutdown
ciscoasa(config-if)# exit
設定 IEEE 802.1Q VLAN Identifier
ciscoasa(config)# interface gigabitEthernet 0/5.200
ciscoasa(config-subif)# vlan 200
ciscoasa(config-subif)# nameif Corp_LAN
ciscoasa(config-subif)# security-level 100
ciscoasa(config-subif)# ip address 192.168.200.254 255.255.255.0
ciscoasa(config-subif)# exit
設定DHCP Server
ciscoasa(config)# dhcpd address 10.10.200.1-10.10.200.250 Corp_LAN
ciscoasa(config)# dhcpd dns 168.95.1.1 8.8.8.8
ciscoasa(config)# dhcpd lease lease_length                      # 預設是3600sec,0 to 1,048,575
ciscoasa(config)# dhcpd domain domain_name
ciscoasa(config)# dhcpd option 3 ip 10.10.200.254(gateway_ip)   # dhcpd option 3 這是定義dhcp client的default gateway
                                                                # 如果不用的話,就會被改用management interface
                                                                # 這interface是不包含DHCP ACK這個選項的
                                                                # 所以不會去幫你做轉送。
ciscoasa(config)# dhcpd enable Corp_LAN                         # 啟用dhcp server
顯示目前的vlan
ciscoasa(config)# show vlan
顯示目前DHCP Server的設定
ciscoasa(config)# show running-config dhcpd
dhcpd option 3 ip 10.10.200.254
!
dhcpd address 10.10.200.1-10.10.200.250 Corp_LAN
dhcpd dns 168.95.1.1 8.8.8.8 interface Corp_LAN
dhcpd enable Corp_LAN
設定ACL - 讓DHCP Client可以正常ping internet
ciscoasa(config)# access-list ACL_OUT permit icmp any any
新增 global_policy
ciscoasa(config)# policy-map global_policy
ciscoasa(config-pmap)# class inspection_default
ciscoasa(config-pmap-c)# inspect icmp 
ciscoasa(config-pmap-c)# inspect icmp error        # Enable NAT on icmp error messages
Reference:

2017年7月14日 星期五

Cisco Router 設定 SSH 連線

設定router只允許ssh連線,並使用本地端認証
Test-RTR#conf t
Test-RTR(config)#line vty 0 4
Test-RTR(config)#login local
Test-RTR(config)#transport input ssh
設定Domain Name
Test-RTR(config)#ip domain-name xxx
Cisco Router 產生一把rsa的key
Test-RTR(config)#crypto key generate rsa
The name for the keys will be: Test-RTR.xxx
Choose the size of the key modulus in the range of 360 to 4096 for your
  General Purpose Keys. Choosing a key modulus greater than 512 may take
  a few minutes.

How many bits in the modulus [512]: 4096
% Generating 4096 bit RSA keys, keys will be non-exportable...
設定SSH Version
Test-RTR(config)#ip ssh version 2
建立使用者及密碼,並做密碼加密
Test-RTR(config)#username admin password 1234567890
Test-RTR(config)#service password-encryption
設定登入特權模式密碼
Test-RTR(config)#enable secret 1234567890
這樣就可以測試登入了
Reference:

2017年7月13日 星期四

Cisco ASA 5512 設定及備份

 備份
  • Cisco Router 透過usb備份的話,是用 copy running-config usbflash1: 
  • Cisco ASA 透過usb備份的話,是用 copy running-config disk1: 
還原出廠預設值
ciscoasa#conf t
ciscoasa(config)#configure factory-default
ciscoasa(config)# reload
System config has been modified. Save? [Y]es/[N]o:  y
Cryptochecksum: cbae301c 8cea4e21 022b5873 ec551723

2709 bytes copied in 0.780 secs
Proceed with reload? [confirm]
ciscoasa(config)#


***
*** --- START GRACEFUL SHUTDOWN ---
Shutting down isakmp
Shutting down webvpn
Shutting down sw-module
第一次進入特權模式會提醒
***************************** NOTICE *****************************

Help to improve the ASA platform by enabling anonymous reporting,
which allows Cisco to securely receive minimal error and health
information from the device. To learn more about this feature,
please visit: http://www.cisco.com/go/smartcall

Would you like to enable anonymous error reporting to help improve
the product? [Y]es, [N]o, [A]sk later: A

No input detected, recording Ask later.
You will be reminded again in 7 days.

If you would like to enable this feature, issue the command
"call-home reporting anonymous".
設定Cisco ASA 機器名稱
ciscoasa(config)# hostname ASA  #字數要小於64
顯示目前interface
#show interface ip brief
設定WAN IP
ciscoasa# conf t
ciscoasa(config)# interface gigabitEthernet 0/0
ciscoasa(config-if)# ip address A.B.C.D 255.255.255.0           #設定WAN IP
ciscoasa(config-if)# nameif outside                             #設定路由會用到
ciscoasa(config-if)# no shutdown
設定路由
ciscoasa(config)# route outside 0.0.0.0 0.0.0.0 Gateway
設定ASA可以用SSH登入
ciscoasa(config)# enable password Password
ciscoasa(config)# username adminstrator password 1234567890
ciscoasa(config)# ssh External_IP 255.255.255.255 outside
ciscoasa(config)# ssh version 2
ciscoasa(config)# aaa authentication ssh console LOCAL
ciscoasa(config)# crypto key generate rsa modulus 4096
WARNING: You have a RSA keypair already defined named .

Do you really want to replace them? [yes/no]: yes
Keypair generation process begin. Please wait...
在MAC裡,需要設定用ssh 指令可以登入, ~/.ssh/config
HOST asa
    HostName FQDN or IP
    User administrator
    KexAlgorithms +diffie-hellman-group1-sha1
or
ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 IP
Reference:

2017年7月7日 星期五

透過 Ansible 來執行 Cisco Router 的指令 (簡易版)

OS Environment

macOS Sierra
version 10.12.5

Ansible Version

ansible 2.3.1.0
  config file =
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Dec 18 2016, 07:03:39) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]

cisco.yml

---
- hosts: cisco
  connection: local
  gather_facts: yes
  tasks:
    - name: run show version on remoter devices
      ios_command:
        provider:
          host: FQDN or IP
          username: 使用者
          password: ssh的連線密碼
          authorize: yes
          auth_pass: 特權模式密碼
        commands:
          - show version
      register: version
    - name: show version
      debug:
        var: version.stdout_lines
      with_items: version.results
authorize 跟 auth_pass 要一起使用,因為這是要進入 特權模式
重點還是要看官方的用法,下面只是參考而已,有些寫法未來可能會被拔掉
這是我執行某一個寫法的時候吐的訊息
 [WARNING]: argument username has been deprecated and will be removed in a future version

 [WARNING]: argument host has been deprecated and will be removed in a future version

 [WARNING]: argument password has been deprecated and will be removed in a future version
Reference:

2017年6月5日 星期一

理解 httping.pl

 httping.pl

這個是參考 /usr/share/cacti/site/scripts/ping.pl 修改的,如果需要用的話,別忘記要安裝 httping 這個套件。
#!/usr/bin/perl

$url = $ARGV[0];
$url =~ s/tcp:/$1/gis;

open(PROCESS, "httping -c 1 $url | grep 'seq' | grep time |");
$ping = ;
close(PROCESS);
$ping =~ m/(.*time=)(.*) (ms|usec)/;


if ($2 == "") {
    print "U";      # avoid cacti errors, but do not fake rrdtool stats
}elsif ($3 eq "usec") {
    print $2/1000;  # re-calculate in units of "ms"
}else{
    print $2;
}
$- 当前页可打印的行数,属于Perl格式系统的一部分
$! 根据上下文内容返回错误号或者错误串
$” 列表分隔符
$# 打印数字时默认的数字输出格式
$$ Perl解释器的进程ID
$% 当前输出通道的当前页号
$& 与上个格式匹配的字符串
$( 当前进程的组ID
$) 当前进程的有效组ID
$* 设置1表示处理多行格式.现在多以/s和/m修饰符取代之.
$, 当前输出字段分隔符
$. 上次阅读的文件的当前输入行号
$/ 当前输入记录分隔符,默认情况是新行
$: 字符设置,此后的字符串将被分开,以填充连续的字段.
$; 在仿真多维数组时使用的分隔符.
$? 返回上一个外部命令的状态
$@ Perl解释器从eval语句返回的错误消息
$[ 数组中第一个元素的索引号
$\ 当前输出记录的分隔符
$] Perl解释器的子版本号
$^ 当前通道最上面的页面输出格式名字
$^A 打印前用于保存格式化数据的变量
$^D 调试标志的值
$^E 在非UNIX环境中的操作系统扩展错误信息
$^F 最大的文件捆述符数值
$^H 由编译器激活的语法检查状态
$^I 内置控制编辑器的值
$^L 发送到输出通道的走纸换页符
$^M 备用内存池的大小
$^O 操作系统名
$^P 指定当前调试值的内部变量
$^R 正则表达式块的上次求值结果
$^S 当前解释器状态
$^T 从新世纪开始算起,脚步本以秒计算的开始运行的时间
$^W 警告开关的当前值
$^X Perl二进制可执行代码的名字
$_ 默认的输入/输出和格式匹配空间
$| 控制对当前选择的输出文件句柄的缓冲
$~ 当前报告格式的名字
$` 在上个格式匹配信息前的字符串
$’ 在上个格式匹配信息后的字符串
$+ 与上个正则表达式搜索格式匹配的最后一个括号
$< 当前执行解释器的用户的真实ID
$ 含有与上个匹配正则表达式对应括号结果
$= 当前页面可打印行的数目
$> 当前进程的有效用户ID
$0 包含正在执行的脚本的文件名
$ARGV 从默认的文件句柄中读取时的当前文件名
%ENV 环境变量列表
%INC 通过do或require包含的文件列表
%SIG 信号列表及其处理方式
@_ 传给子程序的参数列表
@ARGV 传给脚本的命令行参数列表
@INC 在导入模块时需要搜索的目录列表
$-[0]和$+[0] 代表当前匹配的正则表达式在被匹配的字符串中的起始和终止的位置
Reference

2017年5月27日 星期六

Cacti 使用指令來建立流量圖

一開始本來都是使用WEB UI去建立流量圖,但由於新增的是網路設備,覺得用WEB UI太慢了,而且眼睛也會看到快脫窗了,所以就想說沒有沒指令可以下,就可以加圖片了,果真還真的有指令可以用,主要是拿來建來樹狀結構中的圖片。
所有cacti可以用的指令都放在這個目錄底下,/usr/share/cacti/cli
查詢目前有哪些裝置
php -q add_tree.php --list-hosts
查詢目前的樹狀結構
php -q add_tree.php --list-trees
查詢目前裝置上有哪些圖
php -q add_tree.php --list-graphs --host-id=9
查詢這個樹狀結構中的某一個節點,裡面可以找到 parentid
php -q add_tree.php --list-nodes --tree-id=5
查詢目前樹狀結構中,有建立了哪些header,也就是所謂的子目錄,header的id也是等於parentid
php -q add_tree.php --type=tree  --node-type=header --list-nodes --tree-id=5
把流量圖新增到graph trees裡面
php -q add_tree.php --type=node --node-type=graph --tree-id=5 --parent-node=82 --graph-id=184--rra-id=1
Reference: 

2017年5月26日 星期五

修復 Cacti Server 的 MySQL 資料庫


剛建立cacti時,本來都可以正常畫圖,後來我也不知道我幹了什麼事,DB就壞了,翻log就看到這段錯誤
2017-05-24 16:55:31 603 [ERROR] /usr/sbin/mysqld: Table './cacti/graph_template_input_defs' is marked as crashed and should be repaired
2017-05-24 16:55:31 603 [Warning] Checking table:   './cacti/graph_template_input_defs'
2017-05-24 16:55:31 603 [ERROR] /usr/sbin/mysqld: Table './cacti/host_template_graph' is marked as crashed and should be repaired
2017-05-24 16:55:31 603 [Warning] Checking table:   './cacti/host_template_graph'
2017-05-24 16:55:31 603 [ERROR] /usr/sbin/mysqld: Table './cacti/host_template_snmp_query' is marked as crashed and should be repaired
2017-05-24 16:55:31 603 [Warning] Checking table:   './cacti/host_template_snmp_query'
2017-05-24 16:55:31 603 [ERROR] /usr/sbin/mysqld: Table './cacti/plugin_db_changes' is marked as crashed and should be repaired
2017-05-24 16:55:31 603 [Warning] Checking table:   './cacti/plugin_db_changes'
2017-05-24 16:55:31 603 [ERROR] /usr/sbin/mysqld: Table './cacti/weathermap_groups' is marked as crashed and should be repaired
2017-05-24 16:55:31 603 [Warning] Checking table:   './cacti/weathermap_groups'
解法:
第一種: php /usr/share/cacti/cli/repair_database.php (我沒用)
第二種: mysqlcheck -c -u root -p --all-databases
這樣它就好了。
Reference: 

2017年5月16日 星期二

Proxmox - lv的空間異常

Proxmox Version
Proxmox 4.3 (running kernel: 4.4.19-1-pve)
事由
由於在Proxmox機器上裝了一顆500GB和二顆1TB的HDD,一開始我以為裝機系統時,會幫我把lvm的事情都給搞定,後來幸好同事有發現硬碟空間不夠了,才得知原來Proxmox只有分配到500GB硬碟空間,才導致 /dev/pve/data 這個空間不夠用了,感謝同事幫我發現這個問題。
解法方法
Step1: 先用 pvdisplay 確認實體硬碟的容量跟 VG NAME ,確認所有硬碟都有加入到 VG NAME 這個群組。
# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               pve
  PV Size               465.64 GiB / not usable 4.01 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              119202
  Free PE               4052
  Allocated PE          115150
  PV UUID               Jnvsaq-Ys7Y-snQZ-1NbF-2xzb-P0jo-wnf0qU

  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               pve
  PV Size               931.51 GiB / not usable 4.71 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              238466
  Free PE               238466
  Allocated PE          0
  PV UUID               1C3Bjt-a1Rd-weDb-jL70-5qRy-CGU7-LwpOBJ

  --- Physical volume ---
  PV Name               /dev/sdc1
  VG Name               pve
  PV Size               931.51 GiB / not usable 4.71 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              238466
  Free PE               238466
  Allocated PE          0
  PV UUID               xFO4Be-O0ZT-CHcV-hZXK-mwfj-sNiG-TP5p3c
Step2: 接著用 lsblk 來確認目前硬碟的使用狀況,很明顯可以發現,只有 sda 有被使用,其他二顆 sdb1 跟 sdc1 都是沒有被使用的,而且 pve-data_tmeta 跟 pve-data_tdata 容量都是一樣的。
# lsblk
NAME                           MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                              8:0    0 465.8G  0 disk 
├─sda1                           8:1    0  1007K  0 part 
├─sda2                           8:2    0   127M  0 part 
└─sda3                           8:3    0 465.7G  0 part 
  ├─pve-root                   251:0    0 116.3G  0 lvm  /
  ├─pve-swap                   251:1    0     8G  0 lvm  [SWAP]
  ├─pve-data_tmeta             251:2    0    84M  0 lvm  
  │ └─pve-data-tpool           251:4    0 325.4G  0 lvm  
  │   ├─pve-data               251:5    0 325.4G  0 lvm  
  │   ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm  
  │   ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm  
  │   ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm  
  │   ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm  
  │   ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm  
  │   ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm  
  │   ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm  
  │   ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm  
  │   └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm  
  └─pve-data_tdata             251:3    0 325.4G  0 lvm  
    └─pve-data-tpool           251:4    0 325.4G  0 lvm  
      ├─pve-data               251:5    0 325.4G  0 lvm  
      ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm  
      ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm  
      ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm  
      ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm  
      ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm  
      ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm  
      ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm  
      ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm  
      └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm  
sdb                              8:16   0 931.5G  0 disk 
└─sdb1                           8:17   0 931.5G  0 part 
sdc                              8:32   0 931.5G  0 disk 
└─sdc1                           8:33   0 931.5G  0 part 
sr0                             11:0    1  1024M  0 rom 
Step3: 再用 vgdisplay 觀察後,發現 VG Size有 2.27 TiB ,但可用的卻只有Alloc PE / Size 115150 / 449.80 GiB , 而 Free PE / Size 480984 / 1.83 TiB 則是代表還有1.83 TiB可以使用。
# vgdisplay
  --- Volume group ---
  VG Name               pve
  System ID
  Format                lvm2
  Metadata Areas        3
  Metadata Sequence No  85
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                19
  Open LV               11
  Max PV                0
  Cur PV                3
  Act PV                3
  VG Size               2.27 TiB
  PE Size               4.00 MiB
  Total PE              596134
  Alloc PE / Size       115150 / 449.80 GiB
  Free  PE / Size       480984 / 1.83 TiB
  VG UUID               0AyuWc-Dhjx-5x4H-35O0-kG0r-pZbz-I2c6Q9
Step4: 用 lvextend 把 /dev/pve/data 做擴展
# lvextend -L+1T /dev/pve/data
  Size of logical volume pve/data_tdata changed from 455.39 GiB (116580 extents) to 1.44 TiB (378724 extents).
  Logical volume data successfully resized
Step5: 用 vgdisplay 和 lsblk 去檢查看是不是真的有做擴展
# vgdisplay
  --- Volume group ---
  VG Name               pve
  System ID
  Format                lvm2
  Metadata Areas        3
  Metadata Sequence No  90
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                19
  Open LV               11
  Max PV                0
  Cur PV                3
  Act PV                3
  VG Size               2.27 TiB
  PE Size               4.00 MiB
  Total PE              596134
  Alloc PE / Size       410574 / 1.57 TiB
  Free  PE / Size       185560 / 724.84 GiB
  VG UUID               0AyuWc-Dhjx-5x4H-35O0-kG0r-pZbz-I2c6Q9
# lsblk
NAME                           MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                              8:0    0 465.8G  0 disk
├─sda1                           8:1    0  1007K  0 part
├─sda2                           8:2    0   127M  0 part
└─sda3                           8:3    0 465.7G  0 part
  ├─pve-root                   251:0    0 116.3G  0 lvm  /
  ├─pve-swap                   251:1    0     8G  0 lvm  [SWAP]
  ├─pve-data_tmeta             251:2    0    84M  0 lvm
  │ └─pve-data-tpool           251:4    0   1.5T  0 lvm
  │   ├─pve-data               251:5    0 325.4G  0 lvm
  │   ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm
  │   ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm
  │   ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm
  │   ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm
  │   ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm
  │   ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm
  │   ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm
  │   ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm
  │   └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm
  └─pve-data_tdata             251:3    0   1.5T  0 lvm
    └─pve-data-tpool           251:4    0   1.5T  0 lvm
      ├─pve-data               251:5    0 325.4G  0 lvm
      ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm
      ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm
      ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm
      ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm
      ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm
      ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm
      ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm
      ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm
      └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm
sdb                              8:16   0 931.5G  0 disk
└─sdb1                           8:17   0 931.5G  0 part
  └─pve-data_tdata             251:3    0   1.5T  0 lvm
    └─pve-data-tpool           251:4    0   1.5T  0 lvm
      ├─pve-data               251:5    0 325.4G  0 lvm
      ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm
      ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm
      ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm
      ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm
      ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm
      ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm
      ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm
      ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm
      └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm
sdc                              8:32   0 931.5G  0 disk
└─sdc1                           8:33   0 931.5G  0 part
  └─pve-data_tdata             251:3    0   1.5T  0 lvm
    └─pve-data-tpool           251:4    0   1.5T  0 lvm
      ├─pve-data               251:5    0 325.4G  0 lvm
      ├─pve-vm--10003--disk--1 251:6    0    60G  0 lvm
      ├─pve-vm--10000--disk--1 251:7    0    60G  0 lvm
      ├─pve-vm--10002--disk--1 251:8    0    60G  0 lvm
      ├─pve-vm--10002--disk--2 251:9    0   100G  0 lvm
      ├─pve-vm--10001--disk--1 251:10   0    60G  0 lvm
      ├─pve-vm--10001--disk--2 251:11   0   100G  0 lvm
      ├─pve-vm--10004--disk--1 251:12   0    40G  0 lvm
      ├─pve-vm--10004--disk--2 251:13   0    50G  0 lvm
      └─pve-vm--20002--disk--1 251:14   0    70G  0 lvm
sr0                             11:0    1  1024M  0 rom
觀察發現 Alloc PE / Size 410574 / 1.57 TiB 變大了,且 sdb1 跟 sdc1都已經有資料了。
Reference: