网络设备延迟，导致监控设备日志量大增。
01. 现象
今天上午准备登陆下测试环境的zabbix-server服务器查个东西，发现ssh连接不上，报"No space left on device"。
1[C:\~]$ ssh 172.16.131.142
2Last login: Fri Nov 1 11:28:19 2019 from 10.16.75.35
3/root/.pyenv/libexec/pyenv-init: line 131: cannot create temp file for here-document: No space left on device
于是使用ansible跳过去，查看磁盘空间发现根目录已经100%了。
1[root@ansible ~]# ssh 172.16.131.142
2[root@zabbix1 ~]# df -h
3Filesystem Size Used Avail Use% Mounted on
4/dev/vda1 50G 50G 0 100% /
5/dev/mapper/datavg-home_lv
6 343G 178G 148G 55% /home
7/dev/mapper/datavg-swap_lv
8 976M 490M 436M 53% /swap
因为之前遇到过类似情况，所以我猜想还是boot.log满了，去看一下,果然已经占了41个G。
1[root@zabbix1 ~]# cd /var/log/
2[root@zabbix1 log]# du -sh *
3 326M audit
4 441G boot.log
5 54.0K dmesg
6 64.0K dmesg.old
7 74.0K dracut.log
8 850M httpd
9 9824M messages
10104.0K tallylog
1111224K wtmp
12124.0K yum.log
131321M zabbix
查看日志里面的内容，日志在疯狂的写入，只截取部分。
1[root@zabbix1 log]# tail -f boot.log
2 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: RtrPriority 1"
3 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: RtrDeadInterval 12"
4 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: DRouter 0.0.0.0"
5 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: BDRouter 0.0.0.0"
6 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: # Neighbors 1"
7 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11:33:22 devname=BJ-YZ-CO-FW1 devid=FG5H0E5818903326 logid=0103020301 type=event subtype=router level=information vd=root logdesc="Routing log" msg="OSPF: Neighbor 172.16.44.18"
8 Nov 1 11:33:22 172.16.32.2 date=2019-11-01 time=11