日記/2017-08-26

linuxサーバのプロセス制御

CentOS上で動いているdplusの負荷変動（CPU負荷が100%に近づく）と他の影響で
dplusがハングアップ状態になることを防ぐ対策。※dplus だけでなく他にも応用可。

1. dplusのCPU負荷状態の記録を取る

   --- /root/logdplus.sh ---
   #!/bin/bash
   
   sleep 60
   while :
   do
   DT1=`date | cut -c12-20`
   #echo "$DT1"
   top -b -d20 -n1 | grep dplus | grep -v "logdplus" > hukadt.txt
   HUKA=`head -1 hukadt.txt | cut -c41-`
   echo "$DT1 $HUKA"
   echo "$DT1 $HUKA" >> hukadt0.txt
   sleep 10
   done

2. 記録を解析してdplusを再起動させる

   --- /root/dplreset.sh ---
   #!/bin/bash
   DPLF=`ps ax | grep dplus | grep JP1YJQ`
   if test "$DPLF" = ""; then
     /etc/init.d/dplus restart
     sleep 60
   fi
   
   while :
   do
   FUKA=`tail -1 /root/hukadt0.txt | cut -c11-13`
   #echo "$FUKA"
   FUKA=$((FUKA + 0))
   if test $FUKA -gt 20 ; then
     sleep 12
     FUKA=`tail -1 /root/hukadt0.txt | cut -c11-13`
     FUKA=$((FUKA + 0))
     if test $FUKA -gt 20 ; then
       echo "dplus is going to loop ?" >> /root/hukadt0.txt
       echo "dplus is going to loop ?"
       sleep 180
       FUKA=`tail -1 /root/hukadt0.txt | cut -c11-13`
       FUKA=$((FUKA + 0))
       if test $FUKA -gt 20 ; then
         echo "Dplus is stoping" >> /root/hukadt0.txt
         echo "Dplus is stoping" | /usr/local/bin/msend.sh
         /etc/init.d/dplus stop
         echo "waiting for 5 min" >> /root/hukadt0.txt
         echo "waiting for 5 min" | /usr/local/bin/msend.sh
         sleep 300
         echo "Dplus is starting" >> /root/hukadt0.txt
         echo "Dplus is starting" | /usr/local/bin/msend.sh
         /etc/init.d/dplus start
       else
         echo "No. Nothing to do" >> /root/hukadt0.txt
         echo "No. Nothing to do"
       fi
     fi
   fi
   sleep 13
   done

3. 結果（hungupしなかった場合）

   # tail -f /root/hukadt0.txt
   10:59:46 0.0 0.1 18:29.19 dplus
   10:59:56 0.0 0.1 18:29.20 dplus
   11:00:07 0.0 0.1 18:29.21 dplus
   11:00:17 0.0 0.1 18:29.21 dplus
   11:00:27 7.5 0.1 18:29.25 dplus
   11:00:38 84.8 0.1 18:38.76 dplus
   11:00:49 91.7 0.1 18:48.94 dplus
   dplus is going to loop ?
   11:00:59 72.8 0.1 18:57.89 dplus
   11:01:10 89.9 0.1 19:08.13 dplus
   11:01:20 58.5 0.1 19:17.11 dplus
   11:01:31 93.6 0.1 19:27.31 dplus
   11:01:41 82.9 0.1 19:36.26 dplus
   11:01:51 67.4 0.1 19:46.33 dplus
   11:02:02 44.7 0.1 19:55.21 dplus
   11:02:12 67.4 0.1 20:05.04 dplus
   11:02:23 71.5 0.1 20:14.29 dplus
   11:02:33 0.0 0.1 20:17.93 dplus
   11:02:43 0.0 0.1 20:17.93 dplus
   11:02:54 0.0 0.1 20:17.95 dplus
   11:03:04 3.7 0.1 20:17.96 dplus
   11:03:14 0.0 0.1 20:17.97 dplus
   11:03:25 0.0 0.1 20:17.97 dplus
   11:03:35 0.0 0.1 20:17.98 dplus
   11:03:45 0.0 0.1 20:17.98 dplus
   11:03:56 0.0 0.1 20:17.98 dplus
   No. Nothing to do
   11:04:06 0.0 0.1 20:17.98 dplus
   11:04:16 0.0 0.1 20:17.99 dplus

• Prev
• Next
• 日記