In the Spring of 2005 Comcast experienced a major DNS outage. Since then many Comcast users have switched to DNS servers that belong to Verizon and other ISPs. Comcast started taking a lot of flak from its competition upset by Comcast customers using their resources. Finaly, a year later Comcast networking gurus remedied the problem by introducing dynamic DNS. Or so they thought.

Since about late March of 2006 I’ve been having a lot of issues with my Comcast broadband. DNS was sluggish and the network was slow and unstable. Calling Comcast service department was a problem: I have Vonage VoIP that, naturally, uses my Comcast connection. Calling Comcast on my cell phone and spending my expensive Cingular minutes to listen to Michael Bolton while waiting for a service rep became very frustrating very quickly. So I turned to the good old US Postal Service.

Eventually, Comcast people contacted me and, as it turned out, they were well aware of widespread problems caused by their recent DNS changes. Still, they claimed that signal strength to my modem was good and the line was clean. Whatever that means. So I provided them with network performance data I collected over the past few days and theatened to file a BBB complaint. As if by miracle, a few hours later my network was fully functional.

Apparently, if you are a Comcast customer, simply paying your monthly bill is not enough to get service you’ve been promised. You also need to be a computer professional with a very big mouth. However, let’s not dwell on the unpleasant and instead consider the simple Shell script I used to collect and store network performance data produced by ping. MySQL was used to store collected data and GNUPlot was used to visualize it.

#!/bin/sh

# April 8, 2006

# This script uses ping to monitor quality of network connection to
# selected hosts and notifies the admin when packet loss ratio
# exceeds defined thresholds.

# PINGHOST      www.comcast.net 204.127.195.15
# PINGHOST      www.yahoo.com   216.109.112.135
# PINGHOST      www.google.com  64.233.187.99
# PINGHOST      www.mit.edu     18.7.22.83

MYSQL="/usr/bin/mysql"
DBUSER="dbuser"
DBPASS="password"
DBNAME="netmon"

LOST_PACKET_THRESHOLD=10
PACKET_SIZE=64
PACKET_COUNT=10
PING_INTERVAL=1
TIMEOUT=2
PING="/bin/ping"

GNUPLOT="/usr/bin/gnuplot"

cat /var/adm/bin/netmon.sh | grep PINGHOST | grep -v grep | while read LINE
do
        NAME=`echo "$LINE" | awk '{print $3}'`
        IP=`echo "$LINE" | awk '{print $4}'`

        $PING -s $PACKET_SIZE -c $PACKET_COUNT -i $PING_INTERVAL -W $TIMEOUT $IP | egrep "packet loss|rtt min" | while read LINE2
        do
                if [ `echo "$LINE2" | egrep -c "packet loss"` -eq 1 ]
                then
                        echo "$LINE2" | awk '{print $6}' | awk -F'%' '{print $1}' > /tmp/netmon_packet_loss.txt
                        echo "$LINE2" | awk '{print $1}' > /tmp/netmon_packets_sent.txt
                        echo "$LINE2" | awk '{print $4}' > /tmp/netmon_packets_received.txt
                elif [ `echo "$LINE2" | egrep -c "rtt min"` -eq 1 ]
                then
                        echo "$LINE2" | awk -F '/' '{print $5}' > /tmp/netmon_avg.txt
                        echo "$LINE2" | awk -F '/' '{print $6}' > /tmp/netmon_max.txt
                fi
        done

        if [ -f /tmp/netmon_packet_loss.txt ] &&  [ -f /tmp/netmon_avg.txt ] && [ -f /tmp/netmon_max.txt ] && [ -f /tmp/netmon_packets_sent.txt ] && [ -f /
tmp/netmon_packets_received.txt ]
        then
                DATE=`date +'%Y-%m-%d %T'`
                PACKETS_SENT=`cat /tmp/netmon_packets_sent.txt`
                PACKETS_RECEIVED=`cat /tmp/netmon_packets_received.txt`
                PACKET_LOSS=`cat /tmp/netmon_packet_loss.txt`
                AVG=`cat /tmp/netmon_avg.txt`
                MAX=`cat /tmp/netmon_max.txt`

                rm /tmp/netmon_packet_loss.txt /tmp/netmon_avg.txt /tmp/netmon_max.txt /tmp/netmon_packets_sent.txt /tmp/netmon_packets_received.txt

echo "$DATE, $NAME, $IP, $PACKET_SIZE, $PACKET_COUNT, $PING_INTERVAL, $PACKETS_SENT, $PACKETS_RECEIVED, $PACKET_LOSS, $AVG, $MAX"

$MYSQL -u$DBUSER -p$DBPASS $DBNAME < < EOF
INSERT INTO pinger (datetime,hostname,ip_address,packet_size,
packet_count,ping_interval,packet_sent,packet_received,
packet_lost,response_avg,response_max)
values('$DATE','$NAME','$IP','$PACKET_SIZE',
'$PACKET_COUNT','$PING_INTERVAL','$PACKETS_SENT',
'$PACKETS_RECEIVED','$PACKET_LOSS','$AVG','$MAX');
EOF

        fi
done

cat /var/adm/bin/netmon.sh | grep PINGHOST | grep -v grep | awk '{print $3}' | while read hostname
do
$MYSQL --column-names=0 -u$DBUSER -p$DBPASS $DBNAME < /tmp/netmon_gnuplot_${hostname}.dat
SELECT hostname, datetime, packet_lost, response_avg, response_max FROM pinger WHERE hostname LIKE '${hostname}' AND DATE_SUB(CURDATE(),INTERVAL 3 DAY) < =
datetime;
EOF

cat < /tmp/netmon_gnuplot_${hostname}.gnu
set title '$hostname'
set xdata time
set key box
set key bottom right
set size 1.5,1.5
set xlabel 'Date'
set ylabel 'Ping Response' font 'Arial,12'
set autoscale
set timefmt "%Y-%m-%d %H:%M:%S"
set term png color
set output '/WD120GB_01/htdocs/netmon/${hostname}_1.png'
plot '/tmp/netmon_gnuplot_${hostname}.dat' using 2:4 title 'Packet Loss (%)' with linespoints, 
'/tmp/netmon_gnuplot_${hostname}.dat' using 2:5 title 'Average Response (ms)' with linespoints, 
'/tmp/netmon_gnuplot_${hostname}.dat' using 2:6 title 'Maximum Response (ms)' with linespoints
EOF
        $GNUPLOT  /dev/null
done

cat <  /WD120GB_01/htdocs/netmon/index.htm

EOF

cat /var/adm/bin/netmon.sh | grep PINGHOST | grep -v grep | awk '{print $3}' | while read hostname
do
hostname2=`echo "$hostname" | sed 's/.//g'`
cat < > /WD120GB_01/htdocs/netmon/index.htm

<a href="#${hostname2}" target="_self" data-mce-href="#${hostname2}">${hostname}</a>

EOF
done

cat /var/adm/bin/netmon.sh | grep PINGHOST | grep -v grep | awk '{print $3}' | while read hostname
do
hostname2=`echo "$hostname" | sed 's/.//g'`
cat < > /WD120GB_01/htdocs/netmon/index.htm

<img src="https://www.krazyworks.com/wp-admin/${hostname}_1.png" alt="" width="960" height="720" data-mce-src="https://www.krazyworks.com/wp-admin/${hostname}_1.png"><a name="${hostname2}" class="mce-item-anchor"></a>

EOF
done

cat < > /WD120GB_01/htdocs/netmon/index.htm
EOF

rm /tmp/netmon_gnuplot*
chown -R wwwrun:www /WD120GB_01/htdocs/netmon
chmod -R 755 /WD120GB_01/htdocs/netmon

And the end result is a page on my Web server with several graphs showing network performance for the selected hosts. Here’s one of the graphs: