A Simple Process Monitoring Script

Submitted by Igor on February 22, 2012 – 11:28 am8 Comments

Let’s say you are running a data restore. Things are moving along, but network is congested and the backup server is overloaded. You really don’t feel like staring at the restore status for the next several hours and just want to be notified when the process completes. The simplest method of monitoring for processes starting or ending on Unix systems is by using “ps” inside a “while” loop.

In the example below we are monitoring a NetBackup restore running on a Linux box. A typical NetBackup restore process on a Unix system looks something like this:

root      5526     1  1 03:14 ?        00:02:57 tar -x -v -Y -p -P -I 1329909118 -U 0 -k -Q -J clnt_lc_messages=C -J clnt_lc_time=C -J clnt_lc_ctype=C -J clnt_lc_collate=C -J clnt_lc_numeric=C -J restoreid=3451740.001 -J job_total=3 -J client=hpc12prod.de.krazyworks.com -J requesting_client=nbs2.de.krazyworks.com -J browse_client=hpc12prod.de.krazyworks.com -J backup_time=1329452235 -L /usr/openv/netbackup/logs/user_ops/netbac/logs/jbp-21746329909118065321000000023-5eaqEQ.log -J spsrestoreoptions=0 -f - -J verbose=0 -J disallow_server_file_writes=0

root 5526 1 1 03:14 ? 00:02:57 tar -x -v -Y -p -P -I 1329909118 -U 0 -k -Q -J clnt_lc_messages=C -J clnt_lc_time=C -J clnt_lc_ctype=C -J clnt_lc_collate=C -J clnt_lc_numeric=C -J restoreid=3451740.001 -J job_total=3 -J client=hpc12prod.de.krazyworks.com -J requesting_client=nbs2.de.krazyworks.com -J browse_client=hpc12prod.de.krazyworks.com -J backup_time=1329452235 -L /usr/openv/netbackup/logs/user_ops/netbac/logs/jbp-21746329909118065321000000023-5eaqEQ.log -J spsrestoreoptions=0 -f - -J verbose=0 -J disallow_server_file_writes=0

So when we run “ps”, we need to “grep” for a unique string, such as “restoreid”. As long as there are processes on the system that match this string, the monitoring script will sleep for ten minutes and then it will check again. Once the last “restoreid” process disappears, the script will send an email to the sysadmin.

#!/bin/bash
# Notify admin when process is done

while [ `/bin/ps -ef | grep -c [r]estoreid` -gt 0 ]
do
	sleep 600
done
echo "Process finished on `hostname` as of `date +'%Y-%m-%d %H:%M'`" | mailx -s "Message from `hostname`" your_email@domain.com
exit 0

#!/bin/bash

# Notify admin when process is done

while [ `/bin/ps -ef | grep -c [r]estoreid` -gt 0 ]

sleep 600

done

echo "Process finished on `hostname` as of `date +'%Y-%m-%d %H:%M'`" | mailx -s "Message from `hostname`" your_email@domain.com

exit 0

Save the script as, for example, /root/process_monitor.bash and make it executable. Then you can start this process in the background using nohup:

nohup /root/process/monitor.bash /dev/null 2&gt;&amp;1 &amp;

1	nohup /root/process/monitor.bash /dev/null 2>&1 &

Simple scripts like this can save you a lot of time and aggravation in the long run. System administration is all about automation and when it comes to automatic tasks, Unix knows no equals.

happyha31

R they copied from memory to DISK?
Plz tell me where they are copied? WHERE?????
Mark

Is it a better field compared with programming or not
mal_functiongeo

Well, I simply mean that we all have some skills like Horoscope reading, Computer Software Programming, Web Designing, Painting & Drawing, Music, etc.

So, I thought it would be great if we could know one another’s hobbies, interests and soft skills and we needed we could share them amongst us. What do you say?

Let me tell you about myself here:

Job: I am working in a Data Center Operations department and my job involves monitoring network (WANs) and servers which are spread across the world. I work on Veritas Netbackup, HP-OVO, Lotus Notes Client, Remedy, and on some other online monitoring tools.

Skills: Software Programming and Web Designing & Development. I know C, C++, HTML, JavaScript, MS -Frontpage, and I just love Macromedia Flash and its ActionScript. I create animations with Flash.
Interests / Hobbies: Music – I can’t live without it! I listen to soft and slow songs, instrumental music, love songs and blues. I want to learn drawing. I can’t ever give an expression to the figures that take their forms in my imagination. I like to learn foreign languages. But I am well versed in English and these days I am learning Tagalog. My mother tongue is Hindi. I also want to learn how to play the guitar and the piano or keyboard. Well, there are lots of things which I want to do and learn. But what I generally do is that I live and learn!
Hi TabbyD!

Thanks for your post. Nice to meet you too! Wow! You’re, in fact, enjoying your life to its fullest!

I also like traveling. I have run away from my home many times! Hahah!!! The last time I ran away to a mysterious place which is well known for magic, sorcery and such supernatural things. It was so interesting, adventurous, brainstorming and thrilling too!
Hi Mattie!

I have my best friend in Missouri. I wish to see her some day. I’ve not heard from her for some days. She was not well.
Hi Myra!

I used to play volleyball when I was in school. But they would ask me to stand near the net to catch the ball. I was not good at any game and I don’t know and play any game yet.
rashest_hippo

What is the average salary of Storage professionals in Australia? Storage means – SAN, TSM, NetBackup, Omnibackup. This comes in skill shortage in AU, therefore is the salary will be coming down in next couple of years as what had happened in USA?
Ed D

something is wrong with my computer burner and i cant burn pictures and video to cds anymore. i have a feeling that i am going to need a new laptop soon.
my question is can i put all my pictures and videos somewhere online and get to them on another computer so that i can download from there. i was trying with webshots.com i can get to the pictures and save them to the computer but not the video.
is there a site that will let me put my video back on the computer so i can burn it on to a cd. and if so which site?

thank you all very much this is very important to me. my sons first year of life is on this laptop. please help
thank you for all the help im going to get a flash drive. thanks again im completely retarded when it comes to computers
Dom L

Is there a site I can go to or an easy way to assign a value in a batch file? Reason why I use a program (veritas netbackup) and I basically want it to ask for a number, assign that a value, then input that into a command. For instance I want it to ask me for my tape number then i input g00001, then i want it to automatically put it into command line like this “bpexpdate -m g00001 -d 11/9/2008″ and run it.
Jerosh Nagulachandran

If you could point me to some online resources that would be good too. many thanks
Matthew David

i have a job to do for a friend, basically he wants me to back up his system to a HDD other than one connected to his work server. i want to back it up to a HDD at his home. ive had a look at some options online, they say to pay x amount/month for server storage, is there a way of doing it without the need of online servers, simply like FTP it to a HDD at home?

Thanks for any advice!