Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Commands & Shells

Testing SSH Connectivity to Multiple Servers

Submitted by on December 27, 2011 – 2:46 am 8 Comments

Imagine a hypothetical scenario: you support hundreds of remote servers and you need to check which server you can access via SSH and which servers are not letting you log in. Doing this manually is a tedious process that many sysadmins choose to skip. The inevitable outcome is inability to quickly access a system when it really counts. After running into this problem on more than one occasion, I decided to spend an hour to write a very simple script that will work against a list of servers and perform the following tasks:

1. Check DNS for fully-qualified domain names (FQDNs) and IP addresses of your servers
2. Check if the servers are using SafeWord authentication for SSH
3. Check if you have passwordless SSH access to the servers
4. Try multiple passwords to test SSH access

The last step is particularly useful if local accounts are used for SSH access to some servers. Whenever local accounts come into play, there is a good possibility of multiple passwords on different systems.

Using the script

Create a list of hostnames to check, one per line. Update the “hostlist” variable in the “configure” function below to point to the list’s location. Update “username” variable below to show your local/LDAP username. Scroll down to line “for password in ‘password1’ ‘password2’ ‘password3′” and insert your all known local and LDAP passwords for the specified $username. If passwords use special characters, use escape sequences. For example, if the password is ‘big$money’, replace it with ‘big$money’.

The output is of the form:
hostname,dns_status,fqdn,ip,login_type,password_type,password_status

The script is slow and inefficient, written on two cups of coffee at 3am, but it works. It is slow, so you may want to run it overnight if you have a large list of servers. The csv output can be imported into a spreadsheet app.

NOTE: Because the script and its output contain your actual passwords, don’t keep them sitting around. Chmod the script 700 and the output will be 600. Delete them when done. Clear shell history and terminal buffer. It is easy to rewrite this script to read passwords as command-line arguments for better security, but I am feeling lazy today…

#!/bin/bash
#
# www.krazyworks.com
# 2011-11-04
#
# Use this script to check SSH access status to listed servers
#
# WORKFLOW:
# -------------------------------
#
# 1. Use DNS to obtain fully-qualified domain names (FQDNs) and IP addresses of all server in your list
# 2. Check if the servers are using SafeWord authentication for SSH
# 3. Check if you have passwordless SSH access to the servers
# 4. Try multiple passwords to test SSH access
#
# INSTRUCTIONS:
# -------------------------------
#
# Create a list of hostnames to check, one per line. Update the "hostlist" variable in the "configure"
# function below to point to the list's location. Update "username" variable below to show your local/LDAP
# username. Scroll down to line "for password in 'password1' 'password2' 'password3'" and insert your all known
# local and LDAP passwords for the specified $username. If passwords use special characters, use escape sequences.
# For example, if the password is 'big$money', replace it with 'big$money'.
#
# The output is of the form:
# hostname,dns_status,fqdn,ip,login_type,password_type,password_status
#
# The script is slow and inefficient, written on two cups of coffee at 3am, but it works. It is slow, so you may
# want to run it overnight if you have a large list of servers. The csv output can be imported into a spreadsheet app.
#
# NOTE: Because the script and its output contain your actual passwords, don't keep them sitting around. Chmod the
# script 700 and the output will be 600. Delete them when done. Clear shell history and terminal buffer. It is easy
# to rewrite this script to read passwords as command-line arguments for better security, but I am feeling lazy today...

configure() {
	hostlist="/var/adm/bin/server_list_primary.txt"
	hostlist_resolved="${hostlist}_resolved"
	hostlist_processed="${hostlist}_processed"
	username="igor"
	safeword="sw_igor"

	if [ ! -f "${hostlist}" ]
	then
		echo "Host list file $hostlist not found. Exiting..." ; exit 1
	fi

	if [ -f "${hostlist_resolved}" ]
	then
		/bin/rm "${hostlist_resolved}"
	fi

	if [ -f "${hostlist_processed}" ]
	then
		/bin/rm "${hostlist_processed}"
	fi
}

resolve() {
	cat "${hostlist}" | while read line
	do
		if [ `/usr/bin/host ${line} | grep -c "not found"` -eq 0 ]
		then
			/usr/bin/host ${line} | tail -1 | while read line2
			do
				fqdn=$(echo ${line2} | awk '{print $1}')
				ip=$(echo ${line2} | awk '{print $NF}')
				echo "${line},resolved,${fqdn},${ip}" >> "${hostlist_resolved}"
				echo "${line},resolved,${fqdn},${ip}"
			done
		else
			echo "${line},unresolved" >> "${hostlist_resolved}"
			echo "${line},unresolved"
		fi
	done
}

check_safeword() {
	cat "${hostlist_resolved}" | while read line
	do
		if [ `echo $line | grep -c ",unresolved"` -eq 0 ]
		then
			status=2
			host=$(echo $line | awk -F',' '{print $1}')
			status=$(echo "" | ssh -n -q -T -o "BatchMode=yes" ${safeword}@$host echo 2>&1 | grep -ic "safeword" | tail -1)
			if [ ${status} -eq 1 ]
			then
				echo "${line},safeword" >> "${hostlist_processed}"
				echo "${line},safeword"
			else
				echo "${line},notsafeword" >> "${hostlist_processed}"
				echo "${line},notsafeword"
			fi
		else
			echo "${line}" >> "${hostlist_processed}"
		fi
		sleep 1
	done

	/bin/mv "${hostlist_processed}" "${hostlist_resolved}"
}

check_passwordless() {
	cat "${hostlist_resolved}" | while read line
	do
		if [ `echo $line | grep -c ",unresolved"` -eq 0 ]
		then
			if [ `echo $line | grep -c ",safeword"` -eq 0 ]
			then
				status=2
				host=$(echo $line | awk -F',' '{print $1}')
				status=$(ssh -n -T -o "BatchMode=yes" ${username}@$host echo 2>&1 | grep -c denied)
				if [ ${status} -eq 1 ]
				then
					echo "${line},password_requried" >> "${hostlist_processed}"
					echo "${line},password_requried"
				else
					echo "${line},passwordless" >> "${hostlist_processed}"
					echo "${line},passwordless"
				fi
			else
				echo "${line},password_requried" >> "${hostlist_processed}"
			fi
		else
			echo "${line}" >> "${hostlist_processed}"
		fi
		sleep 1
	done

	/bin/mv "${hostlist_processed}" "${hostlist_resolved}"
}

try_password() {
	touch "${hostlist_processed}"
	chmod 600 "${hostlist_processed}"
	cat "${hostlist_resolved}" | while read line
	do
		if [ `echo $line | grep -c ",unresolved"` -eq 0 ]
		then
			if [ `echo $line | grep -c ",safeword"` -eq 0 ]
			then
				if [ `echo $line | grep -c ",passwordless"` -eq 0 ]
				then
					status=1
					host=$(echo $line | awk -F',' '{print $1}')
					for password in 'password1' 'password2' 'password3'
					do
						status=$(expect -c "
						set timeout 5
						spawn ssh ${username}@$host "hostname"
						expect "password:" { send "${password}r" }
						expect eof " | tail -1 | grep -c "ssword")
						if [ $status -eq 0 ]
						then
							echo "${line},${password}" >> "${hostlist_processed}"
							echo "${line},${password}"
							break
						fi
					done
					if [ $status -ne 0 ]
					then
						echo "${line},nopass" >> "${hostlist_processed}"
						echo "${line},nopass"
					fi
					sleep 1
				else
					echo "${line}" >> "${hostlist_processed}"
				fi
			else
				echo "${line}" >> "${hostlist_processed}"
			fi
		else
			echo "${line}" >> "${hostlist_processed}"
		fi
	done
}

configure
resolve
check_safeword
check_passwordless
try_password

echo "Check ${hostlist_processed} for status"

Print Friendly, PDF & Email

8 Comments »

  • Jonathan says:

    My firewall’s been blocking a lot of stuff from a certain computer but some programs that access the internet haven’t been working. I think they might be related but I’m not sure. I need to know how to find out if it’s an actual hacker or something related to one of the programs that’s not working.
    I know I just this question twice in a row and it makes me feel like a dirty spammer but I really need to know and thought I’d more likelt to get a good answer if I put it in two categories.

  • Ryan Dunn says:

    I have got the ip address of a particular computer. Now, can i get access to that machine by connecting to the internet, if yes then how? Please explain and give the source of info where i can get the info and tools to do that.
    Thanx

  • Orbit says:

    How to access Linux machine Remotely From a Windows machine using VNC server?
    Can any one suggest me regarding this?

  • musicistabest says:

    I’ve got SSH access to all of the computers on my network, and rather than call or email users, I’d like to be able to send a message via SSH that would open in a terminal, or some kind of pop up window on the remote computer.

    I don’t need to be able to have them send messages back, but that’s ok too.

  • nick s says:

    I downloaded something from my computer at home and I want to put it on my laptop unfortunately my dorm is far away from house.

    Is there a way I can access my home computer from my laptop and download the files from my home computer?

  • ibjammin44 says:

    Say that you uninstall all web browser programs, such as Google Chrome, Mozilla Firefox, etc..
    Can you still access the internet?
    Or say that you only have one internet browser program, and it is malfunctioning, can you still access the internet to download another browser?

  • blarg blarg says:

    I have a headless mac (OS X) and no extra monitor keyboard. I forgot to enable remote desktop so I can use Chicken of the VNC. I did enable ssh and can access it no problem. How can I enable remote desktop through the command line?

  • Mark says:

    I am using my own personal iPhone and access the web ONLY via Verizon 3G network. If I charge my phone using a USB cord to a port on my work computer, can they monitor websites visited, messages sent, etc.? Again I only use 3G but a manager just said she heard they could track this information if charging your phone via USB port into work computer.

Leave a Reply to nick s Cancel reply

%d bloggers like this: