Most log files located in /var/log are part of the log rotation and will be compressed automatically. However, in many cases various user applications maintain log files outside of /var/log. These logs are not managed …

Imagine a hypothetical scenario: you support hundreds of remote servers and you need to check which server you can access via SSH and which servers are not letting you log in. Doing this manually is a tedious process that many sysadmins choose to skip. The inevitable outcome is inability to quickly access a system when it really counts.

As basic as the task of creating a startup script may sound, even experienced sysadmins sometimes run into problems with having services start at the right time during the boot process or stop during the shutdown. Two major reasons for this: the procedure is a bit convoluted due to linking. Also, writing startup scripts is not something you have to do very often these days.

The Unix “man” pages are an indispensable tool even for the most experienced of sysadmins. Unfortunately, in a terminal window the text may be hard to read and getting a hard copy to take with you down to the datacenter will really test your Unix skills. The following quick script will convert the “man” page of your choice into formatted text and open it in the Web browser.

Recently I ran into a situation where I needed to reboot several HPC cluster compute nodes. A couple of systems were still running user jobs that I did not want to interrupt. I also didn’t want to sit around and watch those jobs, as there was not way of telling how long they could take to complete. The solution was to write a short script that looks something like this…

A number of good and mostly free tools are available for testing RAM hardware. However, most of these applications are designed to operate in stand-alone mode that requires shutting down the OS. Still, tare a few ways to run memory integrity checks from inside the operating environment. Here we will look at using dd and memtester.

We have discussed using expect with sudo in an SSH session. Another common way to run commands on remote servers as root is to use the su – root command, where sudo is not configured. The trick to automating su operations using expect is to launch the su command with the spawn ssh command.

In this example we will show you how to calculate simple moving average using Excel and a shell script. As more data is added to the file, the shell script and the Excel formula will automatically use the specified number of the latest data points to calculate the moving average.

Passwordless SSH works by pre-sharing public encryption keys. It is important to understand that setting up passwordless SSH opens your servers to certain security risks. A compromised user account on one server may allow the attacker to gain access to multiple systems on the network. Try to avoid using passwordless access for accounts with elevated privileges.

Let’s imagine a hypothetical scenario: you have a list of a hundred Linux servers and you need to log into each one of them and remove a local user “roger” and his home directory. Doing this by hand will get tedious and, chances are, you will make a few typos and there will be some collateral damage.

The idea here is simple: you need to connect to hosts 1..100 and execute a command in the background. The trick is to make sure the command continues to run after the SSH session terminates. Simply adding the “&” at the end of the command is not the answer here.

Recently I’ve ran into an unusual problem with SSH on SLES 11 SP1. The system – a small HPC cluster – is configured to use NIS for authentication and automounter maps. Whenever a user logs in, the home directory is automounted from an external NAS. Each user has a ~/.ssh directory with the authorized_keys2 and known_hosts required for password-less SSH access among the cluster nodes.

Imagine this: you created the /etc/ntp.conf file, enabled and started the ntpd service, and… the system time is still off. Is there a way to force time sync with the NTP server? It must be your lucky day. All you have to do is to connect to your box as root and run the following command…

This is a very simple one, but somehow I always forget the correct syntax. I want to use the “find” command to locate files and then use several “exec” statements to perform various operations on whatever “find” finds. In the following example I am making sure that all /etc/*.conf files are owned by root and that they don’t have permissions greater than 644.

Recently I ran into an issue: the sshd service on a Solaris 10 box needed to be used with a custom configuration file. By default, the sshd service will use /etc/ssh/sshd_config. I needed it to use /etc/ssh/sshd_config_custom. I could not just modify the default configuration file. Don’t ask why – it’s complicated. So here’s what I ended up doing and this process is applicable to modifying any other Solaris 10 (and above) service.

Here is a quick example of how to substitute system commands based on who tries to run them. A customer of mine has recently deployed a security audit application on his servers. The application has a few bugs that the vendor has not yet resolved. For example, the security audit agent looks at the output of the “mount” command to see if any filesystems are mounted without the “nosuid” option.

The “not owner” error is displayed on the client system (usually Solaris) when attempting to mount an NFS share from a server. This error may appear even though the share is correctly exported and the client system has full access. If you are getting a “permission denied” error, then this article is not for you and you should check here instead.

Until I branched out a few years ago from supporting Unix server to working with Linux clusters, I never really encountered this issue: you type “reboot”, “init 0″, or “shutdown” as root and… nothing happens. Or the system starts going down but then hangs on unmounting a filesystem or unloading a module. I think this happened once to a colleague of mine who was rebooting a Solaris server, but this is a common problem with Linux.

By default, WordPress will not allow you to upload a file larger than 2Mb. This limit is not set by WordPress itself, but by detaul PHP configuration. This configuration file is usually /etc/php.ini, but normally …

When using RDP, sometimes the session may lock up and you may need to reboot your workstation remotely. This can be done using the “shutdown.exe” command from your local PC. First, make sure you can …

Starting with SuSE 10 and including both openSuSE and SLES an annoying SSH problem appeared. When you SSH to the server (via PuTTY and such) you get the “login” prompts right away, but the “password” …

This is the second installment of a multipart guide for beginner Unix sysadmins supporting HPC clusters. You can view the first part of the guide here.
Searching, Replacing, Comparing
Try to work with a large cluster and …