Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Featured, Security

Encrypting Log Data During Log Rotation

Submitted by on April 9, 2019 – 4:12 pm

Most log files do not contain personally-identifiable information or other sensitive data. And even if they do, encryption of all personal data is not mandatory under GDPR. Still, on occasion, for testing and troubleshooting purposes you may want to log potentially sensitive information. It would be a very good idea not to let these logs get away from you.

You can certainly run the encryption process separately from log rotation via cron or some such. But this would be an extra step with extra problems. You can actually use logrotate directly to handle encryption. Basically, you will tell logrotate use gpg instead of the usual gzip.

Here’s an example of my /etc/logrotate.d/squid:

Expand

/var/log/squid/*.log {
    daily
    rotate 5
    compress
    compresscmd /usr/bin/gpg
    compressoptions --always-trust --encrypt --recipient your_email@domain.com
    compressext .gpg
    notifempty
    missingok
    sharedscripts
    postrotate
      # Asks squid to reopen its logs. (log_rotate 0 is set in squid.conf)
      # errors redirected to make it silent if squid is not running
      /usr/sbin/squid -k rotate 2>/dev/null
      # Wait a little to allow Squid to catch up before the logs is compressed
      sleep 1
    endscript
}

Older kernels warning
Expand

Kernels with logrotate version prior to 3.8 have an issue with the compressoptions not taking spaces. There’s a simple workaround, though: create a script that contains the command (such us gpg) and all the options and put it in /usr/bin/gpg-logrotate (don’t forget to make it executable, obviously):

#!/bin/bash
/usr/bin/gpg --always-trust --encrypt --recipient your_email@domain.com

However, since log rotation runs via cron you would either need to source root profile (along with GPG keys), or temporarily extract the relevant GPG key ID to be used for encryption. From a security standpoint, the latter is safer. Here’s what your gpg-logrotate script may look like in this case:

#!/bin/bash
d=$(mktemp -d)
/usr/bin/gpg --homedir ${d} --import /root/.gnupg/your_public_key.pub
keyid="$(/usr/bin/gpg --homedir=${d} --keyid-format 0xLONG -k | grep -m1 ^pub | awk -F'/' '{print $2}' | awk '{print $1}')"
/usr/bin/gpg --homedir=${d} --always-trust --encrypt --recipient ${keyid}
/bin/rm -rf ${d}

And then the logrotate config file from the example above would look like this:

/var/log/squid/*.log {
    daily
    rotate 5
compress
compresscmd /usr/bin/gpg-logrotate
compressext .gpg
    notifempty
    missingok
    sharedscripts
    postrotate
      # Asks squid to reopen its logs. (log_rotate 0 is set in squid.conf)
      # errors redirected to make it silent if squid is not running
      /usr/sbin/squid -k rotate 2>/dev/null
      # Wait a little to allow Squid to catch up before the logs is compressed
      sleep 1
    endscript
}

Following log rotation, you should see something like this in the log destination:

ls -als /var/log/squid
total 232
  4 drwxr-x---   2 squid squid   4096 Feb 28 16:06 .
  4 drwxr-xr-x. 27 root  root    4096 Feb 24 03:10 ..
204 -rw-r-----   1 squid squid 202423 Feb 28 16:56 access.log
  4 -rw-r-----   1 squid squid   1535 Feb 28 16:06 access.log.1.gpg
  8 -rw-r-----   1 squid squid   6238 Feb 28 16:09 cache.log
  4 -rw-r-----   1 squid squid    519 Feb 28 16:06 cache.log.1.gpg
  4 -rw-r--r--   1 root  root     187 Feb  6 11:06 squid.out

Log rotation frequency

By default, log rotation process would run once a day. This is fine if you specified daily or weekly in your config. If you want to run the process hourly, you would need to copy /etc/cron.daily/logrotate into /etc/cron.hourly/ directory. Naturally, in your logrotate config file you would also need to specify hourly with the desired retention period.

And, I guess, a couple of words on how to set up gpg and keys. If you have an existing key, copy the public portion of it to your server and add it to the keyring:

gpg --import F656FEB192A41CA82B6F73834011ACA1D7833129.asc

You can then list all the keys on your server:

gpg --list-keys
/root/.gnupg/pubring.gpg
------------------------
pub   2048R/5303E22B 2017-07-14
uid                  Your Name <your_email@domain.com>
sub   2048R/8A10B563 2017-07-14

If you don’t already have a key, you should create a passphrase-protected key. It is also a good idea not to keep the private half of the key on the server.

Generating new key

In the following example we create a new passphrase-protected non-expiring private key.

cd ~ && gpg --gen-key
gpg (GnuPG) 2.0.14; Copyright (C) 2009 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 2
DSA keys may be between 1024 and 3072 bits long.
What keysize do you want? (2048) 4096
DSA keysizes must be in the range 1024-3072
What keysize do you want? (2048) 3072
Requested keysize is 3072 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Your Name
Email address: your_email@domain.com
Comment: delete me
You selected this USER-ID:
    "Your Name (delete me) <your_email@domain.com>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest size
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key AADDBFF5 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
pub   3072D/AADDBFF5 2019-02-28
      Key fingerprint = FE29 2FF3 8653 F2C2 4B9A  B9C5 FE39 CF03 AADD BFF5
uid                  Your Name (delete me) <your_email@domain.com>
sub   3072g/6DACF5EB 2019-02-28

You see the key you just created:

gpg --list-secret-keys
/root/.gnupg/secring.gpg
------------------------
sec   3072D/AADDBFF5 2019-02-28
uid                  Your Name (delete me) <your_email@domain.com>
ssb   3072g/6DACF5EB 2019-02-28

As I mentioned, you shouldn’t be keeping your private key on the server. Generate the key on a less-exposed machine, extract the public key, and import it on your server. Here’s an example:

Expand

# Generate passphrase-protected private key as shown above
# on a secure machine

# Extract the public key
gpg --armor --export your_email@domain.com > ~/your_email@domain.com.asc

# Move the public key to the server and import it
gpg --import ~/your_email@domain.com.asc

 

Print Friendly, PDF & Email

Leave a Reply