Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Backups, Featured

Filesystem Syncronization with Lsyncd

Submitted by on November 15, 2014 – 3:05 pm

Lsyncd monitors a local filesystem for changes and mirrors those changes to a filesystem on a remote server. The advantage of lsyncd over rsync is the former’s ability to detect filesystem changes without having to re-scan the entire source and target directory structures. This makes lsyncd faster and more efficient. And you can still use rsync with lsyncd to handle the actual data transfer.

Here’s a quick example of installing and using lsyncd. In this example the source_server and target_server are both RHEL 6.4 64-bit VMs running on ESX host. To make things simple for myself, I was using a root account. However, this is not recommended, unless the filesystems you are synchronizing require root access.

Configure key-based SSH authentication

If using the root account, you must allow direct root SSH on the target server.

Edit target_server:/etc/ssh/sshd_config and set:
PermitRootLogin yes

Save the file and restart SSH:
service sshd restart

Generate SSH key-pair for your account and send the public key to the target_server.
on the source_server:

ssh-keygen -t rsa
ssh-copy-id target_server

test ssh connection:

ssh -qt target_server "hostname"

Install Lua and Lsyncd
yum -y install lua*
yum -y install lsyncd

NOTE: the version of lsyncd currently (Nov 2014) available from RHEL/CentOS repos is 2.1.4. Version 2.1.5 brings back a potentially very valuable functionality of initiating file sync after file attribute change (i.e. file ownership, permissions, etc). If that’s something you may need, here’s how to get lsyncd 2.1.5
cd /tmp
wget http://mirrors.htbindustries.org/CentOS/6/x86_64/lsyncd-2.1.5-1.el6.x86_64.rpm
yum -y install /tmp/lsyncd-2.1.5-1.el6.x86_64.rpm

Before continuing, this would be a good time to do a quick test.
on source_server generate some folders and file for testing:

mkdir /tmp/test1
for i in `seq -w 1 10` ; do mkdir /tmp/test1/dir${i} ; for j in `seq -w 1 10` ; do touch /tmp/test1/dir${i}/file${j} ; done ; done

on the target_server create the base folder:

mkdir /tmp/test1

from source_server run the sync:

lsyncd -rsyncssh /tmp/test1/ root@target_server /tmp/test1/

The next step is to create a configuration file and run lsyncd as a service. I would recommend removing the default configuration file – it’s causes problems with the /etc/init.d/lsyncd start-up script:
rm /etc/sysconfig/lsyncd

Here’s an example configuration file (/etc/lsyncd.conf) for the example above:
settings {
        logfile = "/var/log/lsyncd/lsyncd.log",
        statusFile = "/var/log/lsyncd/lsyncd.status",
        nodaemon = false,
        maxDelays = 3,
        onStartup = true,
        onAttrib = true,
        onCreate = true,
        onDelete = true,
        onModify = true,
        onMove = true,
        delete = true,
}

sync {
        default.rsyncssh,
        source = "/tmp/test1",
        host = "target_server",
        targetdir = "/tmp/test1",
        delay=3,
        rsync = {
                        owner = true,
                        perms = true,
                        checksum = true,
                        compress = false,
                        acls = true,
                        verbose = true,
                        _extra = {"-aKx"},
        }
}

The final steps are to create the log directory and activate the lsyncd service.
mkdir -p /var/log/lsyncd
chkconfig lsyncd on
service lsyncd start
service lsyncd status

Lsyncd can be a bit fiddly to configure, but it is well worth the effort. Hopefully, these instructions will make it a bit easier for you.

Print Friendly, PDF & Email

Leave a Reply