Skip to content

Setting up RRDCached

This document describes the setup of RRDCached for LibreNMS.

Since version 1.5, rrdtool / rrdcached now supports creating rrd files over rrdcached. If you have rrdcached 1.5.5 or above, you can also tune over rrdcached. To enable this set the following config:

poller/rrdtool

lnms config:set rrdtool_version '1.5.5'

This setting has to be the exact version of rrdtool you are running.

NOTE: This feature requires your client version of rrdtool to be 1.5.5 or newer, in addition to your rrdcached version.

High-Availability

Run only one active instance of RRDCached. Keep a failover instance ready for a failure. A recommended setup is to use a network socket for RRDCached and have a load balancer in front of it such as Nginx. See Securing RRDCached for more information.

For HA, you must also use a shared storage for the RRD files over NFS with GlusterFS or similar.

Distributed Poller Support Matrix

Shared FS: Is a shared filesystem required?

Features: Supported features in the version indicated.

G = Graphs.
C = Create RRD files.
U = Update RRD files.
T = Tune RRD files.
Version Shared FS Features
1.4.x Yes G,U
<1.5.5 Yes G,U
>=1.5.5 No G,C,U
>=1.6.x No G,C,U
>=1.8.x No G,C,U,T

It is recommended that you monitor your LibreNMS server with LibreNMS so you can view the disk I/O usage delta. This can be done with the RRDCached plugin.

Installation

Ubuntu and Debian are very similar, the main difference is the location of the PIDFILE.

For info about version of rrdcached to install, see Ubuntu packages

  1. Install rrdcached

    sudo apt-get install rrdcached
    
  2. Edit /etc/default/rrdcached to include:

    BASE_OPTIONS="-B -F -R"
    BASE_PATH=/opt/librenms/rrd/
    DAEMON_GROUP=librenms
    DAEMON_USER=librenms
    DAEMON=/usr/bin/rrdcached
    JOURNAL_PATH=/var/lib/rrdcached/journal/
    PIDFILE=/run/rrdcached.pid
    SOCKFILE=/run/rrdcached.sock
    SOCKGROUP=librenms
    WRITE_JITTER=1800
    WRITE_THREADS=4
    WRITE_TIMEOUT=1800
    
  3. Fix permissions

    chown librenms:librenms /var/lib/rrdcached/journal/
    
  4. Restart the rrdcached service

    systemctl restart rrdcached.service
    

For info about version of rrdcached to install, see Debian packages

  1. Install rrdcached

    sudo apt-get install rrdcached
    
  2. Edit /etc/default/rrdcached to include:

    BASE_OPTIONS="-B -F -R"
    BASE_PATH=/opt/librenms/rrd/
    DAEMON_GROUP=librenms
    DAEMON_USER=librenms
    DAEMON=/usr/bin/rrdcached
    JOURNAL_PATH=/var/lib/rrdcached/journal/
    PIDFILE=/var/run/rrdcached.pid
    SOCKFILE=/run/rrdcached.sock
    SOCKGROUP=librenms
    WRITE_JITTER=1800
    WRITE_THREADS=4
    WRITE_TIMEOUT=1800
    
  3. Fix permissions

    chown librenms:librenms /var/lib/rrdcached/journal/
    
  4. Restart the rrdcached service

    systemctl restart rrdcached.service
    

Note

rrdcached is installed as part of the rrdtool package, but the rrdcached service is not setup by default, unlike the Ubuntu/Debian setup.

The SELinux policy process generates intermediate files, for example rrdcached_librenms.mod and rrdcached_librenms.pp. After a successful installation of the policy module, you can remove these files.

  1. link in the service and reload:

    ln -s /opt/librenms/dist/rrdcached/rrdcached.service /etc/systemd/system/
    systemctl daemon-reload
    
  2. Configure SELinux for RRDCached

    1. Compile the SELinux Policy Compile the SELinux policy module using the following command:

      checkmodule -M -m -o /tmp/rrdcached_librenms.mod /opt/librenms/dist/rrdcached/rrdcached_librenms.te
      

      Explanation: - -M: Enable the module compiler. - -m: Enable the module version format. - -o: Specify the output file name.

    2. Package the Policy Module Package the compiled module into a loadable policy package:

      semodule_package -o /tmp/rrdcached_librenms.pp -m /tmp/rrdcached_librenms.mod
      

      Explanation: - -o: Specify the output file name. - -m: Specify the input file name.

    3. Apply the Policy Module Apply the policy module to the system:

      semodule -i /tmp/rrdcached_librenms.pp
      

      Explanation: - -i: Install the policy module.

  3. Start RRDcached and enable for start at boot

    systemctl enable --now rrdcached.service
    

This example is based on a fresh LibreNMS install, on a minimal CentOS 6 installation. This example uses the Repoforge repository.

rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

Enable the Extra repo

vi /etc/yum.repos.d/rpmforge.repo

Install rrdtool

yum update rrdtool

Disable the [rpmforge] and [rpmforge-extras] repos again

vi /etc/yum.repos.d/rpmforge.repo

Edit the rrdcached config /etc/sysconfig/rrdcached:

# Settings for rrdcached
OPTIONS="-w 1800 -z 1800 -f 3600 -s librenms -U librenms -G librenms -B -R -j /var/tmp -l unix:/run/rrdcached.sock -t 4 -F -b /opt/librenms/rrd/"
RRDC_USER=librenms
mkdir /var/run/rrdcached
chown librenms:librenms /var/run/rrdcached/
chown librenms:librenms /var/rrdtool/
chown librenms:librenms /var/rrdtool/rrdcached/
chkconfig rrdcached on

Restart rrdcached

service rrdcached start

Network RRDCached

For remote RRDCached server make sure you have network option -L in /var/default/rrdcached or rrdcached.unit

Edit /etc/default/rrdcached to include:

NETWORK_OPTIONS="-L"

LibreNMS config

Edit your LibreNMS config by running the following:

poller/rrdtool

lnms config:set rrdcached "unix:/run/rrdcached.sock"

poller/rrdtool

lnms config:set rrdcached "${IPADDRESS}:42217"

Note

Change ${IPADDRESS} to the ip the rrdcached server is listening on.

Verify

Make sure that LibreNMS draws the graphs. This step takes some minutes. After at least one poll cycle (5 mins), check the LibreNMS disk I/O performance delta. Disk I/O can be found under the menu Devices>All Devices>[localhost_hostname]>Health>Disk I/O.

The Ops/sec value usually decreases by 30% to 40%. The exact value depends on many factors.

Verify SELINUX

If you are using SELinux, and you have issue you can verify the policy module is installed by running the following command:

semodule -l | grep rrdcached_librenms

Test Functionality: Ensure LibreNMS can successfully interact with RRDcached without SELinux denials. Check SELinux logs for any denials:

ausearch -m avc -ts recent

Without a denial, the policy module works and LibreNMS can reach RRDcached.

Securing RRCached

According to the man page, under "SECURITY CONSIDERATIONS", rrdcached has no authentication and no security. A unix socket is its only protection. With a network socket, you must secure your rrdcached installation. Put a proxy in front of rrdcached with nginx to allow only specific IPs to connect.

Using the same setup above, using nginx version 1.9.0 or later, you can follow this setup to proxy the default rrdcached port to the local unix socket.

(You can use ./conf.d for your configuration as well)

mkdir /etc/nginx/streams-{available,enabled}

add the following to your nginx.conf file:

#/etc/nginx/nginx.conf
...u
stream {
    include /etc/nginx/streams-enabled/*;
}

Add this to /etc/nginx/streams-available/rrd

server {
    listen 42217;

    error_log  /var/log/nginx/rrd.stream.error.log;

    allow $LibreNMS_IP;
    deny all;

    proxy_pass unix:/run/rrdcached.sock;
}

Replace $LibreNMS_IP with the IP address of the rrdcached server. You can give more than one allow statement. This configuration binds nginx to TCP 42217, the default rrdcached port. It permits the given IP addresses and blocks all others.

Then symlink the config to streams-enabled: ln -s /etc/nginx/streams-{available,enabled}/rrd

and reload nginx service nginx reload