Skip to content

OS Unit Tests and SNMP Simulation

Tests keep LibreNMS correct, now and in the future. A new OS must supply enough test data. Test data for an existing OS is also welcome.

The saved SNMP data is in tests/snmpsim/*.snmprec. The saved database data is in tests/data/*.json. Read this data for sensitive information before you submit it. Replace the data in a consistent way.

We use snmpsim for the unit tests. For OS discovery, we can mock snmpsim. The other tests need an installed and working snmpsim. We run snmpsim in our integration tests. lnms dev:check does not run it by default. To install snmpsim, run pip3 install snmpsim.

Capturing test data

If test data already exists

Test data can already exist for a different device or configuration with the same OS. Each device model has its own file: for a Linksys WRT54G with the OS linksys, the model wrt54g saves linksys_wrt54g.snmprec. LibreNMS tests each file separately.

1. Collect SNMP data

lnms dev:collect-snmprec collects the data for the tests. It captures all the discovery data and the polling data of a device in LibreNMS. Run it with the device ID, IP, or hostname:

lnms dev:collect-snmprec 42

If you leave out the device, the command asks for it. LibreNMS must detect the device as its own OS first. The command does not collect data from a generic device, because it would leave data out.

The command asks for the device model, and then saves the data to tests/snmpsim/<os>_<model>.snmprec. To skip the question, give the model with -M, for example lnms dev:collect-snmprec 42 -M wrt54g.

The command always replaces the file. It does not merge the data with an existing file. Run the command again after you add more support. For more options, read the command line help (lnms dev:collect-snmprec --help).

  • Do not fabricate snmprec files. You must only capture snmprec files from an actual device or convert them from snmpwalk output from an actual device.
  • Do not break the data when you sanitize it. Keep valid syntax, correct data types, and valid OID structure. Remove real passwords, SNMP community strings, and public IP addresses.

2. Save test data

After you collect the SNMP data, run lnms dev:generate-test-data:

  • The argument is a fixture file name prefix. ios matches every fixture starting with ios (ios, iosxe, ios_c3560, ...), while ios_ matches only the ios models. Use all for every fixture.
  • To create a new fixture, give the exact name of the snmprec file, for example lnms dev:generate-test-data linksys_wrt54g.
  • To update existing JSON fixtures, run the same command again.

This command dumps the database entries after discovery and after polling into a JSON file (tests/data/<os>.json). Do not edit this file manually.

Running tests

Note: before the tests, run ./scripts/composer_wrapper.php install in your LibreNMS root directory. This command reads composer.json and installs the necessary dependencies.

After you save your test data, run lnms dev:check to test it.

For the full test suite, enable the database tests and the snmpsim tests: lnms dev:check unit --db --snmpsim

Specific OS

lnms dev:check unit -o osname

Test an OS, but only discovery and polling modules (exluding OS detection)

lnms dev:check unit --os osname --os-modules-only

Specific Module

lnms dev:check unit -m modulename

Test all modules for all os and stop on failure

lnms dev:check unit --db -snmpsim --os-modules-only -f

Using snmpsim for testing

This command runs snmpsim with the test data:

lnms dev:simulate

You can then run SNMP queries against it. Use the file name without .snmprec as the community. Use 127.1.6.1:1161 as the host.

snmpget -v 2c -c ios_c3560e 127.1.6.1:1161 sysDescr.0

Simulate specific device from test data

Add or update a device for a specific snmprec file in your install. The device name is "snmpsim-" followed by the file name.

lnms dev:simulate ios_2960x

Then run lnms device:discover snmpsim-ios_2960x -vv and lnms device:poll snmpsim-ios_2960x -vv. These commands discover and poll the simulated device.

When you stop the command, it removes the simulated devices. To keep them, add --keep.

To simulate more than one device, give more files. You can also give paths to snmprec files, so that the shell can expand a glob pattern:

lnms dev:simulate ios_2960x junos_ex
lnms dev:simulate tests/snmpsim/ruckuswireless*

Snmprec format

An snmprec file stores the SNMP data. The format has three columns: the numeric OID, the type code, and the data. This is an example:

1.3.6.1.2.1.1.1.0|4|Pulse Secure,LLC,MAG-2600,8.0R14 (build 41869)
1.3.6.1.2.1.1.2.0|6|1.3.6.1.4.1.12532.254.1.1

In a test, LibreNMS uses the data of the snmprec file for the SNMP calls. This example gives sysDescr (.1.3.6.1.2.1.1.1.0, 4 = Octet String) and sysObjectID (.1.3.6.1.2.1.1.2.0, 6 = Object Identifier), This data is the minimum for a new snmprec file.

To look up the numeric OID and type of an string OID with snmptranslate:

snmptranslate -On -Td SNMPv2-MIB::sysDescr.0

List of SNMP data types:

Type Value
OCTET STRING 4
HEX STRING 4x
Integer32 2
NULL 5
OBJECT IDENTIFIER 6
IpAddress 64
Counter32 65
Gauge32 66
TimeTicks 67
Opaque 68
Counter64 70

Use a hex encoded string (4x) for each string with a line return.

New discovery/poller modules

A new discovery module or poller module defines its database capture parameters in /tests/module_tables.yaml.

Example workflow

Add initial detection

  1. Add device to LibreNMS. It is generic and device_id = 42
  2. Add initial detection for linksys
  3. Run discovery to make sure it detects properly lnms device:discover -vv 42
  4. Add any additional os items like version, hardware, features, or serial.
  5. Run lnms dev:collect-snmprec 42 -M wrt54g to save linksys_wrt54g.snmprec
  6. Run lnms dev:generate-test-data linksys_wrt54g to create the dumped database data.
  7. Review data. If you modified the snmprec or code (do not modify the json manually) run lnms dev:generate-test-data linksys_wrt54g -m os
  8. Run lnms dev:check unit --db --snmpsim
  9. If the tests succeed submit a pull request

Additional module support or test data

  1. Add code to support module or support already exists.
  2. Run lnms dev:collect-snmprec 42 -M wrt54g. It replaces the snmprec file
  3. Run lnms dev:generate-test-data linksys_wrt54g
  4. Review data. If you modified the snmprec (do not modify the json manually) run lnms dev:generate-test-data linksys_wrt54g -m <module>
  5. Run lnms dev:check unit --db --snmpsim
  6. If the tests succeed submit a pull request

JSON Application Test Writing Using ./scripts/json-app-tool.php

  1. First get a good example of the JSON output over SNMP extend in question.
  2. Read the help via ./scripts/json-app-tool.php -h.
  3. Generate the SNMPrec data via ./scripts/json-app-tool.php -a appName -s > ./tests/snmpsim/linux_appName-v1.snmprec. If the SNMP extend name OID differs from the application name, add the -S flag to override it.
  4. Generate the test JSON data via ./scripts/json-app-tool.php -a appName -t > ./tests/data/linux_appName-v1.json.
  5. Update the generated './tests/data/linux_appName-v1.json' making sure that all the expected metrics are present. This step assumes that LibreNMS collapses and uses everything under .data in the JSON.

In a test run, LibreNMS can fail to detect the app. If the app name and the SNMP extend name OID differ, make sure that -S is correct. Also make sure that includes/discovery/applications.inc.php holds the update.