OS Unit Tests and SNMP Simulation
Tests keep LibreNMS correct, now and in the future. A new OS must supply enough test data. Test data for an existing OS is also welcome.
The saved SNMP data is in tests/snmpsim/*.snmprec. The saved database data is in tests/data/*.json. Read this data for sensitive information before you submit it. Replace the data in a consistent way.
We use snmpsim for the unit tests. For OS discovery, we can mock snmpsim. The other tests need an installed and working snmpsim. We run snmpsim in our integration tests.
lnms dev:checkdoes not run it by default. To install snmpsim, runpip3 install snmpsim.
Capturing test data
If test data already exists
Test data can already exist for a different device or configuration with the same OS. Each device model has its own file: for a Linksys WRT54G with the OS
linksys, the modelwrt54gsaveslinksys_wrt54g.snmprec. LibreNMS tests each file separately.
1. Collect SNMP data
lnms dev:collect-snmprec collects the data for the tests. It captures all the discovery data and the polling data of a device in LibreNMS. Run it with the device ID, IP, or hostname:
lnms dev:collect-snmprec 42
If you leave out the device, the command asks for it. LibreNMS must detect the device as its own OS first. The command does not collect data from a generic device, because it would leave data out.
The command asks for the device model, and then saves the data to tests/snmpsim/<os>_<model>.snmprec. To skip the question, give the model with -M, for example lnms dev:collect-snmprec 42 -M wrt54g.
The command always replaces the file. It does not merge the data with an existing file. Run the command again after you add more support. For more options, read the command line help (lnms dev:collect-snmprec --help).
- Do not fabricate snmprec files. You must only capture snmprec files from an actual device or convert them from snmpwalk output from an actual device.
- Do not break the data when you sanitize it. Keep valid syntax, correct data types, and valid OID structure. Remove real passwords, SNMP community strings, and public IP addresses.
2. Save test data
After you collect the SNMP data, run lnms dev:generate-test-data:
- The argument is a fixture file name prefix.
iosmatches every fixture starting withios(ios,iosxe,ios_c3560, ...), whileios_matches only theiosmodels. Useallfor every fixture. - To create a new fixture, give the exact name of the snmprec file, for example
lnms dev:generate-test-data linksys_wrt54g. - To update existing JSON fixtures, run the same command again.
This command dumps the database entries after discovery and after polling into a JSON file (tests/data/<os>.json). Do not edit this file manually.
Running tests
Note: before the tests, run ./scripts/composer_wrapper.php install in your LibreNMS root directory. This command reads composer.json and installs the necessary dependencies.
After you save your test data, run lnms dev:check to test it.
For the full test suite, enable the database tests and the snmpsim tests: lnms dev:check unit --db --snmpsim
Specific OS
lnms dev:check unit -o osname
Test an OS, but only discovery and polling modules (exluding OS detection)
lnms dev:check unit --os osname --os-modules-only
Specific Module
lnms dev:check unit -m modulename
Test all modules for all os and stop on failure
lnms dev:check unit --db -snmpsim --os-modules-only -f
Using snmpsim for testing
This command runs snmpsim with the test data:
lnms dev:simulate
You can then run SNMP queries against it. Use the file name without .snmprec as the community. Use 127.1.6.1:1161 as the host.
snmpget -v 2c -c ios_c3560e 127.1.6.1:1161 sysDescr.0
Simulate specific device from test data
Add or update a device for a specific snmprec file in your install. The device name is "snmpsim-" followed by the file name.
lnms dev:simulate ios_2960x
Then run lnms device:discover snmpsim-ios_2960x -vv and lnms device:poll snmpsim-ios_2960x -vv. These commands discover and poll the simulated device.
When you stop the command, it removes the simulated devices. To keep them, add --keep.
To simulate more than one device, give more files. You can also give paths to snmprec files, so that the shell can expand a glob pattern:
lnms dev:simulate ios_2960x junos_ex
lnms dev:simulate tests/snmpsim/ruckuswireless*
Snmprec format
An snmprec file stores the SNMP data. The format has three columns: the numeric OID, the type code, and the data. This is an example:
1.3.6.1.2.1.1.1.0|4|Pulse Secure,LLC,MAG-2600,8.0R14 (build 41869)
1.3.6.1.2.1.1.2.0|6|1.3.6.1.4.1.12532.254.1.1
In a test, LibreNMS uses the data of the snmprec file for the SNMP calls. This example gives sysDescr (.1.3.6.1.2.1.1.1.0, 4 = Octet String) and sysObjectID (.1.3.6.1.2.1.1.2.0, 6 = Object Identifier), This data is the minimum for a new snmprec file.
To look up the numeric OID and type of an string OID with snmptranslate:
snmptranslate -On -Td SNMPv2-MIB::sysDescr.0
List of SNMP data types:
| Type | Value |
|---|---|
| OCTET STRING | 4 |
| HEX STRING | 4x |
| Integer32 | 2 |
| NULL | 5 |
| OBJECT IDENTIFIER | 6 |
| IpAddress | 64 |
| Counter32 | 65 |
| Gauge32 | 66 |
| TimeTicks | 67 |
| Opaque | 68 |
| Counter64 | 70 |
Use a hex encoded string (4x) for each string with a line return.
New discovery/poller modules
A new discovery module or poller module defines its database capture parameters in /tests/module_tables.yaml.
Example workflow
Add initial detection
- Add device to LibreNMS. It is generic and device_id = 42
- Add initial detection for
linksys - Run discovery to make sure it detects properly
lnms device:discover -vv 42 - Add any additional os items like version, hardware, features, or serial.
- Run
lnms dev:collect-snmprec 42 -M wrt54gto savelinksys_wrt54g.snmprec - Run
lnms dev:generate-test-data linksys_wrt54gto create the dumped database data. - Review data. If you modified the snmprec or code (do not modify the json manually) run
lnms dev:generate-test-data linksys_wrt54g -m os - Run
lnms dev:check unit --db --snmpsim - If the tests succeed submit a pull request
Additional module support or test data
- Add code to support module or support already exists.
- Run
lnms dev:collect-snmprec 42 -M wrt54g. It replaces the snmprec file - Run
lnms dev:generate-test-data linksys_wrt54g - Review data. If you modified the snmprec (do not modify the json manually) run
lnms dev:generate-test-data linksys_wrt54g -m <module> - Run
lnms dev:check unit --db --snmpsim - If the tests succeed submit a pull request
JSON Application Test Writing Using ./scripts/json-app-tool.php
- First get a good example of the JSON output over SNMP extend in question.
- Read the help via
./scripts/json-app-tool.php -h. - Generate the SNMPrec data via
./scripts/json-app-tool.php -a appName -s > ./tests/snmpsim/linux_appName-v1.snmprec. If the SNMP extend name OID differs from the application name, add the -S flag to override it. - Generate the test JSON data via
./scripts/json-app-tool.php -a appName -t > ./tests/data/linux_appName-v1.json. - Update the generated './tests/data/linux_appName-v1.json' making sure that all the expected metrics are present. This step assumes that LibreNMS collapses and uses everything under .data in the JSON.
In a test run, LibreNMS can fail to detect the app. If the app name and the SNMP extend name OID differ, make sure that -S is correct. Also make sure that includes/discovery/applications.inc.php holds the update.