Tag Archives: MRTG

Disk Usage, Raspberry Pi, and MRTG

by

I’ve used MRTG for many years to keep track of network and disk usage on various platoforms. I like it because it’s easy to set up and doesn’t require a database to be configured or maintained. The simple file structure and image creation means that I can easily have the images synchronized to a central web server even if they are generated on the remote device.

The weekly or monthly views of the disk usage are what I find most useful for recognizing trends of usage. I noticed that last Friday there was a huge jump in usage that I ought to spent a little time discovering what suddenly took over a large chunk of my disk.

sudo du --human-readable --max-depth=2 --one-file-system /

That command gave me a nice list of where the disk usage was allocated. Here’s a small section of the results that looked suspicious.

20G     /var
4.0K    /tmp/ssh-72XFLxsn7URe
8.0K    /tmp/systemd-private-e26b4caee9b7494e847b16a023d67460-systemd-timesyncd.service-Nuv83g
4.0K    /tmp/.Test-unix
8.0K    /tmp/systemd-private-e26b4caee9b7494e847b16a023d67460-systemd-logind.service-xl9zMh
88G     /tmp/mc-wim
4.0K    /tmp/.X11-unix
4.0K    /tmp/.font-unix
4.0K    /tmp/.ICE-unix
156K    /tmp/.qBittorrent
4.0K    /tmp/.XIM-unix
88G     /tmp

The 88G in a single directory under /tmp looks very much like the size bump I got last Friday, and I remember killing a process via a HUP signal directed at Midnight Commander which was trying to open a very large tar file. Deleting that temporary directory recovered the free space properly.

wim@WimPi4:/tmp $ df
Filesystem                             1K-blocks       Used  Available Use% Mounted on
/dev/root                              492885032  235012904  237825328  50% /
devtmpfs                                 1774272          0    1774272   0% /dev
tmpfs                                    1939136          4    1939132   1% /dev/shm
tmpfs                                     775656       2088     773568   1% /run
tmpfs                                       5120          4       5116   1% /run/lock
/dev/mmcblk0p1                            258096      50755     207341  20% /boot
tmpfs                                     387824          0     387824   0% /run/user/1000
tmpfs                                     387824          0     387824   0% /run/user/1002
tmpfs                                     387824          0     387824   0% /run/user/1001
wim@WimPi4:/tmp $ rm -r /tmp/mc-wim/
wim@WimPi4:/tmp $ df
Filesystem                             1K-blocks       Used  Available Use% Mounted on
/dev/root                              492885032  143584828  329253404  31% /
devtmpfs                                 1774272          0    1774272   0% /dev
tmpfs                                    1939136          4    1939132   1% /dev/shm
tmpfs                                     775656       2088     773568   1% /run
tmpfs                                       5120          4       5116   1% /run/lock
/dev/mmcblk0p1                            258096      50755     207341  20% /boot
tmpfs                                     387824          0     387824   0% /run/user/1000
tmpfs                                     387824          0     387824   0% /run/user/1002
tmpfs                                     387824          0     387824   0% /run/user/1001
wim@WimPi4:/tmp $

My MRTG configuration for keeping track of the disk usage on the Raspberry Pi root partition is:

######################################################################
#       Disk Monitoring
######################################################################
Options[_]: printrouter, gauge, noo, nolegend, transparent, pngdate
Factor[_]:
kMG[_]: ,k,M,G,T,P
YLegend[_]: Disk Usage
ShortLegend[_]:  Bytes
LegendO[_]:  Bytes
LegendI[_]:  Bytes

Target[wimpi4_disk]: .1.3.6.1.2.1.25.2.3.1.6.31&.1.3.6.1.2.1.25.2.3.1.5.31:public@wimpi4.local * .1.3.6.1.2.1.25.2.3.1.4.31&.1.3.6.1.2.1.25.2.3.1.4.31:public@wimpi4.local
Title[wimpi4_disk]: Disk Usage -- WimPi4
MaxBytes[wimpi4_disk]: 504720539648

I got the correct size for MaxBytes by looking at the results from snmpwalk. Multiplying the cluster size (1.3.6.1.2.1.25.2.3.1.4.31) by the number of clusters (1.3.6.1.2.1.25.2.3.1.5.31) gets the total size. If I had multiple disks in the same machine that I wanted to monitor I’d be able to find the human readable name for the disk in this same output.

wim@WimPi4:/tmp $ snmpwalk -Os -c public -v 2c localhost 1.3.6.1.2.1.25.2
iso.3.6.1.2.1.25.2.2.0 = INTEGER: 3878272
iso.3.6.1.2.1.25.2.3.1.1.1 = INTEGER: 1
iso.3.6.1.2.1.25.2.3.1.1.3 = INTEGER: 3
iso.3.6.1.2.1.25.2.3.1.1.6 = INTEGER: 6
iso.3.6.1.2.1.25.2.3.1.1.7 = INTEGER: 7
iso.3.6.1.2.1.25.2.3.1.1.8 = INTEGER: 8
iso.3.6.1.2.1.25.2.3.1.1.10 = INTEGER: 10
iso.3.6.1.2.1.25.2.3.1.1.31 = INTEGER: 31
iso.3.6.1.2.1.25.2.3.1.1.37 = INTEGER: 37
iso.3.6.1.2.1.25.2.3.1.1.39 = INTEGER: 39
iso.3.6.1.2.1.25.2.3.1.1.40 = INTEGER: 40
iso.3.6.1.2.1.25.2.3.1.1.66 = INTEGER: 66
iso.3.6.1.2.1.25.2.3.1.1.67 = INTEGER: 67
iso.3.6.1.2.1.25.2.3.1.1.68 = INTEGER: 68
iso.3.6.1.2.1.25.2.3.1.1.72 = INTEGER: 72
iso.3.6.1.2.1.25.2.3.1.2.1 = OID: iso.3.6.1.2.1.25.2.1.2
iso.3.6.1.2.1.25.2.3.1.2.3 = OID: iso.3.6.1.2.1.25.2.1.3
iso.3.6.1.2.1.25.2.3.1.2.6 = OID: iso.3.6.1.2.1.25.2.1.1
iso.3.6.1.2.1.25.2.3.1.2.7 = OID: iso.3.6.1.2.1.25.2.1.1
iso.3.6.1.2.1.25.2.3.1.2.8 = OID: iso.3.6.1.2.1.25.2.1.1
iso.3.6.1.2.1.25.2.3.1.2.10 = OID: iso.3.6.1.2.1.25.2.1.3
iso.3.6.1.2.1.25.2.3.1.2.31 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.37 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.39 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.40 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.66 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.67 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.68 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.2.72 = OID: iso.3.6.1.2.1.25.2.1.4
iso.3.6.1.2.1.25.2.3.1.3.1 = STRING: "Physical memory"
iso.3.6.1.2.1.25.2.3.1.3.3 = STRING: "Virtual memory"
iso.3.6.1.2.1.25.2.3.1.3.6 = STRING: "Memory buffers"
iso.3.6.1.2.1.25.2.3.1.3.7 = STRING: "Cached memory"
iso.3.6.1.2.1.25.2.3.1.3.8 = STRING: "Shared memory"
iso.3.6.1.2.1.25.2.3.1.3.10 = STRING: "Swap space"
iso.3.6.1.2.1.25.2.3.1.3.31 = STRING: "/"
iso.3.6.1.2.1.25.2.3.1.3.37 = STRING: "/dev/shm"
iso.3.6.1.2.1.25.2.3.1.3.39 = STRING: "/run"
iso.3.6.1.2.1.25.2.3.1.3.40 = STRING: "/run/lock"
iso.3.6.1.2.1.25.2.3.1.3.66 = STRING: "/boot"
iso.3.6.1.2.1.25.2.3.1.3.67 = STRING: "/run/user/1000"
iso.3.6.1.2.1.25.2.3.1.3.68 = STRING: "/run/user/1001"
iso.3.6.1.2.1.25.2.3.1.3.72 = STRING: "/run/user/1002"
iso.3.6.1.2.1.25.2.3.1.4.1 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.3 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.6 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.7 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.8 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.10 = INTEGER: 1024
iso.3.6.1.2.1.25.2.3.1.4.31 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.37 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.39 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.40 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.66 = INTEGER: 512
iso.3.6.1.2.1.25.2.3.1.4.67 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.68 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.4.72 = INTEGER: 4096
iso.3.6.1.2.1.25.2.3.1.5.1 = INTEGER: 3878272
iso.3.6.1.2.1.25.2.3.1.5.3 = INTEGER: 3980668
iso.3.6.1.2.1.25.2.3.1.5.6 = INTEGER: 3878272
iso.3.6.1.2.1.25.2.3.1.5.7 = INTEGER: 2630604
iso.3.6.1.2.1.25.2.3.1.5.8 = INTEGER: 9624
iso.3.6.1.2.1.25.2.3.1.5.10 = INTEGER: 102396
iso.3.6.1.2.1.25.2.3.1.5.31 = INTEGER: 123221258
iso.3.6.1.2.1.25.2.3.1.5.37 = INTEGER: 484784
iso.3.6.1.2.1.25.2.3.1.5.39 = INTEGER: 193914
iso.3.6.1.2.1.25.2.3.1.5.40 = INTEGER: 1280
iso.3.6.1.2.1.25.2.3.1.5.66 = INTEGER: 516191
iso.3.6.1.2.1.25.2.3.1.5.67 = INTEGER: 96956
iso.3.6.1.2.1.25.2.3.1.5.68 = INTEGER: 96956
iso.3.6.1.2.1.25.2.3.1.5.72 = INTEGER: 96956
iso.3.6.1.2.1.25.2.3.1.6.1 = INTEGER: 3712960
iso.3.6.1.2.1.25.2.3.1.6.3 = INTEGER: 3815356
iso.3.6.1.2.1.25.2.3.1.6.6 = INTEGER: 15900
iso.3.6.1.2.1.25.2.3.1.6.7 = INTEGER: 2630604
iso.3.6.1.2.1.25.2.3.1.6.8 = INTEGER: 9624
iso.3.6.1.2.1.25.2.3.1.6.10 = INTEGER: 102396
iso.3.6.1.2.1.25.2.3.1.6.31 = INTEGER: 35973379
iso.3.6.1.2.1.25.2.3.1.6.37 = INTEGER: 1
iso.3.6.1.2.1.25.2.3.1.6.39 = INTEGER: 592
iso.3.6.1.2.1.25.2.3.1.6.40 = INTEGER: 1
iso.3.6.1.2.1.25.2.3.1.6.66 = INTEGER: 101509
iso.3.6.1.2.1.25.2.3.1.6.67 = INTEGER: 0
iso.3.6.1.2.1.25.2.3.1.6.68 = INTEGER: 0
iso.3.6.1.2.1.25.2.3.1.6.72 = INTEGER: 0
wim@WimPi4:/tmp $

FlightAware and MRTG

by

Continuing to work on graphing data with MRTG, I finally figured out a valid script for graphing FlightAware messages from my PiAware. This is especially interesting to me because I have been playing around with different antennas and running on a couple of different Raspberry Pi platforms. On one of the platforms I’ve got two receivers, the primary one that listens on the 1090MHz frequency, and the secondary that listens on 978MHz. I’ve been especially interested in knowing how many reports come in on the secondary frequency.

Using the in and out property of MRTG graphs, I’ve got the In (green) representing 978 messages and the Out (blue) representing 1090 messages.

I added the following section to my /etc/snmp/snmpd.conf file:

pass .1.3.6.1.2.1.25.1.9 /bin/sh /usr/local/bin/snmp-dump1090
pass .1.3.6.1.2.1.25.1.10 /bin/sh /usr/local/bin/snmp-dump978
view systemonly included .1.3.6.1.2.1.25

I created two files in the /usr/local/bin directory that snmp references. /usr/local/bin/snmp-dump1090:

#!/bin/bash
if [ "$1" = "-g" ]; then
        echo .1.3.6.1.2.1.25.1.9
        echo gauge
        /bin/grep "dump1090-fa.*5m).*FlightAware" /var/log/piaware.log | /usr/bin/tail -n 1 | /usr/bin/cut -f2 -d"(" | /usr/bin/cut -f1 -d" "
fi

and /usr/local/bin/snmp-dump978:

#!/bin/bash
if [ "$1" = "-g" ]; then
        echo .1.3.6.1.2.1.25.1.10
        echo gauge
        /bin/grep "dump978-fa.*5m).*FlightAware" /var/log/piaware.log | /usr/bin/tail -n1 | cut -f2 -d"(" | /usr/bin/cut -f1 -d" "
fi

I added the following section to my /etc/mrtg.conf file:

######################################################################
#       FlightAware messages
######################################################################
Options[_]: gauge, nopercent, transparent, pngdate
Factor[_]:
MaxBytes[_]: 12500000
YLegend[_]: Messages
ShortLegend[_]: Messages
LegendO[_]: dump1090 messages
LegendI[_]: dump978 messages

# Target[WimPi4_piaware]: `/home/wim/MRTG-PiAware.sh`
Target[WimPi4_piaware]: .1.3.6.1.2.1.25.1.10&.1.3.6.1.2.1.25.1.9:public@WimPi4
Title[WimPi4_piaware]: Pi4 FlightAware messages
PNGTitle[WimPi4_piaware]: Pi4 FlightAware Messages
PageTop[WimPi4_piaware]: Pi4 FlightAware messages

Target[WimPiZeroCamera_dump1090]: .1.3.6.1.2.1.25.1.9&.1.3.6.1.2.1.25.1.9:public@WimPiZeroCamera
Options[WimPiZeroCamera_dump1090]: gauge, nopercent, transparent, pngdate, noi
Title[WimPiZeroCamera_dump1090]: PiZero FlightAware messages
PNGTitle[WimPiZeroCamera_dump1090]: PiZero FlightAware Messages
PageTop[WimPiZeroCamera_dump1090]: PiZero FlightAware messages

I was testing that snmp was responding with data using the snmpget commands, but I was getting zeros back. I could run the scripts directly while I was logged in with my regular account and they were producing results so I was scratching my head.

pi@WimPi4:~ $ snmpget -v 2c -c public wimpi4 .1.3.6.1.2.1.25.1.9
iso.3.6.1.2.1.25.1.9 = Gauge32: 130
pi@WimPi4:~ $ snmpget -v 2c -c public wimpi4 .1.3.6.1.2.1.25.1.10
iso.3.6.1.2.1.25.1.10 = Gauge32: 0

The extra complication that added about six hours to my figuring things out was that the /var/log/piaware.log log files were not all readable, only owner and group. The snmp daemon was not running as a member of any group that had access to that file. I verified the problem with the command:

sudo -u Debian-snmp cat /var/log/piaware.log

I fixed the problem with the quick hack of:

sudo chmod a+r /var/log/pi*

 

Govee H5075 and H5074, Bluetooth Low Energy, and MRTG

by

I have been wanting a method of keeping track of temperatures for a long time. Last week I acquired a Govee H5075 Bluetooth Thermometer Hygrometer. It communicates with an app from Govee on my iPhone using Bluetooth Low Energy (BLE).

I’ve now learned some details on BLE, and have written a program that listens for BLE advertisements from either type of thermometer and logs the temperature and humidity in a text file. The code for my project is available on GitHub. https://github.com/wcbonner/GoveeBTTempLogger

The same program can also be called to get the last value from the log and produce output compatible with MRTG. MRTG is not the best method for graphing these temperatures, because all graphs start with zero on the Y axis, and neither the temperature or humidity is likely to be near zero.

MRTG graph of Temperature and Relative Humidity

My program seems to receive advertisements from each thermometer about every ten seconds. I’ve had a friend running the code in his location with a different set of thermometers and it doesn’t get advertisements nearly as frequently. I don’t know if that’s just because environment is different, or if there’s something else going on.

Monitoring Raspberry Pi with MRTG

by

I’ve used MRTG for simple monitoring for years. It’s easy to get working and dependent on very few packages. It stores it’s data in simple files. This both limits it, and makes it easy to move or duplicate.

I wanted to monitor each of my Raspberry Pi network interfaces because they are connected via WiFi and I can’t monitor a particular switch port for each device. I’ve spent nearly a year searching for the reason that MRTG didn’t enumerate the interfaces before coming up with a simple snippet fixing my problem.

Adding this line to the end of my /etc/snmp/snmpd.conf file and restarting the snmpd allowed me to run cfgmaker and see my network interfaces.

view   systemonly  included   .1.3.6.1.2.1.2

Quick and dirty addition and query:

sudo echo view   systemonly  included   .1.3.6.1.2.1.2 >>/etc/snmp/snmpd.conf
sudo systemctl restart snmpd

/usr/bin/cfgmaker --no-down --zero-speed=100000000 public@localhost
Thanks to https://www.seei.biz/cpu-temperature-of-a-raspberry-pi-via-snmp/ for giving me the simple answer that I’d been trying to figure out for over a year.