Server Monitoring

These are two programs for server monitoring:

Munin

To install munin, you must install the main component and the node component (if you plan to use munin to monitor multiple systems, install the munin-node on each system. For this example, it will only be used on one server).

Install munin-main:

cd /usr/sysutils/munin-main
make
make install

Install munin-node:

cd /usr/sysutils/munin-node
make
make install

Unless this file already existed, a sample configuration file has been placed in /usr/local/etc/munin/munin.conf. After installing the node component, you must add "munin_node_enable="YES" to the rc.conf file. 

Note: If you are reading a Munin "how to" for Linux while attempting to install it on FreeBSD, here is a timesaver:

You won't find init.d or any of the runlevel subdirectories on a BSD system. Instead, the system startup scripts live in /etc/rc.d, and the startup scripts for third-party applications stay in /usr/local/etc/rc.d  (Source)

After several days of attempting to get munin to work, it was concluded that the port for FreeBSD is broken, and only fixable if you install munin as described above and then patch it with this patch

To install the patch, place it in the directory "/usr/local/share/munin" and change the name of it to munin-graph.patch. Change to /usr/local/share/munin and issue this command (for more information on the patch command):

patch < munin-graph.patch

You should get the following output:

Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- munin-graph.dist Mon Sep 5 17:17:18 2005
|+++ munin-graph Mon Sep 5 18:13:48 2005
--------------------------
Patching file munin-graph using Plan A...
Hunk #1 succeeded at 728.
Hunk #2 succeeded at 849.
Hunk #3 succeeded at 925.
Hunk #4 succeeded at 992.
Hunk #5 succeeded at 1011.
Hunk #6 succeeded at 1042.
done

Munin should work correctly at this point.

In the process of installing munin, here are problems you may encounter and their solutions:

Problem: Use of uninitialized value in scalar chomp at /usr/share/munin/munin-update line 753.

Solution: Change line 753 to:

if ($res) { chomp $res; }

or:

chomp $res if ($res);

or:

($res) && chomp($res);

Problem: 

Error writing to /var/run/munin/munin-update.lock, wrong permissions at /usr/local/lib/perl5/site_perl/5.8.7/Munin.pm line 356.
Error writing to /var/run/munin/munin-graph.lock, wrong permissions at /usr/local/lib/perl5/site_perl/5.8.7/Munin.pm line 356.
Error writing to /var/run/munin/munin-html.lock, wrong permissions at /usr/local/lib/perl5/site_perl/5.8.7/Munin.pm line 356.

Solution:

The default user for running munin, as found in its sample configuration files, is user: root and group: wheel. FreeBSD doesn't like muning running as root. If you do, you have to run the cron job and any parts of munin you execute from the command line with the -force-root option (example ./munin-graph -force-root).

You also have to make sure that whatever user name and group you run the script under, that you make sure that all of the permissions directories and files for the files under /var/log/munin-main and /var/run/munin have are owned by that same user name/group, and that all files can be written to by that same user name/group as well. Once you get all of the permissions corrected, you will not get the above errors.

Problem:

Unable to graph /usr/local/var/munin/localhost/localhost-mysql_threads-threads-g.rrd: Garbage ':' after command:
COMMENT: Cur:

Solution:

The official line is, "Colons in COMMENT arguments to rrdtool graph must be escaped with a backslash."

This has been officially labeled an rrdtool error and not a munin error. 

This is an aggravating problem, which is fixed by the munin patch provided at the beginning of this piece, and the patch has not made its way into the ports at this time. This error was produced on all versions rrdtool and munin tried, except when using rrdtool 1.0.49, which does not produce this error, but also will not work with munin because munin uses the rrdtool version database format 1.2.x format -- which give you the error, "can't handle RRD file version 0003"

Problem:

rrd_update(/usr/share/ntop/rrd/interfaces/eth2/IP_SNMPBytes.rrd) error:
can't handle RRD file version 0003

Solution:

There are "Major Changes between 1.0.x and 1.2.x." The worst is that, because arguments to rrdtool graph must be escaped with a backslash, it broke munin, which must be patched with the patch provided at the beginning of this piece to work. To get rid of the "Garbage ':' after command" errors, the last version of rrdtool that worked with munin was 1.0.x builds. If you install a 1.0.x build, you will no longer the "Garbage ':' after command" errors, However, because munin uses the updated database used by rrdtool 1.2.x, you now get an error in your munin-graph.log file to the effect of, "can't handle RRD file version 0003."

While the new version of rrdtool can read files created with rrdtool 1.0.x. It is not possible to read files created by rrdtool-1.2.x with rrdtool-1.0.x

If you are getting the above error, you have to install the patch are you are just going to waste a lot of time and get nowhere.


Monit

To install monit,

cd /usr/sysutils/monit
make
make install

To enable monit you need to add monit_enable="YES" to rc.conf file. Before running monit you have to configure monitrc file. There is example configuration file monitrc.sample, you can find many samples for particular services in examples.html:

/usr/local/etc/monitrc.sample
/usr/local/share/doc/monit/examples.html

Rename monitrc.sample to monitrc and copy to /etc