Projects :: DAPd

Have you ever been in charge of a distributed serverpark, colocated at more than one IP supplier, wanting to monitor all these machines from a single box, checking their system load, logged in users, network throughput, and seeing at a glance what server is doing which tasks ? With a growing serverpark running any of NetBSD, OpenBSD, FreeBSD or even Linux, do you lose track of where your bandwidth goes ? Tired of crashing snmpd's on your servers, and IP colocation admins unwilling to run MRTG on your switchports ? Are you an admin at an ISP, with a server you frequently log on to, to read mail, do daily chores etcetera. Do you know your fellow admins have a server of their own also, and sometimes want to be able to figure out where your admin-friends are ? Have you ever ran rwhod, finding that it's stability and portability is poor, that it has to at least start as root due to privileged portusage, and most of the time also remains in possession of superuser privileges listen to the hostile Internet for incoming (possibly forged) packets ? Tired of seeing other people's boxes on your stats, just because they are in the same subnetwork as you ?
Then the dap daemon is a softwarepackage for you!

What is DAPd?

It is a daemon that runs entirely as a non-privileged user (eg your own user account), sending regular updates of the server status to a specified set of peers in a cluster. You can specify any number of information elements, such as uptime, networkdevice counters (packet/octet), logged in users (via utmp), load averages and memory/swap usage. When the program starts, it binds a userdefined UDP socket on the local machine, listening for incoming updates from its peers, or optionally becoming mute and thus safe from possible attacks from the Internet. All incoming packets are verified and then stored in a local state directory, for use by the client programs. On the client side, there are a number of programs which read the state directory and extract the information needed. One utility reports uptime/load, another expands the 'who' functionaly over the entire cluster, counts the amount of logged in users, locates a specific user by least-idle TTY, and more are to come.

Why run DAPd?

If you have a set of servers for which you want to see information like logged in users, processor load, memory usage, network usage, and you want to see all these statistics on every box (or just one NOC box), you will want to have a sampling program on each server sending information around about its status. If you work at an ISP and would like to be in closer contact with your fellow admins across the network, or even across the Internet. If you want to find me, and the machine I am logged onto, with the least idletime, you can use the 'rwho' utility to locate me using the information my server has sent around the network regarding logged in users. If you think Netsaint or Nagios are too elaborate to take care of a simple task such as monitoring load or memory usage (NRPE and check_by_ssh do not appeal to you), you can have dapd monitor these things for you and let them ring a bell if some critical limits are surpassed. If you want runtime statistics about load/mem/cpu/disk usage and others, nicely graphed and presented in a webinterface, you can use dapd as a basis for measuring things and pushing them into a round robin database.

Features in DAPd

The current features include: The DAP daemon keeps track of, and administrates:

Privileges needed to run DAPd

The DAP daemon runs mostly with no privileges at all. It was designed to use a user UDP port (1513), and gather all possible system statistics without having to be root. However, the statistics are not available on all operating systems, and some need elevated privileges to request specific statistics. Swap and RAM work on Net- and OpenBSD thanks to the uvmexp struct. This does not require any specific privileges. FreeBSD does not have uvmexp which makes it require group kmem to read the kvm. Linux is supposed to have disk IO stats in /proc/stat; however, only the first two IDE disks get accounted for (hda/hdb). Besides, Linux is quite inconsistant when it comes to major/minor node numbering. FreeBSD provides the libdevstats to get disk IO statistics without privileges. The other BSDs need kvm.

Happy Users

The following companies and users chose to adopt the dapd software as a part of their daily admin-tools at their site.

History - Changelog

As with most systemsoftware, there have been changes to the DAPd architecture mostly due to the everchanging open source operating systems it runs on. When kernels evolve, so do the ways in which we interface with them. Below is a list of changes I've made to keep DAPd useful throughout the years. Linux /proc/meminfo Linux kernels 2.2, 2.4 and 2.6 all treat /proc/meminfo differently. DAPd was patched to understand all three versions. Note that there is no more 'shared memory' in /proc/meminfo for Linux 2.6 and up. FreeBSD libdevstats FreeBSD4 has quite a simple interface to the devstats library. DAPd uses this interface. FreeBSD5 however changed significantly with regards to the way disk IO is read. DAPd currently does not support the FreeBSD5 interface (so disk IO statistics are unavailable there).