pmacct

pmacct is an open source and freely distributed software project that contains a small collection of command-line passive network monitoring utilities, which allow users to export, measure, classify, account and aggregate IPv6 and IPv4 traffic. It comprises of the nfacctd, pmacct, pmacctd, sfacctd and uacctd tools.

Features at a glance

Key features include support for Cisco NSEL and Cisco NEL for CGNAT, inspection of tunnelled traffic, MPLS/BGP VPNs rfc4364 support, GeoIP lookups, support for SQL (triggers, data pre-processing, dynamic table naming), support for collecting data through the libpcap library, as well as through Netlink/ULOG, sFlow v2/v4/v5, NetFlow v1/v5/v7/v8/v9, and IPFIX.

Additionally, the software is also capable of exporting data to external utilities, such as GNUPlot, RRDtool, Net-SNMP, Cacti and MRTG, it is suitable for IP carrier, ISP, CDN, IXP, hot-spots and data-center environments, can save data to various backends, including MySQL, PostgreSQL, SQLite, MongoDB, BerkeleyDB, RabbitMQ, memory tables, and flat files.

Among other interesting features, we can mention support for BGP ADD-PATHS, logging of live BGP messages, dumping of BGP tables, visualization of internal routes through an IS-IS/IGP daemon, classification traffic streams, visualization of inter-domain routing plane through a BGP daemon, replication of incoming NetFlow, sFlow and IPFIX packets, and a pluggable architecture.

Getting started with pmacct

Being a command-line software, the pmacct project can only be used from a console/CLI environment. To install it on your GNU/Linux distribution, you can either use the built-in Software Center app of your operating system (e.g. Ubuntu Software Center on Ubuntu) or the source package, which is distributed for free on Softoware and the project’s official website (see the homepage link at the end of the article).

If you decide to install pmacct using the source package, download it, save it on your Home folder, use an archive manager tool to unpack it, open a terminal emulator app and go to the location where you’ve extracted the archive file (e.g. cd /home/softoware/pmacct-1.5.0). Then, run the ‘./configure && make’ command to configure and compile the program, followed by the ‘sudo make install’ command to install the tools system wide.

After installation, add the “--help” option to each of the included tools (see the first paragraph for details) to view their command-line options, the usage message, as well as various real life examples, which will help you to better understand how they work and how you can use them in your work. Please note that pmacct runs on Linux, BSD, Solaris, and embedded systems.

What is new in this release:

• Introduced bgp_daemon_msglog_file config directive to enable streamed logging of BGP messages/events. Each log entry features a time reference, BGP peer IP address, event type and a sequence number (to order events when time reference is not granular enough). BGP UPDATE messages also contain full prefix and BGP attributes information. Example given in QUICKSTART file, chapter XIIf.
• Introduced dump of BGP tables at regular time intervals. The filename, which can include variables, is set by bgp_table_dump_file directive. The output format, currently only JSON, can be set in future via the bgp_table_dump_output directive. The time interval between dumps can be set via the bgp_table_dump_refresh_time directive. Example given in QUICKSTART file, chapter XIIf.
• Introduced support for internally variable-length primitives (likely candidates are strings). Introduced also the 'label' primitive which is a variable-length string equivalent of tag and tag2 primitives. Its value are set via a 'set_label' statement in a pre_tag_map (see examples/ pretag.map.example). If, ie. as a result of JEQ's in a pre_tag_map, multiple 'set_label' are applied, then default operation is append labels and separate by a comma.
• pmacct project has been assigned PEN #43874. nfprobe plugin: tag, tag2, label primitives are now encoded in IPFIX making use of the pmacct PEN.
• Ported preprocess feature to print, MongoDB and AMQP plugins. Preprocess allows to process aggregates (via a comma-separated list of conditionals and checks) while purging data to the backend thus resulting in a powerful selection tier. minp, minb, minf, minbpp, minppf checks have been currently ported. As a result of the porting a new set of config directives are added, ie. print_preprocess and print_preprocess_type.
• print, MongoDB & AMQP plugins: if data (start/base) time is greater than commit time then place in pending queue and after purging event re-insert in cache. Concept ported from SQL plugins.
• MySQL, PostgreSQL plugins: sql_locking_style now supports keyword "none" to disable locking. This method can help in certain cases, for example when grants over the whole database (requirement for "table" locking in MySQL) is not available.
• util.c: open_logfile() now calls mkdir_multilevel() to allow building intermediate directory levels, if not existing. This brings all log files in line with capabilities of print_output_file directive.
• Introduced [u|pm]acctd_flow_tcp_lifetime to defines how long a TCP flow could remain inactive. This is in addition to [u|pm]acctd_flow_lifetime that allows to define the same for generic, ie. non-TCP, flows. Thanks to Stathis Gkotsis for his support.
• Introducing nfacctd_account_options: if set to true account for NetFlow/ IPFIX option records as well as flow ones. pre_tag_map offers sample_type value of 'option' now to split option data records from flow ones.
• nfprobe plugin: support for custom-defined primitives has been introduced in line with other plugins. With such feature it will be possible to augment NetFlow v9/IPFIX records with custom fields (in IPFIX also PENs are supported).
• Built a minimal API, for internal use only, around AMQP. Goal is to make re-use of the same AMQP structures for different purposes (logging, BGP daemon dumps, AMQP plugin, etc.). ! fix, BGP daemon: introduced bgp_peer_info_delete() to delete/free BGP info after a BGP peer disconnects. ! fix, print, AMQP, memory plguins: when selecting JSON output, jansson library json_decref() is used in place of free() to free up memory allocated by JSON objects. Using free() was originating memory leaks. ! fix, AMQP plugin: in line with other plugins QN (query number or in case of AMQP messagess number) in log messages now reflects the real number of messages sent to the RabbitMQ message exchange and not just all messages in the queue. Thanks to Gabriel Snook for reporting the issue. ! fix, IMT plugin: memory leak due to missed calls to free_extra_allocs() in case all extras.off_* were null. Thanks to Tim Jackson for his support resolving the issue. ! fix, pmacctd: if reading from a pcap_savefile, introduce a short usleep() after each buffer worth of data so to give time plugins to process/cache it. ! fix, SQL plugins: SQL handler types now include primitives registry index ! fix, print, AMQP & MongoDB plugins: added free() for empty_pcust allocs ! fix, plugin hooks: improved checks to prevent the last buffer on a pipe to plugins (plugin_pipe_size) could go partly out of bounds. ! fix, nfacctd: improved handling of IPFIX vlen records. ! fix, nfprobe: SEGV if custom primitives are defined but array structure is not allocated. ! fix, nfprobe: wrong length was calculated in IPv6 templates for fields with PEN != 0. ! fix, plugin_common.c: declared struct pkt_data in P_cache_insert_pending to be pointed by prim_ptrs. primptrs_set_all_from_chained_cache() is now safe if prim_ptrs is null. ! fix, nfprobe: tackled the case of coexisting 1) PEN and non-PEN custom primitives and 2) variable and fixed custom primitives. ! fix, plugin_common.c: declared struct pkt_data in P_cache_insert_pending to be pointed by prim_ptrs. primptrs_set_all_from_chained_cache() is now safe if prim_ptrs is null. ! fix, lofging: selected configuration file is now logged. cfg_file is passed through realpath() in order to always log the absolute path. ! fix, print, MongoDB & AMQP plugins: pm_setproctitle() invoked upon forking writer processes in alignment with SQL plugins. ! fix, pmacct client: it's now possible to query and wildcard on primitives internally allocated over what_to_count_2 registry.

What is new in version 1.5.0 RC1:

• Introducing custom-defined aggregation primitives: primitives are defined via a file pointed by aggregate_primitives config directive. The feature applies to NetFlow v9/IPFIX fields only, and with a pre-defined length. Semantics supported are: 'u_int' (unsigned integer, presented as decimal number), 'hex' (unsigned integer, presented as hexa- decimal number), 'ip' (IP address), 'mac' (MAC address)and 'str' (string). Syntax along with examples are available in the 'examples/primitives.lst' file.
• Introducing JSON output in addition to tabular and CSV formats. Suitable for injection in 3rd party tools, JSON has the advantage of being a self- consisting format (ie. compared to CSV does not require a table title). Library leveraged is Jansson, available at: http://www.digip.org/jansson/
• Introducing RabbitMQ/AMQP pmacct plugin to publish network traffic data to message exchanges. Unicast, broadcast, load-balancing scenarios being supported. amqp_routing_key supports dynamic elements, like the value of peer_src_ip and tag primitives or configured post_tag value, enabling selective delivery of data to consumers. Messages are encoded in JSON format.
• pre_tag_map (and other maps): 'ip' key, which is compared against the IP address originating NetFlow/IPFIX or the AgentId field in sFlow, can now be an IP prefix, ie. XXX.XXX.XXX.XXX/NN, so to apply tag statements to set of exporters or 0.0.0.0/0 to apply to any exporter. Many thanks to Stefano Birmani for his support.
• Re-introducing support for Cisco ASA NSEL export. Previously it was just a hack. Now most of the proper work done for Cisco NEL is being reused: post_nat_src_host (field type #40001), post_nat_dst_host (field type #40002), post_nat_src_port (field type #40003), post_nat_dst_port (field type #40004), fw_event (variant of nat_event, field type #40005) and timestamp_start (observation time in msecs, field type #323).
• Introducing MPLS-related aggregation primitives decoded from NetFlow v9/ IPFIX, mpls_label_top mpls_label_bottom and mpls_stack_depth, so to give visibility in export scenarios on egress towards core, MPLS interfaces.
• mpls_vpn_rd: primitive value can now be sourced from NetFlow v9/IPFIX field types #234 (ingressVRFID) and #235 (egressVRFID). This is in addition to existing method to source value from a flow_to_rd_map file.
• networks_file: AS field can now be defined as "_", Useful also to define (or override) elments of an internal port-to-port traffic matrix.
• print plugin: creation of intermediate directory levels is now supported; directories can contain dynamic time-based elements hence the amount of variables in a given pathname was also lifted to 32 from 8.
• print plugin: introduced print_history configuration directive, which supports same syntax as, for example, sql_history. When enabled, time- related variables substitution of dynamic print_output_file names are determined using this value instead of print_refresh_time one.
• Introducing IP prefix labels, ie. for custom grouping of own IP address space. The feature can be enabled by a --enable-plabel when configuring the package for compiling. Labels can be defined via a networks_file.
• mongo_user and mongo_passwd configuration directive have been added in order to support authentication with MongoDB. If both are omitted, for backward compatibility, authentication is disabled; if only one of the two is specified instead, the other is set to its default value.
• Introducing mongo_indexes_file config directive to define indexes in collections with dynamic name. If the collection does not exist yet, it is created. Index names are picked by MongoDB.
• print plugin: introduced print_output_file_append config directive: if set to true allows the plugin to append to an output file rather than overwrite.
• bgp_agent_map: added bgp_port key to lookup a NetFlow agent also against a BGP session port (in addition to BGP session IP address/router ID): it aims to support scenarios where BGP sessions do NAT traverals.
• peer_dst_ip (BGP next-hop) can now be inferred by MPLS_TOP_LABEL_ADDR (NetFlow v9/IPFIX field type #47). This field might replace BGP next-hop when NetFlow is exported egress on MPLS-enabled core interfaces.
• Introducing [nf|pm|sf|u]acctd_proc_name config directives to define the name of the core process (by default always set to 'default'). This is the equivalent to instantiate named plugins but for the core process. Thanks to Brian Rak for bringing this up.
• pre_tag_map: introduced key 'flowset_id' to tag NetFlow v9/IFPIX data records basing on their flowset ID value, part of the flowset header.
• pmacct client: introduced '-V' command-line option to verify version, build info and compile options passed to the configure script; also a new -a option now allows to retrieve supported aggregation primitives and their description.
• Check for mallopt() has been added at configure time. mallopt() calls are introduced in order to disable glibc malloc() boundary checks.
• ! flow_to_rd_map replaces iface_to_rd_map, increasing its scope: it is now possible to map couples to BGP/ MPLS VPN Route Distinguishers (RD). This is in addition to existing mapping method basing on .
• ! fix, nfacctd, sfacctd: Setsocksize() call effectiveness is now verified via a subsequent getsockopt(). If result is different than expected, an informational log message is issued.
• ! fix, building system: removed stale check for FreeBSD4 and introduced check for BSD systems. If on a BSD system, -DBSD is now passed over to the compiler.
• ! fix, tee plugin: transparent mode now works on FreeBSD systems. Patch is courtesy by Nikita V. Shirokov.
• ! fix, peer_dst_ip: uninitialized pointer variable was causing unexpected behaviours. Thanks to Maarten Bollen for his support resolving this.
• ! fix, IMT plugin: selective queries with -M and -N switches verified not working properly. Thanks to Acipia organization for providing a patch.
• ! fix, sql_common.c: src_port and dst_port primitives correctly spelled if used in conjunction with BGP primitives. Thanks to Brent Van Dussen and Elisa Jasinska for flagging the issue.
• ! fix, building system: added library checks in /usr/lib64 for OS's where it is not linked to /lib where required.
• ! fix, print, MongoDB and AMQP plugins: P_test_zero_elem() obsoleted. Instead, the cache structure 'valid' field is used to commit entries to the backend.
• ! fix, nfacctd: in NetFlow v9/IPFIX, if no time reference is specified as part of records, fall back to time reference in datagram header.
• ! fix, MongoDB plugin: mongo_insert_batch() now bails out with MONGO_FAIL if something went wrong while processing elements in the batch and an error message is issued. Typical reason for such condition is batch is too big for the resources, mainly memory, available. Thanks very much to Maarten Bollen for his support.
• ! fix, cfg_handlers.c: all functions parsing configuration directives, and expecting string arguments, are now calling lower_string() so to act as case insensitive.
• ! fix, IPv6 & NetFlow exporter IP address: upon enabling IPv6, NetFlow exporter IP addresses were written as IPv4-mapped IPv6 address. This was causing confusion when composing maps since the 'ip' field would change depending on whether IPv6 was enabled or not. This is now fixed and IPv4- mapped IPv6 addresses are now internally translated to plain IPv4 ones.
• ! fix, nfacctd: NetFlow v9/IPFIX source/destination peer ASN information elements have been found mixed up and are now in proper order.

What is new in version 0.14.0:

• This version integrates an IS-IS daemon, which is being run as a parallel thread within the collector core process.
• It implements a single L2 P2P neighborship, i.e. over a GRE tunnel, P2P Hello, CSNP, and PSNP, and does not send any LSP information out.
• A new aggregation primitive 'etype' is introduced in order to support accounting against the EtherType field of Ethernet frames.
• Support for samples generated on ACL matches in Brocade (sFlow sample type: Enterprise: #1991, Format: #1) is now also introduced.
• Several bugfixes are also included in this release.

What is new in version 0.14.0 RC3:

• The BGP daemon now features an implementation of BGP/MPLS VPNs (rfc4364): it correlates couples to Route Distinguisher (RD) values.
• The print plugin can now write network traffic data to flat-files (print_output_file) featuring formatted or CSV output, dynamic filenames, historical breakdown, and triggers.
• pmacctd now supports the DLT_LOOP link-type (i.e. OpenBSD tunnel interfaces).
• Several enhancements to the uacctd daemon and other minor new features and miscellaneous fixes are included.

What is new in version 0.14.0 RC2:

• A sampling_map feature has been introduced, allowing definition of static traffic sampling mappings.
• Further work on the NetFlow v9/IPFIX sampling includes support for 16 bits SAMPLER_IDs (seen against IOS-XR) and support for (FLOW)_SAMPLING_INTERVAL fields as part of the NetFlow v9/IPFIX data record.
• [ns]facctd_as_new and [ns]facctd_net both feature a new "fallback" option to look up BGP-related primitives against BGP first and, if not successful, against the export protocol.
• Other minor new features and misc.
• Fixes are also included in this release.

What is new in version 0.12.5:

• nfacctd, the pmacct NetFlow collector, has been enhanced to support NAT L3/L4 fields (IPFIX field types 225, 226, 227, 228, 281, and 282), Cisco ASA NetFlow v9 NSEL (field types 40001, 40002, 40003, 40004 152, 153, and 323) and application classification (aligning to Cisco NBAR-NetFlow v9 integration).
• The NetFlow probe plugin, nfprobe, introduces egress IPv4/IPv6 NetFlow v9 templates.
• The BGP thread now supports IPv6 transport and mapping to sFlow/NetFlow agents; BGP peer information is linked into the status table for caching purposes, which results in good CPU savings on bigger deployments.

What is new in version 0.12.4:

• BGP daemon: a new memory model is introduced by which IP prefixes are being shared among the BGP peers RIBs - leading to consistent memory savings whenever multiple BGP peers export full tables due to the almost total overlap of information. Longest match nature of IP lookups required to raise BGP peer awareness of the lookup algorithm.
• Updated INTERNALS document to support estimation of the memory footprint of the daemon.
• BGP daemon: a new bgp_table_peer_buckets configuration directive is introduced: per-peer routing information is attached to IP prefixes and now hashed onto buckets with conflict chains. This parameter sets the number of buckets of such hash structure; the value is directly related to the number of expected BGP peers, should never exceed such amount and is best set to 1/10 of the expected number of peers.
• nfprobe: support has been added to export direction field (NetFlowv9 field type #61); its value, 0=ingress 1=egress, is determined via nfprobe_direction configuration directive.
• nfacctd: introduced support for Cisco ASA bytes counter, NetFlow v9 field type #85. Thanks to Ralf Reinartz for his support.
• nfacctd: improved flow recognition heuristics for cases in which Pv4/IPv6/input/output data are combined within the same NetFlowv9 template. Thanks to Carsten Schoene for his support.
• fix, BGP daemon: bgp_nexthop_followup was not working correctly if pointed to a non-existing next-hop.
• fix, nfv9_template.c: ignoring unsupported NetFlow v9 field types; improved template logging. Thanks to Ralf Reinartz for his support.
• fix, print plugin: support for interfaces and network masks has been added. Numeric variables output converted to unsigned from signed.

What is new in version 0.12.3:

• 'cos' aggregation primitive has been implemented providing support for 802.1p priority. Collection is supported via sFlow, libpcap and ULOG; export is supported via sFlow.
• BGP daemon: TCP MD5 signature implemented. New 'bgp_daemon_md5_file' configuration directive is being added for the purpose of defining peers and their respective MD5 keys, one per line, in CSV format. The map is reloadable at runtime: existing MD5 keys are removed via setsockopt(), new ones are installed as per the newly supplied map. Sample map added in 'examples/bgp_md5.lst.example'.
• BGP daemon: added support for RFC3107 (SAFI=4 label information) to enable receipt of labeled IPv4/IPv6 unicast prefixes.
• nfprobe, sfprobe: introduced the concept of traffic direction. As a result, [ns]fprobe_direction and [ns]fprobe_ifindex configuration directives have been implemented.
• [ns]fprobe_direction defines traffic direction. It can be statically defined via 'in' or 'out' keywords; values can also be dynamically determined through a pre_tag_map (1=input, 2=output) by means of 'tag' and 'tag2' keywords.
• [ns]fprobe_ifindex either statically associate an interface index (ifIndex) to a given [ns]fprobe plugin or semi-dynamically via lookups against a pre_tag_map by means of 'tag' and 'tag2' keywords.
• sfprobe: sfprobe_ifspeed configuration directive is introduced and aimed at statically associating an interface speed to an sfprobe plugin.
• sfprobe: Switch Extension Header support added. Enabler for this development was support for 'cos' and in/out direction. Whereas VLAN information was already supported as an aggregation primitive.
• sfprobe: added support for Counter Samples for multiple interfaces. Sampling function has been brought to the plugin so that Counter Samples can be populated with real bytes/packets traffic levels. ! nfprobe, sfprobe: send buffer size is now aligned to plugin_pipe_size, if specified, providing a way to tune buffers in case of sustained exports. ! fix, addr.c: pm_ntohll() and pm_htonll() routines rewritten. These are aimed at changing byte ordering of 64-bit variables. ! fix, BGP daemon: support for IPv6 global address/link-local address next-hops as part of MP_REACH_NLRI parsing. ! fix, cfg_handlers.c: bgp_daemon and bgp_daemon_msglog parsing was not correct, ie. enabled if specified as 'false'. Thanks to Brent Van Dussen for reporting the issue. ! fix, bgp.c: found a CPU hog issue caused by missing cleanup of the select() descriptors vector. ! fix, pmacct.c: in_iface/out_iface did erroneously fall inside a section protected by the "--disable-l2" switch. Thanks to Brent Van Dussen for reporting the issue.

What is new in version 0.12.1:

• Input/output interfaces (SNMP indexes) have now been implemented natively; it's therefore not required anymore to pass through the (Pre-)tag infrastructure. As a result two aggregation primitives are being introduced: 'in_iface' and 'out_iface'.
• Support for source/destination IP prefix masks is introduced via two new aggregation primitives: src_mask and dst_mask. These are populated as defined by the [nf|sf|pm|u]acctd_net directive: NetFlow/sFlow protocols, BGP, Network files (networks_file) or static (networks_mask) being valid data sources.
• A generic tunnel inspection infrastructure has been developed to benefit both pmacctd and uacctd daemons. Handlers are defined via configuration file. Once enabled daemons will account basing upon tunnelled headers rather than the envelope. Currently the only supported tunnel protocol is GTP, the GPRS tunnelling protocol (which can be configured as: "tunnel_0: gtp, "). Up to 8 different tunnel stacks and up to 4 tunnel layers per stack are supported. First matching stack, first matching layer wins.
• uacctd: support for the MAC layer has been added for the Netlink/ ULOG Linux packet capturing framework.
• 'nfprobe_source_ip' feature introduced: it allows to select the IPv4/IPv6 address to be used to export NetFlow datagrams to the collector.
• nfprobe, sfprobe: network masks are now exported via NetFlow and sFlow. 'pmacctd_net' and its equivalent directives define how to populate src_mask and dst_mask values. ! cleanup, nfprobe/sfprobe: data source for 'src_as' and 'dst_as' primitives is now expected to be always explicitely defined (in line with how 'src_net' and 'dst_net' primitives work). See the UPGRADE doc for the (limited) backward compatibility impact. ! Updated SQL documentation: sql/README.iface guides on 'in_iface' and 'out_iface' primitives; sql/README.mask guides on 'src_mask' and 'dst_mask' primitives; sql/README.is_symmetric guides on 'is_symmetric' primitive. ! fix, nfacctd.h: source and destination network masks were twisted in the NetFlow v5 export structure definition. Affected releases are: 0.12.0rc4 and 0.12.0. ! fix, nfprobe_plugin.c: l2_to_flowrec() was missing some variable declaration when the package was configured for compilation with --disable-l2. Thanks to Brent Van Dussen for reporting the issue. ! fix, bgp.c: bgp_attr_munge_as4path() return code was not defined for some cases. This was causing some BGP messages to be marked as malformed. ! fix, sfprobe: a dummy MAC layer was created whenever this was not included as part of the captured packet. This behaviour has been changed and header protocol is now set to 11 (IPv4) or 12 (IPv6) accordingly. Thanks to Neil McKee for pointing the issue. ! workaround, building sub-system: PF_RING enabled libpcap was not recognized due to missing of pcap_dispatch(). This is now fixed.