The PF (packet filter) firewall package was introduced in OpenBSD 3.0, and has since been ported to the FreeBSD and NetBSD Operating Systems. PF contains a stateful packet inspection engine, the ability to replicate state information to a backup firewall, a flexible self optimizing rule engine, QOS support, and the ability to collect performance metrics. These metrics can be useful for gauging the performance of a firewall platform, and provide a way to trend firewall performance over time. This article will describe several utilities that can be used to monitor the health and performance of a PF firewall.
The pfctl utility provides a command line interface to monitor and control the PF firewall module. pfctl contains options to display firewall configuration information and a variety of performance and status counters. To display the status and performance counters, the “info” parameter can be passed to pfctl’s “-s” (show values) option:
$ sudo pfctl -s info``
Status: Enabled for 2 days 10:33:04 Debug: Urgent
Hostid: 0xdf1a2b73
Interface Stats for tun0 IPv4 IPv6
Bytes In 577197106 0
Bytes Out 30295580 0
Packets In
Passed 564503 0
Blocked 1494 0
Packets Out
Passed 386562 0
Blocked 0 0
State Table Total Rate
current entries 1
searches 2164707 10.3/s
inserts 23601 0.1/s
removals 23600 0.1/s
Counters
match 1226533 5.8/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 0 0.0/s
memory 0 0.0/s
In addition to displaying the counters listed above, the show values option can be used to display firewall rulesets, NAT table entries, state table entries, and ALTQ queues. To get detailed usage data for each show values option, pfctl can be invoked with the “-v” (verbose) option. The following example shows how to get detailed usage data for each firewall rule:
$ sudo pfctl -v -s rules``
@0 scrub in all fragment reassemble
[ Evaluations: 3884714 Packets: 1917431 Bytes: 0 States: 0 ]
@1 block drop in log on tun0 all
[ Evaluations: 1200737 Packets: 1368 Bytes: 170289 States: 0 ]
The pfctl(8) man page contains the full list of values that can be be passed to the show values option, and covers pfctl’s advanced options.
The pftop utility provides a “top” like view of the PF state table. pftop displays source and destination IP addresses, TCP and UDP port numbers, packets and bytes transmitted, the age of a connection, and the time left until a connection will be removed from the state table. pftop contains several options to control how data is displayed, and is invoked by running the “pftop” executable from the command line:
$ sudo pftop``
pftop can be installed from the OpenBSD ports collection, or downloaded from the pftop website.
PF can record network packet headers and data when the log key word is used with a rule. Log files can be valuable for debugging rules, understanding traffic flows, and finding performance bottlenecks. When a packet matches a rule with the log key word, the headers and packet body are sent to the pflog pseudo-device. Once a packet is logged to the pflog pseudo-device, The tcpdump utility can be used to print the packet’s contents in real time:
$ sudo tcpdump -i pflog0 -o -ttt -vv -e -n``
Nov 26 00:58:32.001036 rule 4/0(match): block in on tun0: 1.2.3.4.4087 > 5.6.7.8.www: S [tcp sum ok] (src OS: Windows XP SP1, Windows 2000 SP2+) 3615243359:3615243359(0) win 16384 <mss 1440,nop,nop,sackOK> (DF) (ttl 119, id 7135)
........
This example prints protocol headers, timestamps, and an OS fingerprint (using the signatures available in /etc/pf.os) for each packet logged to the pflog pseudo-device. tcpdump’s “-r” option can be used to process PF log files written to the file system by the pflogd daemon:
$ sudo tcpdump -r /var/log/pflog -o -ttt -vv -e -n dst port 80``
Nov 26 00:58:32.001036 rule 4/0(match): block in on tun0: 1.2.3.4.4087 > 5.6.7.8.www: S [tcp sum ok] (src OS: Windows XP SP
1, Windows 2000 SP2+) 3615243359:3615243359(0) win 16384 <mss 1440,nop,nop,sackOK> (DF) (ttl 119, id 7135)
........
This example uses tcpdump’s filtering capabilities to limit the results to connections with a destination port of 80. tcpdump also allows connections to be filtered by IP address, hostname, Ethernet address, TCP flags, etc.
fwanalog is a firewall log file analysis tool that can be used to parse and summarize the log files from several firewall packages (e.g., PF, Checkpoint, ipfilter, iptables, Cisco PIX ). fwanalog uses the analog log file analysis program, and a simple straight forward configuration file to generate reports. The software is available as source code from the fwanalog website.
The fwanalog package contains several sample configuration files, each used to represent a different firewall platform. The “fwanalog.opts.openbsd3” configuration file contains the definitions required to process PF log files on OpenBSD 3.X platforms. When the fwanalog.sh script is invoked, it will process the contents of the configuration file named “fwanalog.opts” by default. The fwanalog documentation recommends creating a symbolic link to point to the platform specific configuration file:
$ ln -s fwanalog.opts.openbsd3 fwanalog.opts``
The fwanalog.opts configuration file contains a series of variables to define the log file locations, the format of each log file, and the location to store the reports. The customizable variables are summarized below:
outdir - Location to store reports
logformat - Format of the log file to analyze
inputfiles_mask - The log file name (supports wildcards)
inputfiles_dir - The directory with the log files
inputfiles_mtime - The maximum age of the log files to parse
The following configuration file can be used to parse PF log files on the OpenBSD platform:
#!/bin/sh
###########################################################################
#
# User-changeable options for fwanalog.sh
#
# $Id: fwanalog.opts.openbsd,v 1.19 2003/11/25 17:11:31 bb Exp $
#
###########################################################################
outdir="/usr/local/fwanalog/reports"
# The directory where the output goes to, without / at the end. You need write
# permissions, of course, and should secure this directory with permissions,
# minefields, guard dogs etc. It will be created if you don`t have it yet.
logformat="pf_30"
# What log format your firewall writes.
# Currently available options:
# iptables Linux 2.4 iptables (probably in /var/log/messages)
# ipchains Linux 2.2 ipchains (probably in /var/log/messages)
# ipf BSD/Solaris ipfilter (probably in /var/log/ipflog)
# openbsd this was the same as ipf until OpenBSD 2.9; this also
# seems to work on NetBSD
# freebsd FreeBSD`s output format (probably in /var/log/ipflog)
# solarisipf Solaris 8.0 Intel ipf 3.4.20 (using ipmon -sn &)
# pf_30 OpenBSD 3.0 pf binary log format
# fwanalog *must* run on OpenBSD 3.0 for this to work
# (because of the special tcpdump of OpenBSD)
# zynos ZyNOS (ZyXEL, Netgear) logfile
# pix Cisco Pix (tested with version 6.22/IOS)
# watchguard Watchguard Firebox
# fw1 Checkpoint Firewall-One (not fw-1 NG!)
# Feel free to program a parser for your firewall if it is not supported.
# See the comments in iptables() and ipf()
#
# The officially maintained formats are pf_30 and iptables.
inputfiles_mask="pflog*gz" # The name of your logfiles, with a wildcard if you want
inputfiles_dir="/var/log" # The directory where your logfiles are in,
# e.g. /var/log
inputfiles_mtime="30" # How old the logfiles can be
# You can change this to your log rotate interval + 1 day (so you never miss a logfile entry)
inputfiles=`find $inputfiles_dir -maxdepth 1 -name "$inputfiles_mask" -mtime -$inputfiles_mtime | sort -r`
# This should find the names of the logfiles you want to parse
# It MUST return the names in reverse order (chronologically) or you
# will have LOTS of duplicate lines in your log.
onehost=dynip
# Available options: false true dynip
# Default: false
# Set to true if this firewall runs on one machine only and you want to see
# the source hosts (not the protected target hosts) in the Blocked Packet
# Report. This is suggested if you protect one server, but loses information
# if you protect a network.
# Set to "dynip" if your firewall has a dynamic IP address.
# After changing onehost, you must delete everything in $outdir!
sep_hosts=false
# Set to true if you want fwanalog to create a separate, additional report for
# each attacking host IP.
# WARNING: this can run for hours using 100 % CPU and consume lots of hard
# disk space (up to 25 kB per host) so you can easily fill up your server if
# too many packets from different hosts were blocked.
# Also, this makes only limited sense with onehost mode set to true.
# If you set this option after having used fwanalog, some hosts won`t be
# linked in the report. You can create a report for a host with the
# "-a <IP-address>" command line option.
sep_packets=false
# Like sep_hosts, but for blocked packets.
# The corresponding command line option is "-p <packet>"
# Program invocations - add path if needed
analog="analog"
# Full pathname if you need, or "nice analog" if you want to de-priorize it
date="date" # should be GNU date or one which can print the timezone.
# see "timezone" below
grep="grep" # should be GNU grep
egrep="egrep" # should be GNU egrep
zegrep="zegrep" # this is just a shellscript on most systems. If you don`t
# have it, copy it from another Unix-lookalike.
gzcat="gzcat" # needed only on OpenBSD 3.x
sed="sed"
perl="perl"
tcpdump="tcpdump" # needed only on OpenBSD 3.x
timezone=`$date +%z`
# Which timezone the server is in. Correct if the server fwanalog runs on
# is not in the timezone the firewall is in.
# The %z option of date is supported on GNU/Linux and OpenBSD,
# but apparently NOT on FreeBSD so you will have to insert your
# timezone difference (e.g. -0500) yourself or use GNU date.
Once the configuration file is modified to match your desired layout and platform fwanalog.sh can be executed to generate a report:
$ fwanalog.sh``
This will place the reports in the directory assigned to the “outdir” variable. The reports can be viewed with a web browser, or emailed as part of a routine reporting process.
The pfstat utility can be used to collect and graph statistics exported through the /dev/pf pseudo-device. pfstat can be installed from the OpenBSD ports collection, or downloaded from the pfstat website.
pfstat requires the PF “loginterface” global configuration directive to be set in the pf.conf configuration file. This directive enables statistics collection for one of the physical interfaces in the firewall. The following pf.conf entry will collect statistics on the hme0 interface:
set loginterface hme0
Once statistics collecting is enabled, the pfstat utility can be invoked with the “-q” option. This will query the current value of each statistics counter, and printed the result to standard out:
$ pfstat -q``
1101400143 1101219586 483226347 25637411 0 0 496899 3866 325988 0 0 0 0 0 6 1692642 17030 17024 879499 0 2 0 0 0
pfstat uses this data to generate historical utilization graphs, so the data should be collected at periodic intervals if graphs are desired. The following cron job will collect statistics every five minutes, and write the results to “/var/log/pfstat/pfstat”:
*/5 * * * * /usr/local/bin/pfstat -q >> /var/log/pfstat/pfstat
To graph the data that is collected, a pfstat configuration file needs to be created. This file describes the graphs to generate, how to display the data, and where to store the output. The following example shows the pfstat configuration required to graph state table data:
image "/home/matty/pfstat/images/state_table.jpg" {
from 3 months to now
width 800 height 300
left
graph states_entries label "state table entries" color 0 255 0,
graph states_searches label "state table searches" color 255 0 0,
graph states_inserts label "state table insertions" color 0 0 255,
graph states_removals label "state table removals" color 0 0 0
}
The pfstat configuration file contains one or more “image” directives. Each image directive is followed by the file name of the image to generate, and a set of curly braces to control the attributes of the image. The “from” and “to” keywords select the time interval to graph. The value that follows the “from” keyword contains an integer value and a time frame (minutes, hours, days, weeks, months, years) to control how far back pfstat will go when processing data. The “to” keyword controls how pfstat processes new data elements, and the special key word “now” indicates the current time.
The “height” and “width” directives set the size, in pixels, for the height and width of the image to be output. Two directives determine the horizontal alignment of text descriptions: “left” aligns text on the left side of the graph and “right” aligns text on the right side. The “graph” statements control which data is graphed, the label assigned to the graph, and the colors used to create the entries on the graph. As of pfstat version 1.7, pfstat can graph packets, bytes, state table information, and several miscellaneous packet counters. The full list of options that can be passed to the “graph” directive are described in the pfstat(8) man page.
Once the configuration file is created, we can execute pfstat, and pass the configuration and data file as arguments to the “-c” and “-d” options:
$ pfstat -c /etc/pfstat/pfstat.conf -d /var/log/pfstat/pfstat >/dev/null``
Here is a pfstat.conf to graph IPv4, IPv6, and state table information:
image "/var/www/htdocs/pfstat/ipv4_bytes.jpg" {
from 3 months to now
width 800 height 300
left
graph bytes_v4_in label "incoming" color 0 255 0,
graph bytes_v4_out label "outgoing" color 255 0 0
}
image "/var/www/htdocs/pfstat/ipv6_bytes.jpg" {
from 3 months to now
width 800 height 300
left
graph bytes_v6_in label "incoming" color 0 255 0,
graph bytes_v6_out label "outgoing" color 255 0 0
}
image "/var/www/htdocs/pfstat/ipv4_connections.jpg" {
from 3 months to now
width 800 height 300
left
graph packets_v4_in_pass label "passed in v4" color 0 255 0,
graph packets_v4_out_pass label "passed out v4" color 255 0 0,
graph packets_v4_in_drop label "dropped in v4" color 0 0 255,
graph packets_v4_out_drop label "dropped out v4" color 255 0 255
}
image "/var/www/htdocs/pfstat/ipv6_connections.jpg" {
from 3 months to now
width 800 height 300
left
graph packets_v6_in_pass label "passed in v4" color 0 255 0,
graph packets_v6_out_pass label "passed out v4" color 255 0 0,
graph packets_v6_in_drop label "dropped in v4" color 0 0 255,
graph packets_v6_out_drop label "dropped out v4" color 0 0 0
}
image "/var/www/htdocs/pfstat/state_table.jpg" {
from 3 months to now
width 800 height 300
left
graph states_entries label "state table entries" color 0 255 0,
graph states_searches label "state table searches" color 255 0 0,
graph states_inserts label "state table insertions" color 0 0 255,
graph states_removals label "state table removals" color 0 0 0
}
This article provided an introduction to several tools that can be used to monitor a PF firewall. The PF FAQ and application manual pages contain additional details on advanced topic, and additional options that can be used to customize each application.
The following references were used while writing this article:
Ryan would like to thank the OpenBSD, PF, pftop, pfstat, tcpdump, analog and fwanalog developers for their awesome work!
Originally published in the September ‘05 issue of SysAdmin Magazine