PFsense Firewall Visualization using Logstalgia
by danlor on Oct.25, 2016, under Uncategorized
I’ve been playing with Logstalgia off and on for about a year now against my web hosts. It’s a fairly nifty way of monitoring the server and looking for errors. But, it’s not a truly effective tool in this context. I have always wanted to run it against my firewall. Unfortunately it’s not setup to parse other log types. It does however support an alternative log structure. This opens the door just a crack… just enough to let us force feed other datasources.
“Many other people” have done this according to posts I read. Using perl, some have supposedly setup syslog relays using the built-in perl libraries, or simply reformatted/reordered the logs through piping into Logstalgia. None have actually posted their solutions until now. Which is stupid.
I broke down and wrote my first perl script in over a decade. KISS is my motto, and it is reflected here. Feel free to improve. If there is sufficient interest, maybe I’ll add some more options. So, here is how I got mine working.
There are a number of things you need to do to set up your environment. I’m sorry if I have forgotten anything. This was built up over quite some time, and I’m mostly here to share the perl script and my workarounds.
1) Setup a second admin user on your pfsense. The root account does not land on the CLI when using ssh.
2) After you create the new admin account, make sure to grant it read write to the necessary files under /var/log/ (filter.log)
3) Setup ssh key authentication for the new admin account (you case safely skip this if you don’t know how)
3) Install Perl if you don’t have it.
4) Download Logstalgia
5) Compile/install Logstalgia
6) Save the script out to a working folder on your machine
Here is the script:
#!/usr/bin/perl -w
my $filename = 'custom.log';
while (<>) {
chomp;
open(my $fh, '>', $filename) or die "Could not open file '$filename' $!";
my @array = split(/,/);
my @datest = substr($array[1],0,15);
print $fh time(); # date stamp
print $fh "|$array[18]"; # Source IP
print $fh "|$array[19]:$array[21]"; # dest IP:port
print $fh "|$array[6]"; # action
print $fh "|$array[17]"; # Size
if( $array[6] eq 'block' ) {
print $fh "|1"; # Block
} else {
print $fh "|0"; # Pass
};
print $fh "\n";
#print $fh "|$array[8]"; # color(IP4 vs IP6)
#print $fh "|$array[16]"; # Referrer url(Protocol)
#print $fh "|$array[]"; # User Agent
#print $fh "|$array[]"; # Virtual Host
#print $fh "|$array[3]"; # PID/Other(Rule Anchor)
close $fh
};
The script is executed by running:
ssh admin-user@pf.local "clog -f /var/log/filter.log" | perl ./pfsense2logstalgia.pl
This will create a 1-line buffer file wherever you run the script. Clog is used to export out the current log buffer on the pfsense.
After the script is running, tail pipe the buffer into Logstalgia:
tail -f custom.log | logstalgia -sync
Using -g you can customize your target addresses:
tail -f custom.log | logstalgia -sync -g "Main Interface,URI=^12.23.34.45,30" --no-bounce
Have fun!
February 3rd, 2017 on 8:32 pm
Hi Ben! Love you! Miss all of you! Love, Brian and Beth.