Summary
Though in many cases syslog is preferred to transport the pfSense logs to external system, Elastic beats provides quite a niche way to send the logs while modelling the data alongside. This makes it ready-made to send to ElasticSearch directly and get ready-made outcomes like SIEM, performance etc.
Pre-reqs
- A build server (preferably Ubuntu or Fedora) with internet connectivity
- shell access to pfsense server
- Basic knowledge of Elastic Stack (filebeat.yml configurations etc)
- Ensure connectivity is allowed from pfsense machine to your Elastic Stack receiver
Setup Summary
- Connectivity tests
- Install dependencies in build server (vagrant, virtualbox, gmake, go etc)
- Download Elastic Beats source
- Make elastic Beats package for FreeBSD
- Copy binary packages to pfsense server
- Configure Beats to send to destination
- Configure ElasticSearch to view the data
Installation Steps
Connectivity tests
Logon to pfsense server via Shell
ssh root@192.168.1.1
Password for root@pf.localdomain:pfSense - Netgate Device ID: 1a6323512345bf9e165d2
*** Welcome to pfSense 2.4.5-RELEASE (amd64) on pf ***
WAN (wan) -> re0 -> v4: 192.168.1.3/24
LAN (lan) -> re1 -> v4: 192.168.2.1/24
0) Logout (SSH only) 9) pfTop
1) Assign Interfaces 10) Filter Logs
2) Set interface(s) IP address 11) Restart webConfigurator
3) Reset webConfigurator password 12) PHP shell + pfSense tools
4) Reset to factory defaults 13) Update from console
5) Reboot system 14) Disable Secure Shell (sshd)
6) Halt system 15) Restore recent configuration
7) Ping host 16) Restart PHP-FPM
8) Shell
Enter an option: 8
Test connectivity to your ElasticSearch from pfsense
[root@pf.localdomain]/root: ssh -v -p 9200 192.168.1.10 # This is elasticSearch Server
OpenSSH_7.5p1, OpenSSL 1.0.2u-freebsd 20 Dec 2019debug1: Reading configuration data /etc/ssh/ssh_configdebug1: Connecting to 192.168.1.10 [192.168.1.10] port 9200.debug1: Connection established.
Test Internet connectivity from build Server (just ping github.com)
[root@buildserver]/tmp: ping github.com
PING github.com (140.82.118.4):
56 data bytes64 bytes from 140.82.118.4: icmp_seq=0 ttl=57 time=19.443 ms
64 bytes from 140.82.118.4: icmp_seq=1 ttl=57 time=19.348
Install dependencies in build server
- Ensure VirtualBox is installed. (or follow link)
- Ensure Vagrant is installed
[root@buildserver]/tmp: mkdir /tmp/freebsd
[root@buildserver]/tmp: cd /tmp/freebsd/
[root@buildserver]/tmp/freebsd: wget https://releases.hashicorp.com/vagrant/2.2.9/vagrant_2.2.9_x86_64.deb [root@buildserver]/tmp/freebsd: apt install ./vagrant_2.2.9_x86_64.deb
Create Vagrant file for FreeBSD
[[root@buildserver]/tmp/freebsd: vi Vagrantfile
# FreeBSD Vagrant
Vagrant.configure("2") do |config|
config.vm.box = "freebsd/FreeBSD-11.2-RELEASE"
config.vm.guest = :freebsd
config.ssh.shell = "sh"
config.vm.network "private_network", type: "dhcp"
config.vm.synced_folder ".", "/vagrant", id: "vagrant-root", disabled: true
config.vm.box_version = "2018.06.22"
end
Bring up the container and ssh to it
[root@buildserver]/tmp/freebsd: vagrant up
[root@buildserver]/tmp/freebsd: vagrant ssh
Now the FreeBSD container is ready. Just logon to container and switch to root within it and install the dependencies
[root@freeBSDContainer]/tmp/: pkg install git gmake go bashUpdating pfSense-core repository catalogue...pfSense-core repository is up to date.Updating pfSense repository catalogue.....The following 13 package(s) will be affected (of 0 checked):
exit the shell and relogin
Download Elastic Beats source
[root@freeBSDContainer]/root: go get github.com/elastic/beats
package github.com/elastic/beats: build constraints exclude all Go files in /root/go/src/github.com/elastic/beats
Please take of the location
/root/go/src/github.com/elastic/beats Make elastic Beats package for FreeBSD
[root@freeBSDContainer]/tmp: cd /root/go/src/github.com/elastic/beats
[root@freeBSDContainer]/tmp: git checkout v7.8.0
[root@freeBSDContainer]/root/go/src/github.com/elastic/beats/filebeat: gmake
[root@freeBSDContainer]/root/go/src/github.com/elastic/beats/filebeat: file filebeat
[root@freeBSDContainer]/root/go/src/github.com/elastic/beats/filebeat: ./filebeat version
[root@freeBSDContainer]/root/go/src/github.com/elastic/beats/metricbeat: ./metricbeat version
Copy binary packages to pfsense server
- Using scp or other methods, copy the file to pfsense server. Ideally its good to keep everything under /opt/beats/
- configure Beats to send to destination in the relevant *beat.yml (eg filebeat.yml)
- Ideally you shouldn't start filebeat as root, but since this is for test purposes, we will do so. Else you need to create user and setup it correctly with least privileges
[[root@pf.localdomain]/root/go/src/github.com/elastic/beats/filebeat: vi filebeat.yml
# ============================== Filebeat Config START ==============================
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 120s
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
hosts: ["192.168.1.10:9200"]
protocol: "http"
#api_key: "id:api_key"
username: "elastic"
password: "whateverpassword"
# ============================== Filebeat Config END ==============================
Ensure the relevant modules in filebeat are enabled
[root@pf.localdomain]/root/go/src/github.com/elastic/beats/filebeat: mv modules.d/auditd.yml.disabled modules.d/auditd.yml
Start filebeat
[root@pf.localdomain]/root/go/src/github.com/elastic/beats/filebeat: ./filebeat -e
-
check for any errors. otherwise you will see data coming to ElasticSearch cluster in few minutes.
- Configure ElasticSearch to view the data (this includes indices, index-patterns, index-templates)
