Here is an example of a simple but effective blocklist system that I'm using on my pfSense firewalls. pfSense is a very modular firewall that can be expanded with many packages. About blocklists, there is a well-known one called pfBlocklist. Personally, I prefer to avoid installing extra packages on my firewalls because it increases the risk to face potential problems while upgrading (pfSense recommends to disable them before any upgrade). Some packages might also be developed by 3rd parties that have a light security mindset and, therefore, introduce bugs in a core element of the infrastructure.

pfSense has a feature called 'Aliases' that helps to build lists of FQDN, IP addresses, or ports to created powerful rules. Not many people know that it's also possible to configure pfSense to fetch a list of elements from an URL:

It is not mentioned clearly in the web interface but pfSense will regularly revisit the URL to fetch new data and update the list. It's performed via a cronjob:

[2.4.5-RELEASE][admin@pfesx2.xxxxxx.xxx]/etc: grep urltables crontab
30    12    *    *    *    root    /usr/bin/nice -n20 /etc/rc.update_urltables

Once a day, the firewall will check the URL and update the list if required. You can follow this in the firewall system logs:

As you see in the first screenshot, I'm not querying directly my MISP instance because you can't properly validate the returned data. if your MISP instance is in trouble, the returned list could be empty and the URL list zeroed. I prefer to collect data from MISP via a third-party server that will fetch the data and validate them:

curl -d '{"returnFormat":"text","type":"ip-src","category":"Network activity","last":"90d","enforceWarninglist":true,"to_ids":true}' \
     -H "Authorization: xxxx" \
     -H "Accept: application/json" \
     -H "Content-type: application/json"
     -X POST -s -o /tmp/misp-ip.txt \
     https://misp.xxxxxx.xxx/attributes/restSearch \
&& mv /tmp/misp-ip.txt /var/www/html/

The list of IP's will be published only if curl does not return an error. It also helps to finetune the API query to reduce the risk of false positives (ex: by enforcing the MISP Warning Lists or "To IDS" flag). The second advantage is to make the file available to other tools for monitoring and reporting purposes. The same list is fetched by my Splunk instance to create a lookup list:

Now, let's do some reporting by extracting the malicious IP addresses from the firewall logs:

index=firewall host=pfesx2 action=blocked 
| rename src_ip as ip 
| search [|inputlookup misp_ip.csv ]
| stats count by ip

Across the last 24 hours, we clearly spotted an interesting IP that was probably scanning my network: 185.176.27.118.

Here is a brief recap of the flows in place:

As you can see, you can play on both sides with only one list: you automatically block potentially malicious traffic and you get some stats about malicious IP addresses.

A final remark about the performance, the size of data fetched via the URL is limited as explained in the pfSense documentation: "When Save is clicked, up to 3,000 entries from each URL are read from the file and imported into a network type alias". To keep the list of malicious IP efficient, you can apply more filters and return, by example, only IP addresses validated and tagged by the SOC. 

Xavier Mertens (@xme)
Senior ISC Handler - Freelance Cyber Security Consultant
PGP Key