CyberSecurity Project: Transparent Filtering Bridge (+ Extras) 7.0

Updated on
Confirmation that the full list of IP addresses was added to one of the Aliases.

Last time on the Transparent Filtering Bridge project, I started working on building up alias lists for the various firewall rules. Through the use of a Python script, I scraped IP addresses from Pi-Hole DNS block lists. Since Pi-Hole only focuses on blocking domain names, having OPNsense block those specific IPs increases the layers of defense for the network.

Table of Contents

Figuring out the CSV Format

One thing that proved to be surprisingly difficult was formatting the CSV correctly. The columns were the easy part. The problem was that the spreadsheet editor I was using saved CSVs in a way that caused the OPNsense Alias script to error out.

Even switching to Visual Studio Code didn't really help, but it made me realize that the issue could be in the formatting.

So, to brute force the issue, I went and acquired CSVEdit, a decade plus piece of software that's specifically made to handle CSVs. Starting with a copy of the original I made, I then began progressing through the various save options to find the correct combination that would load. After several iterations, I discovered these were the proper settings:

CSV settings for the OPNsense Alias script - Value Separator = Comma, Character Set = System, String (Value) Delimiter = Nothing

After generating a properly formatted CSV, I successfully updated the Alias.json file, then uploaded it to OPNsense.

Validating the Aliases

Once this was concluded, I noticed an immediate issue:

OPNsense Alias Interface with only one IP applied to most aliases.

Despite having large lists of IPs, only one IP address showed up in each alias. This meant I had incorrectly formatted something in the CSV. Examining the script's code revealed this:

if len(row['data'].split(" "))>1:
            item_data = "\n".join(row['data'].split(" "))

So the main issue was the fact that I mistakenly separated out each IP address or range into its own row. This meant that I had to reformat the list of IPs from each IP being on its own line, to being all on one line, with spaces as separation. 

This means I'll have to make some adjustments to my script's code, to have it automatically generate a list in that format.

However, until I make that change, I had to manually edit the lists into the proper format, which was tedious, but not too difficult.

Correctly formatted CSV for the OPNsense Alias script, with space separated IP addresses in the Data column.

Once these changes were made, the new Alias.json was generated and uploaded into OPNsense, then visually inspected in the GUI:

Confirmation that the full list of IP addresses was added to one of the Aliases.

Next Steps

The CSV format information has been uploaded to the Github repo, so all that needs to be done is iterating on my script to do a few things:

  • Ignore IP addresses embedded in URLs.
  • Generate a separate list for IP address ranges.
  • Generate the text files in the proper format (IP addresses separated by spaces).

Once these are complete, they will be tested and uploaded to the repo upon validation.

Related posts