2Replacing the BT Infinity SmartHub with pfsense

When I moved into a new home a year ago I was finally able to join the 21^st cen­tury and ordered BT Infin­ity which is sup­plied with a SmartHub. The SmartHub is actu­ally reas­on­ably decent kit con­sid­er­ing it comes for free, but as with most ISP sup­plied devices it is locked down in some ways, for example you can­’t use your own DNS serv­ers which I prefer to do. In the early days of ADSL (circa 2001) I ran a smooth­wall box in place of a router, and for a range of reas­ons (includ­ing inter­net fil­ter­ing con­trolled by me, rather than an ISP) I decided to go back to a linux-based fire­wall router.

The little Cel­er­on J1900 box I got to do the hard­ware side of things did­n’t want to install smooth­wall, so I installed pfsense instead. I also had to get a vDSL (fibre) modem to con­nect the router to the phone sock­et. I got a net­gear DM200 which is actu­ally a full router that can be switched into “modem only” mode. Get­ting the whole arrange­ment work­ing took quite some fid­dling, so I thought I’d pull all the resources togeth­er in one place

1. Installing pfsense

I needed to install pfsense via USB, so effect­ively by flash drive. This was made pos­sible by the use of a blank pen drive, soft­ware called rufus, and the ‘mem­stick’ down­load of pfsense.

2. Configuring the netgear DM200

To do any­thing with the DM200 you will have to con­nect both a LAN cable to it, and the phone sock­et to it. Your PC should get an address from the net­gear by DHCP but if it does­n’t you will need to manu­ally set you IP address to 192.168.5.x (x being any­thing from 2 — 254). You can then log in via web inter­face at 192.168.5.1. The default login user­name is admin and the pass­word is pass­word. Select the “advanced” tab, and then the “advanced” menu option at the bot­tom left of that page. Under that select the “device mode” option. Change the device mode to “Modem (modem only)” and click apply. You can see more details and screen­shots on the net­gear help pages

3. Login and change the default password

Con­nect your pfsense box to the lan and con­nect to it using web inter­face via it’s lan IP address (which it will dis­play on it’s default boot up screen if you con­nect a dis­play to it). You may have to change your loc­al IP address to achieve this. Login to the pfsense with the user­name admin and pass­word pfsense. Go through the setup wiz­ard and when giv­en the oppor­tun­ity change the default webui pass­word. For more detailed inform­a­tion on steps 1–3 I recom­mend a guide on tecmint

4. Configuring the pfsense box to get a basic connection

I use BT infin­ity and get­ting the right set­tings proved trick­i­er than I had hoped. I had to first con­fig­ure the WAN set­tings cor­rectly and after that, set the cor­rect pro­file for the WAN inter­face. First, go to Inter­faces: WAN and set the following.

Save the changes, and then go to Inter­faces: Assign­ment. Set the WAN inter­face to “PPPOE…” which after sav­ing should show with the phys­ic­al inter­face in brack­ets — in my case it says “PPPOE (em0)”. Save the changes again and hope­fully you will get a connection.

4b. WAN MTU Value

In the WAN Inter­face set­tings you might want to adjust your MTU set­ting to work optim­ally with BT Infin­ity to avoid frag­men­ted pack­ets and pos­sible pack­et loss.  I have writ­ten a [int­link id=“4002” type=“post”]dedicated article[/intlink] on this issue.

5. IPv6 Testing

The set­tings above should be suf­fi­cient to get IPv6 work­ing on your LAN cli­ents — you should also see an IPv6 address for the pfsense LAN inter­face (i.e. one that does­n’t start fe80). Try pinging google.com from a ter­min­al win­dow on a LAN cli­ent — if you get a response from the IPv6 address then all is well. You can also check that all i cor­rect using test-ipv6.com. Thanks to Dan­neh for the set­tings. For more inform­a­tion I recom­mend this red­dit thread.

There is one fur­ther tweak required to make sure IPv6 works fully, you need to allow ICMPv6 pack­ets through the fire­wall. Go to Fire­wall, and then Rules. Add a new rule, set the address fam­ily to IPv6, change the pro­tocol to ICMP, leave “any” selec­ted as the sub­types (unless you want to do a lot more read­ing about spe­cif­ic sub­types). Click Save, and then click “Apply Changes”.

6. Enabling Intel enhanced speed-step

I don’t want my lower powered router run­ning at full tilt all the time — but sadly pfsense doens’t seem to cor­rectly sup­port intel enhanced speed step by default at the moment. To get mine work­ing (and a lower cpu tem­per­at­ure to go with it!) I first had to enable PowerD in Sys­tem -> Advanced -> Mis­cel­laneous -> Enable PowerD. If you want to enable the low­est fre­quen­cies (altho these don’t save much power) you will also need to do the fol­low­ing changes: go to Dia­gnostics, Edit File. Then enter the file path /boot/device.hints. change the bot­tom 2 entries from 1 to 0 (called hint.acpi_throttle.0.disabled and hint.p4tcc.0.disabled). Thanks to SecondEdge and dreamslack­er for these tips. To check this is work­ing you will need to log into the router via SSH, select option 8 (shell) and run sysctl dev.cpu. | grep freq. This took my cpu core tem­per­at­ure from 66C to 57C — not bad for a tiny fan­less sys­tem packed in next to anoth­er PC, a modem, and an 8‑port switch.

7. Port forwarding

Go to fire­wall: NAT and then click the add but­ton. Enter the IP address and port for the des­tin­a­tion and (most likely) the same port for the extern­al port. For more detailed inform­a­tion I recom­mend a post by splurben on the pfsense forums.

8. NAT Reflection

I use my laptop both at home on the LAN and away from home and in both cases want to access vari­ous web inter­faces on the LAN. I use DDNS to get a domain name and wanted to use this to con­nect even when con­nec­ted to the LAN. This requires NAT reflec­tion which can be enabled under sys­tem: advanced: NAT Reflec­tion mode for port for­wards. You may (prob­ably) need to also enable 2 oth­er options on this page: Enable NAT Reflec­tion for 1:1 NAT and Enable auto­mat­ic out­bound NAT for Reflection

9. Adblocking

All of my PC webbrowsers have adb­locked installed, but the same can­’t be said of my android devices as these have to be rooted to install block­ers. So being able to block ads with pfsense is one of the major advant­ages of using it. First, go to sys­tem: pack­age man­ager and then search for pfb­lock­erng and install it. You can then con­fig­ure it using Fire­wall: PFB­lock­erNG. I then used the guide by Fred­Merc to con­fig­ure it. A brief sum­mary of the set­tings I’ve used is as fol­lows. Go to Fire­wall: PFB­lock­erNG and then click on the DNSBL tab, and then click on the DNSBL EasyL­ist tab. Turn on the top EasyL­ist feed and point it to EasyL­ist. Then click the add but­ton, and set the second EasyL­ist feed to EasyP­ri­vacy and turn that on too. List action should be “unbound” and I set the update fre­quency to 1 day. Then click save. Then go to the DNSBL tab and enable the option Enable DNSBL. Finally go to the Gen­er­al tab and enable pfB­lock­erNG.

9b. Adblock fixes

The default PFB­lock­erNG con­fig­ur­a­tion causes prob­lems for the amazon android app. To avoid this, and oth­er issues, it is worth using some whitel­ist­ing. Go to Fire­wall: PFB­lock­erNG and then click on the DNSBL tab, scroll down to cus­tom domain whitel­ist and enter the fol­low­ing (thanks to bchow on the pfsense for­ums)

.amazonaws.com
.amazon-adsystem.com
.amazon.com
.ssl.google-analytics.com
.ssl-google-analytics.l.google.com # CNAME for (ssl.google-analytics.com)
.www.google-analytics.com
.www-google-analytics.l.google.com # CNAME for (www.google-analytics.com)
.www.googleadservices.com
.plex.tv
.gravatar.com
.thetvdb.com
.themoviedb.com
.googleapis.com # 172.217.3.202 is important for amazon app to work
.1e100.net # cname? altname? for googleapis.com
.ad.doubleclick.net # needed for clash of clans?
.g.doubleclick.net # needed for clash of clans?
.q1mediahydraplatform.com # needed for hungryhouse android app?

You may also want to enable the alexa whitel­ist of top sites.

10. Transparent squid proxy

I decided to set up a trans­par­ent squid proxy as much of the brows­ing that we do hits the same sites repeatedly on dif­fer­ent devices, I don’t expect it to make a huge dif­fer­ence, but I can­’t see any good reas­ons not to. Use sys­tem: pack­age man­ager to install squid. Then go to ser­vices: squid proxy serv­er to con­fig­ure it. This is also needed for Squid­Guard if you want to use it, as I do.

11. Web filtering for child safety with SquidGuard

I have young chil­dren in the house and want to block unsuit­able con­tent. This can be achieved with the Squid­Guard pack­age and Shal­la’s Black­lists. Install squid­guard from sys­tem: pack­age man­ager. Then go to ser­vices: squid­guard proxy fil­ter. Go to the black­list tab, enter the address https://www.shallalist.de/Downloads/shallalist.tar.gz and click down­load. Then use the Com­mon ACL tab, click on the plus but­ton and select the cat­egor­ies you wish to block. It is also necesary to set up a dummy tar­get cat­egory due to a bug. For more inform­a­tion see this post on pfsense for­um. Don’t for­get to set the default for all of the lists to allow at the very bot­tom of the lists. Thanks to net­work­inggeek on the pfsense for­ums for this tip. Lastly — it may be worth edit­ing a couple of advanced options so that blocked requests are only cached for a short peri­od of time — that way if you decide to unblock some sites you wont have to clear the browser cache to access those sites — there is more inform­a­tion on the pfsense for­um. I had to whitel­ist the category [blk_BL_sex_lingerie] so that my wife could buy under­wear as the fil­ter was block­ing the under­wear sec­tions on main­stream retail­ers (e.g. Debenhams).

12. Enable U‑PNP for a range of services (gaming, messaging, torrent, etc)

Go to Ser­vices: UPnP & NAT-PMP, tick the top 2 boxes (Enable UPnP & NAT-PMP and Allow UPnP Port Map­ping), and click save.

13. Malicious traffic blocking with SNORT

To block detect and block poten­tially mali­cious traffic you can install the SNORT pack­age. I recom­mend run­ning it without block­ing for the first few weeks as it will block lots of things you don’t want due to large num­bers of false pos­it­ives. I recom­mend using the fol­low­ing sup­pres­sion list to avoid some of the most annoy­ing false positives

#ET P2P Bittorrent P2P Client User-Agent (uTorrent)
suppress gen_id 1, sig_id 2011706
#ET P2P BitTorrent DHT announce_peers request
suppress gen_id 1, sig_id 2008585
#(spp_ssl) Invalid Client HELLO after Server HELLO Detected
suppress gen_id 137, sig_id 1
#ET P2P BitTorrent DHT ping request
suppress gen_id 1, sig_id 2008581
#(http_inspect) SIMPLE REQUEST
suppress gen_id 119, sig_id 32
#(http_inspect) UNKNOWN METHOD
suppress gen_id 119, sig_id 31
#(http_inspect) INVALID CONTENT-LENGTH OR CHUNK SIZE
suppress gen_id 120, sig_id 8
#(http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE
suppress gen_id 120, sig_id 3
#(http_inspect) DOUBLE DECODING ATTACK
suppress gen_id 119, sig_id 2
#(http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED
suppress gen_id 120, sig_id 6
#(http_inspect) IIS UNICODE CODEPOINT ENCODING
suppress gen_id 119, sig_id 7
#(http_inspect) BARE BYTE UNICODE ENCODING
suppress gen_id 119, sig_id 4
#(http_inspect) JAVASCRIPT OBFUSCATION LEVELS EXCEEDS 1
suppress gen_id 120, sig_id 9
#(http_inspect) JAVASCRIPT WHITESPACES EXCEEDS MAX ALLOWED
suppress gen_id 120, sig_id 10
#(http_inspect) UNESCAPED SPACE IN HTTP URI
suppress gen_id 119, sig_id 33
#(http_inspect) U ENCODING
suppress gen_id 119, sig_id 3
#(http_inspect) DOUBLE DECODING ATTACK
suppress gen_id 119, sig_id 2
#(http_inspect) MULTIPLE ENCODINGS WITHIN JAVASCRIPT OBFUSCATED DATA
suppress gen_id 120, sig_id 11
#(http_inspect) HTTP RESPONSE HAS UTF CHARSET WHICH FAILED TO NORMALIZE
suppress gen_id 120, sig_id 4
#FILE-IMAGE Directshow GIF logical width overflow attempt
suppress gen_id 1, sig_id 27525

14. Getting web-access to the modem, through the pfsense box

My Net­gear DM200 modem (in pass-through mode) is only access­ible via a fixed IP address (192.168.5.1). I wanted to be able to access its web inter­face on LAN com­puters. There are some instruc­tions in the pfsense wiki, but these did­n’t work for me at first. There is a help­ful post by user Non­sense on the pfsense for­um

14b. Showing the modem connection statistics on the pfsense dashboard

After some head­scratch­ing I figured out a way to make the modem stat­ist­ics for my net­gear modem show on my dashboard.
This is done by cre­at­ing a cus­tom wid­get with php code.
Go to dia­gnostics and edit file. Cre­ate a new file at the path

/usr/local/www/widgets/widgets/modemstatus.widget.php

with the contents

< ?php $status= file_get_contents("https://username:password@192.168.5.1/RST_statistic.htm"); $status= str_replace("var timereset=\"5\";","var timereset=\"0\";",$status); echo $status ?>

You will need to cus­tom­ise the user­name and pass­word. The above code works for the Net­gear DM200, and prob­ably oth­er net­gear modems and routers. For oth­er makes of hard­ware you will need a dif­fer­ent address for the stat­ist­ics and you may need to do addi­tion­al manip­u­la­tion of the response using php.
Note that I have over-rid­den the default net­gear refresh inter­val — I’ve turned it off as the reload breaks the dash­board dis­play. To get updated num­bers just refresh the pfsense dash­board using your web browser reload button
Now go to the dash­board and add the wid­get and you’re all done.

15. Fixing the certificate warning when logging in

See this guide

16. Ask firefox to use local DNS over HTTPS, instead of bypassing our filters (added April 2020)

In Ser­vices -> DNS Resolver
Add fol­low­ing line to “cus­tom options” field… 

server:local-zone: "use-application-dns.net." always_nxdomain

17. Use domain name of pfsense box for blocked resources instead of IP (added April 2020)

Ser­vices -> Squid­Guard Proxy Fil­ter -> Com­mon ACL
Change “ReDir­ect Mode” to “ext url move (enter URL)”
In the “Redir­ect info” field set “https://your-router-name/sgerror.php?url=403%20&a=%a&n=%n&i=%i&s=%s&t=%t&u=%u”