Sunday, January 20, 2019

Raspberry Pi Pi-hole Network-wide blocking of Ads, tracking, and popups

How-To: In two hours, with no previous experience, you can build a small "DNS Sink Hole" that can block ads, tracking cookies, popups, and email-trackers -- all by using a small $50 computer called a Raspberry Pi. 

But most importantly, questionable sites, such as ransomware and other scams, are blocked at the network layer, long before your browser has a chance to see them.

This works for all devices in your network, including all desktops, laptops, phones, and tablets.

You no longer need to install ad-blocking software.  All the benefits happen for all devices behind your router -- and you do not have to configure them to gain the benefits.


This replaces a previous Keyliner article:
Stopping Tracking Cookies with whack-a-mole - blocking DNS using Acrylic DNS.



A Raspberry Pi becomes a dedicated computer that handles all DNS (Domain Name Service) requests -- taking the function away from your existing routers.  When you type an address, such as "keyliner.com", -- a request goes to your Domain Name Server.  It translates the human-readable name into an IP address.  If the address is nefarious or an ad-network, the packet is discarded, keeping the traffic from reaching your devices.

As of this article, the device blocks 107,000 domains (now 126,000 domains).  Here is a chart showing the normal traffic at my house, with blocked requests in blue.  Of the 13,000 requests, 2,500 were blocked:





Raspberry Pi, you say?


To make this work, you will need to build a small computer using a device called a Raspberry Pi, and then install DNS software called "Pi-Hole" (an open-source, community-developed Domain Name Service supported by hundreds of volunteers).

"I don't know anything about that!" Neither did I!

And yet, with zero experience, I built the PC, installed the operating system, and configured everything -- all in about two hours.  The operating system and DNS software are free.  You can do this! 


What is a Raspberry Pi?

A Pi is a small computer, running Linux and costs about $50.  It has 4 USB ports, an HDMI Video port, an RJ45 wired network jack, Bluetooth, a wireless adapter, and a slot for an SD-card drive. You do not need to know Linux to build this project. 

I found this model on Amazon, which included a case, power-supply.

I also like this model (where you supply your own cell-phone 2.5A charger), or this model.  These are all similar, except for cosmetics, accoutrements, HDMI cables, and the like.  I noted Walmart.com carries the same products, with free shipping for the same cost.

I am using an older Raspberry Pi 2.0.  Version 3b is now available.  Either will work.  A Pi 2.0 can easily support up to 2M DNS transactions per hour -- well within the realm of a 50 workstation network.


You will also need the following:

HDMI cable to connect to your TV or monitor (temporary, just for setup; may be included in your kit)

Short .5 or 1 Meter (2 - 3 ft) Ethernet patch cable ($5)
Wired or wireless Keyboard (borrow from your PC)
Wired or wireless Mouse (borrow from your PC)

And you will need a 16GB (recommended) or 32GB Micro SD card with adapter.  These are often included in your Raspberry Pi kit:


The SD card acts as the Pi's hard drive.  Note, this is the Micro SD card, which is much smaller than a postage stamp.  Buy these at any electronics or office store.  Shop around for the best price (expect about $10).  The drive comes with a standard-sized adapter so you can plug it into your laptop or desktop's SD slot.


Before you start, some research is required.


Important Prerequisites

A.  From your PC, discover your IP-address pool-range with these geeky but easy steps:

From a DOS / Command prompt (windows-R, "CMD"), type this command:

ipconfig (enter)


* Note your IPV4 address, illustrated above.

Yours will probably read something like
192.168.0.10    (mine happens to be 192.168.100.10)

* Note your Default Gateway (mine is 192.168.100.1) - This is your main router.

B.  Decision

If your workstation's ipv4 address is below 10, such as 192.168.0.2  or 192.168.1.3,
write-down, and later use, this fixed IP address: 192.168.0.151

This will be the Pi's new internal IP address, where the first three octets will be the same as your workstation.  


If your displayed IPV4 address is something like 192.168.0.11 (or some number higher than 9), then consider using this IP Address for your soon-to-be-built Pi:  192.168.0.5 (-now in retrospect, it is probably safest to always use a .151 address because this address is beyond most auto-assigned addresses).

Technical notes for those who care:  Your home router assigns automatic DHCP addresses to each workstation on the network using a range or pool of numbers.  This range varies by router manufacturer.  Some start at 2 - through 100, others start at 10 through 150.  The range does not matter, but the Raspberry Pi needs a number from outside that range.  You could log into the router and confirm the exact range, but the steps above are a good-enough approximation and you could probably use .151 in all normal cases. 


C.  You must be able to login to your Router (and, if present, your optional Wireless Router's) Admin screens. Test this now, before going on further:

Open a browser.  In the URL line, type your router's main IP address -- illustrated in the DOS screen above as the "Default Gateway."  The first three octets will be the same as your workstation's address.  The last octet will most likely be a dot-1. 

For example,

192.168.0.1  

(also typical are addresses like this:  192.168.1.1 .  Your network may be different.  The main router's last octet is almost always dot-one.). 


D.  Look on the side of the router for a printed label that shows the admin login ID and password, or you may have recorded the password when the network was first built.

Login with "admin"  (usually, lower-case-a)
Confirm you can login and get to the router's administrative screens.


E.  If you have a separate wireless router, you will also need to see its administrative screens (more details below).  That address is often 192.168.0.2.  Login with similar steps.



You must be able to login to your router's admin screens before continuing.  If not, consider this keyliner article, and this one.  Your ISP or the person who setup your original network may be able to help. 






Raspberry Pi Hardware Setup


A new Raspberry Pi is a small circuit board.  Snap it into the kit's plastic case, and if the kit came with self-adhesive heat-sinks, apply them now. 

Next, download and install the Linux operating system with these steps (this step can be skipped if your raspberry kit came with a pre-installed NOOB operating system; if so, jump to step 4):


1.  From a PC, go to the Raspbian download site and download the "NOOBS Offline and Network Install.zip".  This download is slow and will take several hours (they have a slow network connection and it has to cross an ocean. 1.7Gig file.):

https://www.raspberrypi.org/downloads/noobs/

Save the .ZIP to a known location.

2.  Insert the MicroSD card into your PC's card-reader.

The card must be a 16GB or 32GB card (smaller than 64GB).
If prompted, format the card - format it like you would with any disk or USB thumb drive.


3.  On the PC, using File Explorer, open the .ZIP and copy all files and folders within the .zip to the SD card's root directory.

(Important:  Do not copy the .zip file -- copy the contents inside the zip. 
Use Copy-and-Paste -- not Cut-and-paste -- not click-and-drag
).

 
Details:  To copy, double-click .ZIP to open.  
               On detail side. click the first file/folder. 
               Shift-Click the last file/folder.
               Hover on the highlighted files, "other-mouse-click", choose "Copy" (not cut)

               Find the SD Card drive (On my PC, this showed as Drive G:"
               In the details pane, other-mouse-click and chose "Paste"

Once copied, eject the SD card.

If desired, the .zip file can be deleted as it is no longer needed.


4.  Remove the Micro-SD Card-insert.
  • Insert into the Raspberry Pi's card-slot
  • The SD-card installs "up-side-down," into the board's slot
  • Push until it locks in place

5.  Connect the HDMI cable to your TV or Monitor  (I used my TV).
  • Connect a USB Keyboard (borrow from your desktop; can be wireless)
  • Connect a USB Mouse (can be wireless) If you have a Cat-5 cable and can easily connect to the router, do so now.  Otherwise, you can use a wireless connection for the initial setup.
     
  • For the initial setup, you can use either a wired or wireless connection.  If near the main router, connect an RJ45 network cable to any open port on any router.  (Do not plug in to the router's "uplink" port (do not use the lonely port; plug into the 4 or 8 port areas)).   Connect the other end to the Pi's RJ45 port. 

    If you are using Wireless for the setup, continue with the USB power supply step.  Note: Later, you must switch to a wired connection.
  • Connect the USB 2.5a power supply to the Pi.  (Any 2.5a micro-USB cell charger will work)

The Raspberry Pi will boot; visible on TV.  You may need to switch your TV's INPUT to find the right HDMI port.



Raspberry Pi Operating System Install


6.  When the Pi first-time boots with the new SD-card installed, it will arrive automatically at the Raspian Operating System Installation screen.  Select the top-most Raspberry PI operating system,

[x] Raspian Full (Recommended)

Click the Install button on the ribbon bar. 
Install takes apx 45 minutes.  When done, it will boot to a desktop.

Black Screen:  I had troubles when partway through the install, the TV showed "signal not found."  I realized the TV was routed through the stereo and the stereo would go into power-save mode.  Rebooting the stereo returned the TV's Pi image.


When prompted:
  • Select Country:  (e.g. United States)
  • Language: (e.g. note American English) 
  • TimeZone:  (oddly by City name)
  • Important:  If United States, you must click the [x] US Keyboard option
When prompted for the Admin password:
  • Change the admin password to a password of your choosing
  • Write the password on the checklist above

7.  Network Decision - Wired or Wireless:

If your Pi is connected to a Wired network, allow the Pi to auto-update and patch.

If using a wireless for the install, See the top-menus.  (Most modern Pi's have Wireless built-in)

Right-click the right-side wireless-strength icon.  Configure the SSID on WLan0, etc., connecting the Pi to your wireless network, much like you would any other device.   The wireless connection is temporary and should only be used for the initial setup.  Later, you must change to a wired connection.

7a.  Allow Pi to auto-update and patch. 
  • After patching, the Pi will reboot
  • After patching, you will be forced to re-enter the Location, Keyboard, and Admin password

7b.  At the Raspberry-top-menu icon,
  • Click "Raspberry-pi Configuration", "Interfaces"
  • Enable SSH  (allows remote desktop control - handy for geeks; do it now, while convenient)


At this stage, you have a fully-installed, fully-usable copy of Linux.  Pat yourself on the back because you are good!

The next step is to install the Pi-Hole DNS Server software.


Install Pi-Hole DNS

Once the operating system is installed and patched, install the Pi-Hole software with these steps:

8.  On the Raspberry Pi's top-menu, open the "Terminal Window" (command prompt)

  • Type this case-sensitive command.  Note the "-sSL" -- seems to be very case-sensitive.  Note the split-vertical bar:
  • curl -sSL https://install.pi-hole.net | bash

9.  Answer these prompts:

"This installer will transform your device into a network-wide ad-blocker" 
tab to the OK button and press Enter

  • You may be prompted to: 
    Choose eth0 for the hard-wired port
    (even if you are using wireless to do the install)
  • Accept Google (or OpenDNS) as your upstream DNS Provider
  • Accept the default third-party list; tab for OK
  • Choose IPV4 (not IPV6) for the protocol

!!!  Important:  When prompted
!!!  "do you want to use your current network settings as a static address"
  • -- tab to "No"
  • Press Enter 
  • If you miss this step, press ESC and restart at the CURL step.
(Reason:  You want to set a static, hard-coded IP-address on the wired network). 
  • For an IP Address, you are setting a "Static" / fixed/hard-coded IP address, found in the prerequisite steps:
  • Type the Raspberry Pi's IP-address, from the prerequisites (it was either xx.5, or xx.151).
    Type the full address, appending a trailing "/24" --  (slash /24 sets the subnet mask to 255.255.255.0)

    Examples from your prerequisite/decision:
192.168.0.151/24   or
192.168.0.5/24
192.168.1.151/24 etc.
  • Set the Default "Gateway" to the same address as your workstation's Gateway IP Address.  This is your main router's IP address.  See the checklist, above:

    Typically:
192.168.0.1    (192.168.1.1 etc.)

  • Allow it to install the Web Admin Interface
  • Accept Log Queries, ON  (recommended)
  • (If the install goes "south," reboot the PI and restart the curl command.)


10.  At the "Installation Complete" screen

Step away from the keyboard and
Carefully write down the insufferable "Administrative login/password"
and the set-installed IP address.

For example, my machine showed: 
192.168.100.5/admin   Password: xxxxxxx______________________


11.  Change the Pi's admin password.  Do this now, while it is easy to get to these screens:

From the main desktop, open a terminal window.

Type this command:
pihole -a -p

Follow the prompts to change the password.
* Record this final password in the checklist above.

The Pi-Hole is now fully configured and ready to use. A moment of self-congratulations is in order.



12.  Optional Cleanup Steps:


The Raspian operating system comes pre-installed with extra software that is not needed for this project.  The Raspberry Pi and Pi-hole software will run perfectly-well as-is, but if you are geek, and don't mind spending another hour, consider uninstalling the following programs.  This will make the Pi faster and will leave more space on the drive:

From the main Linux desktop, top-menu, open a Terminal Window.
Type these commands, pressing ENTER after each.  If software is not found, press the up-arrow and double-check the spelling or move to the next command.

Answer with "Y" (capital Y), when prompted:

a.   sudo apt-get purge wolfram-engine
b.   sudo apt-get remove --purge libreoffice* 
c.   sudo apt-get purge sonic-pi
d.   sudo apt-get purge scratch
e.   sudo apt-get purge greenfoot
f.   sudo apt-get purge geany
  g.   sudo apt-get purge nuscratch
h.   sudo apt-get purge python-pygame
i.   sudo apt-get purge pygame
j.   sudo apt-get purge squeak-vm
k.   sudo apt-get purge dillo
 
l.   sudo apt-get purge minecraft-pi
m.   sudo apt-get purge penguinspuzzle
n.   sudo apt-get purge oracle-java8-jkk
o.   sudo apt-get purge oracle-java7-jdk
p.   sudo apt-get purge openjdk-8-jre

If you typed any of the above, reboot then follow with these two commands:

x.  sudo apt-get clean
y.  sudo apt-get autoremove --purge

Update the OS with this command:

z.  sudo apt-get update && sudo apt-get upgrade -y

Optional software is now de-installed.  Apx 2G of disk space is freed.



Pi Final (Production) Wiring Steps:

Using the top-Raspberry menu, shut-down the server.
Unplug the HDMI cable; you will not need the monitor again.
Unplug the Keyboard and Mouse; you will not need them again.


13.  Move the Raspberry Pi to a location near your main router. 

Using a short Cat-5 Network cable, plug in the Raspberry into any available port on your router. A hard-wired connection is required.

For example, my home network looks like this, where the Pi was connected to an 8-port switch.  It could have been easily connected to the DSL or Wireless router's open (yellow) port -- any open port can be used, where there are groups of 4 or 8 network jacks.  Do not plug it into the up-link port (a lonely port, usually a different color):


Plug in the power adapter. 

Give the computer a few minutes to boot and get settled.
Note the activity lights on the Pi's RJ45 network port.

(See this keyliner article for a photo of my home setup)


14.  Initial Test: 

From your PC-workstation, open a DOS / Command Prompt and ping the Pi to see if it is on the network.  Type this command:

PING 192.168.0.151  (or 0.5, or 1.151, etc)

It should reply in xx milliseconds.


Router Setup

The final step is to configure your router(s) to point to this new Domain Service.  You must make these changes in order for the Pi to do its job.  This is a one-time setup.

For most households, your main DSL or Cable-Modem router (the box with a .1 ip-address) is the one which needs to be changed, but some networks may have a second router.

Some people have two routers -a DSL/Cable and a second (often a dual-antenna) wireless router.  The usually have IP Addresses of 192.168.100.1  and 100.2, and it may need to be changed too.  However, this is relatively rare. 

**** If you are unsure, or these instructions are poor, it doesn't hurt to look at both routers for DNS settings.


A.  Login to your DSL / Cable-Modem's administrative screen (see per-requisites, from above)

Typically, type this address in the URL address bar and press enter.
Your address may be different:

192.168.0.1


B.  Login with "admin" and your pre-recorded password.

The main setup screens vary by modem manufacturer.  Several examples are illustrated below:
  • Usually under an Advanced Configuration menu
  • Look for a DNS Setup section
    (or sometimes DHCP/DNS)
  • Look for
    "Dynamic DNS" (or "Auto-DNS", or "use these DNS Servers", depending on modem)
     
  • Change to:
    Static DNS or "Use these DNS Servers"...
  • At the Primary DNS, type the IP address of the Raspberry Pi.
    For example, on my network, 192.168.100.5  (or 192.168.0.151, from your prerequisites)
  • Optionally type a Secondary DNS (not necessarily recommended)
    8.8.8.8
    (Or use an Open DNS address, documented at the end of this article)

    I leave mine blank, as the Pi already defaults to your favorite secondary DNS as part of its initial install.  See the end of this article for more discussion about this. 

    Some routers require a secondary DNS and one must be typed.  If so, use 8.8.8.8.  But, if you want to force all DNS traffic through the Pi, use a dummy secondary address of 127.0.0.1.  I personally like the 127.0.0.1 option.  Again, see the discussion near the end of this article for my reasons. 
See the red-section, directly below for other modem examples.

C.  Important:  Save the changes by clicking this screen's SAVE or APPLY button.  Do this before moving to any other screen.  The router will reboot.


Example Modem Setup Screens:

My Zyxcel DSL router looked/s like this:

Click for larger view


* Some of newer models of routers require a secondary DNS
-- I used Google's  8.8.8.8 -- which is redundant because this is the go-to address used by the Pi:

Click for larger view



* A typical Linksys router looked like this, where in this case, the network was 192.168.1.150 (should have been 192.168.1.151):




* Another version of a linksys router looked like this, where the pi-hole's address of, 192.168.100.5 was added:



* A NetGear Genie AC1450 looked like this, where the Raspberry Pi was the primary and again, Google's DNS was set as a secondary:

Click to enlarge

                                           


D.  If you have a secondary, wireless router (rare for most households), typically at 192.168.100.2, look to see if it needs to be configured. 

Login to that device's admin screen by opening a browser and typing the wireless router's IP Address
typically:  192.168.0.2,   (but could be something like 192.168.1.1 See your prerequisites)

Login to the administrator's screen, again with a default password likely printed on a back label. Snoop-around the setup screens (Basic Setup, Advanced Setup), looking for a DNS Server. 

Usually these routers use the main router for DNS and likely, you will *not* find a DNS Server setting (don't confuse with DHCP -- which is probably disabled).  If DNS settings cannot be found, jump to the Testing steps. 

If a DNS entry is found, make similar DNS changes.

*Note:  If you can't login to the router's admin screen -- and often you can't while passing through a wired network, consider the following:

1.  Use a wireless device to reach the configuration screens.  Or,

2.  With a laptop or desktop,
     Run a temporary hard-wired RJ45 connection directly from the PC
     to any available yellow-port on the wireless router.

3. Reboot the PC to get a new IP address.  IPConfig to see your new IP Address.
4. Try logging into the dot-1, dot-2 IP Address again.


Raspberry Pi and Pi-hole configuration is complete!
I recommend the following tests and recommend logging into the Pi-hole's admin screens.  These topics are covered next.



TESTING

Ublock Origin, illustrated
To properly test, you need to disable ad-blocking software on your test workstation, as Ad-blockers also block traffic.  The difference is they block the traffic *after* it has downloaded where-as the Pi keeps them from ever downloading. 

You may or may not have ad-blocking software installed.  Look in your browser's Tools, Add-Ins menu and look for "adblock-plus" or "uBlock Origin" (the two most commonly used blockers).  If installed, close the Add-in screens and look on your browser's upper-right menu bar, looking for a UBlock Origin or an Adblock-plus icon.  Click the icon and temporarily disable the ad-blocker.

Test 1: 

This test makes sure the network is functioning properly and you have the routers pointing to the right DNS-resolver (the Pi-hole).
  • From your normal workstation, browse to www.google.com.
  • If you arrive, your DNS is working correctly.
Test 2:
  • Browse to Yahoo.com
     
  • Note "holes" in the page -- blank spaces, illustrated below in orange.  There are being snuffed by the Pi-hole.
     
  • Be sure adblockers are disabled or they will distort this test
     
  • Note "holes" in displayed page. These are never transmitted to you; speeding up your page- load times.  A drawback is content providers cannot monitize their articles.  There are moral and ethical considerations; see the end of this article for a discussion.  On the other hand, they are often abusive and sometimes provide malicious content.
Click for larger view

Test 3:
  • Attempt to browse to  http://tag.bounceexchange.com - a nefarious site
  • Note how pi-hole blocks the address.  It may look like this or this, depending on your browser:



    or this:

  • Browse to  didtheyreadit.com  (an email tracking service that uses one-pixel white images on emails to track if you opened the email).  As-of this article, you will likely succeed and arive at the site.
     
  • Consider "Blacklisting" this and other such sites.  See the blacklist later in this article.
  • Note that *all* devices in your network benefit from the Pi.  And, more importantly, none of the devices need to be told about the setup -- it just works.  But if your device (cell phone, tablet, laptop) strays from the network, the Pi's benefits are lost.

Side-notes:  If the domain is on the naughty-list, the Pi dumps the DNS request into a dark hole, hence "pi-hole."  As of this article, over 120,000 domains are in the discard list.  If the address is on the good-boy list, it is handed off to (Google's) Domain Services and Google resolves the address normally.  Some routers require a secondary DNS but Pie-Hole's secondary will win the battle.

Most home routers use your ISP's Domain Name Services, for example, CableOne, Century Link, Comcast, etc., and some ISPs have been known to slip-stream their own advertisments into your data-stream(!), replacing ads with their own.  With the Pi-hole (or Google's DNS, 8.8.8.8) you resolve with a more trustworthy source.


Testing:  Simulate a pi-failure:

Unplug the Raspberry Pi's power and attempt to browse any site from any workstation. 
You will find no internet addresses resolve*.  In other words, the Pi is required to be online -- just like your router is required to be online.  Restart the Pi and give it a few minutes to boot and repeat the test, confirming the network returned to normal.

(* If you typed a secondary DNS in your router, traffic routes to the secondary address when the Pi is offline.  This is good and bad.  The secondary will resolve domain addresses, stopping a catastrophic failure, but you will not know the Pi is offline and will lose the benefits of nefarious-site-blocking.  If your router forces a secondary DNS, consider using a dummy  ip-address of 128.0.0.1.  This will force all traffic through the Pi -- making the Pi, once again, a critical component.)


What happens under the hood:

When your device tries to resolve a blocked domain name, the DNS service drops the request in the hole and discards it.  The target domain does not even know you attempted a call.  No graphics, scripts, or other code will run from that site.  If a page has code that reaches out, from your browser, those domains will be dropped and the code will think no network was available.  This is a win-win for you.

While surfing on my cell phone, I noticed an ugly "webpage not available" in the middle of the article -- this is likely an advertisement and likely that ad is recording your PC's IP address and other information.  The 'page not available' message is the Pi-hole at work, discarding the traffic.  Each application or browser decides how to handle the error in its own fashion.



Most applications show white-space where the ad lived -- with no obvious errors. 

The neat thing about this is the vendor never knew you attempted the connection because it is blocked before the traffic left the house.  You won't be tracked, monitored, or recorded as you read articles, and big advertising graphics won't download.


As seen on the administrative screens, here is a snapshot of recent activity after a few random seconds of activity.  I can see my TV is busy on the network, playing Pandora.  I caught a Nest Thermostat checking on the daily weather.  This traffic was allowed to pass. 

Click for larger view

But "settings-win.data.microsoft.com" was blocked.  This is Microsoft collecting diagnostic data for the Consumer Windows Experience program; see link Infoworld article.  The Pi-hole team decided this was intrusive and added this address to the blocked domain list.  From the admin panel, it could be white-listed with a click.

This report is where I find ad and email tracking sites.


Pi Administrative Login:

Test the administrative login.

From any browser, type the Pi's IP address/admin:

192.168.0.5/admin  (press enter)

On the left-nav, click Login, using the Pi's administrator password (changed and recorded in the steps above).  The Dashboard displays.  There are two areas of particular interest:  White and Black lists.


White Lists:

For sites you want to support in their advertising, such as the NewYork Times, allow them their ad-revenue by adding their domain to the Pi's white-list.

If you decide to keep your browser's ad-blocking software installed, you will also have to add the domain to that program's white-list.  With this said, I would de-install adblockers from your desktop clients -- but leave them installed on laptops that might travel outside of the pi-network.

Add these domains to your whitelist

nexus.officeapps.live.com    (Microsoft; used by Outlook; Media Player)
redire.metaservices.microsoft.com  (reported by Windows Media Player)
 



Black Lists:


Keyliner recommends manually adding the following to the Black List -- especially the Email tracking addresses. These are addresses I have discovered, that have not made it to the Pi's official lists. From the Pi-hole's administrative login screen, manually blacklisted these additional sites. 

When black-listing; always add as a "WildCard":

123banners.com
l90.com
adforce.com
advertising.com
agkn.com
appnet.com
avenuea.com
babator.com
bananatag.com      #email tracking
bluekai.com
bluestreak.com
burstmedia.com
burstnet.com
cirrusinsight.com  #email tracking
clearslide.com     #email tracking
clipix.com
contactmonkey.com  #email tracking
demdex.net
deskun.com         #email tracking
didtheyreadit.com  #email tracking
doubleclick.com
doubleclick.net
dynamicyield.com
engage.com
exelator.com
extreme-dm.com
fastclick.net
filepicker.io
g2crowd.com       #email tracking iko system also velocify
getnotify.com     #email tracking
gigya.com
gmelius.com       #email tracking
gobankingrates.com
go.com
hubspot.com       #email tracking
icanbuy.com
imgis.com
imrworldwide.com
intelliverse.com  #email tracking
keywee.co         #Note the .co, not .com
livehive.com      #email tracking
mail-track.com    #email tracking
minute.ly
newtonmail.com    #email tracking
nr-data.net
optimizely.com
outbrain.com
outreach.com      #email tracking
pagefair.com
pixelsite.info    #email tracking
pubexchange.com
quantserve.com
remail.com        #email tracking
remail.io         #email tracking
rlcdn.com
rocketbolt.com    #email tracking
ru4.com
salesloft.com     #email tracking
sidekick.com      #email tracking, now hubspot
saleshandy.com    #email tracking
scorecardresearch.com
stats.net
streak.comp
sync.optimatic.com
taboola.com
teknosurf.com
tinypass.com
toutapp.com       #email tracking
tru.am
valueclick.com
velocify.com      #email tracking Velocity Pulse
voicefive.com
websidestory.com
w55c.net
yesware.com       #email tracking




De-Installing the Pi:

From the admin screen, you can temporarily disable the Pi for (5-minutes, 10-minutes) while testing.  When disabled, all requests pass through to (Google's) DNS service and all Pi-protection is lost.  Note: This was specified in the Pi-installation screens -- and this is not your router's secondary DNS setting.

To permanently remove the Pi-hole from the network, re-edit the local .1 Gateway router(s), changing the Static DNS field from (192.168.0.5  or 192.168.1.151, etc.)  to Google's DNS: 8.8.8.8. If you have two routers; edit both.  A worse choice would be to return the Routers to "Auto-DNS" -- this would put you at your ISP's mercy.

Once changed, the Pi can be unplugged and removed from the network.


Known Problems:

Some sites, especially those that show the "top 100 celebrity before and after photos" will be blocked.  Reason: These are trolling sites, with obtrusive ads and with possible fly-by installs.  These sites were deemed dangerous and were blocked by the Pi-hole community.  Trust their decision.

Sadly, every other type of web failure will be blamed on the Pi. 

My experience is the Pi has not been wrong.

Use the admin screens to prove this by temporarily disabling the Pi-hole and re-testing the site or page in question (see side-illustration, directly above).  Once the Pi is disabled, retest the page.  If it still malfunctions, then you know the Pi is innocent.  The Pi does not interfere with non-blocked sites.  If the Pi blocks the site, it almost always has a good reason for doing so.  If you trust the blocked site, and insist on arriving (overriding thousands of volunteer's opinions), add the domain to the white list. 




Pi-hole and Ad-blocker Ethics

A word about publishers who need revenue to keep producing content.  Ad-blocking, and the Pi-hole, cut into these revenue streams. But the current model of using third-parties to display ads is broken.  With this, we might want to let the New York Times broadcast ads, but the ad-sites are being blocked as a third-party and it is not possible to allow an exception without allowing the same ad-network across all sites. 

Many publishers now detect ad-blockers, such as Ublock-Origin, and refuse to display the content.  The Pi-hole can sometimes dance around that restriction.  In other words, you can disable the ad-blocker and let the Pi-hole do the blocking undetected.

Ultimately, as the industry matures, publishers will be forced to host the ads on their own site and ad-blockers and the pi-hole will be less effective.  In other words, you can't block the New York Times completely, or none of the content will show.

The other side of this argument is obvious:  Publishers and Advertisers have abused the ads.  Displaying annoying ads, ads that occupy most of the screen, non-dismissable ads, and small articles broken into dozens of pages to force ad-impressions.  Abuse is everywhere.  This is why Pi-hole exists.



Update:
 2019.01 - I completely rebuilt the Pi, with new OS and new versions of Pi, using these same instructions.  The new version looks and acts identically to the old.  And, as before, all is well.  Still very pleased with the device.  I added more bling to this article to make it easier to follow.  This is an admittedly a complex project.

2018.06 - Six months and the PI is still going strong!  Still a fun and recommended project.

2018.03 - My spouse was trying to login to a site to pay a bill.  The site turned out to be a phishing site from an email (never click links in email!).  The pi intercepted.  Spouse complained she could not login.  The Pi saved our checking account that day.



Related Keyliner Articles:
This is the way I used to do this -- manually blocking about 50 high-volume sites.  With this article, I now block 100,000 sites!
Stopping Tracking Cookies with whack-a-mole - blocking DNS using Acrylic DNS.

Learn more about the pi-hole project here:
https://pi-hole.net/2018/01/11/pi-hole-is-open-source-consume-contribute-or-both/#more-9734
and
https://pi-hole.net/2017/05/12/seven-things-you-may-not-know-about-pi-hole


Related Thoughts:
Some routers run Linux under the hood and can be re-programmed.  After reading this article, https://www.ab-solution.info, my co-worker reviewed a similar project, which adds a sink-hole to the router.  The router is the best place to store this process, but not all routers can be re-programmed and this takes skills.  Ultimately, he reported back (unspecified) troubles and abandoned that solution, returning to a simpler Pi-hole.


Instead of Google's DNS 8.8.8.8, you can use openDNS's ip address. 
These are now options on the Pi-hole's installation screens:

208.67.222.222 or
208.67.220.220.

OpenDNS also has a "Home" service that blocks phishing sites, porn, acting much like the PI.  Use these addresses on your router if you don't want to use the PI solution.  The Pi is a tad faster and it subscribes to the OpenDSN list, and others, so little is gained by making this the Pi's secondary address:
  • 208.67.222.123
  • 208.67.220.123
Your comments:

I found this to be an enjoyable and worthwhile project.  I would love to hear your comments. 
If you like the Pi-hole project, donate a few dollars their way; they deserve the support.  See the admin-login screens.

Originally published: 2017.11 - 2019.01.  Rewritten and updated.


1 comment:

  1. The author comments: A friend complained that a site, "The 10 best ways to...." would not load, being blocked by the Pi.

    I said hurray! This was a trolling site, whose sole purpose was to rack-up page-views by showing one sentence at a time, and making you click 'Next" 20 times. Blocked by design.

    ReplyDelete

Comments are moderated and published upon review.