Motivation
Under the twitter hashtag #adblockgate you find news related to the business of the company behind the adblock plus browser plugin for chrome or firefox.
The problem with their business is the secret white list which serve you ads without a chance to avoid them.
The advantage of this solution here is to have a central ad blocking service for all of your devices including smart phones and tablets.
Used Software (under ubuntu 13.04):
- Apache Httpd (for serving the pac file)
- Squid3 (as primary proxy)
- Quintolabs (as ICAP Server for ad blocking)
Install Apache Httpd
apt-get install apache2
Install Squid3
apt-get install squid3
Install Quintolabs
open Download page and choose deb File and run the command (as root)
dpkg -i qlproxy-2.0.0.d746b-ubuntu_i386.deb
Create PAC File for the web clients
Change <your ubuntu host ip> with the ip of your server
nano /var/www/proxy.pac
function FindProxyForURL(url, host) {
// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.com")) {
return "DIRECT";
}
// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.net")) {
return "DIRECT";
}
// avoid proxy for URLs inside your network
if (isInNet(host, "192.168.178.0", "255.255.255.0")) {
return "DIRECT";
}
// the rest goes through your proxy
return "PROXY <your ubuntu host ip>:3128";
}
Configure Squid3
nano /etc/squid3/squid.conf
acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow all http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost http_access deny all http_port 3128 cache_mem 768 MB maximum_object_size_in_memory 1024 KB memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA pid_filename /var/run/squid3.pid buffered_logs on coredump_dir /var/spool/squid3 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880 refresh_pattern . 0 20% 4320 negative_dns_ttl 5 minutes via off ignore_expect_100 on forward_timeout 30 seconds connect_timeout 30 seconds read_timeout 30 seconds request_timeout 30 seconds persistent_request_timeout 1 minutes client_lifetime 20 hours dns_timeout 5 minutes ipcache_size 10240 forwarded_for delete client_db off acl home_network src 192.168.178.0/24 icap_enable on icap_preview_enable on icap_preview_size 4096 icap_persistent_connections on icap_send_client_ip on icap_send_client_username on icap_client_username_header X-Client-Username icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod adaptation_access qlproxy1 allow all adaptation_access qlproxy2 allow all
Patch Quintolabs
To use own ad block lists change the settings file
nano /opt/quintolabs/qlproxy/bin/settings.py
adblock = {
“DataDir” : macro_var + “/spool/adblock”,
“DataFiles” : {
“easylist.txt” : “https://easylist-downloads.adblockplus.org/easylist.txt”,
“easyprivacy.txt” : “https://easylist-downloads.adblockplus.org/easyprivacy.txt”,
“easylistgermany.txt” : “https://easylist-downloads.adblockplus.org/easylistgermany.txt”,
“fanboy-russian.txt” : “https://secure.fanboy.co.nz/fanboy-russian.txt”,
“own-adblock.txt” : “http://ralf.schaeftlein.com/adblock-plus.txt”
}
}
own-adblock.txt can be used as alias for own ad block subscriptions. See further configuration below.
Change the validation of ad block lists to avoid verification errors with different headers
nano /opt/quintolabs/qlproxy/bin/update_adblock.py
change verify function line to this
if re.search(r’\[Adblock\s+Plus’, line) != None:
Configure Quintolabs
Enable ad block module in the global config
nano /etc/opt/quintolabs/qlproxy/qlproxyd.conf
adblock_enabled = yes
and disable the other modules
urlblock_enabled = no
httpblock_enabled = no
contentblock_enabled = no
adultblock_enabled = no
Enable relaxed module for your subnet
nano /etc/opt/quintolabs/qlproxy/policies/relaxed/members.conf
Change the last line to (assuming 192.168.0.0 is the network subnet of your clients)
user_ip_subnet = 192.168.0.0/16
set pass through domains for your antivirus tool by editing
nano /etc/opt/quintolabs/qlproxy/policies/relaxed/exceptions.conf
Change the lines with domain name for your needs like this for bitdefender
# disable filtering for all third level domains of the following servers
domain_name = .microsoft.com
domain_name = .quintolabs.com
domain_name = .bitdefender.com
domain_name = .bitdefender.net
Enable all ad block subscriptions
nano /etc/opt/quintolabs/qlproxy/policies/relaxed/rules/block_ads.conf
use_subscription = easylist.txt
use_subscription = easylistgermany.txt
use_subscription = fanboy-russian.txt
use_subscription = easylist_custom.txt
use_subscription = easyprivacy.txt
use_subscription = own-adblock.txt
Restart Quintolabs to read change configs
/etc/init.d/qlproxy restart
Update ad block list for the first time
/etc/cron.daily/qlproxy_update
Restart Quintolabs to read change ad block lists
/etc/init.d/qlproxy restart
Restart web and proxy server
restart squid3
service apache2 restart
Configure windows browser
- Start internet explorer and open settings.
- Go to connection tab
- click on lan settings
- click second checkbox for “script for automatic configuration” and enter (replace <servername> with your ubuntu hostname)http://<servername>/proxy.pac
- Confirm dialog with ok and close settings with ok
- Restart all browsers
This affect chrome and internet explorer.
Under Firefox
- open settings
- go to advanced
- go to network
- Click on connections
- Set the last line to the same url as step 4 from above and click on the radiobox over the url for “automatic proxy configuration url”
- Click on “reload” button beside the url text field
Remove existing browser plugins
Uninstall chrome and firefox adblock plus plugins
Set proxy under android for the wlan
- Open settings
- Click on wlan (which is enabled)
- Hold the entry with your ssid pressed
- Choose from menu “change network setttings”
- Click on checkbox “show extended settings”
- Choose “manual” as proxy settings
- Choose <servername> (like in the url above) as proxy hostname
- Choose “3128” as proxy port
- Click on save button
Test ad block
Open web pages like the following to see if the ad banners are removed by the proxy server
Check if proxy can be detected by external web sites
Sample sites which looks for http header in the request to see if you are coming via a proxy
http://whatismyipaddress.com/proxy-check
“Proxy server not detected.”
http://www.lagado.com/proxy-test
“This request appears NOT to have come via a proxy.”