Category Archives: Windows

Ad block with Squid3 and Quintolabs instead of Adblock Plus

Motivation

Under the twitter hashtag #adblockgate you find news related to the business of the company behind the adblock plus browser plugin for chrome or firefox.

The problem with their business is the secret white list which serve you ads without a chance to avoid them.

The advantage of this solution here is to have a central ad blocking service for all of your devices including smart phones and tablets.

Used Software (under ubuntu 13.04):

  • Apache Httpd (for serving the pac file)
  • Squid3 (as primary proxy)
  • Quintolabs (as  ICAP Server for ad blocking)

Install Apache Httpd

apt-get install apache2

Install Squid3

apt-get install squid3

Install Quintolabs

open Download page and choose deb File and run the command (as root)

dpkg -i qlproxy-2.0.0.d746b-ubuntu_i386.deb

Create PAC File for the web clients

Change <your ubuntu host ip> with the ip of your server

nano /var/www/proxy.pac


function FindProxyForURL(url, host) {
// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.com")) {
return "DIRECT";
}

// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.net")) {
return "DIRECT";
}
// avoid proxy for URLs inside your network
if (isInNet(host, "192.168.178.0", "255.255.255.0")) {
return "DIRECT";
}

// the rest goes through your proxy
return "PROXY <your ubuntu host ip>:3128";
}

Configure Squid3

nano /etc/squid3/squid.conf


acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access allow all
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all
http_port 3128
cache_mem 768 MB
maximum_object_size_in_memory 1024 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
pid_filename /var/run/squid3.pid
buffered_logs on
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320
negative_dns_ttl 5 minutes
via off
ignore_expect_100 on
forward_timeout 30 seconds
connect_timeout 30 seconds
read_timeout 30 seconds
request_timeout 30 seconds
persistent_request_timeout 1 minutes
client_lifetime 20 hours
dns_timeout 5 minutes
ipcache_size 10240
forwarded_for delete
client_db off
acl home_network src 192.168.178.0/24
icap_enable on
icap_preview_enable on
icap_preview_size 4096
icap_persistent_connections on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Client-Username
icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod
icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod
adaptation_access qlproxy1 allow all
adaptation_access qlproxy2 allow all

Patch Quintolabs

To use own ad block lists change the settings file

nano /opt/quintolabs/qlproxy/bin/settings.py

adblock = {
“DataDir” : macro_var + “/spool/adblock”,
“DataFiles” : {
“easylist.txt” : “https://easylist-downloads.adblockplus.org/easylist.txt”,
“easyprivacy.txt” : “https://easylist-downloads.adblockplus.org/easyprivacy.txt”,
“easylistgermany.txt” : “https://easylist-downloads.adblockplus.org/easylistgermany.txt”,
“fanboy-russian.txt” : “https://secure.fanboy.co.nz/fanboy-russian.txt”,
“own-adblock.txt” : “http://ralf.schaeftlein.com/adblock-plus.txt”
}
}

own-adblock.txt can be used as alias for own ad block subscriptions. See further configuration below.

Change the validation of ad block lists to avoid verification errors with different headers

nano /opt/quintolabs/qlproxy/bin/update_adblock.py

change verify function line to this

if re.search(r’\[Adblock\s+Plus’, line) != None:

Configure Quintolabs

Enable ad block module in the global config

nano /etc/opt/quintolabs/qlproxy/qlproxyd.conf

adblock_enabled = yes

and disable the other modules

urlblock_enabled = no
httpblock_enabled = no
contentblock_enabled = no
adultblock_enabled = no

Enable relaxed module for your subnet

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/members.conf

Change the last line to (assuming 192.168.0.0 is the network subnet of your clients)

user_ip_subnet = 192.168.0.0/16

set pass through domains for your antivirus tool by editing

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/exceptions.conf

Change the lines with domain name for your needs like this for bitdefender

# disable filtering for all third level domains of the following servers
domain_name = .microsoft.com
domain_name = .quintolabs.com
domain_name = .bitdefender.com
domain_name = .bitdefender.net

Enable all ad block subscriptions

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/rules/block_ads.conf

use_subscription = easylist.txt
use_subscription = easylistgermany.txt
use_subscription = fanboy-russian.txt
use_subscription = easylist_custom.txt
use_subscription = easyprivacy.txt
use_subscription = own-adblock.txt

Restart Quintolabs to read change configs

/etc/init.d/qlproxy restart

Update ad block list for the first time

/etc/cron.daily/qlproxy_update

Restart Quintolabs to read change ad block lists

/etc/init.d/qlproxy restart

Restart web and proxy server

restart squid3

service apache2 restart

Configure windows browser

  1. Start internet explorer and open settings.
  2. Go to connection tab
  3. click on lan settings
  4. click second checkbox for “script for automatic configuration” and enter (replace <servername> with your ubuntu hostname)http://<servername>/proxy.pac
  5. Confirm dialog with ok and close settings with ok
  6. Restart all browsers

This affect chrome and internet explorer.

Under Firefox

  1. open settings
  2. go to advanced
  3. go to network
  4. Click on connections
  5. Set the last line to the same url as step 4 from above and click on the radiobox over the url for “automatic proxy configuration url”
  6. Click on “reload” button beside the url text field

Remove existing browser plugins

Uninstall chrome and firefox adblock plus plugins

Set proxy under android for the wlan

  1. Open settings 
  2. Click on wlan (which is enabled)
  3. Hold the entry with your ssid pressed
  4. Choose from menu “change network setttings”
  5. Click on checkbox “show extended settings”
  6. Choose “manual” as proxy settings
  7. Choose <servername> (like in the url above) as proxy hostname
  8. Choose “3128” as proxy port
  9. Click on save button

Test ad block

Open web pages like the following to see if the ad banners are removed by the proxy server

http://www.spiegel.de

http://www.engadget.com/

Check if proxy can be detected by external web sites

Sample sites which looks for http header in the request to see if you are coming via a proxy

http://whatismyipaddress.com/proxy-check

“Proxy server not detected.”

http://www.lagado.com/proxy-test

“This request appears NOT to have come via a proxy.”

Start and stop windows services with powershell scripts

With Powershell you can easily start or stop specific windows services depending on current status. An example is starting the VMWare Services only if they are currently stopped. The command

get-service

list all windows service with their status, short – and display name. Create a new File startVMWare.ps1 and paste the following

foreach ($svc in Get-Service){
  if(($svc.displayname.StartsWith("VMware")) -AND ($svc.Status -eq "Stopped")) {
    echo $svc.DisplayName
    Start-Service $svc.name
  }
}

Sign the script as shown in my previous post to run self signed scripts. Run the script in an powershell with administrator rights.

.\startVMWare.ps1

The script starts only stopped VMWare services. To run this script directly you can write a small dos file startVMWare.cmd with the following content

powershell -file <FULL PATH TO YOUR SCRIPT>\startVMWare.ps1

Start the Dos file with right click and admin rights to execute the vmware start powershell script.

Howto run self signed powershell scripts

Windows command line scripts was for a long time the only way for scripting windows. With the Windows PowerShell you have can write scripts more like an program in object oriented way. Starting with Windows 7 it is preinstalled with version 2.0 but can as well installed under Windows XP or Vista. Windows 8 will ship with version 3.0 which adds windows work flow foundation functionality. Windows 7 ships with an IDE for PowerShell called Windows PowerShell ISE”. Scripts stored in files with .ps1 suffix. A sample hello world looks like this:

echo "hello world"

Save the content in a file called hello.ps1. Start the powershell by searching for powershell in the windows 7 search box above the start button and with right click to run as administrator. Change the current folder with cdto the one where you saved your first powershell script. Run your script with

.\hello.ps1

Unfortunately you get a PSSecurityException because powershell script execution is controlled by an security policy. Like Java signed jars you must sign your scripts and set the policy to execute only signed scripts

Set-ExecutionPolicy Restricted

Powershell accepts self signed certificates and can be created by the makecert command. Makecert is part of the windows SDK. Download the installer and run through the wizard steps. At the last screen deselect all parts and check only the tools under first section called Windows Native Code Development. Open a command shell with shift and right click on the folder C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin. Enter the following commands to create a authority:

makecert -n "CN=PowerShell Local Certificate Root" -a sha1 -eku 1.3.6.1.5.5.7.3.3 -r -sv root.pvk root.cer -ss Root -sr localMachine

and the following command to create a self signed certificate

makecert -pe -n "CN=PowerShell User" -ss MY -a sha1 -eku 1.3.6.1.5.5.7.3.3 -iv root.pvk -ic root.cer

The following command in the powershell show your created certificate:

Get-ChildItem cert:\CurrentUser\My -codesign

Now we can sign our first script with the following command

Set-AuthenticodeSignature .\hello.ps1 @(Get-ChildItem cert:\CurrentUser\My -codesigning)[0]

Run now your signed script with

.\hello.ps1

with prints

hello world

Powershell scripts can use profile scripts to store common functions for own scripts, functions,.. Enter the following command to your standard profile file

$profile

Open the file or create it with a text editor and paste the following

function sign ($filename) {
 $cert = @(gci cert:\currentuser\my -codesigning)[0]
 Set-AuthenticodeSignature $filename $cert
 }

Sign the profile file

Set-AuthenticodeSignature $profile @(Get-ChildItem cert:\CurrentUser\My -codesigning)[0]

Write now your own little powershell script like test.ps1. The sign function can now be used like this

sign .\test1.ps1

Printing via SMB from Ubuntu 10.04 to Windows 7

I have successfully installed Ubuntu 10.04 Beta 1 on my Samsung NC10 with dual boot side by side to windows XP Home. The installation process is very smart and allow you to import settings like desktop background from the installed windows xp home. Printer configuration was a bit complicated. The first thing is to change the workgroup definition in

/etc/samba/smb.conf

workgroup = XYZ

Change XYZ to your windows 7 workgroup name. Next step is to uninstall the “Windows Live Sign-In Assistant” under windows 7 which is the main cause why the windows shares like the printers are not available from ubuntu. After that go to menu item System => administration => Printer. Click on add button and select from the list under network printer the last entry “windows printer via SAMBA”. When you now click on browse will be your workgroup and below your windows 7 pc with the printer shares be available.

Convert VMWare 2.x Image to VirtualBox 3.x (seamless mode)

Windows 7 Home Premium has no XP Mode Feature like in Pro or Ultimate Edition to run an Windows XP virtual machine seamless. Home Premium can be upgraded by Anytime Upgrade within Windows to Professional or Ultimate (90$* or 140$*). You need at least Professional Version for XP Mode. So i looked for alternatives. Under Windows XP was VMWare server 2.x my first choice without seamless mode. Seamless Mode made VM windows be available in host OS as “normal” windows without the rest of the VM OS. One Alternative is VMWare Workstation 7 with seamless mode (176€*). Paralells has currently an beta version available with Coherence (seamless) feature. Virtualbox as open source alternative has seamless mode too. You need only to install the guest additions of virtualbox, reboot after installation and call from menu “seamless mode”.

So i give Virtualbox a try and installed the current 3.0.10 version under Windows 7. VirtualBox can work with VMware VMDK Hard disks out of the box.

Steps to convert from VMware to Virtualbox:

  1. Start VMware image under VMware
  2. Uninstall VMware Tools inside VM
  3. Shutdown VM
  4. Open DOS command shell in VM folder
  5. set path variable to include VMWare server folder: set path=;%PATH%
  6. convert splitted VM hard disk to single file: vmware-vdiskmanager -r source_multiples.vmdk -t 2 single_file.vmdk (use ” to surround file name if file name contains spaces)
  7. copy new single_file.vmdk and single_file-flat.vmdk to \harddisks on host os with virtualbox
  8. Start VirtualBox under host OS
  9. Open in file menu “Virtual media manager”
  10. Go to “Hard Disks” tab
  11. Click on Add Button and choose single_file.vmdk
  12. Close dialog with OK button
  13. Click on “New” button to create a new VM
  14. Click on “Next” button
  15. Enter name for new VM and choose operation system including version (in my example Win_vista” as name, “Microsoft Windows” as OS and “Windows Vista” as version)
  16. Set memory to a suitable value (on my machine i prefer 512MB for good enough performance)
  17. On the next page choose “Use existing hard disk” and select from Drop down the old VMware hard disk image
  18. Click on Finish
  19. Open Settings of new VM to correct problems with different hard ware in VMware and VirtualBox
  20. Go to System => Motherboard and “enable IO APIC”
  21. a had an scsci lsi logic controller under VMware: Go to “Hard Disks” and “Enable Additional Controller” including selection of “SCSI (Lsilogic)”s
  22. Start new VM
  23. Install Virtualbox guest additions from vm window menu under “Devices”
  24. reboot vm
  25. Start for example a IE windows inside vm and call from “Machine” menu “seamless mode”

Currently that doesn’t seems to work with windows vista. Another VM in virtualbox with windows XP Professional works as expected in seamless mode.

Keep in mind that such a migration with Windows Vista as guest os needs another activation because of changed multiple hardware components.

Another possibility is to install the unofficial Virtual PC 6.1 for windows 7 from the knowledge base (32bit / 64bit). As described here you see in the windows 7 start menu the entries of the installed programs under the xp vm to start them directly in seamless mode. Comparing to the windows 7 XP mode their is the requirement for a separate windows xp license for a windows home or starter host os. Windows 7 professional or ultimate include such a license for a virtual windows xp on the same machine. Virtual PC needs a Virtualization hardware support like intel VT-X or AMD-V. The installation of Virtual PC fails if the processor and mainboard don’t fit to this requirement.

Startmenu entry of Virtual PC

Startmenue Virtual PC

Start installed Apps inside XP VM from Windows 7

Start Apps inside XP VM from Windows 7

Seamless started XP VM Application under Windows 7

Seamless started App

*Price are only snapshots from the manufacturer websites without any guarantee and only provided to compare the possibilities.