Tag Archives: ubuntu

VPN Connection from Ubuntu 13.04 to AVM Fritzbox

Pre Requisites

Install Cisco compatible VPN Client with

sudo apt-get install vpnc network-manager-vpnc network-manager-vpnc-gnome

Fritzbox

  • Login into your Fritzbox as admin
  • Go to System -> Fritzbox Users
  • Click on the edit icon from one of your users
  • Check the VPN Checkbox
  • Click on the Link in the last line for showing the VPN settings
  • Leave the Popup with the settings open

Ubuntu

  • Click on the Connections Menu icon in the toolbar on top
  • Go to VPN Connections -> Configure VPN
  • Click on Add Button at the right side
  • Choose from the list under VPN the Cisco compatible VPN client type
  • Click on create button
  • Enter the following values (<Ubuntu>: <Fritzbox>)
    • Connection Name : Fritzbox
    • Gateway: <Server Adress>
    • User Name: <IPSec Identifier>
    • User Password :<User Password>
    • Group Name: <IPSec Identifier>
    • Group Password: <IPSec Pre-Shared Key>
  • Set the Password Option to “Saved” (optional)
  • Click on save button
  • Cut the cable connection if established
  • Establish a mobile connection
  • Choose from VPN menu the new created VPN connection
  • Try accessing the Fritzbox web interface with your favorite browser

Ad block with Squid3 and Quintolabs instead of Adblock Plus

Motivation

Under the twitter hashtag #adblockgate you find news related to the business of the company behind the adblock plus browser plugin for chrome or firefox.

The problem with their business is the secret white list which serve you ads without a chance to avoid them.

The advantage of this solution here is to have a central ad blocking service for all of your devices including smart phones and tablets.

Used Software (under ubuntu 13.04):

  • Apache Httpd (for serving the pac file)
  • Squid3 (as primary proxy)
  • Quintolabs (as  ICAP Server for ad blocking)

Install Apache Httpd

apt-get install apache2

Install Squid3

apt-get install squid3

Install Quintolabs

open Download page and choose deb File and run the command (as root)

dpkg -i qlproxy-2.0.0.d746b-ubuntu_i386.deb

Create PAC File for the web clients

Change <your ubuntu host ip> with the ip of your server

nano /var/www/proxy.pac


function FindProxyForURL(url, host) {
// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.com")) {
return "DIRECT";
}

// avoid anti virus update problems
if (shExpMatch(host,"*.bitdefender.net")) {
return "DIRECT";
}
// avoid proxy for URLs inside your network
if (isInNet(host, "192.168.178.0", "255.255.255.0")) {
return "DIRECT";
}

// the rest goes through your proxy
return "PROXY <your ubuntu host ip>:3128";
}

Configure Squid3

nano /etc/squid3/squid.conf


acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access allow all
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access deny all
http_port 3128
cache_mem 768 MB
maximum_object_size_in_memory 1024 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
pid_filename /var/run/squid3.pid
buffered_logs on
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320
negative_dns_ttl 5 minutes
via off
ignore_expect_100 on
forward_timeout 30 seconds
connect_timeout 30 seconds
read_timeout 30 seconds
request_timeout 30 seconds
persistent_request_timeout 1 minutes
client_lifetime 20 hours
dns_timeout 5 minutes
ipcache_size 10240
forwarded_for delete
client_db off
acl home_network src 192.168.178.0/24
icap_enable on
icap_preview_enable on
icap_preview_size 4096
icap_persistent_connections on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Client-Username
icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod
icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod
adaptation_access qlproxy1 allow all
adaptation_access qlproxy2 allow all

Patch Quintolabs

To use own ad block lists change the settings file

nano /opt/quintolabs/qlproxy/bin/settings.py

adblock = {
“DataDir” : macro_var + “/spool/adblock”,
“DataFiles” : {
“easylist.txt” : “https://easylist-downloads.adblockplus.org/easylist.txt”,
“easyprivacy.txt” : “https://easylist-downloads.adblockplus.org/easyprivacy.txt”,
“easylistgermany.txt” : “https://easylist-downloads.adblockplus.org/easylistgermany.txt”,
“fanboy-russian.txt” : “https://secure.fanboy.co.nz/fanboy-russian.txt”,
“own-adblock.txt” : “http://ralf.schaeftlein.com/adblock-plus.txt”
}
}

own-adblock.txt can be used as alias for own ad block subscriptions. See further configuration below.

Change the validation of ad block lists to avoid verification errors with different headers

nano /opt/quintolabs/qlproxy/bin/update_adblock.py

change verify function line to this

if re.search(r’\[Adblock\s+Plus’, line) != None:

Configure Quintolabs

Enable ad block module in the global config

nano /etc/opt/quintolabs/qlproxy/qlproxyd.conf

adblock_enabled = yes

and disable the other modules

urlblock_enabled = no
httpblock_enabled = no
contentblock_enabled = no
adultblock_enabled = no

Enable relaxed module for your subnet

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/members.conf

Change the last line to (assuming 192.168.0.0 is the network subnet of your clients)

user_ip_subnet = 192.168.0.0/16

set pass through domains for your antivirus tool by editing

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/exceptions.conf

Change the lines with domain name for your needs like this for bitdefender

# disable filtering for all third level domains of the following servers
domain_name = .microsoft.com
domain_name = .quintolabs.com
domain_name = .bitdefender.com
domain_name = .bitdefender.net

Enable all ad block subscriptions

nano /etc/opt/quintolabs/qlproxy/policies/relaxed/rules/block_ads.conf

use_subscription = easylist.txt
use_subscription = easylistgermany.txt
use_subscription = fanboy-russian.txt
use_subscription = easylist_custom.txt
use_subscription = easyprivacy.txt
use_subscription = own-adblock.txt

Restart Quintolabs to read change configs

/etc/init.d/qlproxy restart

Update ad block list for the first time

/etc/cron.daily/qlproxy_update

Restart Quintolabs to read change ad block lists

/etc/init.d/qlproxy restart

Restart web and proxy server

restart squid3

service apache2 restart

Configure windows browser

  1. Start internet explorer and open settings.
  2. Go to connection tab
  3. click on lan settings
  4. click second checkbox for “script for automatic configuration” and enter (replace <servername> with your ubuntu hostname)http://<servername>/proxy.pac
  5. Confirm dialog with ok and close settings with ok
  6. Restart all browsers

This affect chrome and internet explorer.

Under Firefox

  1. open settings
  2. go to advanced
  3. go to network
  4. Click on connections
  5. Set the last line to the same url as step 4 from above and click on the radiobox over the url for “automatic proxy configuration url”
  6. Click on “reload” button beside the url text field

Remove existing browser plugins

Uninstall chrome and firefox adblock plus plugins

Set proxy under android for the wlan

  1. Open settings 
  2. Click on wlan (which is enabled)
  3. Hold the entry with your ssid pressed
  4. Choose from menu “change network setttings”
  5. Click on checkbox “show extended settings”
  6. Choose “manual” as proxy settings
  7. Choose <servername> (like in the url above) as proxy hostname
  8. Choose “3128” as proxy port
  9. Click on save button

Test ad block

Open web pages like the following to see if the ad banners are removed by the proxy server

http://www.spiegel.de

http://www.engadget.com/

Check if proxy can be detected by external web sites

Sample sites which looks for http header in the request to see if you are coming via a proxy

http://whatismyipaddress.com/proxy-check

“Proxy server not detected.”

http://www.lagado.com/proxy-test

“This request appears NOT to have come via a proxy.”

Ubuntu 13.04 on Lenovo Thinkpad X60

Hardware Spec

  • 4GB RAM
  • 120 GB Samsung SSD 830 (AHCI Mode)
  • Dual Core L7500 @ 1.6 Ghz

Used Distro

Ubuntu 13.04 x64 Desktop Edition

Installation

Download the iso file and use unetbootin to install on a usb stick. Leave the stick in the laptop. Reboot. Go into bios by pressing ThinkVantage button and then F1. Choose Startup -> Boot If the usb hdd is excluded then include them with x button and move up with F6. Press F10 to save and exit. Power on the laptop and choose “Install ubuntu” to start installation process.

After installation

recommended additional software (install with “apt-get install <name>”)

  • ntp (for correct date and time)
  • xournal (tablet programm for writing combined with stylus notes)
  • libxss1 (prerequisites for google chrome)
  • lingconf2-4 (prerequisites for google chrome)

Google Chrome

Open Download page and click on chrome download button. Choose “64bit .deb” and click on “accept and install” button

Get rid of anoying shopping hints in unity

apt-get remove unity-lens-shopping

Click on the amazon icon in the unity starter with right click and choose remove from starter

Getting special Hardware to work

Hibernate on closing laptop

Test if it works by running

pm-hibernate

If the laptop suspend to disk and start at the current state after reboot everything is fine to go on.

Edit file /etc/polkit-1/localauthority/50-local.d/com.ubuntu.enable-hibernate.pkla

[Re-enable hibernate by default]
Identity=unix-user:*
Action=org.freedesktop.upower.hibernate
ResultActive=yes

Reboot machine and open Energy menu (click on the battery in the top bar and choose energy settings) to define the hibernate state if the laptop cover is closed.

Gnome Buttons on the left side

gsettings set org.gnome.desktop.wm.preferences button-layout ‘:minimize,maximize,close’

Fingerprint Sensor

sudo apt-add-repository ppa:fingerprint/fingerprint-gui && sudo apt-get update

sudo apt-get install libbsapi policykit-1-fingerprint-gui fingerprint-gui

run the fingerprint.gui:

  1. Show Vendor => Choose STMicroelectronics and click next
  2. Choose a finger and click next
  3. Repeat five times to scan and click next
  4. Click on the test button and see if it works

Now you can login or authenticate for sudo with your fingerprint

Source:

 

Xlib: extension “RANDR” missing on display…

i got in the log of my selenium based integration test against firefox inside a Xvfb server. My first thought was of a missing library but the problem is caused by selenium:

“org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:”

was the line above and i had recently updated firefox on my ubuntu 12.01 server to version 15 by normal updates. The maven dependent selenium plugin version was still the same when i got the error. After updating to the latest version everything works as before:

 

		
			org.seleniumhq.selenium
			selenium-java
			2.24.1
			test
		

Running headless webdriver based selenium junit tests inside jenkins under ubuntu linux

My test setup is for a little wicket 1.5.x based web application with ui tests running during integration-test phase of maven 3.x. The problem here is that my ubuntu 12.01 server has only a console and no gnome or kde running like a desktop linux. Inside eclipse is a test, which starts a browser like firefox to run automated clicks, no problem. I decided to use the webdriver based selenium tests which can use several driver for the different browser. Each driver supports a different browser like chrome, internet explorer or firefox. My tests starts a firefox with explicit locale setting. The wicket application is i18n localized for english an german speaking customer. On the server is a jenkins ci installed with subversion polling to run automated tests during maven build. You can run the scenario with no problems as well under hudson as ci. The solution use xvfb a virtual xwindow system for firefox as desktop. It will automatically started and stopped by jenkins during a job build.

Software Installation on the server

Installation of xvfb

apt-get install xvfb gtk2-engines-pixbuf

Installation of fonts

apt-get install xfonts-base xfonts-75dpi xfonts-100dpi
apt-get install xfonts-scalable xfonts-cyrillic

Installation of tools for testing xvfb

apt-get install x11-apps imagemagick

Testing server installation

1. Console run server

Xvfb -ac :99 -screen 0 1280x1024x16

2. console start firefox

export DISPLAY=:99
firefox http://ralf.schaeftlein.com

3. console make a screenshot

xwd -root -display :99 | convert xwd:- capture.png

And see a result like this when you retrieve the file capture.png via ssh

 

 

 

 

 
 

Jenkins Configuration

Init.d Script for xvfb

Save content as file under /etc/inid.d/xvfb

XVFB=/usr/bin/Xvfb
XVFBARGS="$DISPLAY -ac -screen 0 1280x1024x16"
PIDFILE=/var/hudson/xvfb_${DISPLAY:1}.pid
case "$1" in
  start)
    echo -n "Starting virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS
    echo "."
    ;;
  stop)
    echo -n "Stopping virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE
    echo "."
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
  echo "Usage: /etc/init.d/xvfb {start|stop|restart}"
  exit 1
esac
exit 0

Set rights for jenkins running user on the script

chown jenkins:root /etc/init.d/xvfb
chmod ug+rwx /etc/init.d/xvfb

Add display environment variable to jenkins init.d script

export DISPLAY=:99

Create a new jenkins job

Create a new jenkins job for your web project stored in subversion

 Add to pre and post build step a shell script to start and stop xvfb

 

 

 

 

 

 

 

 

Configure maven pom of the web project

Define special tests in surefire to run as integration tests and jetty as integration application server

			
				org.apache.maven.plugins
				maven-surefire-plugin
				2.4.3
				
					true
				
				
					
						surefire-test
						test
						
							test
						
						
							false
							
								**/itest/**
							
						
					

					
						surefire-itest
						integration-test
						
							test
						
						
							false
							
								**/itest/**
							
						
					
				
			
			
				org.mortbay.jetty
				maven-jetty-plugin
				6.1.26
				
					10
					foo
					9998
					/${project.artifactId}
					true
					${basedir}/src/test/webapp/WEB-INF/web.xml
					
						
							9999
							60000
						
					
				
				
					
						start-jetty
						pre-integration-test
						
							run
						
						
							0
							true
						
					
					
						stop-jetty
						post-integration-test
						
							stop
						
					
				
			

Add dependency to selenium maven artifacts

		
			org.seleniumhq.selenium.client-drivers
			selenium-java-client-driver
			1.0.2
			test
		
		
			org.seleniumhq.selenium
			selenium-java
			2.21.0
			test
		

Record ui steps and write a selenium junit test

Install the selenium ide as firefox plugin

Start firefox and install the xpi file as new plugin. Restart firefox afterwards.

Record ui steps

  1. Open the menu, choose web developer and their selenium ide.
  2. Start inside eclipse the tomcat with your web application
  3. Open in firefox the web application
  4. Click  inside selenium ide on the red record button for start recording
  5. Click through your web application
  6. Click again on the red record button for stop recording
  7. Choose from menu “Export as testcase” and their “Junit4 (Webdriver backed)”
  8. Save file as test.java

Create an new selenium junit test file

  1. Copy the test.java into your eclipse project into the src/test/java folder
  2. Adopt the class definition and setup method like this
public class SeleniumTest {

	private Log log = LogFactory.getLog(getClass());

	protected Selenium selenium;

	@Before
	public void setUp() throws Exception {
		FirefoxProfile profile = new FirefoxProfile();
		// enable german language locale
		profile.setPreference("intl.accept_languages", "de-de,de");
		profile.setEnableNativeEvents(true);
		profile.layoutOnDisk();
		WebDriver driver = new FirefoxDriver(profile);
		String baseUrl = "http://localhost:9999/"; // port jetty surefire integration test
		selenium = new WebDriverBackedSelenium(driver, baseUrl);
	}

– insert java code –

  1. Check into subversion and control jenkins job console.