Running headless webdriver based selenium junit tests inside jenkins under ubuntu linux

My test setup is for a little wicket 1.5.x based web application with ui tests running during integration-test phase of maven 3.x. The problem here is that my ubuntu 12.01 server has only a console and no gnome or kde running like a desktop linux. Inside eclipse is a test, which starts a browser like firefox to run automated clicks, no problem. I decided to use the webdriver based selenium tests which can use several driver for the different browser. Each driver supports a different browser like chrome, internet explorer or firefox. My tests starts a firefox with explicit locale setting. The wicket application is i18n localized for english an german speaking customer. On the server is a jenkins ci installed with subversion polling to run automated tests during maven build. You can run the scenario with no problems as well under hudson as ci. The solution use xvfb a virtual xwindow system for firefox as desktop. It will automatically started and stopped by jenkins during a job build.

Software Installation on the server

Installation of xvfb

apt-get install xvfb gtk2-engines-pixbuf

Installation of fonts

apt-get install xfonts-base xfonts-75dpi xfonts-100dpi
apt-get install xfonts-scalable xfonts-cyrillic

Installation of tools for testing xvfb

apt-get install x11-apps imagemagick

Testing server installation

1. Console run server

Xvfb -ac :99 -screen 0 1280x1024x16

2. console start firefox

export DISPLAY=:99
firefox http://ralf.schaeftlein.com

3. console make a screenshot

xwd -root -display :99 | convert xwd:- capture.png

And see a result like this when you retrieve the file capture.png via ssh

 

 

 

 

 
 

Jenkins Configuration

Init.d Script for xvfb

Save content as file under /etc/inid.d/xvfb

XVFB=/usr/bin/Xvfb
XVFBARGS="$DISPLAY -ac -screen 0 1280x1024x16"
PIDFILE=/var/hudson/xvfb_${DISPLAY:1}.pid
case "$1" in
  start)
    echo -n "Starting virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS
    echo "."
    ;;
  stop)
    echo -n "Stopping virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE
    echo "."
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
  echo "Usage: /etc/init.d/xvfb {start|stop|restart}"
  exit 1
esac
exit 0

Set rights for jenkins running user on the script

chown jenkins:root /etc/init.d/xvfb
chmod ug+rwx /etc/init.d/xvfb

Add display environment variable to jenkins init.d script

export DISPLAY=:99

Create a new jenkins job

Create a new jenkins job for your web project stored in subversion

 Add to pre and post build step a shell script to start and stop xvfb

 

 

 

 

 

 

 

 

Configure maven pom of the web project

Define special tests in surefire to run as integration tests and jetty as integration application server

			
				org.apache.maven.plugins
				maven-surefire-plugin
				2.4.3
				
					true
				
				
					
						surefire-test
						test
						
							test
						
						
							false
							
								**/itest/**
							
						
					

					
						surefire-itest
						integration-test
						
							test
						
						
							false
							
								**/itest/**
							
						
					
				
			
			
				org.mortbay.jetty
				maven-jetty-plugin
				6.1.26
				
					10
					foo
					9998
					/${project.artifactId}
					true
					${basedir}/src/test/webapp/WEB-INF/web.xml
					
						
							9999
							60000
						
					
				
				
					
						start-jetty
						pre-integration-test
						
							run
						
						
							0
							true
						
					
					
						stop-jetty
						post-integration-test
						
							stop
						
					
				
			

Add dependency to selenium maven artifacts

		
			org.seleniumhq.selenium.client-drivers
			selenium-java-client-driver
			1.0.2
			test
		
		
			org.seleniumhq.selenium
			selenium-java
			2.21.0
			test
		

Record ui steps and write a selenium junit test

Install the selenium ide as firefox plugin

Start firefox and install the xpi file as new plugin. Restart firefox afterwards.

Record ui steps

  1. Open the menu, choose web developer and their selenium ide.
  2. Start inside eclipse the tomcat with your web application
  3. Open in firefox the web application
  4. Click  inside selenium ide on the red record button for start recording
  5. Click through your web application
  6. Click again on the red record button for stop recording
  7. Choose from menu “Export as testcase” and their “Junit4 (Webdriver backed)”
  8. Save file as test.java

Create an new selenium junit test file

  1. Copy the test.java into your eclipse project into the src/test/java folder
  2. Adopt the class definition and setup method like this
public class SeleniumTest {

	private Log log = LogFactory.getLog(getClass());

	protected Selenium selenium;

	@Before
	public void setUp() throws Exception {
		FirefoxProfile profile = new FirefoxProfile();
		// enable german language locale
		profile.setPreference("intl.accept_languages", "de-de,de");
		profile.setEnableNativeEvents(true);
		profile.layoutOnDisk();
		WebDriver driver = new FirefoxDriver(profile);
		String baseUrl = "http://localhost:9999/"; // port jetty surefire integration test
		selenium = new WebDriverBackedSelenium(driver, baseUrl);
	}

– insert java code –

  1. Check into subversion and control jenkins job console.

Migrating HSQLDB to MySQL

HSQLDB migration to MySQL is not out of the box supported by mysql. The MySQL Workbench can only handle with dumps made by other mysql server. HSQLDB Transfer Tool is no longer bundled with the standard hsqldb jar file in the 1.8.x or later releases. After downloading and building with

ant hsqldbutil

you get in lib an additional hsqldbutil jar. With the hsqldb.jar, the hsqldbutil.jar and and mysql connector jar in my classpath i tried the Database Manager (From Tools menu choose transfer) from hsql with no luck. The MySQL Migration Toolkit has reached the EOL phase but works perfect for me. After downloading add the hsqldb jar to the lib folder C:\Program Files (x86)\MySQL\MySQL Tools for 5.0\java\lib. I update as well the mysql connector to the latest one.

  1. Start your hsqldb server from command line
    1. java -cp hsqldb-1.8.0.10.jar org.hsqldb.Server -database <path to your hsqldb files>\<database name>
  2. Start MySQL Migration Toolkit
  3. Choose direct migration
  4. Choose as Source a generic jdbc
  5. Enter “org.hsqldb.jdbcDriver” as classname
  6. Enter “jdbc:hsqldb:hsql://localhost” as connection string
  7. Enter “sa” as username and leave password empty
  8. click next
  9. Configure your mysql connection
  10. click next
  11. choose public as schema
  12. click next several time till screen “object creation options”
  13. choose “create script file..” instead of “create objects online”
  14. click next several time till screen “data mapping options”
  15. choose “create script file..” instead of “transfer data online”
  16. click next several time till screen “Summary”
  17. click finish

With the sql you can use tools like phpmyadmin or the mysql workbench to import your data into the mysql server.

Start and stop windows services with powershell scripts

With Powershell you can easily start or stop specific windows services depending on current status. An example is starting the VMWare Services only if they are currently stopped. The command

get-service

list all windows service with their status, short – and display name. Create a new File startVMWare.ps1 and paste the following

foreach ($svc in Get-Service){
  if(($svc.displayname.StartsWith("VMware")) -AND ($svc.Status -eq "Stopped")) {
    echo $svc.DisplayName
    Start-Service $svc.name
  }
}

Sign the script as shown in my previous post to run self signed scripts. Run the script in an powershell with administrator rights.

.\startVMWare.ps1

The script starts only stopped VMWare services. To run this script directly you can write a small dos file startVMWare.cmd with the following content

powershell -file <FULL PATH TO YOUR SCRIPT>\startVMWare.ps1

Start the Dos file with right click and admin rights to execute the vmware start powershell script.

Howto run self signed powershell scripts

Windows command line scripts was for a long time the only way for scripting windows. With the Windows PowerShell you have can write scripts more like an program in object oriented way. Starting with Windows 7 it is preinstalled with version 2.0 but can as well installed under Windows XP or Vista. Windows 8 will ship with version 3.0 which adds windows work flow foundation functionality. Windows 7 ships with an IDE for PowerShell called Windows PowerShell ISE”. Scripts stored in files with .ps1 suffix. A sample hello world looks like this:

echo "hello world"

Save the content in a file called hello.ps1. Start the powershell by searching for powershell in the windows 7 search box above the start button and with right click to run as administrator. Change the current folder with cdto the one where you saved your first powershell script. Run your script with

.\hello.ps1

Unfortunately you get a PSSecurityException because powershell script execution is controlled by an security policy. Like Java signed jars you must sign your scripts and set the policy to execute only signed scripts

Set-ExecutionPolicy Restricted

Powershell accepts self signed certificates and can be created by the makecert command. Makecert is part of the windows SDK. Download the installer and run through the wizard steps. At the last screen deselect all parts and check only the tools under first section called Windows Native Code Development. Open a command shell with shift and right click on the folder C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin. Enter the following commands to create a authority:

makecert -n "CN=PowerShell Local Certificate Root" -a sha1 -eku 1.3.6.1.5.5.7.3.3 -r -sv root.pvk root.cer -ss Root -sr localMachine

and the following command to create a self signed certificate

makecert -pe -n "CN=PowerShell User" -ss MY -a sha1 -eku 1.3.6.1.5.5.7.3.3 -iv root.pvk -ic root.cer

The following command in the powershell show your created certificate:

Get-ChildItem cert:\CurrentUser\My -codesign

Now we can sign our first script with the following command

Set-AuthenticodeSignature .\hello.ps1 @(Get-ChildItem cert:\CurrentUser\My -codesigning)[0]

Run now your signed script with

.\hello.ps1

with prints

hello world

Powershell scripts can use profile scripts to store common functions for own scripts, functions,.. Enter the following command to your standard profile file

$profile

Open the file or create it with a text editor and paste the following

function sign ($filename) {
 $cert = @(gci cert:\currentuser\my -codesigning)[0]
 Set-AuthenticodeSignature $filename $cert
 }

Sign the profile file

Set-AuthenticodeSignature $profile @(Get-ChildItem cert:\CurrentUser\My -codesigning)[0]

Write now your own little powershell script like test.ps1. The sign function can now be used like this

sign .\test1.ps1

Configuring solr, tomcat 7 with mod_jk and apache 2.2

As follow up post to setting up solr i show you how to integrate tomcat into Apache as main web server:

The setup was made under ubuntu 11.10 with the following prerequisites:

  • CATALINA_HOME is /usr/share/tomcat7
  • JAVA_HOME is /usr/lib/jvm/default-java
  • HOSTNAME in my case is ubuntu-vm.localdomain

 

  1. apt-get install libapache2-mod-jk
  2. nano /etc/apache2/workers.properties with the following contents
workers.tomcat_home=/usr/share/tomcat7
workers.java_home=/usr/lib/jvm/default-java
ps=/
worker.list=ajp13
worker.ajp13.port=8009
worker.ajp13.host=localhost
worker.ajp13.type=ajp13
worker.ajp13.lbfactor=1
  1. nano /etc/apache2/mods-available/jk.conf with the following contents

        # Where to find workers.properties
        JkWorkersFile /etc/apache2/workers.properties
        # Where to put jk shared memory
        JkShmFile     /var/log/apache2/mod_jk.shm
        # Where to put jk logs
        JkLogFile     /var/log/apache2/mod_jk.log
        # Set the jk log level [debug/error/info]
        JkLogLevel    info
        # Select the timestamp log format
        JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
        # solr redirect
        JkMount /solr* ajp13

  1. less /etc/apache2/mods-available/jk.load to see the following contents
    1. LoadModule jk_module /usr/lib/apache2/modules/mod_jk.so
  2. run a2enmod jk to see if mod_jk is enabled
    1. Module jk already enabled
  3. less /usr/share/tomcat7/conf/server.xml to see a line like this
    1. <Connector port=”8009″ protocol=”AJP/1.3″ redirectPort=”8443″ />
  4. nano /etc/apache2/sites-available/default and insert a line into the virtualHost:80 section
    1. JKMountCopy On
  5. restart apache2 with service apach2 restart
  6. Open a browser on your client to open http://ubuntu-vm.localdomain/solr/admin/

tech stuff, development news…