All posts by Ralf Schäftlein

Configure logging in Solr

After the base installation of solr under tomcat 7 is now the step to configure seperate logging for solr. Fist step is to create a directory for the log file:

mkdir /var/log/solr
chown tomcat7:adm /var/log/solr

Now delete the following files in the extracted war lib folder (/usr/share/solr/war/WEB-INF/lib)

  • jcl-over-slf4j-1.6.4.jar
  • log4j-over-slf4j-1.6.4 .jar
  • slf4j-api-1.6.4.jar
  • slf4j-jdk14-1.6.4.jar

Copy the following files into this folder

  • slf4j-api-1.6.5.jar
  • slf4j-log4j12-1.6.5.jar
  • log4j-1.2.16.jar

Create a classes folder inside the WEB-INF folder

mkdir /usr/share/solr/war/WEB-INF/classes

Create a log4j.properties in this folder with the following sample content

log4j.rootLogger=WARN, solrLog
log4j.appender.solrLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.solrLog.File=/var/log/solr/solr.log
log4j.appender.solrLog.Append=true
log4j.appender.solrLog.Encoding=UTF-8
log4j.appender.solrLog.DatePattern=’-‘yyyy-MM-dd
log4j.appender.solrLog.layout=org.apache.log4j.PatternLayout
log4j.appender.solrLog.layout.ConversionPattern=%d [%t] %-5p %c – %m%n
log4j.logger.org.apache.solr=INFO

Change folder and file rights to give the tomcat instance user full access

chown -R tomcat7:adm /usr/share/solr/war

Restart tomcat

service tomcat7 restart

Now you will find a solr.log inside/var/log/solr/

Replacing web.xml with Java based Configuration for Servlet 3.x Webapplications

With the Servlet 3.x API as part of JEE 6 come with the possibility to configure a java web application by writing a java class instead of having a web.xml file inside the WEB-INF folder. My sample apache cxf project has a web.xml with a spring configuration listener and a cxf servlet mapping. I use jetty for integration tests inside maven builds and tomcat as application server for deployment and running the web application. The web service sample use Spring 3.1 and Apache CXF 2.6.2.

I migrated my old web project with just a few steps:

Maven Settings

Step 1: Integrate latest Spring Web API

The Spring 3.1 web project contains the SpringServletContainerInitializer class which will be automatically invoked by a servlet 3.x container:

	
		2.6.2
		3.1.2.RELEASE
	
		
			javax
			javaee-api
			6.0
			provided
		
		
			org.apache.cxf
			cxf-rt-frontend-jaxws
			${cxf.version}
		
		
			org.apache.cxf
			cxf-rt-transports-http
			${cxf.version}
		

		
		
			org.apache.cxf
			cxf-rt-transports-http-jetty
			${cxf.version}
		
		
			org.eclipse.jetty
			jetty-webapp
			8.1.1.v20120215
		
		
			junit
			junit
			4.10
			jar
			compile
		
		
			org.springframework
			spring-test
			${spring.framework.version}
			jar
			compile
		
		
			org.springframework
			spring-core
			${spring.framework.version}
			jar
			compile
		
		
			org.springframework
			spring-beans
			${spring.framework.version}
		
		
			org.springframework
			spring-web
			${spring.framework.version}
		

Step 2: Update build plugin for jetty to use servlet 3.x api

The Jetty 8.x supports Servlet 3.x api to support bootstrapping and java based configurations and is used by the jetty plugin:

			
				org.mortbay.jetty
				jetty-maven-plugin
				8.1.5.v20120716
				
					10
					
						/${project.artifactId}
					
					true
					foo
					9999
					
						
							9090
							60000
						
					
				
				
					
						start-jetty
						pre-integration-test
						
							run
						
						
							0
							true
						
					
					
						stop-jetty
						post-integration-test
						
							stop
						
					
				
			

Step 3: Configure Maven War Plugin to ignore web.xml

The maven war plugin throws otherwise a exception if it does not found the web.xml file:

			
				org.apache.maven.plugins
				maven-war-plugin
				2.2
				
					src/main/webapp
					${project.artifactId}
					false
					
					
						
				
			

 

Java Configuration

Step 4: Write Configuration file

Write a class that implement the WebApplicationInitializer interface from Spring Web and override the onStartup Method. Inside that you have to setup spring application context and map the servlet classes to url patterns:

package demo.spring.cfg;

import javax.servlet.ServletContext;
import javax.servlet.ServletException;
import javax.servlet.ServletRegistration;

import org.apache.cxf.transport.servlet.CXFServlet;
import org.springframework.web.WebApplicationInitializer;
import org.springframework.web.context.ContextLoaderListener;
import org.springframework.web.context.support.XmlWebApplicationContext;

public class WebAppInitializer implements WebApplicationInitializer {

	@Override
	public void onStartup(ServletContext container) throws ServletException {
		XmlWebApplicationContext rootContext = new XmlWebApplicationContext();
		rootContext.setConfigLocations(new String[] { "classpath*:applicationContext.xml" });
		container.addListener(new ContextLoaderListener(rootContext));

		ServletRegistration.Dynamic dispatcher = container.addServlet("CXFServlet", CXFServlet.class);
		dispatcher.addMapping("/*");
	}
}

Step 5: Delete web.xml

Delete inside WEB-INF folder the web.xml file and use the mvn clean command to remove it from target folder.

Step 6: Testing

run mvn package to build the war file in your project. Download and extract tomcat 7.x for testing your war file. Put the war file in the webapps folder and run your server with the startup.bat in the bin folder of tomcat.

Xlib: extension “RANDR” missing on display…

i got in the log of my selenium based integration test against firefox inside a Xvfb server. My first thought was of a missing library but the problem is caused by selenium:

“org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:”

was the line above and i had recently updated firefox on my ubuntu 12.01 server to version 15 by normal updates. The maven dependent selenium plugin version was still the same when i got the error. After updating to the latest version everything works as before:

 

		
			org.seleniumhq.selenium
			selenium-java
			2.24.1
			test
		

Common pitfalls installing Oracle XE 11 under Oracle Linux 6

First thing to check is the hostname inside /etc/hosts to ensure the install process can find the net listener. During the installation of Oracle Linux you set the hostname for example to oracle-vm.localdomain. Open with nano /etc/hosts and keep sure you have a line like this.

127.0.0.1 oracle-vm.localdomain localhost localhost.localdomain localhost4 localhost4.localdomain4

Next thing is to keep sure you have shared memory configured. Enter df -k and see if a line starts with shmfs. If not then enter

mount -t tmpfs shmfs -o size=2048m /dev/shm

and change your fstab to have shared memory available after next boot:

shmfs /dev/shm tmpfs size=2048m 0 0

Oracle XE for Linux is only available for X64 systems. Enter uname -a and see if it ends with

x86_64 x86_64 x86_64 GNU/Linux

If not reinstall your system with the x64 version of your linux distribution.

Keep sure your system is up to date with yum update.

After Download of Oracle Database Express Edition 11g unzip the File to a folder. Install the rpm with

rpm -ivH Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm

and enter

/etc/init.d/oracle-xe configure

Check if the install process outputs ends with

Starting Oracle Net Listener…Done
Configuring database…Done
Starting Oracle Database 11g Express Edition instance…Done
Installation completed successfully.

or the log files under

/u01/app/oracle/product/11.2.0/xe/config/log.

On success you have a icon called “Get started with Oracle… .desktop”. Click on the icon and mark the link as trust worthy. It will open a browser

http://localhost:9000/apex/f?p=4950

If asked for a user name and password use SYSTEM and your chosen password as credentials.

Running headless webdriver based selenium junit tests inside jenkins under ubuntu linux

My test setup is for a little wicket 1.5.x based web application with ui tests running during integration-test phase of maven 3.x. The problem here is that my ubuntu 12.01 server has only a console and no gnome or kde running like a desktop linux. Inside eclipse is a test, which starts a browser like firefox to run automated clicks, no problem. I decided to use the webdriver based selenium tests which can use several driver for the different browser. Each driver supports a different browser like chrome, internet explorer or firefox. My tests starts a firefox with explicit locale setting. The wicket application is i18n localized for english an german speaking customer. On the server is a jenkins ci installed with subversion polling to run automated tests during maven build. You can run the scenario with no problems as well under hudson as ci. The solution use xvfb a virtual xwindow system for firefox as desktop. It will automatically started and stopped by jenkins during a job build.

Software Installation on the server

Installation of xvfb

apt-get install xvfb gtk2-engines-pixbuf

Installation of fonts

apt-get install xfonts-base xfonts-75dpi xfonts-100dpi
apt-get install xfonts-scalable xfonts-cyrillic

Installation of tools for testing xvfb

apt-get install x11-apps imagemagick

Testing server installation

1. Console run server

Xvfb -ac :99 -screen 0 1280x1024x16

2. console start firefox

export DISPLAY=:99
firefox http://ralf.schaeftlein.com

3. console make a screenshot

xwd -root -display :99 | convert xwd:- capture.png

And see a result like this when you retrieve the file capture.png via ssh

 

 

 

 

 
 

Jenkins Configuration

Init.d Script for xvfb

Save content as file under /etc/inid.d/xvfb

XVFB=/usr/bin/Xvfb
XVFBARGS="$DISPLAY -ac -screen 0 1280x1024x16"
PIDFILE=/var/hudson/xvfb_${DISPLAY:1}.pid
case "$1" in
  start)
    echo -n "Starting virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS
    echo "."
    ;;
  stop)
    echo -n "Stopping virtual X frame buffer: Xvfb"
    /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE
    echo "."
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
  echo "Usage: /etc/init.d/xvfb {start|stop|restart}"
  exit 1
esac
exit 0

Set rights for jenkins running user on the script

chown jenkins:root /etc/init.d/xvfb
chmod ug+rwx /etc/init.d/xvfb

Add display environment variable to jenkins init.d script

export DISPLAY=:99

Create a new jenkins job

Create a new jenkins job for your web project stored in subversion

 Add to pre and post build step a shell script to start and stop xvfb

 

 

 

 

 

 

 

 

Configure maven pom of the web project

Define special tests in surefire to run as integration tests and jetty as integration application server

			
				org.apache.maven.plugins
				maven-surefire-plugin
				2.4.3
				
					true
				
				
					
						surefire-test
						test
						
							test
						
						
							false
							
								**/itest/**
							
						
					

					
						surefire-itest
						integration-test
						
							test
						
						
							false
							
								**/itest/**
							
						
					
				
			
			
				org.mortbay.jetty
				maven-jetty-plugin
				6.1.26
				
					10
					foo
					9998
					/${project.artifactId}
					true
					${basedir}/src/test/webapp/WEB-INF/web.xml
					
						
							9999
							60000
						
					
				
				
					
						start-jetty
						pre-integration-test
						
							run
						
						
							0
							true
						
					
					
						stop-jetty
						post-integration-test
						
							stop
						
					
				
			

Add dependency to selenium maven artifacts

		
			org.seleniumhq.selenium.client-drivers
			selenium-java-client-driver
			1.0.2
			test
		
		
			org.seleniumhq.selenium
			selenium-java
			2.21.0
			test
		

Record ui steps and write a selenium junit test

Install the selenium ide as firefox plugin

Start firefox and install the xpi file as new plugin. Restart firefox afterwards.

Record ui steps

  1. Open the menu, choose web developer and their selenium ide.
  2. Start inside eclipse the tomcat with your web application
  3. Open in firefox the web application
  4. Click  inside selenium ide on the red record button for start recording
  5. Click through your web application
  6. Click again on the red record button for stop recording
  7. Choose from menu “Export as testcase” and their “Junit4 (Webdriver backed)”
  8. Save file as test.java

Create an new selenium junit test file

  1. Copy the test.java into your eclipse project into the src/test/java folder
  2. Adopt the class definition and setup method like this
public class SeleniumTest {

	private Log log = LogFactory.getLog(getClass());

	protected Selenium selenium;

	@Before
	public void setUp() throws Exception {
		FirefoxProfile profile = new FirefoxProfile();
		// enable german language locale
		profile.setPreference("intl.accept_languages", "de-de,de");
		profile.setEnableNativeEvents(true);
		profile.layoutOnDisk();
		WebDriver driver = new FirefoxDriver(profile);
		String baseUrl = "http://localhost:9999/"; // port jetty surefire integration test
		selenium = new WebDriverBackedSelenium(driver, baseUrl);
	}

– insert java code –

  1. Check into subversion and control jenkins job console.