How does it work?
The Data Export API provides read-only access to all available analytics data. Any data that is displayed in the analytics web interface can be accessed through this API. Nice isn’t it? You can get all your analytics data, for all your monitored websites and use it as you please.

Getting started
To get started basically you need very little — a valid Google Analytics account and a client that would do all the low level work(sending HTTP requests with corresponding headers) or you could write your own small client that fits your needs. For example you could write shell scripts that use cURL under the hood. Anyway, it doesn’t matter, the important thing here is to find a good way to communicate with the Google Analytics servers.

At this point, Google offers 2 solutions: a Java and a JavaScript client. Both are easy to use, but may not fit your needs. That’s why there are also other similar clients in different languages like PHP, Perl, and Ruby.

The entire process of getting Google Analytics data and doing something useful with it can be split into 3 steps:

• Authorization
• Data feed
• Parse XML feed and perform your computations.

In Part 2 I’ll describe several ways of authorizations and when you would use each. But before that I’d like to mention some limitations. Basically Google has several quota policies, which currently are:

• 10,000 requests per 24 hours (this limit applies to a single web resource)
• 10 requests in any given 1-second period (this limit applies across all queries an application makes to the API)
• 4 pending requests at any given time. Be advised that this limit is new and is subject to change
• A query is also limited to pagination limits of 10,000 entries per feed, with a default response of 1,000 entries

Stay tuned!

1. Musings of a SpringOne 2009 Attendee Day 1
2. Musings of a SpringOne 2009 Attendee Day 2
3. Musings of a SpringOne 2009 Attendee Day 3

Read on for day 4.

OSGi and Groovy Jump Start

The presentation started with a 5 minute youtube video preview which I thought was pretty innovative.

The first half was around what is OSGi, some of which I’ve had experience with. I did pick up on a few key concepts that I didn’t know i wasn’t communicating earlier in my own presentations on OSGi.

We covered how a groovy POGO has groovy types in the public API. So if your groovy bundle has the groovylib in it’s classpath then it has to export the groovy package. Also when using Groovy in OSGi, if possible use POJOs for your applications type system since POGOs force you to expose couple groovy types in the bundle’s API.

The examples were done using gradle and this was my first live example of using gradle. This is a key benefit of conferences — being exposed to technologies you may otherwise not have the time to take a peek at.

I liked how the presenter took the path of not making things look complex but rather simplifying the concepts. He did this with concepts such as the OSGi bundle lifecycle.

Some of the tools covered in this session were:

• bnd to create a groovy manifest
• osgi bundle for gradle uses bnd
• junit4osgi

Expand your business with Groovy – Case Study

This session was a case study on how a monolithic application was extended to provide customizations using groovy. A major customer had approached HypericQ wanting a custom dashboard.

HypericQ developed a architecture around attach point’s which are places within the UI where plugins can connect to.

Custom UI was provided by gsp templates for customer specific dashboards.

Groovy templates was used as the templating framework that allowed clients to build their own using GSP pages.

A Web Services API was provided as part of the core platform that enabled several benefits:
- hqapi command line interaction
- allows integration testing of the application
- embedding a groovy console inside the application
- console gives you a HQL way to query the domain and not worry about the end customers database environment

The talk was an interesting exercise in how a dynamic plugin architecture could be implemented. Groovy categories was heavily used to enable this.

Re-factoring a Spring Application for SOA using Spring technologies in 40 min

The current focus of SOA is really software architecture. SOA is neither static nor predictable. The architectural pattern of any application must be reflective of the business. SOA is preparing the stuff that is critical to your business to stand the test of time.

Architecture involves the courage to be wrong while minimizing the impact of being wrong. It is about making educated decisions between speed and perfection. The longer it takes to deliver the software the higher the client’s expectations of that software is.

The foundation of SOA is highly cohesive, loosely coupled modules. One must practice

1. strategy first programming
2. process life-cycle dynamics
3. process collaboration
4. loose coupling promotes a network based architecture over a hierarchical one – composition over inheritance

SOA and OSGi enable embracing change by making versioning a runtime concept.

There is an argument to be made for deploying a process as a whole in BPEL. But what about flexible orchestration via simple messaging. We were given the American football analogy. Each play is scripted, but between plays we need to adjust strategy to meet the overall goal.

Modern event driven architecture was mentioned as a composite process linked together in a linear fashion through the use of rule based pathing.

The second half of this talk was the Spring booking demo from Spring Web Flow project. There are 3 OSGi bundles:

1. war bundle
2. service bundle
3. datasource bundle

The demo was using STS and it’s DM Server tooling. The entire development workflow was very seamless with SpringSource having done an awesome job of the tooling.

We saw an example of a transparent failover. So there is a need to take down the primary database for maintenance work without taking the application down. A backup datasource bundle is started and the primary datasource bundle is stopped. The datasource bundle is a simple datasource definition and an osgi service with ranking. Because of a higher rank the backup datasource bundle takes precendence and the primary datasource bundle can be safely stopped. Bundles can also take advantage of filtering besides ranking.

Spring Integration builds on pipes and filters. You can expose OSGi services as pipes. It provides the ability to change the behavior of a modularized application by changing how the modules are hooked up in the XML as opposed to modifying source code or in cases where the source code is not available.

Conference Wrap Up

Here are my key take aways:

• It is time for Grails
• Need to investigate Spring Faces
• Webgoat
• Spring Web Flow
• Spring Integration
• Spring DM Server

Overall it was a very productive conference and I would recommend it for anyone looking to attend it next year.

The Problem

For instance, say you have two pages – page1.xhtml and page2.xhtml. In your faces-config.xml, you will have 2 entries.

<navigation-case>
	<from-outcome>page1</from-outcome>
	<to-view-id>/pages/page1.xhtml</to-view-id>
	<redirect/>
</navigation-case>
 
<navigation-case>
	<from-outcome>page2</from-outcome>
	<to-view-id>/pages/page2.xhtml</to-view-id>
	<redirect/>
</navigation-case>

Let’s say page1Bean is called in page1 and you want it to display messages in page2. So your bean would have something like this:

public class page1Bean {
	...
	public String page1Action{
		...
		/** Add message */
		FacesMessage facesMessage = new FacesMessage();
		...
		FacesContext.getCurrentInstance().addMessage(null, facesMessage);
		...
		return "page2";
	}
	...
}

Your page2.xhtml has like the following somewhere in it.

<t:messages globalOnly="true">

One would expect that this would display the message on the next page, however these messages are request scoped and so are not available in this second request and you do not see your message unless you remove the <redirect/> on page2.

Fortunately, there is a nifty and quick solution. Basically, it is a phase listener that saves the messages from the previous request and then restores them just before the RENDER_RESPONSE phase of the second request.

The Solution

To solve the problem, save the following class into some package where your faces-config will be able to access it

package com.myproject.web.jsf.phaselistener;
 
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
 
import javax.faces.application.FacesMessage;
import javax.faces.context.FacesContext;
import javax.faces.event.PhaseEvent;
import javax.faces.event.PhaseId;
import javax.faces.event.PhaseListener;
 
/**
 * Enables messages to be rendered on different pages from which they were set.
 * To produce this behaviour, this class acts as a <code>PhaseListener</code>.
 *
 * This is performed by moving the FacesMessage objects:
 *
	<li>After each phase where messages may be added, this moves the messages from
 * the page-scoped FacesContext to the session-scoped session map.
 *</li>
	<li>Before messages are rendered, this moves the messages from the session-scoped
 * session map back to the page-scoped FacesContext.
 *</li>
 * Only messages that are not associated with a particular component are ever
 * moved. These are the only messages that can be rendered on a page that is different
 * from where they originated.
 *
 * To enable this behaviour, add a <code>lifecycle</code> block to your
 * faces-config.xml file. That block should contain a single <code>phase-listener</code>
 * block containing the fully-qualified classname of this file.
 *
 * EDIT: This code was minimally modified by Max Kuipers to address some of the Java 1.6
 * compiler warnings.  All code was originally written by Jesse Wilson.
 *
 * @author <a href="mailto:jesse@odel.on.ca">Jesse Wilson</a>
 * @author <a href="mailto:mkuipers@sourceallies.com">Max Kuipers</a>
 */
 
public class MessageHandler implements PhaseListener {
	private static final long serialVersionUID = 1L;
 
	/**
	 * a name to save messages in the session under
	 */
	private static final String sessionToken = "MULTI_PAGE_MESSAGES_SUPPORT";
 
	/**
	 * Return the identifier of the request processing phase during which this
	 * listener is interested in processing PhaseEvent events.
	 */
	public PhaseId getPhaseId() {
		return PhaseId.ANY_PHASE;
	}
 
	/**
	 * Handle a notification that the processing for a particular phase of the
	 * request processing lifecycle is about to begin.
	 */
	public void beforePhase(PhaseEvent event) {
 
		if(event.getPhaseId() == PhaseId.RENDER_RESPONSE) {
			FacesContext facesContext = event.getFacesContext();
			restoreMessages(facesContext);
		}
	}
 
	/**
	 * Handle a notification that the processing for a particular phase has just
	 * been completed.
	 */
	public void afterPhase(PhaseEvent event) {
 
		if(event.getPhaseId() == PhaseId.APPLY_REQUEST_VALUES ||
				event.getPhaseId() == PhaseId.PROCESS_VALIDATIONS ||
				event.getPhaseId() == PhaseId.INVOKE_APPLICATION) {
 
			FacesContext facesContext = event.getFacesContext();
			saveMessages(facesContext);
		}
 
	}
 
	/**
	 * Remove the messages that are not associated with any particular component
	 * from the faces context and store them to the user's session.
	 *
	 * @return the number of removed messages.
	 */
	private int saveMessages(FacesContext facesContext) {
		// remove messages from the context
		List messages = new ArrayList();
		for(Iterator i = facesContext.getMessages(null); i.hasNext(); ) {
			messages.add(i.next());
			i.remove();
		}
		// store them in the session
		if(messages.size() == 0) {
			return 0;
		}
		Map sessionMap = facesContext.getExternalContext().getSessionMap();
		// if there already are messages
		@SuppressWarnings("unchecked")
		List existingMessages = (List) sessionMap.get(sessionToken);
		if(existingMessages != null) {
			existingMessages.addAll(messages);
		}
		else {
			sessionMap.put(sessionToken, messages); // if these are the first messages
		}
 
		return messages.size();
	}
 
	/**
	 * Remove the messages that are not associated with any particular component
	 * from the user's session and add them to the faces context.
	 *
	 * @return the number of removed messages.
	 */
	private int restoreMessages(FacesContext facesContext) {
		// remove messages from the session
		Map sessionMap = facesContext.getExternalContext().getSessionMap();
		@SuppressWarnings("unchecked")
		List messages = (List)sessionMap.remove(sessionToken);
		// store them in the context
		if(messages == null) {
			return 0;
		}
		int restoredCount = messages.size();
		for(Iterator i = messages.iterator(); i.hasNext(); ) {
			facesContext.addMessage(null, i.next());
		}
 
		return restoredCount;
	}
}

And then add something like the following in your faces-config.xml. Be sure to replace the example package location with your own.

<lifecycle>
	<phase-listener>com.myproject.web.jsf.phaselistener.MessageHandler</phase-listener>
</lifecycle>

The forum post that documented this solution is here.

It was the end of the quarter and teachers were complaining that the test scoring application was returning incorrect results. The application had passed all development tests, passed through qa with flying colors, and has been deployed without any issue for a little while now. To make matters worse, all of the teachers were under lots of pressure to get all of their tests scored before the end of the quarter. When I heard that, a little light went off in my head. With a little digging, we were able to determine that the system had been serving more concurrent users than planned for.

The first thing I try to do when faced with a bug report is find a way to consistently reproduce it. This particular issue is a little harder since it only seems to be happening with multiple users. Luckily, Java makes writing multithreaded code very easy. Since most of the complaints seem to be related to test scoring, I start focusing my attention on that. The plan of attack is to:

1. Get a list of test ID’s.
2. Score each test and save it’s result.
3. Create a separate thread for each ID.
4. Each thread will score the test again, and compare it to the score from step 2. If it doesn’t match, increment a failure count.
5. Perform step 4 a couple of times.
6. Once all of the threads exit, print out the number of failures.

The plan of attack would then be to run this until either a failure occurs or until I feel fairly certain that there isn’t a problem with the scoring service.

Here is the the initial version of the test:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.List;
 
public class ScoringServiceTest {
 
  private int runCount = 0;
  private int scoreFailureCount = 0;
  private List skipFailureList = new ArrayList();
  private ScoringService scoringService = new ScoringService();
 
 
  private synchronized void incrementRunCount() {
    runCount++;
  }
  private synchronized void incrementScoreFailureCount() {
    scoreFailureCount++;
  }
 
  private int getRunCount() {
    return runCount;
  }
  private int getScoreFailureCount() {
    return scoreFailureCount;
  }
 
  private boolean compareScores(ScoringResult a, ScoringResult b) {
    return a.getNumerator().equals(b.getNumerator()) &&
    a.getDenominator().equals(b.getDenominator());
  }
 
 
  private void runTest(List testIdList,  int iterationCount) {
    long start = System.currentTimeMillis();
 
    List threadList = new ArrayList();
    String testId = "";
    for(int i = 0; i < testIdList.size(); i++) {
      try {
      testId = testIdList.get(i);
      ScoringResult score = scoringService.scoreTest(testId);
      Runnable testRunner = new TestRunner(iterationCount, testId, score);
      Thread thread = new Thread(testRunner, testId);
      threadList.add(thread);
 
      }
      catch(Exception exc) {
        System.out.println("Error with ID: " + testId);
        exc.printStackTrace();
      }
    }
 
    System.out.println("Starting threads");
    for(Thread t : threadList) {
      t.start();
    }
 
    System.out.println("Waiting for threads to finish");
    for(Thread t : threadList) {
      try {
        t.join();
      }
      catch(InterruptedException exc) {
        System.out.println("Interrupted while waiting for " + t.getName());
      }
    }
 
    System.out.println("Number of IDs: " + testIdList.size());
    System.out.println("Iterations: " + iterationCount);
    System.out.println("Total: " + getRunCount());
    System.out.println("Elapsed Time: " + (System.currentTimeMillis() - start));
    System.out.println("# Score Failures: " + getScoreFailureCount());
  }
 
  private class TestRunner implements Runnable {
    private String testId;
    private ScoringResult expectedScore;
    private int iterations;
 
    TestRunner(int iters, String tId, ScoringResult score) {
      testId = tId;
      expectedScore = score;
      iterations = iters;
    }
 
    @Override
    public void run() {
 
      for(int i = 0; i < iterations; i++) {
        ScoringServiceTest.this.incrementRunCount();
        ScoringResult scoringResult = ScoringServiceTest.this.scoringService.scoreTest(testId);
        if(!compareScores(scoringResult, expectedScore)) {
          ScoringServiceTest.this.incrementScoreFailureCount();
        }
      }
    } 
  }
 
  public static void main(String[] args) {
    ScoringServiceTest test = new ScoringServiceTest();
    List idList = new ArrayList();
 
    idList.add("Math1");
    idList.add("Science1");
    idList.add("Math2");
    idList.add("Science2");
    idList.add("English1");
    test.runTest(idList, 500);
  }
}

While this did the job at first, it’s usefulness was very short lived. Before long, I was wanting to change the test logic and didn’t want the threading logic getting in the way. The approach I took was to extract the threading code out into a test harness and then create an interface for the testing logic by using the Command pattern. This resulted in the following code, which I have been able to reuse across multiple test scenarios.

The test harness class:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
 
public class MultithreadedTest {
 
  private int runCount = 0;
  private int failureCount = 0;
 
 
  private synchronized void incrementRunCount() {
    runCount++;
  }
  private synchronized void incrementFailureCount() {
    failureCount++;
  }
  public int getRunCount() {
    return runCount;
  }
  public  int getFailureCount() {
    return failureCount;
  }
 
 
  public void runTest(Collection commands,  int iterationCount) {
    long start = System.currentTimeMillis();
 
    List threadList = new ArrayList();
    System.out.println("Loading expected results");
 
    for(TestCommand command : commands) {
      Object result = command.doTest();
      TestRunner testRunner = new TestRunner(iterationCount, command, result);
      Thread thread = new Thread(testRunner);
      threadList.add(thread);
    }
 
    System.out.println("Starting threads");
    for(Thread t : threadList) {
      t.start();
    }
 
    System.out.println("Waiting for threads to finish");
    for(Thread t : threadList) {
      try {
        t.join();
      }
      catch(InterruptedException exc) {
        System.out.println("Interrupted while waiting for " + t.getName());
      }
    }
 
    System.out.println("Number of Tests: " + commands.size());
    System.out.println("Iterations: " + iterationCount);
    System.out.println("Total: " + getRunCount());
    System.out.println("Elapsed Time: " + (System.currentTimeMillis() - start));
    System.out.println("# Failures: " + getFailureCount());
  }
 
 
  private class TestRunner implements Runnable {
    private int iterations;
    private TestCommand command;
    private Object expectedResult;
 
    TestRunner(int iters, TestCommand command, Object expectedResult) {
      this.command = command;
      iterations = iters;
      this.expectedResult = expectedResult;
    }
 
    @Override
    public void run() {
 
      for(int i = 0; i < iterations; i++) {
        MultithreadedTest.this.incrementRunCount();
        try {
          Object result = command.doTest();
          if(!command.compareResults(result, expectedResult)) {
            MultithreadedTest.this.incrementFailureCount();
          }
        }
        catch(Exception exc) {
          exc.printStackTrace();
          MultithreadedTest.this.incrementFailureCount();
        }
      }
    } 
  }
 
 
}

The test command:

package com.sourceallies.multithread;
 
public interface TestCommand {
  public Object doTest();
  public boolean compareResults(Object a, Object b);
}

and finally, the test command for my scenario:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.List;
 
public class ScoringServiceTestCommand implements TestCommand {
  private String testId;
  private ScoringService scoringService;
 
  public ScoringServiceTestCommand(ScoringService service, String id) {
    scoringService = service;
    testId = id;
  }
 
  public Object doTest() {
    return scoringService.scoreTest(testId);
  }
 
 
  public boolean compareResults(Object expectedResult, Object result) {
    ScoringResult sr1 = (ScoringResult)expectedResult;
    ScoringResult sr2 = (ScoringResult)result;
 
    boolean match = sr1.getNumerator().equals(sr2.getNumerator()) &&
        sr1.getDenominator().equals(sr2.getDenominator());
 
    return match;
  }
 
  public static void main(String[] args) {
    List idList = new ArrayList();
    idList.add("Math1");
    idList.add("Science1");
    idList.add("Math2");
    idList.add("Science2");
    idList.add("English1");
 
    ScoringService scoringService = new ScoringService();
 
    List commandList = new ArrayList();
    for(int i = 0; i < idList.size(); i++) {
      commandList.add(new ScoringServiceTestCommand(scoringService, idList.get(i)));
    }
    MultithreadedTest mtest = new MultithreadedTest();
    mtest.runTest(commandList, 500);
  }
}

Overview

The discussion below assumes that you have a Subversion repository that uses the standard trunk/branches/tags structure.  In other words, a repository at http://site.com/project with trunk, branches, and tags subdirectories.

I’ll also assume that you have already checked out a local working copy of trunk, and that you’ve  already created an Eclipse project using this working copy.

When you want to work on a new branch you’ll follow these general steps using Subclipse, which are described in more detail below:

• Create a new branch and switch to it
• Develop in your branch, periodically merging the latest changes from trunk into your branch
• When your development is complete, merge changes from your branch back into trunk
• Delete your branch

Switch

Your local working copy reflects some directory in the remote Subversion repository.  If you initially checkout trunk, then your working copy reflects trunk.  If you initially checkout a branch, it reflects that branch.  Using the svn switch command you can have Subversion modify your working copy to reflect another directory in the repository.  Switching may not sound that exciting but it’s key to branching and merging, as you’ll soon see.

First off, ensure that your working copy has no local changes.  Either commit your changes or revert them.

Next, right-click your project in Eclipse’s Project Explorer, select the Team submenu, and click the Switch to another Branch/Tag/Revision… item.  All of Subclipse’s actions are in this Team menu, so from now on I’ll refer to specific actions like Team > Switch to another Branch/Tag/Revision.

In the To URL field, you specify the URL of the repository directory you want to switch your local working copy to.  If your working copy initially reflects http://site.com/project/trunk, you could specify a branch as http://site.com/project/branches/somebranch.  Generally you’ll want to leave all of the other options in this dialog box as they are.

Create a new Branch

Creating a new branch is almost trivial.  First, switch your local copy to trunk (if it’s not already trunk).  And again, make sure you have no local changes.

Next select Team > Branch/Tag… and specify http://site.com/project/branches/yourbranch in the Copy to URL field.  This will create a branch named yourbranch.

Click Next and leave HEAD revision selected.  Click Next again, enter a commit message like “Created the yourbranch branch”, check the “Switch working copy to new branch/tag” checkbox, and click Finish.

That’s it!  You’ve created your new branch in the repository and switched your working copy to it.  Now you can implement your major new features in your branch and commit them to subversion without affecting trunk.

Merge Trunk into a Branch

While you’re working on your branch, you should periodically merge the latest version of trunk into your branch to make sure you’re not straying too far away.  Eventually, you’ll also need to do this as the last step before merging your branch back into trunk, to ensure you won’t cause any conflicts with trunk.

First you’ll need to switch your working copy to your branch, if necessary.  Subversion will merge the changes from trunk into your working copy, and then you’ll commit your working copy to your branch to fully integrate the changes from trunk.  So it’s vital that you start with your working copy as a mirror of your branch.  And again, ensure you have no local changes.

Click Team > Merge… and select the “Merge a range of revisions” option.  Make sure “Perform pre-merge best practices” is checked and click Next.  On the next page, if it asks you to perform any actions, like updating your working copy, perform them and click Next.

On the next page, make sure you’re merging from trunk and getting all eligible revisions, and click Next.  Leave all of the defaults for conflict handling and click Finish.

After the merge, the results screen will show you what happened.  If you got any conflicts just resolve them as usual.  Then commit any changes to your branch, and you’re done!

Merge a Branch into Trunk

When you’ve finished development in your branch, you need to merge your branch changes back into trunk, so everyone else can share in the glory of your awesome new feature.  This is usually where developers turn pale and run away, but fear not; Subclipse will guide you through.

Make sure you only perform this merge when you are completely finished with your branch!  After merging your branch into trunk, you will no longer be able to use your branch.  In fact, you should delete your branch after merging it into trunk (see next section).

First, merge trunk into your branch and commit any changes to your branch, as described in the previous section.  We need to make sure your branch is consistent with trunk before merging it back.

Next, switch your working copy to trunk.  Remember that Subversion merges changes into your working copy first, and then you commit them back to the repository.  So Subversion will apply your branch changes to your working copy of trunk, and then you commit those changes to trunk.

Click Team > Merge… and select the “Reintegrate a branch” option.  Make sure “Perform pre-merge best practices” is checked and click Next.  On the next page, if it asks you to perform any actions, like updating your working copy, perform them and click Next.

On the next page, make sure you’re merging from your branch and getting all eligible revisions, and click Next.  Leave all of the defaults for conflict handling and click Finish.

After the merge, the results screen will show you what happened.  If you got any conflicts just resolve them as usual.  Then commit any changes to trunk, and you’re done!

Delete a Branch

As previously mentioned, after merging a branch into trunk, the branch can no longer be used.  Subversion keeps track of merges using the svn:mergeinfo property.  Because of the way it uses this property, once a branch is merged into trunk it can no longer be involved in merges properly.  For the full story, please consult the last few paragraphs of this section on branching and merging.

Subclipse does not provide a way to delete an entire branch, so you’ll need to just run this command manually:

svn delete http://site.com/project/branches/yourbranch -m "Removing completed branch"

Background

The specific app I created was an adaptation of the Books example from Chapter 10 of Beginning Java™ EE 6 Platform with GlassFish™ 3. It’s a simple web app that displays a list of books in a database and lets you add new books. Although it’s a pretty trivial app, it does touch on several important Java EE 6 technologies: JPA 2.0, EJB 3.1 and JSF 2.0.

Results

As a baseline, I first created the example from the Book using Java (and put it on github). The three Java classes are mostly identical to the book: Book.java, BookEjb.java and BookController.java. Aside from those, I refactored common elements of the two JSF views into a template, included the Blueprint CSS framework and created a better Maven pom.xml.

Next I rewrote the Java parts in Scala (and put it on github). Initially I did a one-to-one mapping of the Java classes to their Scala counterparts. Once that was complete I did a bit of refactoring to Book.scala and BookEjb.scala, so the versions you’ll see on github now look a bit different.  During this process I did use some hints from a few other related blog posts.

JPA

First up, here’s Book.java:

package com.sourceallies.model;
 
import javax.persistence.*;
 
@Entity
@NamedQuery(name = Book.FindAllBooks, query = "SELECT b FROM Book b")
public class Book {
	public static final String FindAllBooks = "findAllBooks";
 
	@Id
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	private Long id;
 
	@Column(nullable = false)
	private String title;
 
	private Float price;
 
	@Column(length = 2000)
	private String description;
 
	private String isbn;
	private Integer nbOfPage;
	private Boolean illustrations;
 
	public Long getId() {
		return id;
	}
 
	public void setId(Long id) {
		this.id = id;
	}
 
	public String getTitle() {
		return title;
	}
 
	public void setTitle(String title) {
		this.title = title;
	}
 
	public Float getPrice() {
		return price;
	}
 
	public void setPrice(Float price) {
		this.price = price;
	}
 
	public String getDescription() {
		return description;
	}
 
	public void setDescription(String description) {
		this.description = description;
	}
 
	public String getIsbn() {
		return isbn;
	}
 
	public void setIsbn(String isbn) {
		this.isbn = isbn;
	}
 
	public Integer getNbOfPage() {
		return nbOfPage;
	}
 
	public void setNbOfPage(Integer nbOfPage) {
		this.nbOfPage = nbOfPage;
	}
 
	public Boolean getIllustrations() {
		return illustrations;
	}
 
	public void setIllustrations(Boolean illustrations) {
		this.illustrations = illustrations;
	}
 
	public static String getFindallbooks() {
		return FindAllBooks;
	}
}

The Java version has 58 lines of completely useless getters & setters.  You need at least 9 lines per field and more for additional annotations.  IDEs like Eclipse can generate these getters/setters for you, but they still take up space and must be maintained over time.

Now let’s look at the Book entity in Scala:

package com.sourceallies.model
 
import javax.persistence._
import scala.reflect._
 
@Entity
@NamedQuery{val name = "findAllBook", val query = "SELECT b FROM Book b"}
class Book extends Id with Description {
  @Column{val nullable = false}
  @BeanProperty
  var title: String = _
 
  @BeanProperty
  var price: Float = _
 
  @BeanProperty
  var isbn: String = _
 
  @BeanProperty
  var nbOfPage: Int = _
 
  @BeanProperty
  var illustrations: Boolean = _
}
 
/** Defines an id field for entities to use as a primary key. */
trait Id {
  @javax.persistence.Id
  @GeneratedValue{val strategy = GenerationType.IDENTITY}
  @BeanProperty
  var id: Long = _
}
 
/** Defines a description field for entities. */
trait Description {
  @Column{val length = 2000}
  @BeanProperty
  var description: String = _
}

In the code base these are actually three separate files: Book.scala, Id.scala and Description.scala, I just included them all in one place here for convenience.

The Scala version needs only two lines per field: one for the @BeanProperty annotation to generate getters/setters in the compiled class and one to declare the field itself.  This is seven lines less than the Java version per field.

Fields are declared as vars so they are public by default, mutable and every var that is a non-private member of some object implicitly defines a getter and a setter method with it.  So “var x” defines a getter just named “x” and a setter named “x_=”.  This lets you get the field using object.x and set it using object.x = y.

If JPA supported Scala-style accessors & mutators, we’d be done, but it requires the use of get/set methods.  This is why we use @BeanProperty on each field.  This instructs the Scala compiler to generate Java-style get/set methods in the compiled class file so it looks just like a Java entity to JPA.

You may also be wondering why each field is assigned a value of _ in its declaration.  This is just the default value in Scala, just like omitting an initializer in Java: 0 for numeric types, false for booleans, and null for reference types.

The annotations also look a bit different; in Java they’re @Column(nullable = false) but in Scala are @Column{val nullable = false}.  This is just how you do it in Scala.

I also pulled the id and description fields out into their own traits. This makes the entity class shorter and makes common fields reusable and more consistent across entities.  I could have pulled all fields out into their own traits, but in practice there’s probably a limit to how much you’d want to do this, some fields will be specific to a single entity class and not reusable.

EJB

Now let’s look at BookEjb.java:

package com.sourceallies.ejb;
 
import java.util.*;
 
import javax.ejb.*;
import javax.persistence.*;
 
import com.sourceallies.model.*;
 
@Stateless
public class BookEjb {
	@PersistenceContext(unitName = "book")
	private EntityManager manager;
 
	@SuppressWarnings("unchecked")
	public List<Book> findBooks() {
		return manager.createNamedQuery(Book.FindAllBooks).getResultList();
	}
 
	public Book createBook(Book book) {
		manager.persist(book);
		return book;
	}
}

The Java version is very simple: it basically just provides two Book-specific CRUD methods using a JPA EntityManager.

And here is the Scala version:

package com.sourceallies.ejb
 
import javax.ejb._
import javax.persistence._
import com.sourceallies.model._
 
@Stateless
@LocalBean
class BookEjb extends CrudEjb[Book]
 
/** Provides basic CRUD support using an injected JPA entity manager. */
trait CrudEjb[E] {
  @PersistenceContext
  protected var manager: EntityManager = _
 
  def create(entity: E): E = {
    manager persist entity
    entity
  }
 
  def readAll()(implicit m: Manifest[E]) = manager createNamedQuery ("findAll" + m.erasure.getSimpleName) getResultList
 
  def read(id: Long)(implicit manifest: Manifest[E]): E = manager.find(manifest.erasure, id).asInstanceOf[E]
 
  def update(entity: E): E = manager merge entity
 
  def delete(entity: E) { manager remove entity }
}

Again, these are two separate files in the actual code base: BookEjb.scala and CrudEjb.scala.

In the Scala version I created the CrudEjb trait that provides full CRUD support for any JPA entity, so the concrete EJB can be defined simply as BookEjb extends CrudEjb[Book].  In several places we need the Class object of the entity, which is difficult to obtain due to Java’s erasure of generic types at runtime.  Scala’s Manifests are a great solution to this problem.

The Scala version requires the @LocalBean annotation while the Java version does not (not sure why).  Also the EntityManager is defined as a var without the Java-style getters and setters.  The container is able to inject the EntityManager using the Scala-style mutator (bean.manager = x) but it does log an error about it not having a set method.

JSF

And now the BookController.java:

package com.sourceallies.controller;
 
import java.util.*;
 
import javax.ejb.*;
import javax.faces.bean.*;
 
import com.sourceallies.ejb.*;
import com.sourceallies.model.*;
 
@ManagedBean
@RequestScoped
public class BookController {
	@EJB
	private BookEjb ejb;
	private Book book;
	private List<Book> bookList;
 
	public Book getBook() {
		if (book == null)
			book = new Book();
		return book;
	}
 
	public void setBook(Book book) {
		this.book = book;
	}
 
	public List<Book> getBookList() {
		if (bookList == null)
			bookList = ejb.findBooks();
		return bookList;
	}
 
	public void setBookList(List<Book> bookList) {
		this.bookList = bookList;
	}
 
	public String doNew() {
		return "newBook.xhtml";
	}
 
	public String doCreateBook() {
		book = ejb.createBook(book);
		bookList = ejb.findBooks();
		return "listBooks.xhtml";
	}
}

This Java version is simply a Book-specific CRUD wrapper around BookEjb, and provides Book and List<Book> objects to the JSF views.

And finally, BookController.scala:

package com.sourceallies.controller
 
import javax.ejb._
import javax.faces.bean._
 
import com.sourceallies.model._
import com.sourceallies.ejb._
 
@ManagedBean
@RequestScoped
class BookController {
  @EJB
  private var ejb: BookEjb = _
 
  private val book = new Book
 
  def getBook = book
 
  def getBookList = ejb.readAll
 
  def doNew() = "newBook.xhtml"
 
  def doCreateBook() = {
    ejb create book
    "listBooks.xhtml"
  }
}

The Scala version is basically just the Java version converted to Scala, no major changes, just looks a bit cleaner due to Scala’s concise syntax.  As with the EJB, we get an error logged about not having a set method for injecting BookEjb, but the container does actually inject an instance properly.

Conclusion

Even though this was a simple example, I think it shows a few advantages of using Scala instead of Java in Java EE apps.  You can forget about useless getters/setters in JPA entity classes, use traits to reuse common JPA entity field definitions and mixin common behavior in EJBs (example: BookEjb could also mixin Logging trait and other traits).  And advanced Scala features like Manifests help you write better code faster.

This example only scratched the surface of what Scala can do, and didn’t even use things like closures, case classes, pattern matching or Scala’s functional programming capabilities (using Scala collections of JPA entities would help a lot).

The fact that we even started a book club was a big positive because it is one great way to geek out with very smart people. You get the opportunity to voice your opinion on a certain topic and hear counter-points or similar views that expand your own perspective.

We went through 2 books in the span of about 6 months. The first book was Test Driven Development: By Example by Kent Beck. It’s a very good book by itself. But we found it didn’t make for very good book club discussion. The material didn’t really provide much fodder for critical thinking and exploration in my opinion. But it was a start.

We then picked up Effective Java by Joshua Bloch as our second book. Java is currently my bread and butter and this book was very appropriate with it’s tip based approach. The material was a little dense in some areas, but overall an extremely good book for a book club discussion because it provided enough room for critical thinking and discussion. Unfortunately though well intentioned I thought we fell off the wagon due to a few different reasons:

• We met in a cafeteria location which was noisy and had it’s own share of distractions making it difficult to have any sustained discussion
• Not a whole lot of discussion around the topic because each of us read a different chapter in the book and we didn’t really have common ground
• We had participants in the discussion who hadn’t read the book and would understandably drive the conversations off topic
• We didn’t have a standing meeting time which we were committed to. Sometimes I felt we were scrambling just to get a common meeting time organized. We did get better about this towards the later half of last year.

Going forward into 2010
Learning from last year, we are making some changes to derive more out of the book club this year.

• Meet in a quiet location – This year we will be meeting in a conference room with little distractions
• A once a week standing meeting – This way everyone in the group is aware of the weekly commitment and can be prepared
• Be more pro-active about our book selection – Maintain a list of potentially interesting books and select books at the regular meeting
• More structured discussion format – We all have views and opinions on what we’ve read. So all of us must have the opportunity to voice our thoughts
• Book review – This year we are going to attempt to publish a blog post reviewing each book that we discussed at the book club
• Limit the book club to a maximum of 6 members – any more and the 1-hour will be insufficient for everyone to participate fully

We’ve already had two meetings so far this year with the above format and I thought it went very well. And am looking forward to a more rewarding book club this year.

Say you get a jar file. After using the jar for a while you realise that there is a bug in a class in the jar file. Unfortunately you also find out that the jar is no longer supported and there is no way you will get a fix from the author (who is long gone fishing).

In order to solve this issue, you first need to get the source of the class. If you are lucky enough and the author did not obfuscate the class file you can decompile it with a decompiler (my favourite one is JD-GUI).

After you get the source, you modify it, recompile and tell the java interpreter to load your new class first before the old one. Easy right?

Let me give you an example

Say we have this “outdated” Spider class

alex@tractorash:~/work/java$ pwd
/home/alex/work/java
alex@tractorash:~/work/java$ cat sai/Spider.java

package sai;
 
class Spider{
 
    private static final String URL = "http://www.google.com/search?q=best+software+2009";
 
    public String getURL(){
        return URL;
    }
 
}

Now, we have the Main class:

package sai;
 
class Main{
 
    public static void main(String[] args){
        Spider spider = new Spider();
        System.out.println(spider.getURL());
    }
 
}

Now, if we compile and run the Main class we will get:

alex@tractorash:~/work/java$ javac sai/Main.java
alex@tractorash:~/work/java$ java sai.Main
http://www.google.com/search?q=best+software+2009

As we expected. Suppose we want to search for http://www.google.com/search?q=best+software+2010. The only way to do that is create another sai.Spider class and return the needed string.

So, here we go

alex@tractorash:~/h$ pwd
/home/alex/h
alex@tractorash:~/h$ cat sai/Spider.java

package sai;
 
class Spider{
 
    private static final String URL = "http://www.google.com/search?q=best+software+2010";
 
    public String getURL(){
        return URL;
    }
}

We compile our new class

alex@tractorash:~/h$ javac sai/Spider.java

Then, switch to previous path:

alex@tractorash:~/h$ cd ~/work/java/
alex@tractorash:~/work/java$ java -cp /home/alex/h:/home/alex/work/java sai.Main
http://www.google.com/search?q=best+software+2010

So what did we do? We modified the classpath (http://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/classpath.html) and told the JVM to load our new class first, since it’s first in our list of directories we specified as classpath.

This way we “tricked” the JVM in loading our new class first so we could get the desired output.

What we’ve done so far is called replacing and patching application classes.

You will not be able to modify/replace system classes like java.lang.Integer using this method.

You are probably wondering why would you ever want to replace java.lang.Integer. Let me give you an example. As you might know the Integer class is immutable, in other words once the object is initialised you cannot modify it’s value. So, let’s add a new method called setValue(int newVal) which will modify the current value of the Integer object.

As you can imagine, applying the method used in the Spider class example will not work. The reason is: to load system classes and libraries, the JVM does not use the CLASSPATH variable, but the BOOTCLASSPATH. The boot strap class loader uses the boot class path to load the main/system classes.

What’s interesting is that you can specify/modify the BOOTCLASSPATH as well. In order to to that we have to use the -X argument for the java launcher:

alex@tractorash:~$ java -X
-Xbootclasspath:<directories and zip/jar files separated by :>
set search path for bootstrap classes and resources
-Xbootclasspath/a:<directories and zip/jar files separated by :>
append to end of bootstrap class path
-Xbootclasspath/p:<directories and zip/jar files separated by :>
prepend in front of bootstrap class path

We are interested in the -Xbootclasspath/p: option since we want to prepend the directory that contains the patched Integer class.

First, let’s add that new method. Get the source of java.lang.Integer. You can get it either from the src.zip file that comes with the JDK, or download it from online.

Switch to our working directory:

alex@tractorash:/$ cd /home/alex/work/java/

Add the method:

public void setValue(int value){
    this.value = value;
}

If you are modifying a Java 6 version of Integer class you have to remove the final keyword from the value variable.

Was this:

private final int value;

Has to be this:

private int value;

Now we need a test class where we will test the new functionality. Let’s create the IntegerTest class:

public class IntegerTest{
    public static void main(String[] args){
        Integer tInt = new Integer(1);
        System.out.println("Initial value: " + tInt);
        tInt.setValue(25);
        System.out.println("New value: " + tInt);
    }
}

Compiling patched versions of system classes can be a little tricky.

First let’s compile our Integer class:

alex@tractorash:~/work/java$ javac java/lang/Integer.java
java/lang/Integer.java:571: warning: sun.misc.VM is Sun proprietary API and may be removed in a future release
if (!sun.misc.VM.isBooted()) {
^
Note: java/lang/Integer.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning

We get the warning because we use a sun.* class in our new class. Since sun.* packages and classes are proprietary to Sun, it is strongly advised not to use them, but in our case we just reuse them (so we can ignore the warning).

Now, we have our Integer.class file. We have to use it when we execute our test class.

Now compile IntegerTest.java:

alex@tractorash:~/work/java$ javac -Xbootclasspath/p:/home/alex/work/java/lang/:/usr/lib/jvm/java-6-sun-1.6.0.15/jre/lib/rt.jar IntegerTest.java
./java/lang/Integer.java:571: warning: sun.misc.VM is Sun proprietary API and may be removed in a future release
if (!sun.misc.VM.isBooted()) {
^
Note: ./java/lang/Integer.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning

Copy Integer.class to the directory where your IntegerTest class is:

alex@tractorash:~/work/java$ cp ./java/lang/Integer.class .

So IntegerTest.class and Integer.class are in the same directory.

Finally run IntegerTest:

alex@tractorash:~/work/java$ java -Xbootclasspath/p:/home/alex/work/java:/usr/lib/jvm/java-6-sun-1.6.0.15/jre/lib/rt.jar IntegerTest
Initial value: 1
New value: 25

Success! Now you have a mutable Integer class.

I recommend you read the Java Covert book.

However, I have to warn you that this post is for academic purposes only. Patching/removing/replacing Java core classes can be against the Java license agreement. Please consult the license before going down this path.

mysql> DESC ipaddresses;
+----------+---------------------+------+-----+---------+----------------+
| FIELD    | Type                | NULL | KEY | DEFAULT | Extra          |
+----------+---------------------+------+-----+---------+----------------+
| id       | int(10) UNSIGNED    | NO   | PRI | NULL    | AUTO_INCREMENT | 
| A        | tinyint(3) UNSIGNED | NO   |     | NULL    |                | 
| B        | tinyint(3) UNSIGNED | NO   |     | NULL    |                | 
| C        | tinyint(3) UNSIGNED | NO   |     | NULL    |                | 
| D        | tinyint(3) UNSIGNED | NO   |     | NULL    |                | 
+----------+---------------------+------+-----+---------+----------------+
5 rows IN SET (0.00 sec)
 
mysql> SELECT * FROM ipaddresses WHERE id=1
+----+----+----+----+-----+
| id | A  | B  | C  | D   |
+----+----+----+----+-----+
|  1 | 10 | 20 | 30 | 131 |
+----+----+----+----+-----+
1 row IN SET (0.02 sec)

As I started manipulating these addresses I found it awkward to do common binary math (like bitwise ANDs). I decided instead to store these 32-bit values as unsigned integers (of length 32). To make my life easier yet, MySQL and PHP both have native functions to convert IP addresses between my old and new formats to make this migration extremely easy.

In MySQL:

mysql> SELECT INET_ATON('10.20.30.131');
+---------------------------+
| INET_ATON('10.20.30.131') |
+---------------------------+
|                 169090691 | 
+---------------------------+
1 row IN SET (0.00 sec)
 
mysql> SELECT INET_NTOA(169090691);
+----------------------+
| INET_NTOA(169090691) |
+----------------------+
| 10.20.30.131         | 
+----------------------+
1 row IN SET (0.00 sec)

In PHP:

$ip_string="10.20.30.131";
$ip_integer = ip2long($ipstring);
echo $ip_integer; ## 169090691
echo long2ip($ip_integer); ## 10.20.30.131

In another example of inefficiency, I used to gather all sorts of info from the user and use PHP to make multiple SQL queries before modifying my data. I’ve since learned about stored procedures – the database does all the work and I only need to make one SQL call from PHP. This presented me with an unexpected problem, however:

mysql> CALL getAddress();
+----+----+----+----+----+
| id | A  | B  | C  | D  |
+----+----+----+----+----+
|  1 | 10 | 20 | 30 | 131|
+----+----+----+----+----+
1 row IN SET (0.00 sec)
 
Query OK, 0 rows affected (0.00 sec)

The string “Query OK, 0 rows affected (0.00 sec)” is a second result set, which the PHP mysql library can’t handle. The fix here is to use the mysqli library instead and access only the result sets you need. In my case I’m only concerned with the first result set so I added a bunch of wasteful while loops after each of my CALL queries.

while (mysqli_next_result()){}

Lastly, PHP’s ip2long function treats 32-bit numbers as signed ints, so I had to specifically treat them as unsigned ints:

echo $ip_integer; ## BAD
echo sprintf(%u, $ip_integer); ## GOOD

Sonar is a web based code quality analysis tool for Maven based Java projects. It covers a wide area of code quality check points which include: Architecture & Design, Complexity, Duplications, Coding Rules, Potential Bugs, Unit Test etc. Sonar has a rich set of features like what you would get with different tools such as Covertura, PMD, FindBugs, Check Styles combined.

http://sonar.codehaus.org/

Setting up Sonar

• Download the latest release of Sonar from http://sonar.codehaus.org/downloads/.
• Unzip it to a folder, e.g. C:\Sonar
• Run C:\Sonar\bin\windows-x86-32\StartSonar.bat for Windows. There are several options available for other operationg system such as Linux, Solaris.
• Browse to http://localhost:9000 . The browser will display a “No projects have been analysed” message. This is a sanity check to see if Sonar is running properly.
• From within your project, run the following maven command: mvn clean install sonar:sonar
• Refesh the browser. The code quality related reports such as Complexity, Duplication, Voilation, Rules of Compliance etc will be displayed.
• To customize the features using the admin console, login/password is admin/admin.