Source Allies Blog » Design Patterns

Developing a multithreaded test harness

Tim Bierbaum — Fri, 05 Mar 2010 17:19:19 +0000

You can’t ignore the fact that web servers are multithreaded. We can hide as much as we want, but sooner or later you’ll find yourself in the situation where your application works fine during development and testing; but once it hits production you start hearing about “funny” things happening. While there are plenty of tools that can be used to simulate multiple users, they aren’t always the easiest to run locally and they seem to take to much time to modify while hot on the trail of a multithreading bug. Here I’ll discuss the approach I took when I was recently faced with this situation.

It was the end of the quarter and teachers were complaining that the test scoring application was returning incorrect results. The application had passed all development tests, passed through qa with flying colors, and has been deployed without any issue for a little while now. To make matters worse, all of the teachers were under lots of pressure to get all of their tests scored before the end of the quarter. When I heard that, a little light went off in my head. With a little digging, we were able to determine that the system had been serving more concurrent users than planned for.

The first thing I try to do when faced with a bug report is find a way to consistently reproduce it. This particular issue is a little harder since it only seems to be happening with multiple users. Luckily, Java makes writing multithreaded code very easy. Since most of the complaints seem to be related to test scoring, I start focusing my attention on that. The plan of attack is to:

Get a list of test ID’s.
Score each test and save it’s result.
Create a separate thread for each ID.
Each thread will score the test again, and compare it to the score from step 2. If it doesn’t match, increment a failure count.
Perform step 4 a couple of times.
Once all of the threads exit, print out the number of failures.

The plan of attack would then be to run this until either a failure occurs or until I feel fairly certain that there isn’t a problem with the scoring service.

Here is the the initial version of the test:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.List;
 
public class ScoringServiceTest {
 
  private int runCount = 0;
  private int scoreFailureCount = 0;
  private List skipFailureList = new ArrayList();
  private ScoringService scoringService = new ScoringService();
 
 
  private synchronized void incrementRunCount() {
    runCount++;
  }
  private synchronized void incrementScoreFailureCount() {
    scoreFailureCount++;
  }
 
  private int getRunCount() {
    return runCount;
  }
  private int getScoreFailureCount() {
    return scoreFailureCount;
  }
 
  private boolean compareScores(ScoringResult a, ScoringResult b) {
    return a.getNumerator().equals(b.getNumerator()) &&
    a.getDenominator().equals(b.getDenominator());
  }
 
 
  private void runTest(List testIdList,  int iterationCount) {
    long start = System.currentTimeMillis();
 
    List threadList = new ArrayList();
    String testId = "";
    for(int i = 0; i < testIdList.size(); i++) {
      try {
      testId = testIdList.get(i);
      ScoringResult score = scoringService.scoreTest(testId);
      Runnable testRunner = new TestRunner(iterationCount, testId, score);
      Thread thread = new Thread(testRunner, testId);
      threadList.add(thread);
 
      }
      catch(Exception exc) {
        System.out.println("Error with ID: " + testId);
        exc.printStackTrace();
      }
    }
 
    System.out.println("Starting threads");
    for(Thread t : threadList) {
      t.start();
    }
 
    System.out.println("Waiting for threads to finish");
    for(Thread t : threadList) {
      try {
        t.join();
      }
      catch(InterruptedException exc) {
        System.out.println("Interrupted while waiting for " + t.getName());
      }
    }
 
    System.out.println("Number of IDs: " + testIdList.size());
    System.out.println("Iterations: " + iterationCount);
    System.out.println("Total: " + getRunCount());
    System.out.println("Elapsed Time: " + (System.currentTimeMillis() - start));
    System.out.println("# Score Failures: " + getScoreFailureCount());
  }
 
  private class TestRunner implements Runnable {
    private String testId;
    private ScoringResult expectedScore;
    private int iterations;
 
    TestRunner(int iters, String tId, ScoringResult score) {
      testId = tId;
      expectedScore = score;
      iterations = iters;
    }
 
    @Override
    public void run() {
 
      for(int i = 0; i < iterations; i++) {
        ScoringServiceTest.this.incrementRunCount();
        ScoringResult scoringResult = ScoringServiceTest.this.scoringService.scoreTest(testId);
        if(!compareScores(scoringResult, expectedScore)) {
          ScoringServiceTest.this.incrementScoreFailureCount();
        }
      }
    } 
  }
 
  public static void main(String[] args) {
    ScoringServiceTest test = new ScoringServiceTest();
    List idList = new ArrayList();
 
    idList.add("Math1");
    idList.add("Science1");
    idList.add("Math2");
    idList.add("Science2");
    idList.add("English1");
    test.runTest(idList, 500);
  }
}

While this did the job at first, it’s usefulness was very short lived. Before long, I was wanting to change the test logic and didn’t want the threading logic getting in the way. The approach I took was to extract the threading code out into a test harness and then create an interface for the testing logic by using the Command pattern. This resulted in the following code, which I have been able to reuse across multiple test scenarios.

The test harness class:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
 
public class MultithreadedTest {
 
  private int runCount = 0;
  private int failureCount = 0;
 
 
  private synchronized void incrementRunCount() {
    runCount++;
  }
  private synchronized void incrementFailureCount() {
    failureCount++;
  }
  public int getRunCount() {
    return runCount;
  }
  public  int getFailureCount() {
    return failureCount;
  }
 
 
  public void runTest(Collection commands,  int iterationCount) {
    long start = System.currentTimeMillis();
 
    List threadList = new ArrayList();
    System.out.println("Loading expected results");
 
    for(TestCommand command : commands) {
      Object result = command.doTest();
      TestRunner testRunner = new TestRunner(iterationCount, command, result);
      Thread thread = new Thread(testRunner);
      threadList.add(thread);
    }
 
    System.out.println("Starting threads");
    for(Thread t : threadList) {
      t.start();
    }
 
    System.out.println("Waiting for threads to finish");
    for(Thread t : threadList) {
      try {
        t.join();
      }
      catch(InterruptedException exc) {
        System.out.println("Interrupted while waiting for " + t.getName());
      }
    }
 
    System.out.println("Number of Tests: " + commands.size());
    System.out.println("Iterations: " + iterationCount);
    System.out.println("Total: " + getRunCount());
    System.out.println("Elapsed Time: " + (System.currentTimeMillis() - start));
    System.out.println("# Failures: " + getFailureCount());
  }
 
 
  private class TestRunner implements Runnable {
    private int iterations;
    private TestCommand command;
    private Object expectedResult;
 
    TestRunner(int iters, TestCommand command, Object expectedResult) {
      this.command = command;
      iterations = iters;
      this.expectedResult = expectedResult;
    }
 
    @Override
    public void run() {
 
      for(int i = 0; i < iterations; i++) {
        MultithreadedTest.this.incrementRunCount();
        try {
          Object result = command.doTest();
          if(!command.compareResults(result, expectedResult)) {
            MultithreadedTest.this.incrementFailureCount();
          }
        }
        catch(Exception exc) {
          exc.printStackTrace();
          MultithreadedTest.this.incrementFailureCount();
        }
      }
    } 
  }
 
 
}

The test command:

package com.sourceallies.multithread;
 
public interface TestCommand {
  public Object doTest();
  public boolean compareResults(Object a, Object b);
}

and finally, the test command for my scenario:

package com.sourceallies.multithread;
 
import java.util.ArrayList;
import java.util.List;
 
public class ScoringServiceTestCommand implements TestCommand {
  private String testId;
  private ScoringService scoringService;
 
  public ScoringServiceTestCommand(ScoringService service, String id) {
    scoringService = service;
    testId = id;
  }
 
  public Object doTest() {
    return scoringService.scoreTest(testId);
  }
 
 
  public boolean compareResults(Object expectedResult, Object result) {
    ScoringResult sr1 = (ScoringResult)expectedResult;
    ScoringResult sr2 = (ScoringResult)result;
 
    boolean match = sr1.getNumerator().equals(sr2.getNumerator()) &&
        sr1.getDenominator().equals(sr2.getDenominator());
 
    return match;
  }
 
  public static void main(String[] args) {
    List idList = new ArrayList();
    idList.add("Math1");
    idList.add("Science1");
    idList.add("Math2");
    idList.add("Science2");
    idList.add("English1");
 
    ScoringService scoringService = new ScoringService();
 
    List commandList = new ArrayList();
    for(int i = 0; i < idList.size(); i++) {
      commandList.add(new ScoringServiceTestCommand(scoringService, idList.get(i)));
    }
    MultithreadedTest mtest = new MultithreadedTest();
    mtest.runTest(commandList, 500);
  }
}

Taking Advantage of Spring MVC’s Default Behavior

Travis Klotz — Thu, 11 Feb 2010 16:00:55 +0000

Over the last several months I have worked on several content heavy websites for one of our clients. When I say “content heavy”, I mean that 80%-90% of the pages in the application are static, or at least mostly static, a customer name, membership number, etc may need to be floated in, but not big data tables with dynamic data being pulled from the database. The marketing department manages this content with their content management system and publish fully formed HTML pages (layout, look and feel, etc is controlled in the CMS) which are then pulled into our /WEB-INF/jsp/content directory by our build process.

Our applications treat these HTML pages as JSPs (simple rename in the build script). This lets the marketing team work from a cheat sheet of EL expressions such as ${customerName} and keeps the IT department out of the day to day content management work. One of our goals with these systems was to easily and seamlessly deal with both static pages and very dynamic pages requiring custom controllers to be built. With just a little bit of work Spring MVC makes it easy to provide this functionality and also provides a sane set of defaults for building out these web sites one page at a time.

Under the hood, Spring MVC can be very complicated with its Handlers, Handler Mappings, Controllers, Views, ViewResolvers, etc. Thanks to the Annotation based version of Spring MVC introduced in Spring 2.5, today we are really only going to need to worry about Controllers and ViewResolvers. The only way to truly make handling both static pages and controller based pages seamless is to route all page requests through the Spring MVC servlet. At the same time, we don’t want to make changes to the application configuration every time the marketing department creates a new page or even a new folder full of pages. This requires we configure Spring MVC to have a very specific default behavior.

View Resolver

First up is the view resolver. We can’t specify all the views ahead of time and the only way the marketing department communicates new pages to us is by publishing them into our JSP directory, Springs InternalResourceViewResolver is the perfect fit.

 class="org.springframework.web.servlet.view.InternalResourceViewResolver">
     name="prefix" value="/WEB-INF/jsp/content/" />
     name="suffix" value=".jsp" />
>

Assuming the build script pulls the published HTML files into the projects /WEB-INF/jsp/content directory and renames them with a “.jsp” extension, then all the pages should be available with a view that matches the path to the page, minus the ViewResolver’s specified prefix and suffix. For example, if we have a page stored at /WEB-INF/jsp/content/members/profile.jsp, A controller can access that page by returning a view name of members/profile.

The Default Controller

Now that there is a view resolver capable of seeing all the pages produced by the marketing department, a controller must be created and mapped so that it will produce the correct view name.

package com.sourceallies.base;
 
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
 
@Controller
public class DefaultController {
 
    @RequestMapping("/**/*")
    public void defaultRequest(){}
}

That was easy. It would be hard to come up with a simpler controller, in fact it doesn’t look like it does anything at all! In reality it is taking advantage of Spring MVC’s default behavior for just about everything. With annotation based controllers, request mapped methods have a variety of possible return types. Strings are interpreted as view names, and ModelView objects are used like they were in previous versions of Spring MVC. But if the method has no return type then we get some interesting behavior. On void methods (or String methods that return null) Spring MVC looks at the request URI, strips off the app context and any file extensions, and uses whatever is left over as the view name. For example, if my application is mapped to /corporate/ and my request URL is /corporate/members/profile.html then Spring MVC will generate a view name of “members/profile”. The other thing to note is the RequestMapping, /**/* is the most generic mapping there is. It will match any request that comes through the Spring MVC servlet. This is how to avoid having to add new mappings for every new page. This is exactly what we want to use when working with the ViewResolver we defined earlier. All of the pages from the CMS live relative to each other, and the URLs requested from the client should match the file structure from the CMS.

So at this point I now have a more complicated version of the normal servlet request process. Its usefulness becomes apparent when marketing needs a new piece of data on a specific page and I need to create a controller.

@Controller
public class ProfileController {
 
    @RequestMapping("/members/profile.html")
    public void testRequest(Model model) {
        Profile profile = ...LookupProfile
 
        model.addAttribute("profile", profile);
    }
}

All I have to do is create a new controller and map it to the page name from the marketing department. And by having the requst method be void, I once again don’t have to worry about view names. Because the request mapping is more specific than the one on the DefaultController, Spring MVC chooses this controller when the profile page is requested. This setup allows the marketing department to very quickly prototype entire sites while at the same time not having to worry about IDEs, app servers, etc. They get to stay inside the CMS and the IT department only has to write custom code and configuration for the pages that need it.

Building Multi-Criteria Search Queries in Hibernate

Vaithi Subramanian — Wed, 09 Dec 2009 18:02:53 +0000

In this post I am going to show how to write queries multi-criteria search screens. There are two approaches for making this possible.

HQL for building the Query
Building Query using Criteria API

HQL for building the Query

Here I am going to show 2 approaches to building the HQL and try to point out the better approach.

Approach I:String concatenation
This approach uses String concatenation and setting up the values directly in the query.

if (startDate != null) {
   if (firstClause) {
      query = query + " where ";
   }
   else {
      query = query + " and ";
   }
 
   query += " s.date >= '" + startDate + "'";
}

Using the above approach there might be a chance of SQL Injection attack and using string concatenation is inherently error-prone.

Approach 2: Criteria as Named Parameters

In this one we create two map to hold parameter name and value which could be binded to the HQL during the execution.

Here is brief example of the approach.

public List search() {
   StringBuilder aQuery = 
      new StringBuilder(" from document p where p.id is not null ");
   HashMap parameterMap = new HashMap();
   HashMap parameterListMap =  new HashMap();
 
   //Collection Criteria
   if (countyList != null) {
      buildCollectionCriterion(aQuery, parameterListMap, "listID", countyList);
   }
 
   //Date Criteria
   if (startDate!= null || endDate != null) {
      buildDateCriterion(aQuery, parameterMap, "dateField",startDate, endDate);
   }
 
   Query query = hibSession.createQuery(aQuery.toString());
 
   for (String key : parameterMap.keySet()) {
      query.setParameter(key, parameterMap.get(key));
   }
 
   for (String key : parameterListMap.keySet()) {
      query.setParameterList(key, parameterListMap.get(key));
   }
 
   List results = query.list();
}
 
//Helper Methods for different type of criteria
 
protected void buildCollectionCriterion(StringBuilder aQuery, 
                                        Map parameterListMap, 
                                        String aFieldName, 
                                        Collection aList) {
   if (aList != null && !aList.isEmpty()) {
      aQuery
         .append(" and p.")
         .append(aFieldName)
         .append(" in (:")
         .append(aFieldName)
         .append(")");
      parameterListMap.put(aFieldName, aList);
   }
}
 
public void buildDateCriterion(StringBuilder aQuery, 
                               Map parameterMap, 
                               String aFieldName, 
                               Date aStartDate, 
                               Date anEndDate) {
   if (aStartDate != null && anEndDate != null) {
      aQuery
         .append(" and ( p."
         .append(aFieldName)
         .append(" between :aStartDate and :anEndDate)");
      parameterMap.put("aStartDate", aStartDate);
      parameterMap.put("anEndDate", anEndDate);
   }
   else if (aStartDate != null) {
      aQuery
         .append(" and  (p.")
         .append(aFieldName)
         .append(" >= :aStartDate)");
      parameterMap.put("aStartDate", aStartDate);
   } 
   else if (anEndDate != null) {
      aQuery
         .append(" and (p.")
         .append(aFieldName)
         .append(" <=:anEndDate");
      parameterMap.put("anEndDate", anEndDate);
   }
}

Building Query using Criteria API

public List search() {
        Criteria c = hibSession.createCriteria(Document.class);
        c.add(Restrictions.notNull("id"));
 
        if(countryList != null) {
                c.add(Restrictions.in("listId", countryList));
        }
 
        if(startDate != null) {
                c.add(Restrictions.ge("dateField", startDate);
        }
 
        if(endDate != null) {
                c.add(Restrictions.le("dateField", endDate);
        }
 
        return c.list();
}

Conclusion

The Hibernate Criteria API is a powerful and elegant library which is well adapted for implementing multi-criteria search functionality and also HQL queries must be built ‘on-the-fly’. Using it in appropriate circumstances will result in cleaner, clearer, more reliable and more maintainable code.