Joe Developer
ikvm and freemarker in C#
I've been doing a fair bit of .NET work lately, mostly C#, and I've a need for a templating engine for dynamic text generation. Coming from the Java realm my first thought was of Freemarker. I've used FM a decent amount, am comfortable with it, and want the same fucntionality in a C# app.
Perhaps there's something out there, I've searched around a bit, but couldn't find just the thing to get the job done.
Enter ikvm. ikvm is an interesting project in that it aims to do .NET<->Java interop in a native way, offering a lot of options. One of the options is the simplest - use your Java libraries directly in your .NET code by translating the java bytecode into msil (Microsoft Intermediate Language - the .NET bytecode equiv).
Turns out this works exactly as advertised, and within an hour or so I was using .NET pojos (poNos?) in a little console app with a little FM template hard coded.
There are a few problems though, the largest of which is the difference between the javabeans naming convention for properties and the .NET convention.
For example, if I have a Java class Person with a property name, I'd have:
public String getName(){...}
public void setName(String value){...}
in C#, I'd have:
public string Name
{
get
{
return this.name;
}
set
{
this.name = value;
}
}
note that "value" is an implicitly typed object of the type that the property is... in this case a string. It's a different syntax to get used to, but you can do everything you'd do in a getter and setter in what is an arguably more readable syntax.
So I use the ikvm tool ikvmc to translate my bytecode (had to fiddle a bit to also translate all the runtime dependencies) into freemarker-2.3.8wdeps.dll (like a jar file but I "compiled" all the dependencies into one library - bad for complex dependencies but easier for a quick test).
Simple, examples go off without a hitch. (using primatives in the FM context), but when I put an instance of Person in, things don't go so well. I try to use ${person.Name} in my FM template and it blows up exactly the way you'd think it would - FM attempts to find a method called "getName" on the Person class, it's not there on the .NET class, so things don't work. Obviously. It occurs to me that the .NET way with properties is much easier to think about what with the not having to create getName from name to call the right method, but that's the way things are. (see the getter/setter versus Property bit above)
Quick solution - create a little decorator that extends HashMap, overrides get(Object key), use the .NET reflection system there and return the right value. By the way, I extended HashMap in .NET via the wonder that is the IKVM.GNU.Classpath library that ships with ikvm. Thanks to the folks who make this little bit of magic possible.
This creates a number of issues as far as I'm concerned, starting with how absolutely kludgy it is. In my template rather than ${person.Name} I have to use ${person['Name']}. This creates all sorts of other issues, as I'd have to do far more messy magic to get this to work for complex object graphs (say, the Address property of a Person expression like ${person.Address.ZipCode}).
The "right" answer is, I think, to create a different object wrapper implementation for FM and use that in the FM configuration, one that can do all the nice Method/Property caching for me(as the FM implementation does for performance), resolve indexed properties, and the like. I'll be working on that in my copious free time as I love the power that FM gives, but I'm living (not unhappily) in a .NET world quite a bit these days.
The short story is - ikvm and the GNU classpath project rock, as I was able to start to get the job done using a great Java library from within a .NET application in a matter of a couple of hours. Amazing. I love code re-use, and since there are so many great OSS Java libs out there, I'm sure now that some of them will be available to me in .NET - a boon to productivity and surely a more less clunky interop than web services, a .NET embedded JVM (though ikvm does that too as a fully embeddable .NET JVM implementation), or Java->JNI->COM->.NET.
Note too - this says nothing of the performance of the generated msil... right now it's about POC and functionality, not billion transactions per second on a 486.
For the curious, below is the command line I used to get my version of FM and dependencies compiled to a dll. There are a TON of warnings with this configuration, but hey, I just DL'ed ikvm a few hours ago, it's going to take me a minute to get the kinks worked out with the tool.
set PATH=c:\java\lib\ikvm.32\ikvm-0.32.0.0\bin;%PATH% ikvmc -out:freemarker-2.3.8wdeps.dll -target:dll ant.jar dom4j-1.6.jar javax.servlet.jar javax.servlet.jsp.jar jaxen-core.jar jaxen-jdom.jar jdom.jar log4j-1.2.11.jar xalan-2.6.0.jar xerces-2.4.0.jar xerces.jar xercesImpl.jar xml-apis.jar freemarker.jarAnd here's my shady little test code in C#
using System;
using System.Collections.Generic;
using System.Text;
using freemarker;
using freemarker.template;
using java.io;
namespace freemarker_test
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting...");
Configuration config = new Configuration();
Person p = new Person();
p.Name = "Brian";
string templateText = "number: ${testNumber}, name: ${person['Name']}";
StringReader reader = new StringReader(templateText);
Template t = new Template("fooTemplate", reader, config);
java.util.Map map = new java.util.HashMap();
map.put("testNumber", 1);
map.put("person", new DecoratingMap(p));
StringWriter writer = new StringWriter();
t.process(map, writer);
Console.WriteLine("Text: " + writer.toString());
Console.WriteLine("Press any key to exit...");
Console.ReadLine();
}
}
public class Person
{
private string name;
public string Name
{
get { return name; }
set { name = value; }
}
}
//a terrible little hack until a full on .NET object wrapper can be created for FM
public class DecoratingMap : java.util.HashMap
{
private object o;
public DecoratingMap(object subject)
{
this.o = subject;
}
public override object get(object key)
{
System.Reflection.PropertyInfo p = o.GetType().GetProperty((string)key);
object ret = p.GetValue(o, null);
return ret;
}
}
}
Posted at 10:49PM Dec 30, 2006 by saibrian in General | Comments[702]
jBPM upgrade - 3.0 to 3.1 a BIG step
I recently upgrade an app that relies heavily on jBPM from 3.0 to 3.1, and it is a big step (3.1.2 specifically). I'm compiling a list of things I liked, things I didn't and things I learned.
Improvements in 3.1
- TaskInstance is easier to extend in a cleaner fashion In 3.0 you could extend TaskInstance (something you almost have to do to support a number of use cases) via a config change in the jbpm.properties file. As far as we could tell though, the only way to finish the job was with a table-per-subclass mapping. In 3.1 TaskInstance is mapped with a discriminator column such that you can easily extend it with table per hierarchy without hacking the distributed mapping file.
- Transparent persistence of "hibernatable" domain objects as process instance variables
This is just plain nice, and keeps me from a)writing a bunch of coverters or b)using all sorts of work arounds. In a nutshell, if your class is persistable by hibernate AND your app and jBPM are sharing a hibernate configuration, you can save your domain objects as process variables transparently with no extra effort. This level of integration is very nice.
There is a problem with it in the 3.1.2 release (I posted on this the other day), but Tom Baeyens says it is fixed in CVS so it will work as expected in the future without modification. - They refactored JbpmConfiguration and the whole configuration section The previous JbpmConfiguration class was a pain to work with... lots of static references to it, hard to plug into. The deprication of that class and the config system that replaced it is easier to use and plug into.
- Cleaner integration with Spring and a single hibernate session factory via the latest SpringModules release (0.5) See the previous point. The 3.1 integration between Spring and jBPM is much easier to achieve. Made seamless with the use of the SpringModules project. There isn't much documentation on getting this done (I'm going to post an example with more info here in a bit), but it is possible to integrate the 2 and have a nice single TX manager and hibernate config that is all configured the way you're used to seeing it done. The down side is that you have to use callbacks with a template... long story short, I don't particularly care for that, but at least now there's a clean way to get the job done. Great work from the SpringModules gents.
- The consistent use of the jbpmContext object to get jBPM related tasks done In 3.1 you'll most frequently be using the jbpmContext to get the most common things done related to tasks, processes, process definitions, etc. It is still delegating to the other modules in the system, but having those delegations on a single class makes it easier for devs new to the product to get things done. You can still get at the modules (TaskMgt etc) to do more advanced work, but the convenience on jbpmContext is nice.
- Automatically using the latest version of a subprocess rather than the version that was active at deployment time
Finally. Another hack I can stop with... in 3.0 when your process instantiated a child process it was the version of the child process that was deployed when you deployed the parent. Read that again. This doesn't seem too bad, actually makes sense right? Doesn't make sense for one huge reason - it keeps large, long running processes from being flexible.
I don't know about you, but I know a few business users that have told me that business processes change. Now for a variety of reasons you really need a process instance to execute based on the same process definition for the life of its execution. Trust them - this is the right thing. With that constraint though, you can have issues if your process is long on the calendar as changing a subprocess would have no effect on long running processes that called it. At that point it only served as a way to make a long process definition shorter. The new implementation supports the old way through the use of an added attribute when you call a subprocess, but by default (if I'm reading the docs right) will use the most recent version of the subprocess.
This is a powerful feature. Given the constraint that you can't change the definition of a process for a particular instance, you can instead compose multiple processes into one larger one, buying yourself the flexibility to change a portion of the larger, long-running process by deploying a new portion of a sub process. Now you don't get any flexibility out of this if you don't design your process carefully, but with the capability at least supported you can truly have the flexibility required in business to with your solid workflow engine :) Thanks jBPM team.
Points of frustration
- I didn't like the limitations of the old BeanShell scripts in decisions - the way the scripting context was initialized and run was a bit limiting and forced some work arounds I didn't like. Fortunately or unfortunately (depending on how many processed you had written), the decision condition style has changed.
The new decision expression language is very much like jsf (to appease the Seam users that are using JPDL to define page flows). This is nice and clean but still limiting in power.
By limiring I mean you really get one expression - in the old style and the new. Sure you can do a lot in one expression, but no complex scripting was supported. The transparent domain object access makes this less of a limiting factor overall, but there are times when I'd like to be able to execute just a wee bit more logic than a single expression gives me...
There are answers to this for sure... The prettiest answer is to use Drools (now of JBoss)... Really though, I'd like to just be able to execute a little bit of code in a script (with a context I can control/manipulate prior to execution) rather than learn yet another framework. Drools is great, and is certainly the most robust answer, but seems like overkill when 3 statements and a return value would suffice in a lot of cases. - The business calendar is STILL in a properties file. The business calendar and the supporting expressions based on timers are great, and overall it is a powerful feature in a workflow engine. If you're developing an app for in-house use you MAY be able to get away with the business calendar being defined in a properties file, but when you're designing an app to release, and run-time configurability is a must, having the business calendar in a properties file is rather limiting. I plan to hack this at some point to make it available in the DB as an option - it's the only way to go.
- The actual XML that is the process definition still isn't stored in the DB that I've seen... One of the first things we did in our app was create a domain object with a big ole clob that would be used to store the process definitions. Now I'm all for pretty pictures in the GPD (and I'll get to that), but processes change - it would be nice to have them persisted by default rather than having to build that capability every time it's needed.
- jbpmContext likes to commit transactions and close the hibernate session. Grrr. We had a bit of work to do to set up SpringModules to keep this from happening, as it wreaks havoc if you're using the OpenSessionInView pattern/Spring filter to manage sessions. (Yes, I know about the hibernate 3.1 getCurrentSession() technique. Not using it in this app.) I WANT the jbpm activities in the same session, with the same tx manager... It makes the most sense if you're going to embed jBPM, don't want to get involved with it's hibernate session etc, want simple process management or really just want an easy to configure state machine, but if you're doing a full integration it is a bit of a pain until you get things working.
Once it's working though... works like a champ. Exactly like you'd want it to. - Swimlanes still only fire once per process instance. This is a big flexiblity loss as far as I'm concerned. I've submitted a Jira issue, and will have a patch available for this shortly. It seems that the preferred method is to use the identity module with some custom implementation and assignment expressions to get the job done.
Custom swimlanes are much more flexible though, and allow a great deal of re-use if you code them carefully. We have only a handful of swimlane implementations that handle all of the use cases we're tackling right now, and I have only one more implementation to round out the set. If you haven't looked at creative uses for swimlanes, I urge you to. With non-singular execution they offer all the task assignment power you could want without the constraints of the identity module. The proposed swimlane extension gives lots of power when coupled with creative swimlane implementations. - The docs, and the javadoc, are a bit out of date and sparse. I can't complain too much though as the code itself is great documentation, is easy to follow, and generally tells me everything I want to know. They're kind enough to distribute a .classpath and .project file for us eclipse users, so integrating the source into an eclipse env for debugging is a 1 minute process.
And before you say it - yes, I'm looking at adding to the docs where I can and giving back - its the least I can do I suppose for such a wonderful tool.
For the dev new to jBPM though the lack of documentation and examples can be truly frustrating, particularly where the guide says "(document this)". Still better documented than most OSS projects though. - the GPD (Graphical Process Designer) is still a drag. Sorry guys. Quite frankly I don't use it, preferring like most devs I think to just write the xml... but it IS a great marketing tool. The PHBs of the world love a pretty picture...
That said, it isn't really usable for anything serious if the intention is to generate all but the most linear, simple process definition. If you think you'll DL some WYSIWYG tool and be cranking out that complex process, you're sadly mistaken. Constructs that are a requirement in a moderately complex process aren't covered in the tool, and it is clunky and non-intuitive.
I know, I know, its OSS - just fix it... If I relied on it for daily use I certainly would. Since I only use it to show the pretty picture of what I'm going to hand author anyway I'm not that concerned. I'm not bashing it - most workflow tools have some nice GUI so they (JBoss) sort of have to supply one - but at this point it's a checkbox to mark off on an RFP more than a fully functional tool.
That said, it's a lot better now than it was, and kudos to the gents writing it, but there's clearly more effort being placed in the engine (which is a great decision IMHO).
So use the tool to show a pretty picture for selling jBPM to your boss, but take the time to learn JPDL until the tool catches up.
Things I'm learning/looking at
There are a number of exciting new features in 3.1 and beyond that I'm looking forward to using... haven't used them yet, but I'll post when I do.- Async nodes - we've no use case for them right now, but terribly useful when you look at web service orchestration without the annoyance that is BPEL.
- Drools integration - though I'm not all for having all the devs know it, it is a powerful integration that needs to be considered
- The whole messaging sub-system... looks neat and I'm going to play with it a bit in the near future.
- There appears to a whole form generating thing coming in 3.2. I'm interested to see how this plays out since we've already implemented this internally. Without care and attention it's probably going to be too constraining for the really hard stuff, but we'll see - I've been surprised before.
however it shakes out, at least there will be more example code out there related to tieing a web UI to the core engine... we spent a fair bit of time figuring that out, and it's a REALLY common question on the forum - looking forward to it.
Random Thoughts
jBPM is still my favorite workflow engine, suited to a variety of tasks and solving a host of problems.Though we're not using it for page flow (still a great thought), we're using it for all sorts of other things, and with a few extensions it is hands down the most powerful and flexible process tool I've used.
Hats off to the jBPM team for continuing to bring a great tool to the community.
If you're looking for a powerful, easy to use, extendable, and embeddable workflow engine you can't do much better than jBPM. And no, I don't work for them - just love the product after working with others in the field and spending the last 18 months with jBPM.
Posted at 01:00PM Sep 09, 2006 by saibrian in jBPM |
jBPM 3.1 Domain Object auto persistence
I just went through an upgrade from jBPM 3.0 to 3.1.2. There are a number of changes, some large, some small. One of the welcome changes is more exposure of the underlying persistence mechanism - Hibernate.
jBPM allows you to store variables in a process instance - and if you write a converter to translate from a primative (or wrapper) to your type, you can easily persist user defined types in your process.
In an action handler it looks like:
executionContext.setVariable("foo", myInstanceOfFoo);
Where you've written and reistered a converter for whatever class myInstanceOfFoo is. The converter registration was SO 3.0, particularly if your type, we'll call it com.example.Foo is persisted by Hibernate. You ended writing boilerplate code.In 3.1.x the engine is smart enough to use the hibernate configuration that it's already using to see if your type is persistable by hibernate, and if so, will use its internal session factory to save and retrieve your variable when you reference it in the process.
That's neat. There's a problem though... what if your domain object implements Serializable? As it turns out, this will cause you all sorts of issues when you'd like transparent persistence if you're relying on the default configuration for variable handling.
This is due to the way jBPM looks for a way to handle persisting a variable.
jBPM first figures out how to save a variable by looking in the config file: org/jbpm/context/exe/jbpm.varmapping.xml.
As the engine is doing its magic, it is checking against each type of built in persistence strategy to figure out which one to use for your variable. When it finds one that matches, that's what it uses for saving and fetching your object. For fun, look in the DB and you'll notice there's a column in the variable_instance table that tells you what strategy it uses.
When your object is serializable, the engine by default will use the org.jbpm.context.exe.converter.SerializableToByteArrayConverter to persist it. That's not actually what I'm looking for, and I'm not sure when that WOULD be the desired behavior, but since the engine is looking at persistence strategies in order and that's found before the hibernate persisters, that's what's used...
The "desired behavior", at least in my case, was to have Hibernate persist things, and have jBPM just go to MY DB tables to rehydrate objects as needed, not pull them from some serialized blob in the DB that is frequently out of date with the domain. I want to be able to have the "variables" that jBPM is using be accurate reflections of the state of the data when the process access it.
The fix? Take the 3 hibernate based variable instance converters and move them to the top of the config file, rather than the bottom. (ok, so one of them is EJB3 based, but the same issue applies if you're in that space). jBPM will then attempt to use Hibernate persistence for your object BEFORE it attempts to serialize it.
Rebuild jBPM (or point to your modified var mapping file from a user supplied jbpm.config file), and off you go.
After that, I have to say that the variable handling is perfect - exactly what I was looking for. It is intuitive and allows a deep integration between the objects in the domain and the workflow engine.
Assuming your object is "hibernatable", say an instance of Foo, then in a handler you can "save" a foo in the process instance like so:
executionContext.setVariable("myFoo", instanceOfFoo);
Fetching the fully hydrated object that is fresh from your domain in a later node/script/handler is then as easy as:
Foo myFoo = (Foo)executionContext.getVariable("myFoo");
... some interesting stuff.
This makes it easy to use your persistent domain objects, from your tables, in a transparent manner within a process - a powerful feature.Just be warned - without a custom config OR a modification of their config, you'll have no end of wondering what's going on if you happen to have domain objects that are Serializable.
An additional note - see the first comment from Tom Baeyens (the creator of jBPM) - this issue is fixed in CVS - so if you're running head you'll not have this, and I look forward to un-doing my change upon the next release.
Posted at 11:34PM Sep 07, 2006 by saibrian in Gotcha | Comments[1]
XDoclet, Struts, and Spring integration code generation
Did I mention I'm not that fond of writing config files?
So I'm working on this project - standard Struts app on the front end, with a bunch of Spring magic behind it. I like Spring. Struts - well it's Struts... I do like the ability to integrate the 2 though (is there anything Spring doesn't integrate with?) using the DelegatingRequestProxy method.
This method goes like so - replace the classes in struts-config.xml with this DelegatingActionProxy. When Struts accesses one, the url is used as the lookup in Spring - and the class returned (your actual action) has been all Spring injected/AOPed etc...
This is great, save 2 things.
- struts-config.xml doesn't explicitly tell you what class a URL is mapping to - not good for a dev that's new to a codebase, particularly if there isn't a strong naming convention etc in place
- Its yet another Spring file to manage - and a rather boring one at that. I'm not wanting to inject lots of special things - just beans.
Here's the struts-spring-config.xml template:
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
"http://www.springframework.org/dtd/spring-beans.dtd">
<!--
| XDoclet generated spring-config file to configure the dependencies
| into delegated proxy actions. This is used with the modified
| struts-config template that will optionally write the delegation class
| rather than the actual class to the struts-config file.
+-->
<!--
| Author Brian Greene
| brian@sourceallies.com
+-->
<beans>
<!-- ========== Manual Action Configurations =================================== -->
<XDtMerge:merge file="struts-beans-config.xml">
<!--
Define your action bean dependencies in a file called struts-beans-config.xml and place
it in your merge directory.
-->
</XDtMerge:merge>
<!-- ========== Action Mapping Dependency Declarations =================================== -->
<XDtClass:forAllClasses type="org.apache.struts.action.Action">
<XDtClass:forAllClassTags tagName="struts:action">
<XDtClass:ifDoesntHaveClassTag tagName="struts:action" paramName="no-spring">
<bean name="<XDtClass:classTagValue tagName="struts:action" paramName="path"/>"
class="<XDtClass:fullClassName/>">
<XDtClass:forAllClassTags tagName="spring:depend">
<property name="<XDtClass:classTagValue tagName="spring:depend" paramName="name"/>">
<ref bean="<XDtClass:classTagValue tagName="spring:depend" paramName="bean"/>" />
</property>
</XDtClass:forAllClassTags>
</bean>
</XDtClass:ifDoesntHaveClassTag>
</XDtClass:forAllClassTags>
</XDtClass:forAllClasses>
</beans>
Simple little template. It generates something that looks like:
.
.
.
<bean name="/login" class="com.sourceallies.myproject.web.action.LoginAction">
<property name="userService">
<ref bean="userService"/>
</property>
</bean>
.
.
.
So included in my list of Spring config files, I include this generated file. I could have used auto wiring - but wanted to be more explicit at this stage. We're not done yet though - the struts config file is still going to reference LoginAction.
Since I'm generating struts-config.xml, I'm off to modify the template a bit to support this new feature. This is one of the things I like about XDoclet - for most simple extensions you don't really have to do any work - make up a tag and go.
Here's how my new made up tag (and attribute) looks on LoginAction:
/** * @author BGreene * @struts.action * path="/login" * useSpringProxy="true" * * @spring.depend * name="userService" * bean="userService"There are 2 new things here - the useSpringProxy="true" attribute of the @struts.action tag, and the @spring.depend tag.
useSpringProxy="true" is used to tell the new struts-config.xml template that I want to use the DelegatingActionProxy.
@spring.depend is what is used to generate the spring-struts-config.xml file above.
The generated struts-config file looks like (excerpt from <action-mappings>):
. . <!--using the spring proxy. Actual class: com.sourceallies.myproject.web.action.LoginAction --> <action path="/login" type="org.springframework.web.struts.DelegatingActionProxy" unknown="false" validate="true" > <forward name="login" path=".login" redirect="false" /> . .Note - the modified template includes a comment to tell you the Actual name of the class the URL is mapping to - lest you'd have to dig even more to find it. Again, a new dev on the project may not know WHY the DelegatingActionProxy is the class - but if you know anything about Struts you'll read the comment (hopefully) and at least know what class is servicing a URL.
Finally, the new struts_config_xml.xdt template. Please note if you're going to use this, you'll need to actually replace the template file in the xdoclet-apache jar, as it relates to other templates etc...
<?xml version="1.0" encoding="UTF-8" ?> <!DOCTYPE struts-config PUBLIC "-//Apache Software Foundation//DTD Struts Configuration 1.2//EN" "http://jakarta.apache.org/struts/dtds/struts-config_1_2.dtd"> <struts-config> <!-- ========== Data Sources Definitions =================================== --> <XDtMerge:merge file="struts-data-sources.xml"> <!-- Define your Struts data sources in a file called struts-data-sources.xml and place it in your merge directory. --> </XDtMerge:merge> <!-- ========== Form Bean Definitions =================================== --> <form-beans> <XDtMerge:merge file="xdoclet/modules/apache/struts/resources/struts_dynaform.xdt"> </XDtMerge:merge> <XDtClass:forAllClasses type="org.apache.struts.action.ActionForm"> <XDtClass:forAllClassTags tagName="struts:form" superclasses="false"> <form-bean name="<XDtClass:classTagValue tagName="struts:form" paramName="name"/>" type="<XDtClass:fullClassName/>" /> </XDtClass:forAllClassTags> </XDtClass:forAllClasses> <XDtMerge:merge file="struts-forms.xml"> <!-- If you have non XDoclet forms, define them in a file called struts-forms.xml and place it in your merge directory. --> </XDtMerge:merge> </form-beans> <!-- ========== Global Exceptions Definitions =================================== --> <XDtMerge:merge file="global-exceptions.xml"> <!-- Define your exceptions in a file called global-exceptions.xml and place it in your merge directory. --> </XDtMerge:merge> <!-- ========== Global Forward Definitions =================================== --> <XDtMerge:merge file="global-forwards.xml"> <!-- Define your forwards in a file called global-forwards.xml and place it in your merge directory. --> </XDtMerge:merge> <!-- ========== Action Mapping Definitions =================================== --> <XDtConfig:ifConfigParamNotEquals paramName="Controller" value=""> <action-mappings type="<XDtConfig:configParameterValue paramName="Controller"/>"> </XDtConfig:ifConfigParamNotEquals> <XDtConfig:ifConfigParamEquals paramName="Controller" value=""> <action-mappings> </XDtConfig:ifConfigParamEquals> <XDtClass:forAllClasses type="org.apache.struts.action.Action"> <XDtClass:forAllClassTags tagName="struts:action"> <XDtClass:ifHasClassTag tagName="struts:action" paramName="useSpringProxy"> <!-- using the spring proxy. Actual class: <XDtClass:fullClassName/> --> </XDtClass:ifHasClassTag> <action path="<XDtClass:classTagValue tagName="struts:action" paramName="path"/>" <XDtClass:ifHasClassTag tagName="struts:action" paramName="useSpringProxy"> type="org.springframework.web.struts.DelegatingActionProxy" </XDtClass:ifHasClassTag> <XDtClass:ifDoesntHaveClassTag tagName="struts:action" paramName="useSpringProxy"> <XDtClass:ifHasClassTag tagName="struts:action" paramName="type"> type="<XDtClass:classTagValue tagName="struts:action" paramName="type"/>" </XDtClass:ifHasClassTag> <XDtClass:ifDoesntHaveClassTag tagName="struts:action" paramName="type"> type="<XDtClass:fullClassName/>" </XDtClass:ifDoesntHaveClassTag> </XDtClass:ifDoesntHaveClassTag> <XDtClass:ifHasClassTag tagName="struts:action" paramName="name"> name="<XDtClass:classTagValue tagName="struts:action" paramName="name"/>" <XDtClass:ifHasClassTag tagName="struts:action" paramName="attribute"> attribute="<XDtClass:classTagValue tagName="struts:action" paramName="attribute"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action" paramName="prefix"> prefix="<XDtClass:classTagValue tagName="struts:action" paramName="prefix"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action" paramName="suffix"> suffix="<XDtClass:classTagValue tagName="struts:action" paramName="suffix"/>" </XDtClass:ifHasClassTag> scope="<XDtClass:classTagValue tagName="struts:action" paramName="scope" default="request" values="request,session"/>" <XDtClass:ifHasClassTag tagName="struts:action" paramName="input"> input="<XDtClass:classTagValue tagName="struts:action" paramName="input"/>" </XDtClass:ifHasClassTag> </XDtClass:ifHasClassTag> <XDtConfig:ifConfigParamGreaterOrEquals paramName="Version" value="1.1"> <XDtClass:ifHasClassTag tagName="struts.action" paramName="roles"> roles="<XDtClass:classTagValue tagName="struts.action" paramName="roles"/>" </XDtClass:ifHasClassTag> </XDtConfig:ifConfigParamGreaterOrEquals> <XDtClass:ifHasClassTag tagName="struts:action" paramName="parameter"> parameter="<XDtClass:classTagValue tagName="struts:action" paramName="parameter"/>" </XDtClass:ifHasClassTag> unknown="<XDtClass:classTagValue tagName="struts:action" paramName="unknown" values="true,false" default="false"/>" validate="<XDtClass:classTagValue tagName="struts:action" paramName="validate" values="true,false" default="true"/>" > <XDtClass:forAllClassTags tagName="struts:action-set-property"> <set-property <XDtClass:ifHasClassTag tagName="struts:action-set-property" paramName="id"> id="<XDtClass:classTagValue tagName="struts:action-set-property" paramName="id"/>" </XDtClass:ifHasClassTag> property="<XDtClass:classTagValue tagName="struts:action-set-property" paramName="property"/>" value="<XDtClass:classTagValue tagName="struts:action-set-property" paramName="value"/>" /> </XDtClass:forAllClassTags> <XDtClass:forAllClassTags tagName="struts:action-exception"> <exception key="<XDtClass:classTagValue tagName="struts:action-exception" paramName="key"/>" type="<XDtClass:classTagValue tagName="struts:action-exception" paramName="type"/>" <XDtClass:ifHasClassTag tagName="struts:action-exception" paramName="className"> className="<XDtClass:classTagValue tagName="struts:action-exception" paramName="className"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action-exception" paramName="handler"> handler="<XDtClass:classTagValue tagName="struts:action-exception" paramName="handler"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action-exception" paramName="path"> path="<XDtClass:classTagValue tagName="struts:action-exception" paramName="path"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action-exception" paramName="scope"> scope="<XDtClass:classTagValue tagName="struts:action-exception" paramName="scope"/>" </XDtClass:ifHasClassTag> /> </XDtClass:forAllClassTags> <XDtClass:forAllClassTags tagName="struts:action-forward"> <forward <XDtClass:ifHasClassTag tagName="struts:action-forward" paramName="className"> className="<XDtClass:classTagValue tagName="struts:action" paramName="className"/>" </XDtClass:ifHasClassTag> <XDtClass:ifHasClassTag tagName="struts:action-forward" paramName="contextRelative"> contextRelative="<XDtClass:classTagValue tagName="struts:action-forward" paramName="contextRelative"/>" </XDtClass:ifHasClassTag> name="<XDtClass:classTagValue tagName="struts:action-forward" paramName="name"/>" path="<XDtClass:classTagValue tagName="struts:action-forward" paramName="path"/>" redirect="<XDtClass:classTagValue tagName="struts:action-forward" paramName="redirect" default="false" values="true,false"/>" /> </XDtClass:forAllClassTags> </action> </XDtClass:forAllClassTags> </XDtClass:forAllClasses> <XDtMerge:merge file="struts-actions.xml"> <!-- If you have non XDoclet actions, define them in a file called struts-actions.xml and place it in your merge directory. --> </XDtMerge:merge> <XDtMerge:merge file="actions.xml"> </XDtMerge:merge> </action-mappings> <XDtMerge:merge file="struts-controller.xml"> <!-- Define your Struts controller in a file called struts-controller.xml and place it in your merge directory. --> </XDtMerge:merge> <XDtMerge:merge file="struts-message-resources.xml"> <!-- Define your Struts message-resources in a file called struts-message-resources.xml and place it in your merge directory. --> </XDtMerge:merge> <XDtMerge:merge file="struts-plugins.xml"> <!-- Define your Struts plugins in a file called struts-plugins.xml and place it in your merge directory. --> </XDtMerge:merge> </struts-config><Finding the modifications to the template is left as an excercise for the reader.
This is being used with success against XDoclet 1.2.2, and using the 1.2.2 XDoclet-apache lib with the rest of xdoclet 1.2.3.
Posted at 11:02PM Jul 21, 2006 by saibrian in Tools and automation (Sloth) |
XDoclet 1.2.3 and Listeners in TLDs
I'm a big lover of XDoclet. The 1.2.x stream has been quiet for a while, and I thought I'd read that dev on 1.x had stopped in favor of the rewrite for XDoclet 2. So the issues in 1.2.2 were just there unless you wanted to fix them. The beauty of OSS is that you can fix them - but digging into a lib like XDoclet can be a task.
Many of the issues though, are simply related to a template that needs a tweak. Those are easier to fix assuming some patience. The XDoclet templating syntax is a bit tedious, but is rather clear in its verbosity. I've tweaked a template or 2, or looked at them to understand what's available and how its working when XDoclet In Action and the docs don't address the issue.
So to the crux of the matter - I wanted to generate some mappings for Hibernate Components, and the generation facilities in 1.2.2 didn't quite do the job. My friend Google lets me know there's a new release of XDoclet, so I'll do the upgrade and all will be well - right?
The hibernate mappings for components look perfect, but I'm greeted with a nice stack trace in the browser...
A Listener is failing on startup. A listener that was recently depricated... I look in web.xml - no Listener declaration... Confusion and frustration ensue.
The problem:
My listener is showing up in the XDoclet generated TLD.
The JSP 1.2 spec allows Listener declarations in TLDs, but in XDoclet 1.2.2, they weren't generated in the TLD(s). This was "fixed" in XDoclet 1.2.3. The offending code in the XDoclet template:
<XDtClass:forAllClasses type="javax.servlet.http.HttpSessionActivationListener,javax.servlet.http.HttpSessionAttributeListener,javax.servlet.http.HttpSessionBindingListener,javax.servlet.http.HttpSessionListener,javax.servlet.ServletContextAttributeListener,javax.servlet.ServletContextListener" abstract="false"&th; <listener&th; <listener-class&th;<XDtClass:fullClassName/&th;</listener-class&th; </listener&th; </XDtClass:forAllClasses>Why is this code "offending?" Neither the TLD template nor the web.xml template make a distinction regarding listeners that should or should not be included. Actually, they don't even use the @web.listener tag now... All Listeners are blindly included.
Maybe I'm odd - but I'm all for writing some custom tags in a project and letting XDoclet nicely generate a TLD for the project's tags. In this scenario I'm not shooting for a redistributable taglib - I'm looking for a taglib for this project. If I need to use it again I'll pull it into the utility project we keep internally, but until then it belongs right where it is. As such, listeners that are used for taglib initialization are (were) marked with @web.listener, and generated into web.xml.
Now I admit, I'm lazy. The build target for the web generation is as concise and simple as possible, including **/*Listener, **/*Tag, and the other regulars. With the new template I'm forced to either develop custom tags in a different project (maybe not a bad idea), or contorting my build.xml file to generate the web.xml and TLD artifacts with very specifc filesets...
This too may be a good idea, but is another point of maint... when a new dev adds a Listener, I want them to follow the naming convention and use the appropriate @web.listener tag and trust the code gen system to do the rest of the work. Now I to document, explain, and remember that the files used for these generation tasks have to be explicitly enumerated....
Long story short - if you upgrade to XDoclet 1.2.3, watch out for this one.
Posted at 10:18PM Jul 21, 2006 by saibrian in Gotcha |
jBPM/JPDL GUI - a middle of the road approach
I've been using jBPM for a while, and really like it as a BPM tool. I've actually found a few uses outside the original purpose, but that's another post.
Like most tools though, I'm forced to code in XML. I'm not saying I don't want to code in XML, but it can get a bit tedious. Mix that with the love that the analysts and process owners have for pictures, and you'll be wanting a GUI for jBPM.
There is a GUI for jBPM. I've played with it in various incarnations and releases for a while, but always found it lacking. I don't particularly want a flowchart of my process, rather I wanted a structure that showed the semantics of my process that did some other work for me.
With a large process to author and a deadline for completion approaching, I did what any joe developer would do - I wrote a tool that would allow me to author the process in a tool and then export the XML.
This is a small screenshot of the editor:
Why write a different tool? A couple of reasons.
- Dave Thomas says the issue with a GUI is WYSIAYG - What You See Is All You Get. If there is a feature in the system that you know how to use but the tool doesn't expose it, you can't use it unless...
- I needed a tool wouldn't lose data in my process definition - even if it didn't support editing it in the GUI, it should keep the XML that I started with.
- The last time I used the JPDL editor from JBoss, it was rather tied to their way of doing things. I'm not doing things that way, and a tool should be as unobtrusive as possible.
- I hadn't written a fat client in a while, and it looked like a fun problem :-)
Is is a flow chart? No. can I use it to create one? Certainly. The tool doesn't hide the process language from the developer - and it shouldn't. The holy grail of management drawing pictures that turn into software is a myth, and I'm not aiming to chase it.
Posted at 07:38AM Jul 13, 2006 by saibrian in Tools and automation (Sloth) | Comments[1]
Code Generation
I like code generation. A lot. build time, run time, all the time.
Latest Code generation tool: apt-jelly.
The short story - a wrapper around the annotation processor in JDK5 that exposes the annotations to a series of templates for code generation.
Originally it just used Jelly scripting (yeck), and I stayed away, but now it supports FreeMarker - much more friendly.
Now I'm a big fan of XDoclet - I've had it generating 1000's of LOC and config for EJB 2.1 projects (lifesaver), and currently use it to generate all sorts of artifacts (hbm, struts-config, a struts-config-spring integration bean file...), but it is a bit restrictive, and writing templates that have more than mediocre transformations (tag handlers) is possible, but a bit more involvement than I'm looking for.
Contrast that with FreeMarker templates... Classes and types are first class citizens, macros are a really possibility, and writing an annotation to process is a snap.
Death knell for XDoclet with me - the 1.x branch is no longer under active dev... like many OS projects, the lead has moved on to a complete re-write to address architectural issues in the 1.x line, but the 1.x line is the one that has all the cool plugins.... Heck, I even got templates from Oracle's tech support at one point. I'm not giving up on XDoclet, and will probably use it for some time to come (with a patched version of XJavadoc to support JDK5 source files), but new heavy lifting will be done with apt-jelly.
apt-jelly is short of documentation and examples (I'll post an example here at some point), but it is open source, and relatively easy to pick up. Now I just need a FreeMarker debugger...
apt-jellyis the sourceforge site. It is a seriously fledgling project, but having used it, I think there's some potential there until/unless Sun or some such gifts us with an easier way to use apt for code generation.
Posted at 10:23PM Jul 12, 2006 by saibrian in Tools and automation (Sloth) |
DBUnit Ant Task and Foreign Key Constraints - Fix
I have a fix for the DBUnit Foreign Key constraint on ant import issue. The next few paragraphs are history for those that haven't used this wonderful tool. If you're just looking for the fix, skip the next few paragraphs.
So I use DBUnit. Not really in the full way it was intended for - database unit testing, but as a nice, simple, Ant scriptable tool to move data in and out of the DB....
It works really well - point to a DB, specify the standard JDBC stuff, and wa-la - you have a nice, DB agnostic XML export of data. Lots of folks use this bit of the tool. I need to have several sets of data that can be reloaded into the DB on command - the standard stuff consisting of a few user accounts, roles, the stuff of drop down lists, etc. These exports are easy to version control, and great for resetting a test DB, moving new seed data to a staging/testing instance, and getting a new development environment going. Not to mention that the import will run against different DBs, allowing easy importing of the seed data into a new platform.
If that sounds like a commercial for the technique - it is :) I like it, and having started to use it I never want to explain how to do the backup and restore a SQLServer DB or the like.
Then that day comes where I am a bit more picky about the tables I do and don't want exported, and I get the standard Foreign Key Constraint issue. Having manually specified the tables, I am responsible for exporting them in an order that will respect FK constraints on import. This messing about gets more and more annoying, but we live with it.
Then we start wanting to export and import ALL data in the DB... and even letting DBUnit export all the tables, it can't sort out the right order to ensure that FK constraints are respected... I know they're valid in the DB. What I'd really like is to turn off the checking for the import. Most DBs allow that right? Yup.
MySQL though, only allows it on a session specific basis - the life of one JDBC connection. I tried a number of ways to activate it globally to no avail. DBUnit though, gives no option to get at the connection its using.
TA DA - it's open source, so I can figure this out... 3 iterations later, I have a solution I'm happy with. Didn't have to really modify the DBUnit code, and I'm able to get the job done by adding a new kind of "step", intercepting the connection that DBUnit will use to perform the load BEFORE it does its magic.
The code is in 2 little classes.
This is the extension to the DBUnitTask that we're going to use instead of the standard DBUnitTask, it allows additions to the steps that DBUnit is going to execute.
/*
* DBUnitTaskExt.java
* Created on: Jul 9, 2006
*
*/
package org.dbunit.ant;
import org.dbunit.ant.DbUnitTask;
/**
* This extension to the DBUnitTask allows the addition of <code>Script</code>s
* to the DBUnit task.
* This is useful for executing pre and post scripts when performing data-import and export.
* @author Brian Greene
*
*/
public class DBUnitTaskExt extends DbUnitTask
{
public void addScript(Script s)
{
getSteps().add(s);
}
}
This is the actual class that does the script parsing and executing. Happily, I was able to "borrow" and only slightly modify a bit of code from Ant 1.6.5's SQLExec class.
/*
* Script.java
* Created on: Jul 9, 2006
*/
package org.dbunit.ant;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.sql.Statement;
import java.util.StringTokenizer;
import org.apache.tools.ant.Project;
import org.apache.tools.ant.taskdefs.SQLExec;
import org.apache.tools.ant.taskdefs.SQLExec.DelimiterType;
import org.dbunit.DatabaseUnitException;
import org.dbunit.ant.DbUnitTaskStep;
import org.dbunit.database.IDatabaseConnection;
/**
* This class can be used to insert scripts into the series of Steps that are
* executed by the DBUnit task.
*
* This class borrows some functionality from Ant's SQLExec task related to the
* actual parsing of the SQL.
*
* @author Brian
*
*/
public class Script implements DbUnitTaskStep {
private File src;
private String delimiter = ";";
private Connection conn;
/*
* (non-Javadoc)
*
* @see org.dbunit.ant.DbUnitTaskStep#execute(org.dbunit.database.IDatabaseConnection)
*/
public void execute(IDatabaseConnection connection)
throws DatabaseUnitException {
try {
conn = connection.getConnection();
if(src == null || !src.exists())
throw new DatabaseUnitException("The src attribute must be supplied.");
executeScript(src);
} catch (SQLException e) {
throw new DatabaseUnitException("Error in getConnection()... :"
+ e.getMessage(), e);
}
}
/**
* This method was pulled and slightly modified from the SQLExec task from
* Ant 1.6.5
*
* @param f
* the File containing the script.
*/
private void executeScript(File f) {
try {
StringBuffer sql = new StringBuffer();
String line = "";
BufferedReader in = new BufferedReader(new FileReader(f));
while ((line = in.readLine()) != null) {
line = line.trim();
if (line.startsWith("//")) {
continue;
}
if (line.startsWith("--")) {
continue;
}
StringTokenizer st = new StringTokenizer(line);
if (st.hasMoreTokens()) {
String token = st.nextToken();
if ("REM".equalsIgnoreCase(token)) {
continue;
}
}
sql.append(" " + line);
// SQL defines "--" as a comment to EOL
// and in Oracle it may contain a hint
// so we cannot just remove it, instead we must end it
if (line.indexOf("--") >= 0) {
sql.append("\n");
}
String delimiterType = SQLExec.DelimiterType.NORMAL;
String delimiter = ";";
if ((delimiterType.equals(DelimiterType.NORMAL) && sql
.toString().endsWith(delimiter))
|| (delimiterType.equals(DelimiterType.ROW) && line
.equals(delimiter))) {
execSQL(sql.substring(0, sql.length() - delimiter.length()));
sql.replace(0, sql.length(), "");
}
}
// Catch any statements not followed by ;
if (!sql.equals("")) {
execSQL(sql.toString());
}
} catch (Exception e) {
// TODO: handle exception
}
}
/**
* This method was pulled and slightly modified from the SQLExec task from
* Ant 1.6.5 Exec the sql statement.
*/
protected void execSQL(String sql) throws SQLException {
// Check and ignore empty statements
if ("".equals(sql.trim())) {
return;
}
ResultSet resultSet = null;
try {
//log("SQL: " + sql, Project.MSG_VERBOSE);
boolean ret;
int updateCount = 0, updateCountTotal = 0;
Statement statement = conn.createStatement();
ret = statement.execute(sql);
updateCount = statement.getUpdateCount();
resultSet = statement.getResultSet();
do {
if (!ret) {
if (updateCount != -1) {
updateCountTotal += updateCount;
}
} else {
// we always print results.
printResults(resultSet);
}
ret = statement.getMoreResults();
if (ret) {
updateCount = statement.getUpdateCount();
resultSet = statement.getResultSet();
}
} while (ret);
// log(updateCountTotal + " rows affected", Project.MSG_VERBOSE);
// we're always printing results.
StringBuffer line = new StringBuffer();
line.append(updateCountTotal + " rows affected");
// log(line.toString(), Project.MSG_INFO);
SQLWarning warning = conn.getWarnings();
while (warning != null) {
// log(warning + " sql warning", Project.MSG_VERBOSE);
warning = warning.getNextWarning();
}
conn.clearWarnings();
} catch (SQLException e) {
// log("Failed to execute: " + sql, Project.MSG_ERR);
// we've no concept of continue on error - we always fail.
throw e;
} finally {
if (resultSet != null) {
resultSet.close();
}
}
}
/**
* This method was pulled and slightly modified from the SQLExec task from
* Ant 1.6.5 print any results in the result set.
*
* @param rs
* the resultset to print information about
* @throws SQLException
* on SQL problems.
* @since Ant 1.6.3
*/
protected void printResults(ResultSet rs) throws SQLException {
if (rs != null) {
// log("Processing new result set.", Project.MSG_INFO);
ResultSetMetaData md = rs.getMetaData();
int columnCount = md.getColumnCount();
StringBuffer line = new StringBuffer();
for (int col = 1; col < columnCount; col++) {
line.append(md.getColumnName(col));
line.append(",");
}
line.append(md.getColumnName(columnCount));
// log(line.toString(), Project.MSG_INFO);
line = new StringBuffer();
while (rs.next()) {
boolean first = true;
for (int col = 1; col <= columnCount; col++) {
String columnValue = rs.getString(col);
if (columnValue != null) {
columnValue = columnValue.trim();
}
if (first) {
first = false;
} else {
line.append(",");
}
line.append(columnValue);
}
// log(line.toString(), Project.MSG_INFO);
line = new StringBuffer();
}
}
// log("", Project.MSG_INFO);
}
/*
* (non-Javadoc)
*
* @see org.dbunit.ant.DbUnitTaskStep#getLogMessage()
*/
public String getLogMessage() {
return "Script step";
}
public File getSrc() {
return src;
}
public void setSrc(File src) {
this.src = src;
}
}
Usage in the build file:
<target name="data-import" depends="init, recreate-db">
<taskdef name="dbunit" classname="org.dbunit.ant.DBUnitTaskExt" classpathref="classpath.main" />
<dbunit driver="${jdbc.driver}" url="${jdbc.url}" userid="${jdbc.userid}" password="${jdbc.password}">
<classpath refid="classpath.main" />
<script src="etc/sql/${data.movement.script.prefix}.preScript.sql"/>
<operation type="INSERT" src="${dumpFile}.xml" format="xml" />
</dbunit>
</target>
This usage is specialized to only perform one insert based on a parameter(used in the build to load several different datasets based on the dumpFile parameter), as the recreate-db dependency will handle the initial setup of the DB (gotta love the Hibernate tools to generate those scripts :) ). I suppose you could use it to just string together a bunch of scripts though.
So here's the etc/sql/${data.movement.script.prefix}.preScript.sql for MySQL to disable FK checking:
SET @@FOREIGN_KEY_CHECKS=0;Now, I have read that there are ways to append this to the MySQLConnectorJ connection URL, but try as I might it didn't always work. And besides, that just solves the problem for MySQL, not the other DBs I'm going to use. Those of you familiar with DBUnit will recognize the "operation" elements - you can list as many as you want. I just added the option to add as many scripts as I want into the mix, using the same JDBC connection. This also allows an easy way to do pre-export, post-export, or pre/post import work in additon to the work related to the FK problem. Is there a better way to solve the problem? I'm curious to know. I do know that this is a more concise way to chain a bunch of scripts together rather than using a whole mess of SQL tasks (a by product of the solution). If this solves the issue for you, great. If you think there's a more elegant solution that will still allow me to do the job between multiple DBs, please enlighten me - I'm always up for an easier solution. (this is in the lazy category for a reason).
Posted at 09:32PM Jul 12, 2006 by saibrian in Tools and automation (Sloth) |