Thursday, September 10, 2009

Spring 3.0 AOP: Cheatsheet

Continuing with insights and tips around Spring 3.0, here is a collection of code samples I've established "so far" with my (rather brief) prototyping using that framework. As with all of my cheatsheets, I'm not presenting these things as any kind of definitive reference; they are only from my own experience, done only in a "sandbox" so far -- we've not yet put these things into production. Your contributions and corrections to this cheatsheet, and any of my other ones, are welcome.


Dependencies

The following libraries are needed:

org.springframework.asm-3.0.0.M3.jar
org.springframework.beans-3.0.0.M3.jar
org.springframework.context-3.0.0.M3.jar
org.springframework.core-3.0.0.M3.jar
org.springframework.expression-3.0.0.M3.jar
org.springframework.aop-3.0.0.M3.jar
antlr-3.0.1.jar commons-logging.jar
com.springsource.org.aopalliance-1.0.0.jar
com.springsource.org.aspectj.runtime-1.6.5.RELEASE.jar
com.springsource.org.aspectj.weaver-1.6.5.RELEASE.jar

Concepts

Some key terminology and concepts, pulled from the Spring 3.0 Reference doc - I'll just copy key pieces verbatim first and then paraphrase at the end:
  • Aspect: a modularization of a concern that cuts across multiple classes. Transaction management is a good example of a crosscutting concern in J2EE applications. In Spring AOP, aspects are implemented using regular classes (the schema-based approach) or regular classes annotated with the @Aspect annotation (the @AspectJstyle).
  • Join point: a point during the execution of a program, such as the execution of a method or the handling of an exception. In Spring AOP, a join point always represents a method execution.
  • Advice: action taken by an aspect at a particular join point. Different types of advice include "around," "before" and "after" advice. Many AOP frameworks, including Spring, model an advice as an interceptor, maintaining a chain of interceptors around the join point.
  • Pointcut: a predicate that matches join points. Advice is associated with a pointcut expression and runs at any join point matched by the pointcut (for example, the execution of a method with a certain name). The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.
  • Introduction: declaring additional methods or fields on behalf of a type. Spring AOP allows you to introduce new interfaces (and a corresponding implementation) to any advised object. For example, you could use an introduction to make a bean implement an IsModified interface, to simplify caching. (An introduction is known as an inter-type declaration in the AspectJ community.)
  • Target object: object being advised by one or more aspects. Also referred to as the advised object. Since Spring AOP is implemented using runtime proxies, this object will always be a proxied object.
  • AOP proxy: an object created by the AOP framework in order to implement the aspect contracts (advise method executions and so on). In the Spring Framework, an AOP proxy will be a JDK dynamic proxy or a CGLIB proxy.

So an aspect is a class that contains methods referred to as "advice". The advice is executed as specified by the annotation (before, after, etc.) and the "pointcut", or predicate (what classes/methods match the expression). The point in the application where the advice is applied is a "join point". The runtime object containing these join points (i.e., being advised by one or more aspects) is the "target" or "advised object". Spring uses proxying to accomplish the advising (see section 8.6.1 - really, read it, it's short and helpful), and will use JDK dynamic proxies if all advised objects implement at least one interface. Finally, "introductions" are a mechanism where advised objects can be transparently made to implement new interfaces (to retrofit things like caching, comparability, serialization, immutability, etc.), with the advice providing an implementation of the interface, and being handed a reference to the advised object that it can use to invoke the new interface methods.


Problem Space

Recommendations in the following discussions are in bold-face.

The proof-of-concept application is a trivial tongue-in-cheek model of a "family service", where my son Connor and my Siberian Husky Nika would each like to get my attention. It happens. Spring IoC is used to declaratively determine who wins; since that's a static configuration, it's for now the same answer every time.

So the basic idea is a shown in the UML: we have a FamilyService with a FamilyMember property, and two FamilyMembers, each with a name property.

Note that all classes implement an interface. This is in general a good idea for testability, etc. but in particular with Spring AOP, I'd favor this approach because it facilitates use of JDK dynamic proxies instead of CGLIB proxies. Use of CGLIB should consider these things: (1) more library dependencies, (2) final methods cannot participate in AOP, and (3) constructors are called twice due to CGLIB implementation details.

Short of other insights, I would for now recommend implementing at least one interface for any classes that participate in Spring AOP.

A detailed explanation of the above issue around CGLIB vs JDK dynamic proxying, as well as very clear presentation in general around Spring 3.0, can be found in the Spring Reference doc.



Enabling AOP

The injection of the family member getting my attention is done like this:

<bean id="family-service" class="spring.FamilyService">
 <property name="member">
     <ref local="nikadog"></ref>
 </property>
</bean>

This is typical Spring injection; nothing special here. The more interesting stuff is to now add some AOP. Do this by adding an "aop" namespace to the beans declaration tag and enabling what I would recommend as the preferred flavor of Spring AOP (AspectJ-style annotations plus Spring auto-proxying):
 
 <beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:aop="http://www.springframework.org/schema/aop"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">
 ....
   <aop:aspectj-autoproxy/>
 </beans>

The Spring reference doc again provides a clear, useful discussion around the various "flavors" you can use with Spring AOP. Given the tradeoffs described there, I recommend the AspectJ-style annotations with Spring auto-proxying for the following reasons:
  1. Their advice to "use the simplest thing that works" is IMO sound guidance
  2. So, for now, assume that AspectJ-annotations will suffice until you find otherwise
  3. If you need to gain more power, you can migrate easily from the AspectJ-annotations to full-blown AspectJ itself

The limitations to be aware of include these:

  1. AOP can only be applied to Spring-managed beans (therefore, Spring-declare any classes that will leverage AOP)
  2. AOP mechanisms can be applied only to public method-level executions (excluding constructors)
  3. Methods in AOP-enabled classes that have aspects declared will not see those aspects applied if invoked by other
    methods in that same class (i.e., do not use self-invocation within AOP-enabled classes)

Limitations #2 and #3 are a result of Spring's proxy approach to AOP. See section 8.6.1 in the reference doc for a detailed discussion. Note that a workaround for #3 is discussed in that section, but it involves a tight coupling to Spring and makes the application class acutely aware that it is being AOP-managed, neither of which is a recommended practice.


Simple AOP

To create an aspect with some advice that executes before all public methods in the spring package (see reference doc, section 8.2.3.4 for examples around expression syntax):

import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.*;
@Aspect
public class MyAspect {
 /**
  * Log a message before entry to specified methods
  */
  @Before("execution(* spring..*(..))")
  private void logOnEntry(JoinPoint jp) {
     // JoinPoint argument is optional, but recommended as quite useful
     System.out.println("Before " + getSimpleName(jp));
  }
 /**
  * Returns the simple name of the class being advised, as per the given
  * join point.
  */
  private String getSimpleName(JoinPoint jp) {
     return jp.getTarget().getClass().getSimpleName();
  }
 }

The aspect must be managed by Spring to be put into play - add this entry to the configuration file:

<bean id="myAspect" class="spring.MyAspect"> </bean>

As you can see, you can also access the target object via the JoinPoint argument. As such, you can cast it to your application type and execute its public methods.

You can also execute advice after a method or around a method. The @After is analogous to the above example. The "execution" part of the expression is the "pointcut designator" or PCD; there are several others, but "execution" is probably what will be used the most. Since it's indicating a method-level execution, you can alter the method-portion of the expression to e.g. address only setters:

@Before("execution(* spring..set*(..))")

To address executions at a type level (i.e. all public methods in a given type), use the "within" PCD:

@After("within(*.*Service)")

You can also use the Spring bean ID declaration as an alias for a given type; so, given this bean declaration:

<bean id="family-service" class="spring.FamilyService"></bean>

...you can specify this join point using the "bean" PCD:

@After("bean(family-service)")


Reusable Pointcuts

Reusable pointcuts can be established:

/**
 * Establish a pointcut on execution of any and all public methods
 */
@Pointcut("execution(* *.*(..))")
public void anyMethod() {}

Pointcuts can be combined to form conditional expressions, using AND, OR and NOT operators (&&, ||, and !):

@Pointcut("within(com.xyz.service..*)")
private void inServiceModule() {}
@Pointcut("anyMethod() && inServiceModule()")
private void serviceOperation() {}

To intercept the throwing of an exception resulting in a method exit, using the "anyMethod" pointcut established above:

/**
 * Intercept execution when any method exits by throwing any exception
 */
 @AfterThrowing(pointcut = "anyMethod()", throwing = "ex")
 private void trackExceptions(JoinPoint jp, Exception ex) {
     System.out.println("Exception thrown from "           + getSimpleName(jp) + ": " + ex);
 }

To intercept the value returned from a specified pointcut:

 /**
  * Establish a pointcut on execution of specified method
  */
  @Pointcut("execution(* spring.Nika.getName())")
  public void getNikadogName() {}
 /**
  * Intercept the return from specified pointcut
  */
  @AfterReturning(pointcut = "getNikadogName()", returning = "name")
  private void getNikaName(String name) {
      System.out.println("after returning from nikadog, pointcut name: "
           + name + "...");
  }


Beyond the Simple

The @Around PCD is bit more sophisticated:

 /**
  * Establish a pointcut on execution of specified method
  */
  @Pointcut("execution(* FamilyService.getMember(..))")
  public void getMember() {}
 /**
  * Wrap specified method to get profiling information. Can also use this pattern
  * for tracing, caching, etc.
  */
  @Around("getMember()")
  private FamilyMember doProfiling(ProceedingJoinPoint pjp)  throws Throwable {
    long start = System.currentTimeMillis();
    FamilyMember member = (FamilyMember) pjp.proceed();
    System.out.println("Elapsed time to get member: " +        (System.currentTimeMillis() - start));
     return member;
 }

In the above snippet, the ProceedingJoinPoint facilitates setting things up before the advised object method is executed (i.e. start the stopwatch, check the cache for a hit, etc.), then supports (optionally!) executing the method, and finally returning the appropriate value - so you can see that this decorates the object execution with your own proxying code. Note that you can easily check a cache for a "hit", then simply return the cached value if it's there without executing the method, otherwise executing the method and stuffing it in the cache for subsequent requests. Using the "args" PCD, you can (e.g.) examine incoming argument values to qualify what specific object in the cache is being requested.

Note that this approach to caching relies on the aspect to manage the cache. An alternate approach will be presented in the "Introductions" section below.

Suppose our service class was throwing an exception, and we want to apply a retry strategy since the problem involves waiting for some remote resource to become available (e.g. the database is spinning up):

@Around("getMember()")
 private FamilyMember retry(ProceedingJoinPoint pjp) throws Throwable {
     int numAttempts = 0;
     int maxRetries = 5;
     Throwable t = null;
     do {
        numAttempts++;
        try {
            return (FamilyMember)pjp.proceed();
        } catch (DatabaseNotReadyException ex) {
            System.out.println("=========== Caught exception from "
                 + getSimpleName(pjp)
                + "; retry attempt #" + numAttempts + "==============");
            t = ex;
        }
     } while (numAttempts <= maxRetries);
              throw t;
   }

Introductions

A technique described above for caching relies on the aspect itself to manage the cache. An alternate approach is to implement an interface that prescribes caching functionality:

public interface Cacheable {
     public boolean isCached();
     public void save(Object obj);
     public Object fetch();
}
But we may be motivated to regard this cacheability as a cross-cutting concern, one that should be maintained and managed (enable/disable, track hits, monitor throughput gains, flush the cache, etc.) separately from the pure business logic (vs simply having FamilyService implement the interface explicitly, i.e. such that one's business classes are not responsible for determining their own "cacheability" - that's an orthogonal concern). If so, we can use AOP "introductions" to retrofit the caching.

An implementation supporting the FamilyService (and, for that matter, other services in general) might begin life like this:

public class CacheableImpl implements Cacheable {
      private boolean cached = false;
      private Object cachedObject = null;
      public boolean isCached() {
        return cached;
     }
      public void save(Object obj) {
        cachedObject = obj;
        cached = true;
     }
      public Object fetch() {
        return cachedObject;
     }
}
An "introduction" is a way of retrofitting an implementation of an interface onto the target object. Since it's AOP, the target object does not and need not know it's being decorated like this. The somewhat cryptic syntax to make it happen is encapsulated in the aspect:
@DeclareParents(value = "spring.FamilyService", defaultImpl = CacheableImpl.class)
 public static Cacheable mixin;
 .......
 @Around("getMember() && this(service)")
 public FamilyMember checkCache(ProceedingJoinPoint jp, Cacheable service) throws Throwable {
     if (service.isCached())  {
        System.out.println(" ---> Returning cached member");
        return (FamilyMember) service.fetch();
     }
     System.out.println(" ---> Member NOT cached; call service and save result...");
     FamilyMember member = (FamilyMember) jp.proceed();
     service.save(member);
     return member;
 }
Now, instead of relying on the aspect to manage caching - arguably a questionable responsibility - the details of caching are factored out to give us better cohesion. This is just one possible use of the AOP introduction.
Resources Spring 3.0.M3 Reference Documentation, by Rod Johnson et. al., 2009 Spring Framework

Wednesday, September 9, 2009

Spring Test Framework Cheatsheet

This documents some prototyping done with the Spring Test Framework (I'll refer to this as the STF). Effectively it is a paraphrase/summary of what's written up in the Chapter 10 of the Spring Framework Reference document, version 3.0.M3, with some example code snippets that I was able to get working. Consult that writeup for further detail.

Here are the high-level features of the STF that I'd consider most interesting:
  1. How to fetch the Spring application context object regardless of environment (web container, test environment, etc.)
  2. How to configure test-specific Spring contexts
  3. Simplified access to context beans
  4. Automatic rollback of test case transactions
  5. Auto-rollback without extending a superclass
  6. Odds-n-ends

How to fetch the Spring application context

This first tip is not directly related to the STF, but was discovered during my experiments with it, so I make note of it here.

 In a previous post, I presented a pattern for determining at runtime how to get a reference to the application context that would first look for a web context, and if that failed simply produce a classpath-based context. This pattern requires explicit knowledge not only of what environment is in play, but also knowledge of exactly what Spring context is appropriate for that environment. A simpler pattern can be used as follows - first, provide a class that implements ApplicationAwareContext:


public class ApplicationContextProvider 
    implements ApplicationContextAware { 
    private static ApplicationContext context;
    public static ApplicationContext getApplicationContext() {
        return context;
    }
    public void setApplicationContext(ApplicationContext context) {
        this.context = context;
    }
}

Declare that class as managed by Spring in the context file:

<bean id="appCtxProvider" 
    class="com.twc.registry.common.ApplicationContextProvider">
</bean>

Now, any class that wants a reference to the context instance can do so at runtime, whether in a web or a test/standalone environment:

ApplicationContext context = 
    ApplicationContextProvider.getApplicationContext();
MyBean myBean = (MyBean)context.getBean("my-bean");

That code snippet can of course be encapsulated and parameterized as needed, to facilitate keeping third-party framework specifics in a single point of change.

This precludes the need for a ServletContextListener as described in my previous post, simplifies the mechanics around fetching beans in the context, and eliminates the need for application awareness of explicit subclasses of the Spring ApplicationContext.

In what follows, I'll present an even simpler method for injecting dependencies directly into test classes, such that a reference to the application context is not needed.


How to configure test-specific Spring contexts

If you need to specify certain test fixtures or other test-specific dependencies, it'd be useful to both have these injected (to facilitate reuse and simplify test setup) but keep the configuration for this injection separate from production classes. Using the STF, a convenience idiom supports doing this.

First, to use the STF, add org.springframework.test-3.0.0.M3.jar to your classpath.

Configure a test-specific Spring context using the following annotation:

package com.mybiz;
@ContextConfiguration
public class MyTest {
   @Test
    public void testThis() {
          ApplicationContext ctx = 
              ApplicationContextProvider.getApplicationContext();
          MyBean myBean = (MyBean ) context.getBean("my-bean");
          // now do some tests with myBean...
    }
}

This presumes the existence of a Spring configuration file named com/mybiz/MyTest-context.xml. To otherwise specify the location:

@ContextConfiguration("/WEB-INF/MyTestConfig.xml")

Multiple resources can be specified using comma-separated lists. The default is for subclasses to inherit superclass' resource lists and optionally override bean entries; this inheritance behavior can also be disabled.


Simplified access to context beans

Now, instead of using the ApplicationContextProvider manually as in the above snippet, we can simply do this:

@Autowired  // or can annotate a setter instead of a field
private MyBean myBean;

If there are  multiple MyBean's defined in the Spring context, this type-based wiring will not suffice (without further qualification, which we ignore for now). Instead use injection by name:

@Resource
private MyBean myBean;

However, it appears (empirically...) that @Resource also wires by type, since in the above working example, I hadn't configured a bean with an ID of "myBean". It also "just works" this way:

private MyBean myBean;
@Resource  // also works with @Autowired!
private void setTheBean(MyBean theBean) {  
// notice the field and/or method are both private - but injection still works!
    myBean = theBean;
}


Automatic rollback of test case transactions

By far the most valuable feature of the STF is the ability to execute integration tests against a database without changing the database state - i.e., all transactions are automatically rolled back. Making this work is reasonably simple - first, we'll need Spring context entries to specify a data source and a transaction manager:

<bean id="dbcp-dataSource" 
    class="org.apache.commons.dbcp.BasicDataSource" 
    destroy-method="close">
    <property name="driverClassName" value="${database.driver}"/>
    <property name="url" value="${database.url}"/>
    <property name="username" value="${database.user}"/>
    <property name="password" value="${database.password}"/>
</bean>
<bean id="txnMgr" 
    class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
    <property name="dataSource" ref="dbcp-dataSource"/>
</bean>
<tx:annotation-driven transaction-manager="txnMgr"/>
<aop:aspectj-autoproxy/>

This context configuration file must be loaded using @ContextConfiguration; we must declare @Transactional either at the class or method level; and since our transaction manager is not named "transactionManager" (the default) -- or if we would ever wish to disable auto-rollback at the class level -- we'll need to add a @TransactionConfiguration annotation:

@ContextConfiguration(locations = {"/WEB-INF/MyTestConfig.xml"})
@Transactional
@TransactionConfiguration(transactionManager="txnMgr", defaultRollback=true)
public class MyDaoTest extends AbstractTransactionalTestNGSpringContextTests {
.......
// override auto-rollback on per-method basis
    @Test
    @Rollback(false)
    public void keepThisTransactionInPlace() {
        ......
    }
    @BeforeTransaction  
// likewise, @AfterTransaction follows the transaction
    public void doBeforeTxn()
    {
// do whatever setup needed, if any, before the transaction executes
    }
}

To gain numerous conveniences, we've extended a Spring class specific to the TestNG framework (also available are Junit-specific superclasses). We could have instead added a handful of cruft to get around doing this extension, if e.g. you need to extend your own test superclass or for some other reason you're adverse to this. But the added cruft is so noisy that you'll be motivated to simply encapsulate it in your own superclass anyway, or if that's not feasible then reuse becomes a copy-paste headache. If possible, simply extending the Spring-provided convenience classes is probably the simplest thing to do.


Auto-rollback without extending a superclass

But if you must know, here's the cruft you'll need to avoid extending AbstractTransactionalTestNGSpringContextTests:

@ContextConfiguration(locations = {"/WEB-INF/MyTestConfig.xml"})
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class, 
    DirtiesContextTestExecutionListener.class,
    TransactionalTestExecutionListener.class})
@Transactional
@TransactionConfiguration(transactionManager="txnMgr")
public class MyDaoTest implements IHookable, ApplicationContextAware {
    private ApplicationContext applicationContext;
    private final TestContextManager testContextManager;
    private Throwable testException;
    public MyDaoTest() {
        this.testContextManager = new TestContextManager(getClass());
    }
    public final void setApplicationContext(
        ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }
    @BeforeClass(alwaysRun = true)
    protected void springTestContextPrepareTestInstance() 
    throws Exception {
        this.testContextManager.prepareTestInstance(this);
    }
    @BeforeMethod(alwaysRun = true)
    protected void springTestContextBeforeTestMethod(Method testMethod) 
    throws Exception {
        this.testContextManager.beforeTestMethod(this, testMethod);
    }
    public void run(IHookCallBack callBack, ITestResult testResult) {
        callBack.runTestMethod(testResult);
        this.testException = testResult.getThrowable();
    }

    @AfterMethod(alwaysRun = true)
    protected void springTestContextAfterTestMethod(Method testMethod) 
    throws Exception {
        try {
            this.testContextManager.afterTestMethod(
                this, testMethod, this.testException);
        }
        finally {
            this.testException = null;
        }
    }

Ugly, ugly, ugly. Encapsulate!


Odds-n-Ends

There are numerous other facilities in the STF; consult Chapter 10 for the full discussion. Worth mentioning here are the following:
  1. automatic caching of loaded application contexts, to save startup time on per-method basis, once for each test fixture. Optionally instruct the framework to reload the context in case changes to state of beans in that context that are not meant to be preserved across all tests in a given fixture.
  2. JDBC utility class to facilitate querying database state.
  3. Mock objects ready-to-go for JNDI and Servlet API.
Overall, two thumbs up: this is definitely a new item in my toolbox.

Friday, August 21, 2009

A Quick Look at PesterCat

My research into UI-testing tools continues. A nice resource that has been guiding my efforts, although a bit out of date, does have a nice compilation of candidates for functional testing, load-testing, and etc.

A next candidate, PesterCat, tests at the HTTP request-response level. So it's moreless HTTPUnit with a nice GUI around it. This precludes both the "identify the element to click on" model and the "add pause here" stuff needed in both Selenium and Tellurium; imo, this is a Good Thing. The playback doesn't actually launch a browser at all, it just runs the HTTP requests (so it runs very quickly relative to the launch-a-browser approach, which is also nice) and lets you check the HTTP response for what you're looking for. Assertions can be done with Xpath, regular expressions, etc.; I've not explored that aspect fully, but was able to succeed on a simple assertion without too much trouble. So one (possibly show-stopping) downside of this is one can't test what the user-facing stuff looks like, can't test what the Javascript has done after the response is received, etc. So, perhaps not well-suited to AJAX-centric webapps.

The GUI is simple, clean and intuitive. Just like you'd expect for a $50 tool - not bogged down with too many features, so the GUI is still usable.

In theory it can be automated, since the scripts are exported to XML that can then be run from Ant scripts - but the evaluation license doesn't support saving anything so I couldn't test drive that part. The problem I see is that sending requests at the HTTP level calls for data-specific values that make tests less repeatable - e.g. instead of clicking on "My FooBar" and hitting "Delete", you might have to send a request with the ID of the object, as represented on the back end (since the ID is the unique identifying attribute, a typical setup). And, when the test is next executed, that ID could be different (especially if you're creating test data in your tests; you'd have to e.g. parse the returned HTML to figure out the ID), so you can't rely on that break of encapsulation (as usual).

PesterCat requires SSL to be turned off and must be configured as the browser proxy, but only for recording of tests, not for playback (since there's no browser needed for playback). The proxying noticeably slows things down when recording a test (that's with Firefox 2.0; this might be faster with 3.x). It is considerably easier constructing tests than with Selenium or Tellurium - since it's simply replicating HTTP requests, and you're not goofing around with HTML ID's, Xpath, pauses, etc. - but as noted, it's also more narrow in how it can be applied.

If it weren't for how slow the proxy is, this tool would actually be quite useful for monitoring HTTP requests, since you can filter out the GIF's, JS, PNG, CSS, etc. requests, all of which clutter the FireBug console pretty badly. We're also going to look into how useful PesterCat might be in fleshing out the URLs needed to construct JMeter tests, which last time I checked was one of the harder-than-I-wanted-it-to-be aspects of that load-testing tool.

A Quick Look at Tellurium

Tellurium builds on top of Selenium to present a more object-oriented abstraction, potentially resulting in more maintainable test code, and perhaps even making it easier to construct the tests to begin with; both of these things would require more familiarity on my part to judge fairly.

While Tellurium provides an IDE (TrUMP), it is not used for the same purpose as the Selenium IDE - i.e. rather than click-and-record of sequential events, it facilitates click-and-generate of (Groovy) code snippets for a given UI component or hierarchical collection of components. These become the basis of so-called "UI Module" classes, to which the developer can add methods that access the UI component. Test classes - whether in TestNG, Junit, Groovy, etc. - then leverage that set of methods to write test cases. This approach encapsulates the complex lookup (XPath, etc.) needed around UI components with access methods around those components, and supports separating the concerns of UI component vs tests written against that component. This of course sets the stage for reuse of the UI components in different test classes. Since Tellurium provides a relative location lookup model, it insulates the test cases against changes to the UI (i.e. an absolute XPath expression could break if any of the enclosing path components changes, e.g. when the UI is laid out differently).

Good Things and Random Notes

Most notably, Tellurium provides an OO approach. This is good for quick construction of test code and, more importantly, ease of maintenance on that codebase over time.

As a convenience wrapper around Selenium, Tellurium starts up the Selenium server automatically when one runs a test case.

To use TrUMP, the Tellurium IDE, SSL must be turned off; this is however probably a good idea anyway for testing purposes. The IDE is used by turning Record on, clicking on one or more UI components (e.g. a form, a table, link, etc.), and then clicking on Generate in the IDE. This (ultimately) produces a set of snippets, with any components that are part of a hierarchy in the page represented as such. Each snippet presents all attributes in the given component; you can next proceed to customize things (include/exclude snippets, rename for better abstraction, etc.). Next, you'd export the snippets to a Groovy class that represents the given UI Module, and as mentioned add methods to support access by test classes.

One roadblock that took some time to solve was dealing with iframes; the browser instance became confused, losing track of its connection to Tellurium. To deal with this type of thing, set useMultiWindows = true in the TelluriumConfig.groovy file.

When things go wrong - i.e. unexpected failures in your test case - a screenshot is automatically saved to facilitate revisiting things in a post-mortem.

Criticism

As of this writing, TrUMP is not compatible with Firefox 3.5.2; I don't know yet about other 3.x releases.

The export to Groovy is a heavy-handed "save as file" approach; I would prefer just a quick export-to-window-in-TrUMP model so I could more quickly copy-and-paste into my IDE.

I had no luck using TrUMP to capture webpage elements that were generated by post-processing AJAX, i.e. links constructed dynamically by JavaScript based on the data returned. Nor did I succeed with TrUMP in capturing Dojo menus that appear from dropdown buttons (a typical Dojo approach). In both of these cases I had to do things manually, using various approaches like Selenium, FireBug "Inspect" (in particular leveraging the Search functionality in 3.5), and old-fashioned examination of the page source. The good news is that once a particular UI component's location has been fleshed out, that effort is reusable since the resulting UI module is reusable.

While Tellurium provides an OO approach - which IMO is a Good Thing - there are as such some (not unreasonable) assumptions around just what behaviors certain objects should have. For example, it's not uncommon in Dojo widgets to find onclick event handlers (moreless) attached to TD tags; but from an OO point of view, a TD should not be clickable. This appeared to be show-stopper until I (quite by accident) stumbled on an attribute that could be retrofitted to a UI module component description that instructs it to "respond" to click events (i.e. add respond:["click"] to the generated Groovy UI Module description).

The "quite by accident" serendipity described above should be better; in fairness, Tellurium is a nascent product, but I hope that soon some first-class JavaDoc and other reference documentation is provided (or, it could be that I just need to poke around more; but, again, this shouldn't take any effort at all to find - it "should" just jump out at me).

Conclusion

All in all, I'm giving a qualified two thumbs up. Tellurium out-of-the-gate shows great potential, and I personally prefer it to Selenium. It's moreless the kind of thing that I'd probably have ended up doing myself to get the level of abstraction I'd want; I'm grateful to the folks at Google to have done it for me.

There is still a great deal of facility I've yet to explore with this tool. It appears to have a fair amount of power, and I look forward to writing up more insights and tips around Tellurium.


Tuesday, August 4, 2009

Selenium IDE Tips

This is a quick cheatsheet for various Selenium (IDE-based) commands. There's nothing too jaw-dropping here; just a collection of tips that I've found useful.

General Usage

Here is a suggested development technique with caveats. There are lots of ways to accomplish the same thing; this is just my own approach:
  1. Start by creating a directory that is named by the use case you're testing.
  2. Create a test suite in that directory, naming it TestSuite.html. Using this convention will help distinguish the test suite from its test cases, all of which should use an HTML suffix.
  3. Create test cases one by one, immediately give the test case a title via the Properties menu item (right click over the test case object), copy that name onto your clipboard before hitting OK, and then immediately save it to the use case directory by pasting that name into the file-save dialog and appending .html.
  4. Do not expect to be able to re-use general purpose test cases from other use-case directories (e.g. a login sequence) - I've had to duplicate these in each use case that I use it in. This might be just me doing something wrong; I might need to revisit it to confirm. Obviously it sure would be nice if we could reuse things this way.
  5. Do not expect to "Add Test Case" using an existing test case which you then alter, without altering the other uses of it within your test suite. Using my techniques here, one test case == one HTML file; modifying it will modify all instances of it within that use case.
  6. Save often - ideally save test cases as soon you alter them, and save the test suite as soon as you add/delete test cases to/from it. If you change test suites, the IDE will not warn you that you have unsaved changes.
The following are some coding tips. Of course, consult the Selenium documentation for the definitive and exhaustive reference:


To confirm there's a DIV that contains an onClick handler with a given name and some template text of a given value:

<div onclick="doSomething()">foo</div>

...use this:

assertElementPresent
//div[@onclick[contains(., 'doSomething') ]][contains(., 'foo')]

The @onclick can be replaced with any attribute of the given DIV tag. The bracketed "contains" conditions can be chained, and are AND'd together.

To check/uncheck a checkbox in a table with a given ID that contains a cell with the given template text:

<table id="myID">
<tr>
    <td nowrap="nowrap">
        <input type="checkbox">check me</input>
    </td> 
</tr>
</table>

...use this:

click
//table[@id='myID']//td[contains(.,'check me')]//input


Avoid the annoying 30-second delay after clicking a link that use a JavaScript command as the HREF:

<a href="javascript:doSomething()">Do Something</a>

Set the timeout as needed:

setTimeout
1000

This will still get logged in the Selenium IDE as an ERROR, which unfortunately will result in the test being marked as failed. This of course makes it difficult to determine that your tests are in fact all green, since these failures show up as red. But I've just gotten into the habit of clearing the log before I start a test suite, and just ignoring the "[error] Timed out after 1000ms" log messages. If any other log messages are at ERROR level, these tell me there is something to look into; else, I consider my test suite to have passed.

To select an item from a menu:

select
menu
label=three

...use this:

select
menu
label=three


To enter a carriage-return character in an input field with a given ID:

keyPress
inputFieldID
13

So the number 13 is the HTML code for the carriage return character. Here's a reference of more character codes.


To click on a link with the given template text:

<a href="foobar">Click Me</a>

...use this:

click
//a[contains(text(),'Click Me')]


To consume an alert:

storeAlert



Friday, July 17, 2009

Using Spring-iBatis to access LDAP - Part 2

Introduction

In the first half of this series, I described our experience with the latest Spring-LDAP module, concluding that an alternative mechanism would be needed due to deployment issues. I promised an alternative iBatis-over-LDAP solution, and in this article I provide demonstration code samples.
With this approach in place, we can view an LDAP repository as an ordinary SQL-based database, albeit with its own variant of SQL. Other advantages include the standard iBatis benefits, in particular the ability to declaratively specify SQL queries and as such manage them from a single point of change.

What follows is a description of how I put a quick-and-dirty prototype together, in particular using Spring 3.0.

Dependencies

The key to this solution is a JDBC driver that wraps LDAP repositories, created by Octet String (and now owned by Oracle). Once we have a JDBC driver, in theory we should be able to layer iBatis over it.

Download version 2.1 of that driver:
https://sourceforge.net/projects/myvd/files/

Put these jars from that download into your runtime classpath:

jdbcLdap.jar
ldap.jar

You'll need the standard set of jars for Spring 3.0 - for the minimal Spring 3.0 prototyping I've done so far, here is the set that has worked for me:

commons-lang.jar
commons-logging.jar
antlr-3.0.1.jar
org.springframework.asm-3.0.0.M3.jar
org.springframework.beans-3.0.0.M3.jar
org.springframework.context-3.0.0.M3.jar
org.springframework.core-3.0.0.M3.jar
org.springframework.expression-3.0.0.M3.jar

Add in iBatis:

iBatis-2.3.4.jar

Add what's needed to support Spring 3.0 over iBatis:

commons-dbcp.jar 
com.springsource.org.apache.commons.pool-1.4.0.jar
org.springframework.jdbc-3.0.0.M3.jar
org.springframework.orm-3.0.0.M3.jar
org.springframework.transaction-3.0.0.M3.jar

The last jar listed above is the Spring 3.0 version of transactions. The 2.5.6 version is required for Spring-LDAP, and since the classes in both jars are largely (if not completely) the same, this raises a warning flag about classloading conflicts. I didn't get far enough into a test execution to hit any problems; instead I found that deploying Spring-LDAP with Spring 3.0 AOP causes a runtime error out of the gate (during Spring initialization), and at that point I bailed out on use of Spring LDAP (please refer to my first article for the details).

Finally, though it's not necessary for this example, I include the Spring-AOP libraries, if for nothing else to demonstrate that we won't see the same runtime error as happened using Spring-LDAP with Spring 3.0. You can safely exclude these for your own prototyping of this solution (but if so, just be sure to remove the AOP references in the Spring configuration file described below):

com.springsource.org.aopalliance-1.0.0.jar
com.springsource.org.aspectj.runtime-1.6.5.RELEASE.jar
com.springsource.org.aspectj.weaver-1.6.5.RELEASE.jar
org.springframework.aop-3.0.0.M3.jar


Code Artifacts


Now you're ready to do some coding. First, construct the domain object:

package com.mybiz.model;

public class User {

    private String uid;

    public User() {}
   
    public String getUid() { return uid; }
    public void setUid(String uid) { this.uid = uid; }

    @Override
    public String toString() { return this.uid; }
}

Construct the DAO for the domain object. I recommend using an interface and its implementation - because (1) using interfaces is in general a good idea for testability, polymorphism, and etc., and (2) in particular with AOP-proxied classes in Spring, I'd favor this approach because it facilitates use of JDK dynamic proxies instead of CGLIB proxies. Use of CGLIB involves these things: (1) more library dependencies, (2)  final methods cannot participate in AOP, and (3) constructors are called twice due to CGLIB implementation details. Again, use of an interface is an optional step; if you exclude it, you'll need the CGLIB jars (which I haven't used - and plan to avoid using - so I can't point you to the exact set needed).

In either event, here's the DAO interface and implementation:

package com.mybiz.model;

import java.util.List;

public interface UserDao {
    public List getAllUsers();
}

package com.mybiz.model;

import com.ibatis.sqlmap.client.SqlMapClient;

import java.sql.SQLException;
import java.util.List;

public class MyUserDao implements UserDao { 

    // injected via Spring
    private SqlMapClient sqlMapClient;
    public void setSqlMapClient(SqlMapClient sqlMapClient) { this.sqlMapClient = sqlMapClient; }
    private SqlMapClient getSqlMapClient() { return this.sqlMapClient; }

    public List<user> getAllUsers()
    {
        String query = "selectAll";
        try {
            return (List<user>)getSqlMapClient().queryForList(query);
        } catch (SQLException e) {
            System.err.println("Query '" + query + "' failed: " + e);
            throw new IllegalStateException(e);
        }
    }
}

The database properties are described in a standalone file, to facilitate easy modifications to suit different environments (test, development, production). The key here is to specify the ignore_transactions parameter in the URL; adding the search_scope as a default scope is an optional convenience so that this won't be needed in every one of your SQL queries. Let's name the file database.properties in the com/mybiz/model directory; you'll of course need to adjust the parameters here to match your environment:

database.driver=com.octetstring.jdbcLdap.sql.JdbcLdapDriver
database.url=jdbc:ldap://127.0.0.1:389/dc=mybiz,dc=com?search_scope:=subTreeScope&ignore_transactions:=true
database.user=cn=Manager,dc=mybiz,dc=com
database.password=verycleverpassword

Following through with my convention from the first half of this series, I've bold-faced the relevant configuration pieces so you can compare how things are specified. Above, we see that the database connection parameters (LDAP URL, base node specification, principal and password) have moved from the Spring configuration (when using Spring-LDAP, as per my previous article) and into this file.

Next, layer iBatis over the database connection. We need first the SQL Map Configuration file; it doesn't need to do anything but point to your collection of SQL Maps. Let's call it SqlMapConfig.xml in the com/mybiz/model directory:

<!DOCTYPE sqlMapConfig
        PUBLIC "-//ibatis.apache.org//DTD SQL Map Config 2.0//EN"
        "http://ibatis.apache.org/dtd/sql-map-config-2.dtd">

<sqlMapConfig>
    <sqlMap resource="com/mybiz/model/SqlMap.xml"/>
</sqlMapConfig>

Here is the specified SQL Map. As per the above specification, it should be named SqlMap.xml in the com/mybiz/model directory:

<!DOCTYPE sqlMap
    PUBLIC "-//ibatis.apache.org//DTD SQL Map 2.0//EN"
    "http://ibatis.apache.org/dtd/sql-map-2.dtd">

<sqlMap namespace="user">
  <typeAlias alias="user" type="com.mybiz.model.User"/>
  <select id="selectAll" resultClass="user">
    SELECT uid  FROM dc=mybiz,dc=com  where objectclass='person'
  </select>
</sqlMap>

Here, we have bold-faced the LDAP attribute to be fetched, the node from which the search begins, and the search qualifier. While these query parameters plus the database connection parameters were all in one place using Spring-LDAP, here they've been split up into separate files. It's your judgment as to whether this is a good thing or not; personally, I consider these things as separate concerns, so I prefer it. However, you might prefer single point of change for all things database-related and/or all things configuration-related. More importantly, we can now use iBatis' parameterized substitution to declare a query, keeping it readably intact while varying its parameter values dynamically. With Spring-LDAP, we can likewise achieve dynamic query construction, but the model is more procedural.

In the above SQL Map, you'll notice the interesting use of an LDAP base specification instead of a table name (and again, you'll need to adjust this query to match up with your LDAP repository structure). The docs for the JDBC-LDAP bridge do mention that one can map things to an aliased table name, but for our purposes here, transparency and simplicity are more important.

The next step is to configure iBatis with Spring. I included transactional and AOP support, though these are optional for this exercise. Some explanations:
  • Though LDAP itself is not transactional (hence the ignore_transactions parameter above), you might be in an environment where you'll also be accessing normal relational databases, for which you'll want Spring's transactional support. Since we're now solely Spring 3.0, there is in either event no fear of classloading conflicts. However, to make this a bare-bones proof-of-concept, you can exclude the tx namespace declaration in the beans tag, the transaction manager bean and the tx:annotation-driven tag.
  • This Spring config file also configures in AOP solely to demonstrate that there is no longer any deployment conflict as was seen with combining Spring-LDAP and Spring-3.0-AOP; it's not essential for the iBatis-over-LDAP solution per se. As I mentioned above, if you specify AOP in this Spring configuration, you'll need the Spring-AOP runtime jars. Again, to make this a bare-bones proof-of-concept, you can exclude the aop namespace declaration in the beans tag and the aop:aspectj-autoproxy tag.
  • For that matter, Spring itself is not essential to layering iBatis over LDAP - I was able to remove Spring from the stack entirely and get the same successful result. However, Spring provides various benefits to an iBatis application, so I include it here for completeness (and, frankly, because I rely on Spring to help me). If you exclude it, you'll need to configure iBatis manually; I do not include those details here.
Let's name the file beans.xml, in the com/mybiz/model directory:

<beans xmlns="http://www.springframework.org/schema/beans"   
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
       http://www.springframework.org/schema/tx
       http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

    <!-- OPTIONAL: AOP -->
    <aop:aspectj-autoproxy/>

    <!-- support for referencing properties from the database.propeties file  in the data source bean, below -->
    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:com/mybiz/model/database.properties"/>
    </bean>

    <!-- OPTIONAL: transaction manager - needed more for regular databases, not for LDAP -->
    <bean id="txnMgr" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="dbcpDataSource"/>
    </bean>
    <!-- OPTIONAL -->
    <tx:annotation-driven transaction-manager="txnMgr"/>

    <!-- Apache DBCP data source, configured with properties from database.properties file -->
    <bean id="dbcpDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="${database.driver}"/>
        <property name="url" value="${database.url}"/>
        <property name="username" value="${database.user}"/>
        <property name="password" value="${database.password}"/>
    </bean>

    <!-- iBatis SQL Map client, configured with the config-file location and the data source   -->
    <bean id="sqlMapClient" class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
        <property name="configLocation" value="classpath:com/mybiz/model/SqlMapConfig.xml"/>
        <property name="dataSource" ref="dbcpDataSource"/>
    </bean>

    <!-- inject the iBatis SQL Map client into the application DAO -->
    <bean id="userDao" class="com.mybiz.model.MyUserDao">
        <property name="sqlMapClient" ref="sqlMapClient"/>
    </bean>
    
</beans>

You'll notice there are no bold-faced sections in the Spring configuration file (i.e., no application-specific configurations are specified). The database connection and database query configuration information are now in separate files dedicated to these responsibilities alone, vs being embedded in what's potentially a rather large Spring configuration. Again, your take on this may vary from mine: I consider this a good thing, but you might prefer all configuration information to be in a single file.

Finally, we provide a test driver program:

package com.mybiz.demo;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import com.mybiz.model.User;
import com.mybiz.model.UserDao;
import java.util.List;

public class IBatisOverLdapDemo {

    public static void main(String[] args) {

        ApplicationContext context = new ClassPathXmlApplicationContext(
                "com/mybiz/model/beans.xml");
        UserDao dao = (UserDao) context.getBean("userDao");

        List<user> users = dao.getAllUsers();
        for (User user : users) {
            System.out.println(user);
        }
    }
}

Executing this program, with database connection information, queries, and etc. tailored to your LDAP environment, should yield a list of attribute values as expected.

Conclusion

Using iBatis-over-LDAP instead of JNDI-LDAP, you can:
  • configure, connect to and query both RDBMS and LDAP databases in the same manner
  • be unencumbered by JNDI-LDAP programming models
  • gain all other benefits that iBatis has to offer, since the LDAP-ness has been (mostly) abstracted away. For example, fine-tuning queries for performance can be done without changing Java code.
Using iBatis-over-LDAP instead of Spring-LDAP, you gain all of the above advantages, plus these:
  • all SQL is established declaratively (vs being constructed programmatically), facilitating maintenance and debugging
  • as a result, the DAO needs fewer properties - i.e., with Spring LDAP, the DAO needs things like the LDAP base, LDAP attribute(s) to be returned and a search-filter qualifier to configure the search method. However, using iBatis, all of these things are simply part of the SQL statement specified in the SQL Map file.
Spring-LDAP has a lot to offer, including numerous features that this solution lacks, and for many applications will be very suitable. In my opinion, its strongest value-add is the compensating transactional control in an LDAP environment. If our application was more demanding than it is, I'd be motivated to dig further into the Spring-LDAP feature set and would likely be all the more impressed - in my opinion, what the Spring team has been doing is the best thing to ever happen to J2EE, and their LDAP work is likely no exception. But for our application needs - very simple CRUD + Spring 3.0 AOP - the Spring LDAP module is not the right choice. The iBatis-over-LDAP solution is clean and simple, meeting our requirements with a minimum of fuss.

Resources

Using Spring-iBatis to access LDAP - Part 1
, by Gary Horton, July 2009
MyVD Virtual Directory: JDBC-LDAP Bridge
Spring LDAP 1.3.x Reference Documentation, by Mattias Arthursson, Ulrik Sandberg, Eric Dalquist, 2009 Spring Framework
Spring Framework 3.0.M3 Reference Documentation, by Rod Johnson, et. al, 2009 Spring Framework

Thursday, July 16, 2009

Using Spring-iBatis to access LDAP - Part 1

Introduction

As the Spring team points out, the standard JNDI-LDAP programming model is cumbersome, at best - and there's no question that the Spring-LDAP module simplifies that problem considerably, though not as completely as some would prefer. In a 2008 article, Colin Lu describes a homegrown facade using an iBatis-style framework to address his concerns with Spring-LDAP. Our own experience with Spring-LDAP has been a mixed bag, and in this 2-part series, I'll discuss another approach that leverages iBatis-over-LDAP directly. This first article starts with the JNDI-LDAP approach, looks at the improvements available with Spring-LDAP, and describes our attempt to leverage Spring-LDAP in a Spring 3.0 environment. Since that attempt was not successful, my followup article will present an alternate solution that layers Spring-iBatis over LDAP.

JNDI-LDAP Example

Ill start with the plain-vanilla JNDI-LDAP model, and evolve it in two steps - first to Spring-LDAP, and secondly using Spring-iBatis. As presented by Sunil D. Patil in his 2007 article, an example class might look something like this (key pieces are bold-faced so you can keep track of them as we evolve the program):

public class PlainLDAPDemo {
    private static final String LDAP_BASE = "dc=mybiz,dc=com";
    private static final String LDAP_USERNAME = "uid";
    private static final String LDAP_PROVIDER_URL = "ldap://127.0.0.1:389";
    private static final String LDAP_SECURITY_AUTHENTICATION = "simple";
    private static final String LDAP_SECURITY_PRINCIPAL = "cn=Manager,dc=mybiz,dc=com";
    private static final String LDAP_SECURITY_CREDENTIALS = "verycleverpassword";

    public static void main(String[] args) {

        Hashtable env = new Hashtable();
        env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        env.put(Context.PROVIDER_URL, LDAP_PROVIDER_URL);
        env.put(Context.SECURITY_AUTHENTICATION, LDAP_SECURITY_AUTHENTICATION);
        env.put(Context.SECURITY_PRINCIPAL, LDAP_SECURITY_PRINCIPAL);
        env.put(Context.SECURITY_CREDENTIALS, LDAP_SECURITY_CREDENTIALS);

        DirContext ctx = null;
        NamingEnumeration results = null;
        try {
            ctx = new InitialDirContext(env);
            SearchControls searchControls = new SearchControls();
            searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);
            results = ctx.search(LDAP_BASE, "(objectclass=person)",
                    new String[]{LDAP_USERNAME}, searchControls);

            List<string> users = new ArrayList<string>();
            while (results.hasMore()) {
                SearchResult searchResult = (SearchResult) results.next();
                Attributes attributes = searchResult.getAttributes();
                Attribute attr = attributes.get(LDAP_USERNAME);
                String cn = (String) attr.get();
                users.add(cn);
            }
            Collections.sort(users);
            for (String user : users) {
                System.out.println(" UID = " + user);
            }

        } catch (NamingException e) {
            throw new RuntimeException(e);
        } finally {
            if (results != null) {
                try { results.close(); } catch (Exception ignored) {}
            }
            if (ctx != null) {
                try { ctx.close(); } catch (Exception ignored) {}
            }
        }
    }
}

As you can see, the bold-faced portions comprise the application-specific configuration, and are a small part of the overall program - most of the rest is boilerplate that can and should be factored out. That's what the Spring team has done for us.

Spring-LDAP Example

Mr. Patil does an excellent job of moving you from this verbose beginning to a lean Spring-LDAP treatment of the same program, so I won't repeat those details here. The transformation I came up with includes a test driver program, a DAO and a Spring configuration file. Here is my test driver:

public class SpringLDAPDemo {

    public static void main(String[] args) {

        Resource resource = new ClassPathResource("beans.xml");
        BeanFactory factory = new XmlBeanFactory(resource);
        UserDao ldapContact = (UserDao) factory.getBean("ldapContact");
        List<String> users = ldapContact.getAllUsers();

        Collections.sort(users);
        for (String user : users) {
            System.out.println(" UID  = " + user);
        }
    }
}

This is noticeably simpler client access. Here is the DAO:

public class UserDao { 

    private LdapTemplate ldapTemplate;
    private String ldapBase;
    private String ldapUsername; 
    private String searchQualifier; 

    public void setLdapTemplate(LdapTemplate ldapTemplate) { this.ldapTemplate = ldapTemplate; }

    public String getLdapBase() { return ldapBase; }

    public void setLdapBase(String ldapBase) { this.ldapBase = ldapBase; }

    public String getLdapUsername() { return ldapUsername; }

    public void setLdapUsername(String ldapUsername) { this.ldapUsername = ldapUsername; }

    public String getSearchQualifier() { return searchQualifier; }

    public void setSearchQualifier(String searchQualifier) { this.searchQualifier = searchQualifier; }

    public List getAllUsers() {

        return this.ldapTemplate.search(getLdapBase(), getSearchQualifier(),
                new AttributesMapper() {
                    public Object mapFromAttributes(Attributes attrs)
                            throws NamingException {
                        return attrs.get(getLdapUsername()).get();
                    }
                });
    }
}

Again, much cleaner than the original - perhaps even reusable. All of the application-specific configuration and most of the cruft has been factored out, thanks to Spring. Here's the Spring configuration that encapsulates application-specific details:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <bean id="contextSource"
          class="org.springframework.ldap.core.support.LdapContextSource">
        <property name="url" value="ldap://127.0.0.1:389"/>
        <property name="userDn" value="cn=Manager,dc=mybiz,dc=com"/>
        <property name="password" value="verycleverpassword"/>
        <property name="pooled" value="true"/>
    </bean>
    <bean id="ldapTemplate" class="org.springframework.ldap.core.LdapTemplate">
        <constructor-arg ref="contextSource"/>
    </bean>
    <bean id="ldapContact" class="UserDao">
        <property name="ldapTemplate" ref="ldapTemplate"/>
        <property name="ldapBase" value="dc=mybiz,dc=com"/>
        <property name="ldapUsername" value="uid"/>
        <property name="searchQualifier" value="(objectclass=person)"/>
    </bean>
</beans>

Note that I'm working with the latest Spring-LDAP release, 1.3.0, and as such my packages and property names are slightly different from the 2.0.1-based example in Mr. Patil's article - in particular, please note the bold-italic sections above.

The Problem

Now, I'm also working with Spring 3.0 to gain the latest-and-greatest dependency injection, AOP and iBatis integration. Since the Spring LDAP Reference document claims that "...Spring LDAP 1.3 is supported on Spring 2.0 and later", I initially assumed I'd be OK. However, when my program is deployed as a standalone, it actually fails with this:

java.lang.ClassNotFoundException: org.springframework.dao.EmptyResultDataAccessException

I used an Ivy configuration to fetch dependencies for org.springframework.ldap, and in checking back with the artifacts from that retrieval, I found the missing class is available only in the 2.5.6 distribution (in particular, I needed org.springframework.transaction-2.5.6-SEC01.jar), so I would need to do some mixing-and-matching of 3.0 and 2.5.6 jars in my deployment to get this to work.

Yes, I had misgivings about this; the last thing I want is to experiment with combinations of libraries that might cause classloading or other runtime mismatches later on - a costly misadventure in a production environment. So my next step was to test drive the Spring-LDAP standalone with the 3.0 core and the 2.5.6 transaction libraries; this worked out just fine - the missing class is of course available and my simple test program succeeded without a problem. Next, I combined this program with another test driver that applies Spring AOP - but now this message appears:

Cannot convert value of type [$Proxy15 implementing org.springframework.ldap.core.LdapOperations,org.springframework.beans.factory.InitializingBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised] to required type [org.springframework.ldap.core.LdapTemplate] for property 'ldapTemplate': no matching editors or conversion strategy

Removing Spring-LDAP from the mix of course remedies my AOP program - this is no surprise since the LdapTemplate is implicated in the error message above. Checking a little deeper, I find that the 2.5.6 transaction jar has many (if not all) of the same classes as the 3.0 transaction jar that I need to support my AOP configurations. So there's a growing doubt that I will succeed in a full-blown deployment, given the obvious concern around classloading problems, let alone how quickly I encountered the error as seen above.

Analysis of the Problem

Now, I could certainly take steps at this point to Google the error message (which, in fairness, I did - but I found no solution), experiment further, query the forums, and etcetera - but the cost/benefit of further effort here is a question mark. My mission is not to leverage Spring-LDAP; my mission is to establish a clean alternative to some existing JNDI-LDAP code. If Spring-LDAP worked without a hitch in my 3.0 deployment, I would choose that direction and move on. In fairness, and impressively enough, "it just works" has been the case for everything else I've tried so far with Spring 3.0 - injection, iBatis integration, AOP and in particular AOP-based transaction management. But here are some points about our application requirements, rationalizing why I'm reluctant to put much more effort into Spring-LDAP:
  1. Our LDAP-based use cases are dirt simple; this part of the application is not enterprise-scale. We need simple CRUD operations, nothing more. The path of least resistance here is to just wrap that CRUD in our own DAO facade and be done with it.
  2. The application itself, however, is enterprise-scale, and we're prototyping modern technologies like Spring to facilitate a rewrite of the web tier. The legacy web tier is a mix-and-match of more than a few technologies - Struts 1.x, Dojo, (massive amounts of) JavaScript, iFrames, JSTL, JSP, etc. - and is a poster child for a "how many different ways can we do the same thing?" approach. We are motivated this time to keep the technology selection down to a bare minimum, establish a minimal set of reusable patterns, choose frameworks that "just work" and think twice before going with things that have problems out of the gate. The resistance we're meeting here with Spring-LDAP in a 3.0-AOP context gives us cause for pause.
  3. While we could get around the deployment problems with Spring 3.0 AOP and 2.5.6 LDAP modules by isolating the LDAP component as a web service (e.g.), this begins to feel like overkill. This would add an extra network hop to retrieve information, and gives us one more point of failure (the new service) - this is probably not worth it, given #1 above, and especially since we'd be doing this only to enable use of Spring-LDAP. That's the tail wagging the dog.
Consider an Alternative Solution?

As such, I am now motivated to explore a second alternative to JNDI-LDAP, that of wrapping Spring-iBatis around a JDBC-LDAP bridge driver. But first, here are some points further validating my conclusion on #3 above, i.e. reluctance to use Spring LDAP in a separate service for our application:
  1. Clearly Spring-LDAP is a vast improvement over the JNDI-LDAP programming model. But, while the code is cleaner and simpler, it'sjust not that much code to worry about for our application (see #1 above).
  2. Spring-LDAP provides the ability to execute some business logic either before or after a search (but, we can also do this using regular Spring AOP, if I understand this feature correctly).
  3. We can optionally implement a method in the Spring-LDAP inner class that converts LDAP object into custom Java objects (but, we can also do this using an iBatis approach).
  4. Spring-LDAP provides the ability to create dynamic filters (but, this obscures the query being constructed since it's done in Java code. An iBatis approach simply externalizes the SQL in one place, supporting dynamic queries using parameterized templating).
  5. Likewise, basic CRUD idioms in Spring-LDAP are available; but these are also constructed procedurally in java code. Again, an iBatis approach is our preference.
  6. Spring-LDAP provides simplified "dynamic authentication" (i.e. an arbitrary user logging in to the LDAP system, as opposed to an implicitly declared principal as done in the example). However, this is not one of our requirements - the use of a single declaratively specified principal (via Spring configuration) is all we need.
  7. Spring-LDAP provides "compensating transaction support" - which is a solid improvement over the transaction-impaired LDAP environment - but we do not need this. From the Spring-LDAP docs: "The client side transaction support will add some overhead in addition to the work required by the original operations. While this overhead should not be something to worry about in most cases, if your application will not perform several LDAP operations within the same transaction (e.g. a modifyAttributes followed by a rebind), or if transaction synchronization with a JDBC data source is not required (see below) there will be nothing to gain by using the LDAP transaction support."
The list of Spring-LDAP features goes on, but for our application needs, none are compelling enough to warrant deploying it as a separate service (which for now is the only solution we have to get around the deployment issue with 3.0 AOP). In my next post, I'll present our solution that layers Spring-iBatis over the JDBC-LDAP bridge.

Resources

Spring Framework 1.3.x Reference Documentation, by Mattias Arthursson, Ulrik Sandberg, Eric Dalquist, 2009 Spring Framework
Simplify directory access with Spring LDAP, by Sunil D. Patil, 2007 JavaWorld
Extending Spring LDAP with an iBATIS-style XML Data Mapper, by Colin (Chun) Lu, 2008 JavaWorld
SpringSource Enterprise Bundle Repository
MyVD Virtual Directory: JDBC-LDAP Bridge
Simple Authentication Using Spring LDAP, by Mattias Arthursson, 2009 Spring Framework

Thursday, July 9, 2009

Spring 3.0 Cheatsheet: Integration with iBatis

This is the next in a series of tips, insights and recommendations around Spring 3.0. This time, I offer our experiences to-date around use of Spring's ORM facility, or more accurately its integration with iBatis (which is not, strictly speaking, an OR mapper).

iBatis - General
  • Use iBatis "implicit result mapping", i.e. adding an alias to each column in the select field list, to match up with Java object properties so iBatis will automatically map them for you (and as such, there's no need for resultMap constructs). For example, the following maps database column names (that might need to follow database naming conventions) to the property names of the corresponding Java object:
<sql id="allFieldsFromWebapp">
      APP_ID as appID,
      DISPLAY_NAME as displayName,
      DESCRIPTION as description,
      URL as url
    from WEBAPP
</sql>

Now, instead of referring to resultMap id's, reference the class to be populated:

<typeAlias alias="WebApp" type="com.mybiz.model.WebApp"/>
....
<select id="selectAll" resultClass="WebApp">
    select  <include refid="allFieldsFromWebapp"/>
</select>
<select id="selectById" parameterClass="String"  resultClass="WebApp">
    select <include refid="allFieldsFromWebapp"/>
    where APPID = #id#
</select>

Using the <sql>, <include> and <typeAlias> tags, we gain opportunity for reuse in the SQL map file.
  • Add JDBC type as suffix to placeholder parameters in iBatis queries, as needed:

UPDATE person SET
title = #title#,
given_names = #givenNames#,
last_name = #lastName#,
date_of_birth = #dateOfBirth:DATE#
WHERE id = #id#

In the above statement, #dateOfBirth:DATE# specifies day, month and year only - since, in this case, there's no need for hours, minutes and seconds.

  • To monitor iBatis-generated SQL queries, add log4j.logger.java.sql=DEBUG to your log4j configuration file; however, this shows the precompiled statements only, without parameter substitution.
iBatis-Spring: General Database
  • Factor connection details out of the iBatis SQL map configuration and into Spring beans configuration instead, to simplify the iBatis configuration (since these details often vary between environments (eg development, test, production)) and to consolidate application-wide configuration information into a single place. Put these connection details into e.g. an Apache DBCP DataSource bean. Use the Spring PropertyPlaceholderConfigurer bean to support the externalization.
In the Spring beans file:

    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:database.properties"/>
    </bean>

    <bean id="dbcp-dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="${database.driver}"/>
        <property name="url" value="${database.url}"/>
        <property name="username" value="${database.user}"/>
        <property name="password" value="${database.password}"/>
    </bean>

In database.properties, which exists in the default package of the classpath as per the above specification:


database.driver=oracle.jdbc.OracleDriver
database.url=jdbc:oracle:thin:@my-dev-host.mybiz.com:1521:XE
database.user=system
database.password=foobar


Now the SQL map configuration file is not concerned with connection details; it simply deals with SQL maps:

<sqlMapConfig>
  <sqlMap resource="com/mybiz/persistence/WebApp.xml"/>
</sqlMapConfig>

  • Initialize your DAO using Spring's SQL map client factory, handing it both the location of the iBatis SQL map configuration file and the DBCP data source bean as initialization parameters.
In the DAO, provide for setter-injection of an SQL map client, to be used elsewhere in your code to invoke iBatis queries:

package com.mybiz.persistence;

import com.ibatis.sqlmap.client.SqlMapClient;

public class MyDao implements DaoBase {

    private SqlMapClient sqlMapClient;

    public SqlMapClient getSqlMapClient() {
        return this.sqlMapClient;
    }

    public void setSqlMapClient(SqlMapClient sqlMapClient) {
        this.sqlMapClient = sqlMapClient;
    }

    .....

    public MyObjectCollection getAll() throws SQLException {
        MyObjectCollection coll= new MyObjectCollection ();
        coll.set((List) getSqlMapClient().queryForList("selectAll"));
        return coll;
    }
}

In the Spring beans configuration:

    <bean id="sqlMapClient" class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
        <property name="configLocation" value="classpath:com/twc/registry/persistence/WebAppDao.xml"/>
        <property name="dataSource" ref="dbcp-dataSource"/>
    </bean>

    <bean id="webapp-dao" class="com.twc.registry.persistence.WebAppDao">
        <property name="sqlMapClient" ref="sqlMapClient"/>
    </bean>


The SQL map client will automagically read in the SQL map configuration information.
Note that with the Spring XML beans file under ./WEB-INF, you must use the "classpath:" prefix to reference the iBatis SQL map configuration file.

Alternately, one can use the Spring SQL map client class instead of the iBatis version (see section 14.5 in the the Spring Reference Doc). The primary advantage here, as far as I can tell, is that formerly checked SQLExceptions are now wrapped in unchecked exceptions. However, I'd consider this optional, and for that matter undesirable in certain situations - e.g., in a RESTful web service where specific checked exceptions are caught so they can be mapped to corresponding HTTP response codes (and in just such a situation, I've chosen the iBatis SQL map client instead of the Spring version). The debate around documenting checked (and unchecked) exceptions as an important part of specifying program behavior is beyond the scope of this post; bottom line, sometimes wrapping exceptions that you can't do anything about with unchecked ones is useful, but sometimes your methods will intentionally throw their own checked exceptions that clients are meant to handle explicitly.

iBatis-Spring: Transactions
  • Use the Spring TransactionManager and annotation-driven transaction management to establish simple, declarative transactional behavior. Use the JDBC data source transaction manager unless you expect to be managing multiple resources in "global" transactions or want the application server to manager transactions (e.g. to take advantage of advanced features like transaction suspension, etc.), in which case use the JTA transaction manager. Downside of JTA transactions is lack of ability to test outside the container (since it requires JNDI and possibly other container functionality).You'll need to provide the AOP autoproxy tag and associated namespace attributes to get declarative transactions to work.
In the Spring beans file - note the transaction manager references the previously established DBCP data source:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xsi:schemaLocation="
       http://www.springframework.org/schema/tx
       http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       ">
    <bean id="txnMgr" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="dbcp-dataSource"/>
    </bean>
    <tx:annotation-driven transaction-manager="txnMgr"/>
    <aop:aspectj-autoproxy/>

  • Annotate DAO methods as appropriate (i.e. non-read-only unless you care about non-repeatable reads, etc.) with @Transactional. This tells Spring that a transaction is required and to start one if it's not already executing within a transactional context (because the default propagation is REQUIRED). That method body will execute as one transactional unit and will auto-commit if it completes successfully. If a runtime exception is thrown, the transaction will be rolled back. This becomes increasingly important as your java method executes more than one operation (multiple updates, delete + insert, etc.)
    @Transactional
    public void delete(String objId) {
        getSqlMapClient().delete("deleteById", objId);
    }
  • Use of Spring AOP means use of proxies; so, if you want to use JDK dynamic proxying instead of CGLIB proxying, your DAOs should implement an interface that specifies its public methods (as implied above with MyDao implements DaoBase). Dynamic proxying is recommended for these reasons: with CGLIB,
    • there are more library dependencies
    • final methods cannot participate in AOP
    • constructors are called twice due to CGLIB implementation details (see the Spring Reference manual, section 9.5.5).
  • While using interfaces supports annotating methods in the interface only - i.e. you don't need to maintain these in the implementation - this is considered risky by the Spring team; so, at a minimum annotate concrete classes, and optionally annotate interfaces also (the former for the functionality, optionally the latter for documentation).
  • How do you know you're getting transactional behavior? You can monitor transaction activity by turning on a log4j appender to debug for TransactionInterceptor; then, add and remove @Transactional to various methods in your interface to observe the logging statements. In a log4j.properties file:

log4j.logger.org.springframework.transaction.interceptor.TransactionInterceptor=debug


  • While transaction behaviors can be configured programmatically, this couples the code to Spring Framework and as such is not preferred.
  • Beware of self-invocation where you expect transactional behavior - unless there's already a transactional context, a new one will not be created. Consider the use of so-called AspectJ mode if you expect self-invocations to be wrapped with transactions as well (set tx:annotation-driven mode attribute to "proxy" - but this will require use of CGLIB libraries).

Resources


Spring and iBatis Tutorial
http://www.cforcoding.com/2009/06/spring-and-ibatis-tutorial.html

iBatis Wiki
http://opensource.atlassian.com/confluence/oss/display/IBATIS/Converting+iBATIS+DAO+to+Spring+DAO

iBatis FAQ
http://opensource.atlassian.com/confluence/oss/display/IBATIS/How+do+I+reuse+SQL-fragments

Spring Reference Manual
http://static.springsource.org/spring/docs/3.0.0.M3/spring-framework-reference/pdf/spring-framework-reference.pdf

Monday, July 6, 2009

Spring 3.0 Cheatsheet: Application Context

This is the first in a series of posts around patterns and recommendations for Spring 3.0.


Use a Facade to provide Application Context objects

We need to use a Spring ApplicationContext object to retrieve dependency-injected objects. But we want to encapsulate our use of Spring into a single point of change, so we use a facade to do this:

public class SpringFacade {

    private static FileSystemXmlApplicationContext applicationContext;

    public static ApplicationContext getApplicationContext(String fileName) {
        if (applicationContext == null) {
            applicationContext = FileSystemXmlApplicationContext(fileName);
        }
        return SpringFacade.applicationContext;
    }
}
.....
Service svc = (MyService)SpringFacade.getApplicationContext().getBean("myService");

Use a Factory to support both Standalone and Web Application Context objects

While we want a plain vanilla standalone Application Context in a test environment, we need a web-aware flavor in a web deployment. This has the following advantages:

  1. web-centric scopes (i.e. request and session - see Spring reference doc section 4.4.4)
  2. multiple lifecycle events after initial program startup (section 4.8.3)
  3. web-centric resources like the ServletContext (5.4)
  4. view resolvers (17.2.1)
  5. Spring-centric dependency injection in a JSF environment (19.3)
  6. and etcetera

So we need a factory mechanism to decide at runtime which way to go - assuming of course we want to test our web classes, which of course we do. After all, one of Spring's advantages is loose coupling to facilitate testability.

Now, while it's true that I could probably figure out a way (aka a hack) to convince a test class to use a web-aware Spring context, and not have to bother with this factory nonsense, I'd rather spend my time figuring out a way to do things that have good design sensibilities.

First, use a ServletContextListener to cache a web-aware implementation:

public class SpringInitializer implements ServletContextListener {

    private static WebApplicationContext springContext;
    private Logger logger = Logger.getLogger(getClass().getSimpleName());

    public void contextInitialized(ServletContextEvent event) {
       logger.info("In a web environment: get a Spring web application context");
       springContext = WebApplicationContextUtils.getWebApplicationContext(event.getServletContext());
    }

    public void contextDestroyed(ServletContextEvent event) {
        springContext = null; // facilitate garbage collection to support hot redeployments
    }

    public static ApplicationContext getApplicationContext() {
        return springContext;
    }
}

Next, configure the web deployment descriptor to (1) specify the location of the config file(s), (2) load the context listener that will access those locations and (3) load the listener (Spring reference manual, section 4.8.5):
<?xml version="1.0" encoding="UTF-8"?>
 <web-app version="2.5">
    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/myBeans.xml, /WEB-INF/yourBeans.xml, /WEB-INF/**/*Context.xml</param-value>
    </context-param>
.........
    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>
    <listener>
        <listener-class>com.mybiz.SpringInitializer</listener-class>
    </listener>
.........
  </web-app>

Now the facade takes on factory-type responsibilities to decide dynamically which context to return:
public class SpringFacade {
    private static FileSystemXmlApplicationContext applicationContext;

    public static ApplicationContext getApplicationContext() {
       ApplicationContext context = SpringInitializer.getApplicationContext();
       if (context == null) {
           factory.logger.info("Returning class-path-XML app context");
           context = new ClassPathXmlApplicationContext("WEB-INF/beans.xml");
       } else {
          factory.logger.info("Returning web app context");
       }
    }
}



Next, I'll post an approach to using iBatis with Spring.