Friday, March 19, 2010

JMS Synchronous Request/Reply

So far I've introduced some basics around JMS 1.1, including the Common Interface and Security/Concurrency. Continuing through the JMS 1.1 spec, my next drill-down will be around the Request/Reply idiom. Just so you know, these articles track my own learning curve around JMS, so you are invited to correct me and comment as needed.

Section 2.10 of the spec describes Request/Reply as an arrangement where a client sends a message to a remote service of some kind, expecting a reply. The request message header specifies a destination to which the service can (optionally!) respond with some information, or a confirmation that a requested action has been performed, or...etc. A well-behaved service not only replies, but the reply includes the ID of the request message (typically) so the requestor can correlate the reply with the previous request. JMS additionally provides basic helper implementations (QueueRequestor and TopicRequestor) that encapsulate some of the details involved here, including creating the temporary queue or temporary topic to which the reply is sent. In this series of posts, I'll trace my own prototyping, starting with the QueueRequestor (which is a synchronous messaging style) and evolving with some asynchronous alternatives, adding functionality in stages.

I'll use increasingly sophisticated levels of remote service, with various flavors of requestors associated with each. The remote services include one that simply listens on a request queue; another that does that plus notifies requestors about exceptions or invalid messages; and a third that does that plus uses message filtering on an asynchronous shared queue so that only the original requestor receives the reply. This post will describe the basic remote service and a basic requestor.

The remote service listens asynchronously on a system-wide queue and replies to requests at the reply-to destination specified in the JMSReplyTo header field. It is not terribly robust - it assumes the requestor will be sending non-null text messages; if a given message is either null or non-text, an exception is thrown. The service constructs a reply with the correlation ID (JMSCorrelationID header field) set to the request message ID (JMSMessageID header field), sends the reply and acknowledges the request.

This class provides a hook for subclasses to further decorate the message (e.g. setting properties, etc.), because we just happen to know we'll be using that in a later prototype extending this remote service.

The requestor connects to the service queue, sends a request using the QueueRequestor (which blocks on a dedicated temporary channel), and verifies that the reply has a correlation ID that matches the request message ID.

As an aside, the QueueRequestor and TopicRequestor classes are, interestingly, the only concrete classes in the JMS 1.1 distro - in other words, JMS providers need not implement anything here. Though, it wouldn't be surprising if provider-specific request/reply classes are available; it's depends on your needs as to whether proprietary mechanisms are appropriate, as always.

Here's the code for the service:

package com.mybiz.jms.activemq.server.requestreply.replier;

import com.mybiz.jms.activemq.server.requestreply.connection.AsyncConsumerConnectionStuff;
import com.mybiz.jms.activemq.server.requestreply.connection.ConnectionStuff;
import com.mybiz.jms.activemq.server.requestreply.util.MessageUtil;
import org.apache.activemq.ActiveMQConnection;

import javax.jms.Connection;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.MessageProducer;
import javax.jms.TextMessage;

public class RemoteService implements MessageListener {

    // using an ActiveMQ provider
    public static final String url = ActiveMQConnection.DEFAULT_BROKER_URL;
    public static final String serviceQueueName = "ServiceQueue";

    // ConnectionStuff is just a convenience composite object that 
    // encapsulates a destination, the session for that destination and 
    // the connection for that session.
    private ConnectionStuff serviceStuff;

    public RemoteService() {}

    public RemoteService(String theUrl, String serviceQueueName)
            throws JMSException {

        // establish the connection, session and destination, and start the connection
        serviceStuff = new AsyncConsumerConnectionStuff(theUrl, serviceQueueName, this);
        System.out.println("Service is waiting for a request...");
    }

    private boolean shutdown = false;
    protected void shutdown() throws Exception {

        // all done - release connection. No need to release any other objects,
        // that's done automatically when releasing the connection.
        getServiceQueueConnection().close();
    }

    protected Connection getServiceQueueConnection()
    {
        return serviceStuff.getConnection();
    }

    // assumes non-null TextMessage; sends a reply to the specified reply-to. 
    // Throws an exception if any problem occur.
    public void onMessage(Message requestMsg) {

        try {

            TextMessage requestTextMsg = (TextMessage) requestMsg;
            if (requestTextMsg.getText() == null) {
                   throw new IllegalStateException("NULL message received");
               }

            // MessageUtil is a convenience class that prints out 
            // information about messages, and provides various other conveniences
            MessageUtil.examineReceivedRequest(requestTextMsg);
            sendReply(serviceStuff, requestMsg, "Normal Service Reply",
                    requestMsg.getJMSReplyTo());

        } catch (Exception e) {

            throw new IllegalStateException(e);
        }
    }

    // Send the given reply to the given destination and acknowledge the given request message.
    protected void sendReply(ConnectionStuff stuff,
                             Message requestMessage, String reply,
                             Destination destination) throws JMSException {

        // construct and send the reply, with correlation ID set to message ID
        TextMessage replyMsg = stuff.getSession().createTextMessage();
        replyMsg.setJMSCorrelationID(requestMessage.getJMSMessageID());
        replyMsg.setText(reply);

        // allow subclasses to do extra stuff; default impl does nothing
        replyMsg = decorateMessage(requestMessage, replyMsg);

        MessageProducer producer = stuff.getSession().createProducer(destination);
        producer.send(replyMsg);
        MessageUtil.examineSentReply(replyMsg);

        requestMessage.acknowledge();
    }

    /**
     * Provides support for subclasses to further fill in the reply, e.g. 
     * by setting various properties. We just happen to know that this will 
     * be needed for the full demo - see MessageSelectorRemoteService.
     */
    protected TextMessage decorateMessage(Message requestMessage, 
            TextMessage replyMsg)
            throws JMSException {

        return replyMsg;
    }

    public static void main(String[] args) throws Exception {

        RemoteService service = new RemoteService(url, serviceQueueName);
        run(service);
    }

    /**
     * Runs the given remote service; subclasses should call this to run 
     * themselves - so that in case this demo is extended (which it will be), 
     * they won't need to duplicate the same execution code.
     */
    public static void run(RemoteService service) throws Exception {

        while (!service.shutdown) {
            // this service could support receiving a message that it reacts to by
            // setting shutdown to true; or you could set it manually in 
            // the debugger; or you could just not worry about it. This is 
            // just a demo.
            Thread.currentThread().sleep(1000);
        }
        service.shutdown();
        System.out.println("Done - exiting");
    }
}

And, here's the requestor code. The comments explain how to demo things (assuming the ActiveMQ provider is already running), and describes what we might think about to improve the functionality:

package com.mybiz.jms.activemq.server.requestreply.requestor.sync;

import com.mybiz.jms.activemq.server.requestreply.connection.SyncRequestorConnectionStuff;
import com.mybiz.jms.activemq.server.requestreply.replier.RemoteService;
import com.mybiz.jms.activemq.server.requestreply.util.MessageUtil;

import javax.jms.JMSException;
import javax.jms.TextMessage;

public class VanillaSyncRequestor {

//    Demo:
//
//    Start the remote service, then run the requestor - console printouts 
//    reflect request and reply on both sides of messaging provider.
//
//    Now kill the remote service, then run the requestor - request is made, 
//    but no reply just yet. Start up the service "after a while", and the 
//    reply is received.
//
//    What does it need:
//
//    This is a simple request-reply demonstration. It will not know if 
//    the remote service throws an exception (which can happen, as noted 
//    above, if the request is either null or is not a text message). Though 
//    it has a dedicated temporary channel, it can possibly block for indefinite
//    periods of time - e.g. if the service goes down, it will block until that 
//    service is back online; worse, if the service throws an exception, it can 
//    block "forever". Blocking until the service comes back online can be 
//    considered a good thing, i.e. the request will "eventually" be handled -
//    and, it can be considered a bad thing, e.g. if many clients are blocking 
//    on a wait for an offline service, this consumes system resources.

    /**
     * Send a text message using the given connection stuff, wait synchronously 
     * for a reply.
     */
    public void sendRequest(SyncRequestorConnectionStuff stuff) throws JMSException {

        TextMessage requestMessage = stuff.getSession().createTextMessage(
                "Do this as soon as you're online!");

        // this convenience method sends the request, at which point the 
        // JMSMessageID is set in the header of that message; it acknowledges 
        // the reply and then confirms that the JMSCorrelationID in the 
        // reply matches the JMSMessageID, throwing an exception if that is 
        // not the case.
        MessageUtil.processRequestAndReply(stuff.getRequestor(), requestMessage);
    }

    public static void main(String[] args) throws JMSException {

        VanillaSyncRequestor requestor = new VanillaSyncRequestor();

        // get the connection, session and QueueRequestor - on return, 
        // the connection will have been started
        SyncRequestorConnectionStuff serviceQueueStuff =
                new SyncRequestorConnectionStuff(
                RemoteService.url, RemoteService.serviceQueueName);

        requestor.sendRequest(serviceQueueStuff);

        // requestor is done - release connection. No need to release any 
        // other objects, that's done automatically when releasing the connection.
        serviceQueueStuff.getConnection().close();
    }
}

I'm omitting the code for MessageUtil and ConnectionStuff - these can be regarded as black-boxes that "just work" as noted, for purposes of this article. But I will offer the printouts that occur when the suggested demo steps are taken:

1 - Start up the ActiveMQ provider (you'll of course need to download and install this first):

$ cd $ACTIVEMQ_HOME/apache-activemq-5.3.0/bin 
$ ./activemq
.........
(lots of noisy startup messages, omitted here...)
Loading message broker from: xbean:activemq.xml 
..........
INFO | Listening for connections at: tcp://localhost:61616 
.......... (etcetera etcetera)

2 - Start the service (I've done this in my IDE; just run the RemoteService class):

Mar 19, 2010 12:15:47 PM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://localhost:61616
Service is waiting for a request...

3 - Run VanillaSyncRequestor; observe that it sends a request and immediately gets a reply:

INFO: Successfully connected to tcp://localhost:61616
12:16:06.937 Sending request for service 'Do this as soon as you're online!'...
12:16:08.750 Request for service 'Do this as soon as you're online!' has been sent...
    Message ID: ID:080301h0114dl-3388-1269022561656-0:0:1:1:1
    Correlation ID: null
    Reply to:   temp-queue://ID:080301h0114dl-3388-1269022561656-0:0:1
----------------------------------------------
12:16:08.750 Received reply 'Normal Service Reply'
    Message ID: ID:080301h0114dl-3375-1269022545562-0:0:1:1:1
    Correlation ID: ID:080301h0114dl-3388-1269022561656-0:0:1:1:1
    Reply to:   null
----------------------------------------------

4 - Stop the service; run the requestor again - note that it sends the request but then appears to stop - in fact, it's blocking, waiting for the service to come back online:

INFO: Successfully connected to tcp://localhost:61616
12:30:07.875 Sending request for service 'Do this as soon as you're online!'...

5 - Start up the service - the printout there looks the same as above, since it grabs the message waiting in the queue and replies. At that point, thankfully, the requestor now completes, albeit "some time later", as can be seen by the timestamps:

12:31:57.937 Request for service 'Do this as soon as you're online!' has been sent...
    Message ID: ID:080301h0114dl-3906-1269023399953-0:0:1:1:1
    Correlation ID: null
    Reply to:   temp-queue://ID:080301h0114dl-3906-1269023399953-0:0:1
----------------------------------------------
12:31:57.937 Received reply 'Normal Service Reply'
    Message ID: ID:080301h0114dl-3964-1269023512671-0:0:1:1:1
    Correlation ID: ID:080301h0114dl-3906-1269023399953-0:0:1:1:1
    Reply to:   null
----------------------------------------------

As the comments in VanillaSyncRequestor indicate, there is room for improvement. Next, I'll demonstrate a client that blocks "forever", and take some steps to at least partially mitigate that. Then we'll move on to use asynchronous messaging in a request/reply approach.

References

JMS Home Page
JMS Downloads
JMS FAQ
JEE 5 JMS Tutorial
JEE 5 JMS Tutorial - Example Code
JEE 5 Online Javadoc
JMS 1.1 Online Javadoc
Apache ActiveMQ

Monday, March 15, 2010

Case Study: Add Functionality Without Changes to Code

What I'll describe here is an exercise I've completed that adds functionality to an existing codebase without changing that codebase. The product is not dynamically "pluggable", service-oriented, etc., so I don't have those levers to address the problem; instead, I've used Aspect Oriented Programming (AOP), JSF phase listeners, and the Observer design pattern to add to the codebase in a modular way. This gives me an easy way to remove that code if that's called for - and, for this exercise, that's the primary goal.


Our team's product cycle is transitioning with one release in test cycle, moving soon to a Beta audience; and with some new functionality targeted for the next release. These releases are being managed in separate SCM trees; however, due to various infrastructure limitations, we're being asked to wait on checking in changes to the test-cycle branch. But I don't want this to stop me from moving forward on that release.

There are various well-known problems, of course, if one makes "too many" changes to a  codebase before checking things in:
  1. There could be a conflict-resolution exercise between my changes and my teammates' changes to the same files.
  2. If a bug is found in Beta, that code will need a bugfix and a patch - but if I've intermingled new stuff in the code needing bugfixes, I have a potentially error-prone, tedious and time-consuming version control exercise.
There are SCM strategies to address #2; but for the sake of argument, let's just say that my constraint is to develop the new functionality with minimal - if not zero - impact to existing code, to avoid both potential problems from above.

For that matter, I wanted to challenge myself to see just how much I could do with this type of constraint - so I gave myself some "rules":
  1. keep the code changes as modular as possible so they can easily be added, for that matter easily removed if called for;
  2. ideally, no changes at all are made to existing code;
  3. if for some reason, I must change existing code, it must be done in such a way that new classes, declarations, etc. can be introduced, but existing code must not make reference to the new artifacts
So, e.g., I can use aspects, includable XML files, listeners of various flavors, and the like. Looking more closely at my "rules", it appears that #2 and #3 actually are the "how-to" supporting the "what" of #1 - i.e., the true goal here is to make it easy to add or remove the functionality if called for.

Here's the new functionality to be added:
  1. At various points during the customer's use of the product, a certain process should be launched to accomplish a particular goal. I'm intentionally being vague since the details don't really matter here.
  2. For each version of the product, that process should be executed just once without any confirmation from the user that it should proceed; but once it's been executed that first time (for a particular product release), the user should be issued some kind of popup confirmation dialog (this is a webapp UI) that offers a choice: allow the process to proceed with due caution, or just bail out.
  3. Every time a new product release is installed, the warning mechanism resets, i.e. the very next time the user launches the process, no such confirmation dialog appears.
The relevant pieces of the technology stack that I'll be "modifying" includes:
  1. A JSF-based web tier, using in particular the IceFaces component set;
  2. Spring 3.0
Breaking things down, it seems I'll need to do things like this:
  1. Persist some information around the product version, to be generated at build time and deployed with product installation;
  2. intercept the rendering of the UI button that launches the process so that the confirmation dialog can be associated with it, such that when the button is clicked, the confirmation will appear (giving the user a choice of proceeding or not) - but the confirmation appears only if the process has already executed for this product release;
  3. persist information after each process execution indicating this execution has occurred for a given product installations; and
  4. interpret the lack of that persisted information to indicate the process has not executed for this installation. This will be the initial condition, and the condition after each new product release is installed.

What I did to address #1 is trivial, and frankly out of scope for the purposes of this post. Likewise, #4 is reasonably straightforward; no additional details are needed around this.

To address #2 - intercept the rendering of the user action - my initial instinct tells me this sounds like an aspect...well, to be honest, I was looking for some excuses to use aspects - not only because I think this is a good approach to adding functionality in a modular way, but because I want to get better at AOP. A solution for #3 might also use an aspect, so that after the process has been run, the process-execution-history is updated.

As it turned out, the most interesting problem was intercepting the rendering of the UI button to attach a confirmation dialog dynamically. Note that sometimes the confirmation should be there - if the process has already been run - and sometimes, the process should just get launched without any warning to the user.

Let's first establish the confirmation dialog to be used. I'm using IceFaces, which provides a panelConfirmation tag that I use like this:

    <!--
    Warn user if process has already executed for this release
    -->
    <ice:panelConfirmation 
       id="warnProcessExecuted" 
       message="#{msgs['confirm.rerun']}"
       acceptLabel="Yes" cancelLabel="No"/>

The straightforward way to use this is to simply add it as an attribute to the existing IceFaces button that launches the process:

<ice:commandButton id="launchProcess"
     value="Launch Process"
     immediate="true"
     confirmationPanel="warnProcessExecuted"
     actionListener="#{eventsManager.startProcess}"/>

But this violates my self-imposed constraint described by rule #2 above: "ideally, no changes at all are made to existing code".  So my initial thought is to do this:
  1. intercept the handling of the action after user clicks on the button;
  2. find the JSF component associated with the button; and
  3. dynamically add the confirmationPanel attribute to that button component.
Doing things this way also would factor out the conditional part: if the process has not executed yet, there's no need to add the confirmation. But it turns out that trying to add the attribute during the action handling is too late - what I want is to intercept the rendering of the button. This sounds like I could use a JSF phase listener that listens for the beginning of the RENDER_RESPONSE phase to do this. This worked out just fine, but the thing about it that bothered me was that this happens several times for each action, and the method that I use to attach the attribute gets called each time. In this case, no harm is done; but I'd like to find a more general solution so my code execution is more deterministic and less at the mercy of the JSF lifecycle - bottom line, so that the method of interest is called only once at the beginning of the RENDER_RESPONSE phase. Here's how I did it:

First I create a reusable hierarchy of JSF phase listeners. The superclass provides the PhaseListener implementation, and additionally provides a registration mechanism - with which it remembers any interested listeners for a given phase. Each registered listener implements a PhaseObserver interface, and is notified at the beginning and end of each JSF phase:

public class PhaseMonitor implements PhaseListener {

    private static Map<JsfPhaseId, Set<PhaseObserver>> observers =
        new HashMap<JsfPhaseId, Set<PhaseObserver>>();

    // notify all registered observers for this phase that the phase has begun
    public void beforePhase(PhaseEvent event) {

        JsfPhaseId phaseId = JsfPhaseId.getJsfPhaseId(event.getPhaseId());
        Set<PhaseObserver> these = observers.get(phaseId);
        if (these != null) {
            for (PhaseObserver observer : these) {
                observer.notifyBeforePhase(phaseId);
            }
        }
    }

    // notify all registered observers for this phase that the phase has ended
    public void afterPhase(PhaseEvent event) {

        JsfPhaseId phaseId = JsfPhaseId.getJsfPhaseId(event.getPhaseId());
        Set<PhaseObserver> these = observers.get(phaseId);
        if (these != null) {
            for (PhaseObserver observer : these) {
                observer.notifyAfterPhase(phaseId);
            }
        }
    }

    // observer pattern: interested observers implement the PhaseObserver interface and register their interest
    public static void register(JsfPhaseId phase, PhaseObserver observer) {

        Set<PhaseObserver> these = observers.get(phase);
        if (these == null) {
            these = new HashSet<PhaseObserver>();
        }
        these.add(observer);
        observers.put(phase, these);
    }

    protected PhaseId getMyPhaseId() { // subclasses will override this for each JSF phase
        return null;
    }

    public PhaseId getPhaseId() {
        return getMyPhaseId();
    }
}


The JsfPhaseId is a first-class enum, provided not only so the observers can reference a simple enum constant (which the JSF PhaseID does not provide - it is not an enum (!!)), but so that client code need not be tightly coupled to JSF. Granted, clients will use the JsfPhaseId, which uses JSF, so the deployment will be coupled to JSF - but at least I've encapsulated my usage of JSF by providing this facade:

public enum JsfPhaseId
{
    APPLY_REQUEST_VALUES, INVOKE_APPLICATION, PROCESS_VALIDATIONS,
    RENDER_RESPONSE, UPDATE_MODEL_VALUES, RESTORE_VIEW, UNKNOWN;

    public static JsfPhaseId getJsfPhaseId(PhaseId phaseId) {

        if (phaseId.equals(PhaseId.APPLY_REQUEST_VALUES)) {
            return JsfPhaseId.APPLY_REQUEST_VALUES;
        } else if (phaseId.equals(PhaseId.INVOKE_APPLICATION))  {
            return JsfPhaseId.INVOKE_APPLICATION;
        } else if (phaseId.equals(PhaseId.PROCESS_VALIDATIONS)) {
            return JsfPhaseId.PROCESS_VALIDATIONS;
        } else if (phaseId.equals(PhaseId.RENDER_RESPONSE)) {
            return JsfPhaseId.RENDER_RESPONSE;
        } else if (phaseId.equals(PhaseId.UPDATE_MODEL_VALUES)) {
            return JsfPhaseId.UPDATE_MODEL_VALUES;
        } else if (phaseId.equals(PhaseId.RESTORE_VIEW)) {
            return JsfPhaseId.RESTORE_VIEW;
        }
        return JsfPhaseId.UNKNOWN;
    }
}

Each phase-specific subclass of PhaseMonitor does this:

public class RenderResponsePhaseMonitor extends PhaseMonitor {

    private PhaseId phaseId = PhaseId.RENDER_RESPONSE;
    protected PhaseId getMyPhaseId() {
        return phaseId;
    }
}

This is repeated for the other JSF phases. Now I need to add these phase listeners to the code, but in a way that minimizes impact that existing code. So, I do not want to modify my existing JSF configuration file by declaring these listeners; instead, I add a new config file using the web-tier deployment descriptor (web.xml):

<context-param>
        <param-name>javax.faces.CONFIG_FILES</param-name>
        <param-value>/WEB-INF/faces-config-application.xml, /WEB-INF/faces-config-listeners.xml</param-value>
</context-param>

That config file looks like this:

<faces-config xmlns="http://java.sun.com/JSF/Configuration">
    <!-- listen for phase events to facilitate phase listening, notification -->
    <lifecycle>
        <phase-listener>com.mybiz.web.jsf.lifecycle.ApplyRequestValuesPhaseMonitor</phase-listener>
        <phase-listener>com.mybiz.web.jsf.lifecycle.InvokeApplicationPhaseMonitor</phase-listener>
        <phase-listener>com.mybiz.web.jsf.lifecycle.ProcessValidationsPhaseMonitor</phase-listener>
        <phase-listener>com.mybiz.web.jsf.lifecycle.RenderResponsePhaseMonitor</phase-listener>
        <phase-listener>com.mybiz.web.jsf.lifecycle.RestoreViewPhaseMonitor</phase-listener>
        <phase-listener>com.mybiz.web.jsf.lifecycle.UpdateModelValuesPhaseMonitor</phase-listener>
    </lifecycle>
</faces-config>

The interested observers implement this interface:

public interface PhaseObserver {

    public void notifyBeforePhase(JsfPhaseId phaseId);
    public void notifyAfterPhase(JsfPhaseId phaseId);
}

With these levers in place, I can now implement an observer that will be notified once and only once at the beginning of the RENDER_RESPONSE phase, at which time it will determine if a confirmation for the user is needed; if so, it will find the JSF component of interest and attach the confirmation panel, and, if the user elects to proceed, updating some persistent history "somewhere" with information about this execution; else it sets the attribute value for the confirmation panel to an empty string so that no such dialog pops up. Since I don't want to minimize my changes to existing code, I add this functionality with an aspect:

@Aspect
public class ExecutionAspects implements PhaseObserver {

    public ExecutionAspects() {
        PhaseMonitor.register(JsfPhaseId.RENDER_RESPONSE, this);
    }

    private ExecutionHelper executionHelper;  // injected via Spring
    public void setExecutionHelper(ExecutionHelper theHelper) {
        executionHelper = theHelper;
    }

    /**
     * Establish a pointcut that describes a method which we know will be called when the button is rendered
     */
    @Pointcut("execution(* com.mybiz.view.ViewHelper.isLaunchProcessButtonShowing(..))")
    public void isLaunchProcessButtonShowing() {
    }

    // here's how we enforce the "once and only once" constraint:
    private boolean needConfirmationForThisRequest = true;
    public void notifyBeforePhase(JsfPhaseId phaseid) {
        needConfirmationForThisRequest = true;
    }
    public void notifyAfterPhase(JsfPhaseId phaseid)
    {
        needConfirmationForThisRequest = false;
    }

    /**
     * Intercept rendering of button to detect whether or not the process has already been run for this release
     */
    @Around("isLaunchProcessButtonShowing()")
    private boolean interceptRender(ProceedingJoinPoint pjp) throws Throwable {
        if (needConfirmationForThisRequest)
        {
            // get UI component, add confirmation panel if needed
            HtmlCommandButton button =
                    (HtmlCommandButton) FacesUtils.findComponent(FacesUtils.getFacesContext().getViewRoot(),
                            "launchProcess");  // search from root of JSF component tree for the launch button ID
            if (executionHelper.getProcessHasExecutedForThisProduct()) {
                // attach the confirmation
                button.setPanelConfirmation("warnProcessExecuted");
            } else {
                // set the confirmation attribute to an empty string so no warning will appear
                button.setPanelConfirmation("");
            }
            needConfirmationForThisRequest = false;
        }
        // return value as normal
        return (Boolean)pjp.proceed();
    }

    /**
     * Establish a pointcut that intercepts the user action of launching the process
     */
    @Pointcut("execution(* com.mybiz.view.EventsHelper.startProcess(..))")
    public void startProcess()
    {
    }

    /**
     * Intercept launching of process to facilitate updating the execution history after it's done
     */
    @Around("startProcess()")
    private void interceptProcess(ProceedingJoinPoint pjp) throws Throwable
    {
        pjp.proceed();
        // update history so user can be warned that it's already been run (next time it's requested)
        executionHelper.updateConfMergeHistory();
    }
}

Now the aspect needs to be added to runtime execution; I already have a Spring context file for the existing product functionality, but again I don't want to change that file. Instead, I modify the web-tier deployment descriptor to add a new one:

<context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>/WEB-INF/beans-all.xml, /WEB-INF/beans-execution.xml</param-value>
</context-param>

That Spring context file manages the ExecutionAspects class, supplying the ExecutionHelper dependency:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
       ">

    <!--
    Configuration file used to maintain execution information - i.e. as a control for when the process
    is run with or without warning, we need to maintain information about what release is this
    product, and compare that to what release was the last execution run against.
    -->
    <bean id="executionHelper" scope="session" class="com.mybiz.aspects.ExecutionHelper">
    </bean>

    <aop:aspectj-autoproxy proxy-target-class="false"/>

    <bean id="execution-aspect" class="com.mybiz.aspects.ExecutionAspects">
        <property name="executionHelper" ref="executionHelper"/>
    </bean>

</beans>

The ExecutionHelper is of minimal interest here; it simply does CRUD on "some persistence mechanism" to manage information about what is the current product release and what particular release has the process last been executed against. It provides a convenience method (getProcessHasExecutedForThisProduct(), as seen above) encapsulating all of that information with a boolean indicating exactly what it's name suggests.


So let's revisit my initial goals and see how well I've done:
  1. keep the code changes as modular as possible so they can easily be removed if called for;
  2. ideally, no changes at all are made to existing code;
  3. if for some reason, I must change existing code, it must be done in such a way that new classes, declarations, etc. can be introduced, but existing code must not make reference to the new artifacts (this facilitates rule #1)
#1: are my changes modular? In the sense that they can be removed very easily if needed, yes - here's what I'd need to do to remove this functionality: change this -->

<param-value>/WEB-INF/beans-all.xml, /WEB-INF/beans-execution.xml</param-value>

to this -->

<param-value>/WEB-INF/beans-all.xml</param-value>nclude src="/WEB-INF/includes/panel-confirmation.jspx"/>

Without the single additional declaration of Spring context that attaches the new aspect to the runtime, the aspect will not get instantiated, let alone executed. All of the existing code - well, most of it anyway - remains untouched. The new code can be left as an addition to the codebase, ready for activation whenever product management calls for that.

You'll notice I said most of the existing code is unchanged. Beyond the addition of the 2nd Spring context, as noted here, and the additional JSF Config file that declares the hierarchy of phase listeners, I did need to make one change to the JSF file that declares the command button; I needed to include the new JSF file that declares the confirmation within the same form. Apparently this is a constraint of the component set, and it is not surprising. So I ended up doing this:


<ice:commandButton id="launchProcess"
    value="Launch Process"
     immediate="true"
    actionListener="#{eventsManager.startProcess}"/>
             
 <!-- confirmation panel must be in same form as the button that references it. The
  "launchProcess" button, above, is managed dynamically to point to this confirmation
  in an aspect. For better modularity, include the panelConfirmation snippet:
  -->
  <ui:include src="/WEB-INF/includes/panel-confirmation.jspx"/>

So I didn't quite succeed with rule #2 -- I did have to make some changes to existing code. But for all intents and purposes, it's not an issue -- since the change to the code is a declaration that is not referenced by existing code. I'd argue likewise for the addition of the JSF Config file with the hierarchy of phase listeners and built-in observer mechanism - this is, in my opinion, a worthwhile addition to any JSF application, one that I'll likely continue doing from this point forward. I just happened to leverage it with an aspect that registers as an interested observer of the RENDER_RESPONSE phase. That is the essence of modularity.

Again, not quite successful with rule #3 - since the changes to the web.xml do make reference to the new JSF Config and Spring context files. But again, in spirit that goal was only in place to facilitate success on rule #1 - and bottom line, I can add or remove this new functionality without breaking a sweat.

Friday, March 12, 2010

JMS Security, Concurrency and Triggers

This is the 2nd in a series of posts around JMS 1.1, focusing on things that can guide design decisions when putting a JMS application together. In this writeup, I'll discuss security, concurrency and the use of triggers.


Security

JMS 1.1, section 2.7: JMS does not provide features for controlling or configuring message integrity or message privacy. It is expected that many JMS providers will provide such features. It is also expected that configuration of these services will be handled by provider specific administration tools. Clients will get the proper security configuration as part of the administered objects they use.

Decision: does the application need authentication, confidentiality, etc.?

Idioms: Any client authentication is handled by the Connection object. A JMSSecurityException is thrown when authentication credentials are rejected by the provider, or whenever any security restriction prevents a method from completing.

Caveat: Any security measures applied will not be portable across JMS providers. Remember that authentication and confidentiality exercises are relatively heavyweight; combined with the heavy lifting of setting up network connectivity, this motivates minimizing the number of connections in use (and, for that matter, of using connection caches).


Concurrency

Of the six top-level JMS objects, only Destination, ConnectionFactory and Connection are intended for multi-threaded access.  The Session, MessageProducer and MessageConsumer are intended for single-thread use only. See the JMS 1.1 spec, section 2.8 for an in-depth rationale around this restriction - bottom line, this makes it easier for the typical JMS client, with concurrency possible if needed by using multiple sessions.

JMS providers must prevent concurrent access to a given client's state that could result in messages being lost or processed redundantly - whether that's done by an exception being thrown, or blocking the offending client, or otherwise.

Idioms: Note that this doesn't mean multiple threads can't use a given Session object - it just means the developer must ensure the access is not concurrent. Using one thread only is the simplest way to ensure this; otherwise, explicit synchronization is called for. The standard approach to setting up asynchronous delivery is using a single thread for setup while the Connection is stopped, then use that same thread to start the Connection. Concurrent access to a session's producers and consumers is also not allowed. If a client uses one thread to produce messages and other threads to consume them, use of a separate session for the producing thread is called for. Once a connection has been started, do not use any Session method except close; using a separate thread of control to close a Session is allowed. This restriction applies in particular to setting up multiple message listeners - all of these must be established before the connection is started.

Caveat: While a client may have multiple sessions, JMS does not define the behavior around concurrent QueueReceivers for the same Queue; relying on a given provider's support for this is not portable. Note that the Session serializes execution of asynchronous deliveries, using a single thread to run all MessageListeners.  One consequence of this is that a session with asyncronous listeners cannot be used to also receive messages synchronously.

Recommendation: Consider the effect of the session's serial execution on your target throughput. For higher throughput via concurrency, use multiple sessions. However, note that it is not considered reliable to use a single consumer with application-level multi-threading logic to concurrently process messages from a topic (due to lack of adequate transaction facility in JMS). JMS does, however, provide a special facility for creating MessageConsumers that can consume messages concurrently, via application server support; see section 8 in the spec. The application developer presents a single-threaded program if using this facility.


Triggers

A trigger is e.g. a threshold of waiting messages, a length of time that has gone by, a time of day, etc., which is used to wake up a client so it will process any waiting messages. Keep in mind that any such mechanism, if available in the provider you use, is not specified by the JMS spec, and as such is not portable.

Caveat: any trigger mechanisms will not be portable across JMS providers.


In the next several posts in this series, I'll discuss things like request/reply, message ID, timestamp, message expiration and message priority.

JMS Common vs Domain-Specific Interfaces

Reading through the JMS 1.1. spec, I'm motivated to in fact re-read it, since certain concepts are referenced before being introduced. This isn't unusual for a spec, and that's not a critique; but bottom line, I decided to go through the spec a second time, this time collating related information around given concepts in one place, and ordering things in a way that doesn't assume prior knowledge.

This is the first in a series of posts in which I'll focus on concepts and facilities from the spec that call for some kind of design decision (as opposed to more general behaviorial issues, etc.) - since I'm in the process of making those kinds of decisions with my current development efforts.

I'll start with some introductory basics around use of JMS; listing decisions to be made with guidance around each, plus recommended practices, programming idioms and caveats. First I address only the use of "JMS Common" vs "Domain-Specific" interfaces, since that writeup is long enough for its own post.



JMS supports both queue-based (aka Point-to-Point, or PTP) and topic-based (aka Publish/Subscribe, or Pub/Sub) models (aka "domains"). As of JMS 1.1, so-called "common interfaces" are available that encapsulate this distinction (aka "unification of messaging domains") - while the legacy domain-specific APIs are preserved for backwards compatibillity.

As per JMS 1.1, section 2.5: The JMS common interfaces provide a domain-independent view of the PTP and Pub/Sub messaging domains. JMS client programmers are encouraged to use these interfaces to create their client programs.

Here's a table illustrating the difference:


JMS Common Interfaces
PTP-specific
Pub/Sub-specific
ConnectionFactory
QueueConnectionFactory
TopicConnectionFactory
Connection
QueueConnection
TopicConnection
Destination
Queue Topic
Session
QueueSession TopicSession
MessageProducer
QueueSender TopicPublisher
MessageConsumer
QueueReceiver, QueueBrowser
TopicSubscriber

And here are some definitions:

ConnectionFactory - an administered object used by a client to create a Connection
Connection - an active connection to a JMS provider
Destination - an administered object that encapsulates the identity of a message destination
Session - a single-threaded context for sending and receiving messages
MessageProducer - an object created by a Session that is used for sending messages to a destination
MessageConsumer - an object created by a Session that is used for receiving messages sent to a destination

Decision: which one to use?

Recommendation: use the JMS common interface, in particular if you wish to enclose send/receive of messages from both domains (i.e. queue and topic) within a single transaction. From section 11.4.1: (use of JMS Common API) simplifies the client programming model, so that the client programmer can use a simplified set of APIs to create an application...using (JMS Common) methods, a JMS client can create a transacted Session, and then receive messages from a Queue and send messages to a Topic within the same transaction. There are additional benefits to providers in terms of opportunities for certain optimizations in their implementations.

Caveat: Be aware that in future JMS releases, the domain-specific APIs may be deprecated. Keep in mind that PTP and Pub/Sub messaging system behaviors will of course be different, even though you're using the same API, since the semantics of each domain are different. An unpleasant side-effect of this is the fact that, since the common interface defines e.g. some queue-specific methods - and since the topic-specific classes inherit from that interface - there are some methods available that just aren't appropriate. If the application calls any of these methods, an IllegalStateException is thrown. Why this isn't an OperationUnsupportedException is another question.

Here is the list of those methods:

Interface Method
QueueConnection createDurableConnectionConsumer
QueueSession createDurableSubscriber
createTemporaryTopic
createTopic
unsubscribe
TopicSession createQueueBrowser
createQueue
createTemporaryQueue

Note that there are also JMS Common Interfaces available for JTS services, as described in section 8.6 of the spec. However, be aware that JMS providers are not required to support JTS, so use of this is not portable across providers.



In the next post, I'll touch on more basics, to include security and concurrency.

Thursday, March 4, 2010

TestNG Diagnostics

Just a quick note about getting diagnostics from unit tests - if using TestNG, you can configure the dump of detailed test information into the command shell from which you run your tests. Use the verbose property, which you can access via a TestNG object or as part of the testng ant task. This will provide useful insights around failures (stack trace, etc.). For example:

<property environment="env"/>
<testng classpathref="my.classpath"
            outputDir="${my.outputdir}"
            haltOnfailure="true"
            verbose="${env.verbose}">

Here I've used an environment variable to facilitate dynamic levels of verbosity. As per the TestNG Javadoc:

verbose - the verbosity level (0 to 10 where 10 is most detailed) Actually, this is a lie: you can specify -1 and this will put TestNG in debug mode (no longer slicing off stack traces and all).

Given this, I can run my tests like so:

% export verbose=5
% cd /home/mystuff/java/projects/quantum-gravity-dongle/tests
% ant runtests

...and I'll get the diagnostics I need to quickly locate test problems.


Thursday, September 10, 2009

Spring 3.0 AOP: Cheatsheet

Continuing with insights and tips around Spring 3.0, here is a collection of code samples I've established "so far" with my (rather brief) prototyping using that framework. As with all of my cheatsheets, I'm not presenting these things as any kind of definitive reference; they are only from my own experience, done only in a "sandbox" so far -- we've not yet put these things into production. Your contributions and corrections to this cheatsheet, and any of my other ones, are welcome.


Dependencies

The following libraries are needed:

org.springframework.asm-3.0.0.M3.jar
org.springframework.beans-3.0.0.M3.jar
org.springframework.context-3.0.0.M3.jar
org.springframework.core-3.0.0.M3.jar
org.springframework.expression-3.0.0.M3.jar
org.springframework.aop-3.0.0.M3.jar
antlr-3.0.1.jar commons-logging.jar
com.springsource.org.aopalliance-1.0.0.jar
com.springsource.org.aspectj.runtime-1.6.5.RELEASE.jar
com.springsource.org.aspectj.weaver-1.6.5.RELEASE.jar

Concepts

Some key terminology and concepts, pulled from the Spring 3.0 Reference doc - I'll just copy key pieces verbatim first and then paraphrase at the end:
  • Aspect: a modularization of a concern that cuts across multiple classes. Transaction management is a good example of a crosscutting concern in J2EE applications. In Spring AOP, aspects are implemented using regular classes (the schema-based approach) or regular classes annotated with the @Aspect annotation (the @AspectJstyle).
  • Join point: a point during the execution of a program, such as the execution of a method or the handling of an exception. In Spring AOP, a join point always represents a method execution.
  • Advice: action taken by an aspect at a particular join point. Different types of advice include "around," "before" and "after" advice. Many AOP frameworks, including Spring, model an advice as an interceptor, maintaining a chain of interceptors around the join point.
  • Pointcut: a predicate that matches join points. Advice is associated with a pointcut expression and runs at any join point matched by the pointcut (for example, the execution of a method with a certain name). The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.
  • Introduction: declaring additional methods or fields on behalf of a type. Spring AOP allows you to introduce new interfaces (and a corresponding implementation) to any advised object. For example, you could use an introduction to make a bean implement an IsModified interface, to simplify caching. (An introduction is known as an inter-type declaration in the AspectJ community.)
  • Target object: object being advised by one or more aspects. Also referred to as the advised object. Since Spring AOP is implemented using runtime proxies, this object will always be a proxied object.
  • AOP proxy: an object created by the AOP framework in order to implement the aspect contracts (advise method executions and so on). In the Spring Framework, an AOP proxy will be a JDK dynamic proxy or a CGLIB proxy.

So an aspect is a class that contains methods referred to as "advice". The advice is executed as specified by the annotation (before, after, etc.) and the "pointcut", or predicate (what classes/methods match the expression). The point in the application where the advice is applied is a "join point". The runtime object containing these join points (i.e., being advised by one or more aspects) is the "target" or "advised object". Spring uses proxying to accomplish the advising (see section 8.6.1 - really, read it, it's short and helpful), and will use JDK dynamic proxies if all advised objects implement at least one interface. Finally, "introductions" are a mechanism where advised objects can be transparently made to implement new interfaces (to retrofit things like caching, comparability, serialization, immutability, etc.), with the advice providing an implementation of the interface, and being handed a reference to the advised object that it can use to invoke the new interface methods.


Problem Space

Recommendations in the following discussions are in bold-face.

The proof-of-concept application is a trivial tongue-in-cheek model of a "family service", where my son Connor and my Siberian Husky Nika would each like to get my attention. It happens. Spring IoC is used to declaratively determine who wins; since that's a static configuration, it's for now the same answer every time.

So the basic idea is a shown in the UML: we have a FamilyService with a FamilyMember property, and two FamilyMembers, each with a name property.

Note that all classes implement an interface. This is in general a good idea for testability, etc. but in particular with Spring AOP, I'd favor this approach because it facilitates use of JDK dynamic proxies instead of CGLIB proxies. Use of CGLIB should consider these things: (1) more library dependencies, (2) final methods cannot participate in AOP, and (3) constructors are called twice due to CGLIB implementation details.

Short of other insights, I would for now recommend implementing at least one interface for any classes that participate in Spring AOP.

A detailed explanation of the above issue around CGLIB vs JDK dynamic proxying, as well as very clear presentation in general around Spring 3.0, can be found in the Spring Reference doc.



Enabling AOP

The injection of the family member getting my attention is done like this:

<bean id="family-service" class="spring.FamilyService">
 <property name="member">
     <ref local="nikadog"></ref>
 </property>
</bean>

This is typical Spring injection; nothing special here. The more interesting stuff is to now add some AOP. Do this by adding an "aop" namespace to the beans declaration tag and enabling what I would recommend as the preferred flavor of Spring AOP (AspectJ-style annotations plus Spring auto-proxying):
 
 <beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:aop="http://www.springframework.org/schema/aop"
   xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">
 ....
   <aop:aspectj-autoproxy/>
 </beans>

The Spring reference doc again provides a clear, useful discussion around the various "flavors" you can use with Spring AOP. Given the tradeoffs described there, I recommend the AspectJ-style annotations with Spring auto-proxying for the following reasons:
  1. Their advice to "use the simplest thing that works" is IMO sound guidance
  2. So, for now, assume that AspectJ-annotations will suffice until you find otherwise
  3. If you need to gain more power, you can migrate easily from the AspectJ-annotations to full-blown AspectJ itself

The limitations to be aware of include these:

  1. AOP can only be applied to Spring-managed beans (therefore, Spring-declare any classes that will leverage AOP)
  2. AOP mechanisms can be applied only to public method-level executions (excluding constructors)
  3. Methods in AOP-enabled classes that have aspects declared will not see those aspects applied if invoked by other
    methods in that same class (i.e., do not use self-invocation within AOP-enabled classes)

Limitations #2 and #3 are a result of Spring's proxy approach to AOP. See section 8.6.1 in the reference doc for a detailed discussion. Note that a workaround for #3 is discussed in that section, but it involves a tight coupling to Spring and makes the application class acutely aware that it is being AOP-managed, neither of which is a recommended practice.


Simple AOP

To create an aspect with some advice that executes before all public methods in the spring package (see reference doc, section 8.2.3.4 for examples around expression syntax):

import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.*;
@Aspect
public class MyAspect {
 /**
  * Log a message before entry to specified methods
  */
  @Before("execution(* spring..*(..))")
  private void logOnEntry(JoinPoint jp) {
     // JoinPoint argument is optional, but recommended as quite useful
     System.out.println("Before " + getSimpleName(jp));
  }
 /**
  * Returns the simple name of the class being advised, as per the given
  * join point.
  */
  private String getSimpleName(JoinPoint jp) {
     return jp.getTarget().getClass().getSimpleName();
  }
 }

The aspect must be managed by Spring to be put into play - add this entry to the configuration file:

<bean id="myAspect" class="spring.MyAspect"> </bean>

As you can see, you can also access the target object via the JoinPoint argument. As such, you can cast it to your application type and execute its public methods.

You can also execute advice after a method or around a method. The @After is analogous to the above example. The "execution" part of the expression is the "pointcut designator" or PCD; there are several others, but "execution" is probably what will be used the most. Since it's indicating a method-level execution, you can alter the method-portion of the expression to e.g. address only setters:

@Before("execution(* spring..set*(..))")

To address executions at a type level (i.e. all public methods in a given type), use the "within" PCD:

@After("within(*.*Service)")

You can also use the Spring bean ID declaration as an alias for a given type; so, given this bean declaration:

<bean id="family-service" class="spring.FamilyService"></bean>

...you can specify this join point using the "bean" PCD:

@After("bean(family-service)")


Reusable Pointcuts

Reusable pointcuts can be established:

/**
 * Establish a pointcut on execution of any and all public methods
 */
@Pointcut("execution(* *.*(..))")
public void anyMethod() {}

Pointcuts can be combined to form conditional expressions, using AND, OR and NOT operators (&&, ||, and !):

@Pointcut("within(com.xyz.service..*)")
private void inServiceModule() {}
@Pointcut("anyMethod() && inServiceModule()")
private void serviceOperation() {}

To intercept the throwing of an exception resulting in a method exit, using the "anyMethod" pointcut established above:

/**
 * Intercept execution when any method exits by throwing any exception
 */
 @AfterThrowing(pointcut = "anyMethod()", throwing = "ex")
 private void trackExceptions(JoinPoint jp, Exception ex) {
     System.out.println("Exception thrown from "           + getSimpleName(jp) + ": " + ex);
 }

To intercept the value returned from a specified pointcut:

 /**
  * Establish a pointcut on execution of specified method
  */
  @Pointcut("execution(* spring.Nika.getName())")
  public void getNikadogName() {}
 /**
  * Intercept the return from specified pointcut
  */
  @AfterReturning(pointcut = "getNikadogName()", returning = "name")
  private void getNikaName(String name) {
      System.out.println("after returning from nikadog, pointcut name: "
           + name + "...");
  }


Beyond the Simple

The @Around PCD is bit more sophisticated:

 /**
  * Establish a pointcut on execution of specified method
  */
  @Pointcut("execution(* FamilyService.getMember(..))")
  public void getMember() {}
 /**
  * Wrap specified method to get profiling information. Can also use this pattern
  * for tracing, caching, etc.
  */
  @Around("getMember()")
  private FamilyMember doProfiling(ProceedingJoinPoint pjp)  throws Throwable {
    long start = System.currentTimeMillis();
    FamilyMember member = (FamilyMember) pjp.proceed();
    System.out.println("Elapsed time to get member: " +        (System.currentTimeMillis() - start));
     return member;
 }

In the above snippet, the ProceedingJoinPoint facilitates setting things up before the advised object method is executed (i.e. start the stopwatch, check the cache for a hit, etc.), then supports (optionally!) executing the method, and finally returning the appropriate value - so you can see that this decorates the object execution with your own proxying code. Note that you can easily check a cache for a "hit", then simply return the cached value if it's there without executing the method, otherwise executing the method and stuffing it in the cache for subsequent requests. Using the "args" PCD, you can (e.g.) examine incoming argument values to qualify what specific object in the cache is being requested.

Note that this approach to caching relies on the aspect to manage the cache. An alternate approach will be presented in the "Introductions" section below.

Suppose our service class was throwing an exception, and we want to apply a retry strategy since the problem involves waiting for some remote resource to become available (e.g. the database is spinning up):

@Around("getMember()")
 private FamilyMember retry(ProceedingJoinPoint pjp) throws Throwable {
     int numAttempts = 0;
     int maxRetries = 5;
     Throwable t = null;
     do {
        numAttempts++;
        try {
            return (FamilyMember)pjp.proceed();
        } catch (DatabaseNotReadyException ex) {
            System.out.println("=========== Caught exception from "
                 + getSimpleName(pjp)
                + "; retry attempt #" + numAttempts + "==============");
            t = ex;
        }
     } while (numAttempts <= maxRetries);
              throw t;
   }

Introductions

A technique described above for caching relies on the aspect itself to manage the cache. An alternate approach is to implement an interface that prescribes caching functionality:

public interface Cacheable {
     public boolean isCached();
     public void save(Object obj);
     public Object fetch();
}
But we may be motivated to regard this cacheability as a cross-cutting concern, one that should be maintained and managed (enable/disable, track hits, monitor throughput gains, flush the cache, etc.) separately from the pure business logic (vs simply having FamilyService implement the interface explicitly, i.e. such that one's business classes are not responsible for determining their own "cacheability" - that's an orthogonal concern). If so, we can use AOP "introductions" to retrofit the caching.

An implementation supporting the FamilyService (and, for that matter, other services in general) might begin life like this:

public class CacheableImpl implements Cacheable {
      private boolean cached = false;
      private Object cachedObject = null;
      public boolean isCached() {
        return cached;
     }
      public void save(Object obj) {
        cachedObject = obj;
        cached = true;
     }
      public Object fetch() {
        return cachedObject;
     }
}
An "introduction" is a way of retrofitting an implementation of an interface onto the target object. Since it's AOP, the target object does not and need not know it's being decorated like this. The somewhat cryptic syntax to make it happen is encapsulated in the aspect:
@DeclareParents(value = "spring.FamilyService", defaultImpl = CacheableImpl.class)
 public static Cacheable mixin;
 .......
 @Around("getMember() && this(service)")
 public FamilyMember checkCache(ProceedingJoinPoint jp, Cacheable service) throws Throwable {
     if (service.isCached())  {
        System.out.println(" ---> Returning cached member");
        return (FamilyMember) service.fetch();
     }
     System.out.println(" ---> Member NOT cached; call service and save result...");
     FamilyMember member = (FamilyMember) jp.proceed();
     service.save(member);
     return member;
 }
Now, instead of relying on the aspect to manage caching - arguably a questionable responsibility - the details of caching are factored out to give us better cohesion. This is just one possible use of the AOP introduction.
Resources Spring 3.0.M3 Reference Documentation, by Rod Johnson et. al., 2009 Spring Framework

Wednesday, September 9, 2009

Spring Test Framework Cheatsheet

This documents some prototyping done with the Spring Test Framework (I'll refer to this as the STF). Effectively it is a paraphrase/summary of what's written up in the Chapter 10 of the Spring Framework Reference document, version 3.0.M3, with some example code snippets that I was able to get working. Consult that writeup for further detail.

Here are the high-level features of the STF that I'd consider most interesting:
  1. How to fetch the Spring application context object regardless of environment (web container, test environment, etc.)
  2. How to configure test-specific Spring contexts
  3. Simplified access to context beans
  4. Automatic rollback of test case transactions
  5. Auto-rollback without extending a superclass
  6. Odds-n-ends

How to fetch the Spring application context

This first tip is not directly related to the STF, but was discovered during my experiments with it, so I make note of it here.

 In a previous post, I presented a pattern for determining at runtime how to get a reference to the application context that would first look for a web context, and if that failed simply produce a classpath-based context. This pattern requires explicit knowledge not only of what environment is in play, but also knowledge of exactly what Spring context is appropriate for that environment. A simpler pattern can be used as follows - first, provide a class that implements ApplicationAwareContext:


public class ApplicationContextProvider 
    implements ApplicationContextAware { 
    private static ApplicationContext context;
    public static ApplicationContext getApplicationContext() {
        return context;
    }
    public void setApplicationContext(ApplicationContext context) {
        this.context = context;
    }
}

Declare that class as managed by Spring in the context file:

<bean id="appCtxProvider" 
    class="com.twc.registry.common.ApplicationContextProvider">
</bean>

Now, any class that wants a reference to the context instance can do so at runtime, whether in a web or a test/standalone environment:

ApplicationContext context = 
    ApplicationContextProvider.getApplicationContext();
MyBean myBean = (MyBean)context.getBean("my-bean");

That code snippet can of course be encapsulated and parameterized as needed, to facilitate keeping third-party framework specifics in a single point of change.

This precludes the need for a ServletContextListener as described in my previous post, simplifies the mechanics around fetching beans in the context, and eliminates the need for application awareness of explicit subclasses of the Spring ApplicationContext.

In what follows, I'll present an even simpler method for injecting dependencies directly into test classes, such that a reference to the application context is not needed.


How to configure test-specific Spring contexts

If you need to specify certain test fixtures or other test-specific dependencies, it'd be useful to both have these injected (to facilitate reuse and simplify test setup) but keep the configuration for this injection separate from production classes. Using the STF, a convenience idiom supports doing this.

First, to use the STF, add org.springframework.test-3.0.0.M3.jar to your classpath.

Configure a test-specific Spring context using the following annotation:

package com.mybiz;
@ContextConfiguration
public class MyTest {
   @Test
    public void testThis() {
          ApplicationContext ctx = 
              ApplicationContextProvider.getApplicationContext();
          MyBean myBean = (MyBean ) context.getBean("my-bean");
          // now do some tests with myBean...
    }
}

This presumes the existence of a Spring configuration file named com/mybiz/MyTest-context.xml. To otherwise specify the location:

@ContextConfiguration("/WEB-INF/MyTestConfig.xml")

Multiple resources can be specified using comma-separated lists. The default is for subclasses to inherit superclass' resource lists and optionally override bean entries; this inheritance behavior can also be disabled.


Simplified access to context beans

Now, instead of using the ApplicationContextProvider manually as in the above snippet, we can simply do this:

@Autowired  // or can annotate a setter instead of a field
private MyBean myBean;

If there are  multiple MyBean's defined in the Spring context, this type-based wiring will not suffice (without further qualification, which we ignore for now). Instead use injection by name:

@Resource
private MyBean myBean;

However, it appears (empirically...) that @Resource also wires by type, since in the above working example, I hadn't configured a bean with an ID of "myBean". It also "just works" this way:

private MyBean myBean;
@Resource  // also works with @Autowired!
private void setTheBean(MyBean theBean) {  
// notice the field and/or method are both private - but injection still works!
    myBean = theBean;
}


Automatic rollback of test case transactions

By far the most valuable feature of the STF is the ability to execute integration tests against a database without changing the database state - i.e., all transactions are automatically rolled back. Making this work is reasonably simple - first, we'll need Spring context entries to specify a data source and a transaction manager:

<bean id="dbcp-dataSource" 
    class="org.apache.commons.dbcp.BasicDataSource" 
    destroy-method="close">
    <property name="driverClassName" value="${database.driver}"/>
    <property name="url" value="${database.url}"/>
    <property name="username" value="${database.user}"/>
    <property name="password" value="${database.password}"/>
</bean>
<bean id="txnMgr" 
    class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
    <property name="dataSource" ref="dbcp-dataSource"/>
</bean>
<tx:annotation-driven transaction-manager="txnMgr"/>
<aop:aspectj-autoproxy/>

This context configuration file must be loaded using @ContextConfiguration; we must declare @Transactional either at the class or method level; and since our transaction manager is not named "transactionManager" (the default) -- or if we would ever wish to disable auto-rollback at the class level -- we'll need to add a @TransactionConfiguration annotation:

@ContextConfiguration(locations = {"/WEB-INF/MyTestConfig.xml"})
@Transactional
@TransactionConfiguration(transactionManager="txnMgr", defaultRollback=true)
public class MyDaoTest extends AbstractTransactionalTestNGSpringContextTests {
.......
// override auto-rollback on per-method basis
    @Test
    @Rollback(false)
    public void keepThisTransactionInPlace() {
        ......
    }
    @BeforeTransaction  
// likewise, @AfterTransaction follows the transaction
    public void doBeforeTxn()
    {
// do whatever setup needed, if any, before the transaction executes
    }
}

To gain numerous conveniences, we've extended a Spring class specific to the TestNG framework (also available are Junit-specific superclasses). We could have instead added a handful of cruft to get around doing this extension, if e.g. you need to extend your own test superclass or for some other reason you're adverse to this. But the added cruft is so noisy that you'll be motivated to simply encapsulate it in your own superclass anyway, or if that's not feasible then reuse becomes a copy-paste headache. If possible, simply extending the Spring-provided convenience classes is probably the simplest thing to do.


Auto-rollback without extending a superclass

But if you must know, here's the cruft you'll need to avoid extending AbstractTransactionalTestNGSpringContextTests:

@ContextConfiguration(locations = {"/WEB-INF/MyTestConfig.xml"})
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class, 
    DirtiesContextTestExecutionListener.class,
    TransactionalTestExecutionListener.class})
@Transactional
@TransactionConfiguration(transactionManager="txnMgr")
public class MyDaoTest implements IHookable, ApplicationContextAware {
    private ApplicationContext applicationContext;
    private final TestContextManager testContextManager;
    private Throwable testException;
    public MyDaoTest() {
        this.testContextManager = new TestContextManager(getClass());
    }
    public final void setApplicationContext(
        ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }
    @BeforeClass(alwaysRun = true)
    protected void springTestContextPrepareTestInstance() 
    throws Exception {
        this.testContextManager.prepareTestInstance(this);
    }
    @BeforeMethod(alwaysRun = true)
    protected void springTestContextBeforeTestMethod(Method testMethod) 
    throws Exception {
        this.testContextManager.beforeTestMethod(this, testMethod);
    }
    public void run(IHookCallBack callBack, ITestResult testResult) {
        callBack.runTestMethod(testResult);
        this.testException = testResult.getThrowable();
    }

    @AfterMethod(alwaysRun = true)
    protected void springTestContextAfterTestMethod(Method testMethod) 
    throws Exception {
        try {
            this.testContextManager.afterTestMethod(
                this, testMethod, this.testException);
        }
        finally {
            this.testException = null;
        }
    }

Ugly, ugly, ugly. Encapsulate!


Odds-n-Ends

There are numerous other facilities in the STF; consult Chapter 10 for the full discussion. Worth mentioning here are the following:
  1. automatic caching of loaded application contexts, to save startup time on per-method basis, once for each test fixture. Optionally instruct the framework to reload the context in case changes to state of beans in that context that are not meant to be preserved across all tests in a given fixture.
  2. JDBC utility class to facilitate querying database state.
  3. Mock objects ready-to-go for JNDI and Servlet API.
Overall, two thumbs up: this is definitely a new item in my toolbox.