Wednesday, June 30, 2010

Using KeyMan To Generate Keys and Certs

This post continues my series around Jetty. Previous posts discuss setting up a basic embedded Jetty application, and then some adjustments to repeat that exercise in a Cygwin environment. My next step is to configure SSL...now, I recall earlier adventures using Java's keytool to manage this, and have hoped for something better. Jetty's documentation pointed me to KeyMan, which is by far a nicer way to go - it provides a decent intuitive GUI to help create, delete, and otherwise manage keys and certificates, among other things. Here's an outline of how to use KeyMan to create a PKCS#12 keystore with a self-signed certificate and public-private key pair:

  • Download, install (unpack zip, etc.), read the README.txt. I did nothing with the km.setup file, but did edit the km.bat as instructed. Turns out that, since I'm on cygwin, that wasn't needed; instead, I execute the km program. Click on the "New" icon to create a new "token" (i.e. repository for keys, certs, etc.):



  • Choose the PKCS#12 Token from the next dialog, and hit the checkmark ("Complete Dialog") to proceed:



  • Next, you need to store a key and a certificate in this token. Select "Actions -> Generate Key" from the token management window that appears:



  • The default algorithm is RSA-1024; that's strong enough for my needs. Click the Complete Dialog checkmark...this takes a second to complete, offering a cool little progress bar while you wait. The new key shows up in the All Certificate Items viewport of the token management window; now we need a certificate to go with it. Click "Actions -> Create Certificate...". Self-signed is good enough for my needs. Click checkmark and fill in the fields as needed (only "Your name" is required):




  • A verification appears when you check "Complete Dialog" here, with the option to label this certificate. Enter a label if you wish, and again move on with the checkmark:



  • Save the token to a file by selecting File -> Save. This first prompts you for a passphrase, then a file location.
Prove to yourself that the keystore (token, repository, whatever) is really there and that you can view it in human-friendly form by first exiting the program, restarting it and selecting the "Open existing..." icon, then "Local resource..." and "Open a file...". Browse to the file location you just saved to, enter the passphrase, and you should see your token listed in the Private Certificates category (in the dropdown). Click right on that item and you'll see all the informational details entered when you created the certificate.

Next, I'll see about using that keystore for my Jetty SSL setup. Meanwhile, here are some useful links around KeyMan, SSL and Jetty's SSL instructions:

Solaris Keytoolhttp://java.sun.com/j2se/1.4.2/docs/tooldocs/solaris/keytool.html
Windows Keytoolhttp://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/keytool.html
KeyManhttp://www.alphaworks.ibm.com/tech/keyman
OpenSSLhttp://www.openssl.org/docs/HOWTO/
OpenSSL FAQhttp://www.openssl.org/support/faq.html
Jetty SSLhttp://docs.codehaus.org/display/JETTY/How+to+configure+SSL

Followup: Basic Embedded Jetty in Cygwin

In a recent post, I described how to get a basic Jetty web application going, using the embedded approach. Since then, I've reproduced it in a Cygwin environment, and here I'll comment on that exercise.

As it turns out, specifying this type of Java startup in Cygwin (or in Windows XP per se) will not work:

java -server -Dbasedir=/usr/local/mywebapp/war -cp /usr/local/mywebapp/war/WEB-INF/lib/* com.mybiz.MyJettyWebServer

That's because the wild-card expression apparently is not supported in a DOS-based environment - even if I enclose the above classpath in quotes. Instead, I'd need to provide a semi-colon-delimited list (not colon-separated - I'm in XP) of all jars under ./WEB-INF/lib. This is not the kind of thing I'd like to do; maintaining that kind of list would be a headache as the webapp evolves. Additionally, keep in mind that I've told Jetty to start up with an exploded warfile location:

        WebAppContext appContext = new WebAppContext();
        File warPath = new File(System.getProperty("basedir"));
        appContext.setWar(warPath.getAbsolutePath());
        HandlerList handlers = new HandlerList();
        handlers.setHandlers(new Handler[]{ appContext, new DefaultHandler() });
        jetty.setHandler(handlers);
        jetty.start();

This will result in a web-level classloader to load all the jars under WEB-INF/lib, which is arguably redundant, since I'm explicitly setting my classpath to the same thing. That in turn will cause loader constraint violations when running the webapp in an IDE such as Intellij, if the run configuration you're using there points to the same classpath (since that application-level classloader loads the classes first, and then the webapp-level classloader tries to do the same thing). I'll defer solving the Intellij problem for now, and just address basic command line startup.

Given maintenance cost concerns, I'm motivated to load the minimal number of jars needed to get Jetty going, then allow it to load the rest of what it needs from WEB-INF/lib. In my particular setup, that minimal set includes my application jar and three Jetty jars:

MyApp-1.0.jar
jetty-6.1.21.jar
jetty-util-6.1.21.jar
servlet-api-2.5-20081211.jar

I figured out this minimal set by just trying to start up the WebServer class and seeing what classdef-not-found problems I had - then searching for the necessary jar by setting up a bash function that I can reuse:

findclass () { find . -name '*.jar' -o -type f |xargs -i bash -c "jar -tvf {}| tr / . | grep -i "$@" && echo {}"; }

...and subsequently invoking it like this:

findclass <dot-delimited-classname>

Once I have all my dependencies figured out, I can invoke my Jetty program with that minimal set, and rely on the webapp-level classloader to do the rest when the embedded Jetty webserver starts:

java -server -Dbasedir=/usr/local/mywebapp/war -cp "MyApp-1.0.jar;jetty-6.1.21.jar;jetty-util-6.1.21.jar;servlet-api-2.5-20081211.jar" com.mybiz.MyJettyWebServer

Note that I've wrapped the classpath in quotes, and, as mentioned, used semi-colons instead of colons.

Wednesday, June 23, 2010

Send/Receive Notification About an Exception

From Java Message Service, Second Edition, by Mark Richards, Richard Monson-Haefel, and David A. Chappell - here's a lightweight mechanism for notifying interested consumers about an exception (in a JMS context):

try {
    ...
} catch (Exception up) {
    Message message = session.createMessage();
    message.setStringProperty("Exception", up.getMessage());
    publisher.publish(message);
    throw up;
}

Interested consumers can receive this notification like this:
public void onMessage(Message message) {
    System.out.println("Exception: " + message.getStringProperty());
}

This uses a simple Message object, which contains no payload - only JMS headers and properties. As noted in the book, when simple notification is all that is needed, use of the Message type is the most efficient means to do this.

Basic Embedded Jetty Setup: JSF 1.2 Webapp

Here are some basic code snippets I've used (to-date ... subject to change) to get an IceFaces-based (JSF 1.2) webapp deployed as an embedded Jetty webapp - using Maven for building. The artifacts include a configuration file, a minimal bootstrap class, the pom and the file used as the maven assembly-plugin descriptor.

The configuration file is jetty.xml, and doesn't require much. It lives under the ./etc directory of my project:

<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="Server" class="org.mortbay.jetty.Server">

    <Set name="ThreadPool">
        <New class="org.mortbay.thread.QueuedThreadPool">
            <!-- initial threads set to 10 -->
            <Set name="minThreads">10</Set>
            <!-- the thread pool will grow only up to 200 -->
            <Set name="maxThreads">200</Set>
            <!-- indicates that having 20 and below, the pool will be considered low on threads -->
            <Set name="lowThreads">20</Set>
            <!-- The number of queued jobs (or idle threads) needed before the thread pool is grown (or shrunk) -->
            <Set name="SpawnOrShrinkAt">2</Set>
        </New>
    </Set>

    <Call name="addConnector">
        <Arg>
            <New class="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- the ip address or domain to bind -->
                <Set name="host">
                    <SystemProperty name="jetty.host"/>
                </Set>
                <!-- the port to use/bind, defaults to 8080 if property not set -->
                <Set name="port">
                    <SystemProperty name="jetty.port" default="8080"/>
                </Set>
                <!-- the time in milliseconds when a connection is considered idle -->
                <Set name="maxIdleTime">300000</Set>
                <!-- the number of acceptors (their job is to accept the connection and dispatch to thread pool) -->
                <Set name="Acceptors">2</Set>
                <!-- should the connection statistics be turned on? (Not advisable in production) -->
                <Set name="statsOn">false</Set>
                <!-- the confidential port -->
                <Set name="confidentialPort">8443</Set>
                <!-- indicates the minimum number of connections when the server is considered low on resources -->
                <Set name="lowResourcesConnections">5000</Set>
                <!-- when low on resources, this indicates the maximum time a connection must be idle to not be closed -->
                <Set name="lowResourcesMaxIdleTime">5000</Set>
            </New>
        </Arg>
    </Call>

    <!-- Stops the server when ctrl+c is pressed (registers to Runtime.addShutdownHook) -->
    <Set name="stopAtShutdown">true</Set>
    <!-- send the server version in the response header? -->
    <Set name="sendServerVersion">true</Set>
    <!-- send the date header in the response header? -->
    <Set name="sendDateHeader">true</Set>
    <!-- allows requests(prior to shutdown) to finish gracefully -->
    <Set name="gracefulShutdown">1000</Set>

</Configure>

This configuration file is referenced by the bootstrap class, using it to configure Jetty. This class also sets the context for the webapp, points to the top-level directory of the exploded WAR content, sets a webapp context and a default context as handlers for Jetty, and starts up the webserver:

public class MyJettyWebServer {

    public static void main(String[] args) throws Exception {

        Server jetty = new Server();

        // configure Jetty by pointing to config file(s)
        String[] configFiles = { "etc/jetty.xml" };
        for (String configFile : configFiles) {
            XmlConfiguration configuration = new XmlConfiguration(new File(configFile).toURI().toURL());
            configuration.configure(jetty);
        }

        // set the context for the webapp
        WebAppContext appContext = new WebAppContext();
        appContext.setContextPath("/mycontext");

        // point to the top-level directory of the exploded WAR content
        File warPath = new File(System.getProperty("basedir"));
        appContext.setWar(warPath.getAbsolutePath());

        // set a webapp context and a default context as handlers for Jetty
        HandlerList handlers = new HandlerList();
        handlers.setHandlers(new Handler[]{ appContext, new DefaultHandler() });
        jetty.setHandler(handlers);

        // start up the webserver
        jetty.start();
    }
}


The pom specifies IceFaces 1.8.2, being careful to exclude the EL API jar wherever that's brought in transitively, as per http://www.icefaces.org/docs/v1_8_0/htmlguide/devguide/appendixA.html; and it specifies Jetty and Log4J artifacts. Note that JSP support is explicitly specified:

    ....
    <!-- use JAR packaging for embedded Jetty -->
    <packaging>jar</packaging>
    ....
    <dependencies>
        <dependency>
        <dependency>
            <groupId>org.icefaces</groupId>
            <artifactId>icefaces</artifactId>
            <version>1.8.2</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.el</groupId>
                    <artifactId>el-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.icefaces</groupId>
            <artifactId>icefaces-comps</artifactId>
            <version>1.8.2</version>
        </dependency>
        <dependency>
            <groupId>org.icefaces</groupId>
            <artifactId>icefaces-facelets</artifactId>
            <version>1.8.2</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.el</groupId>
                    <artifactId>el-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-api</artifactId>
            <version>1.2_12</version>
        </dependency>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-impl</artifactId>
            <version>1.2_12</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jetty</artifactId>
            <version>6.1.21</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jetty-util</artifactId>
            <version>6.1.21</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jsp-2.1-jetty</artifactId>
            <version>6.1.21</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>${log4jVersion}</version>
        </dependency>
    </dependencies>
</project>

The assembly-plugin descriptor has a few things worth mentioning. In the dependency sets, I lay down my dependencies in the standard webapp location, ./WEB-INF/lib:

        <dependencySet>
            <unpack>false</unpack>
            <scope>runtime</scope>
            <outputDirectory>war/WEB-INF/lib</outputDirectory>
        </dependencySet>

I also specify several destinations for my dev-time files to deal with Jetty-related stuff:

        <!--
        Jetty deployment: configuration file location
        -->
        <fileSet>
            <directory>etc</directory>
            <outputDirectory>/usr/local/mywebapp/etc</outputDirectory>
            <fileMode>0644</fileMode>
        </fileSet>
        <!--
        Jetty deployment: webapp deployment location - hmmm, this one might not be needed...
        -->
        <fileSet>
            <directory>webapps</directory>
            <outputDirectory>/usr/local/mywebapp/webapps</outputDirectory>
            <fileMode>0644</fileMode>
        </fileSet>
        <!--
        Jetty deployment: production-time exploded warfile location
        -->
        <fileSet>
            <directory>src/main/webapp</directory>
            <outputDirectory>/usr/local/mywebapp/war</outputDirectory>
            <fileMode>0644</fileMode>
        </fileSet>

Finally, to start the program:

java -server -Dbasedir=/usr/local/mywebapp/war -cp /usr/local/mywebapp/war/WEB-INF/lib/* com.mybiz.MyJettyWebServer

Browse to localhost:8080 and your webapp should appear.

You'll notice I did not use the maven-jetty plugin (nor the jetty-maven plugin - yes, there are two different ones, each with different names, schemas and behaviors). The good news about the plugin is that it shields you from much of the configuration/deployment exercises you'll need; that's also the bad news. I needed to understand explicitly what dependencies, etc. I'd need for production, so I chose to do things manually. The good news around this is that I've done most of the heavy lifting to grease the skids for future embedded-Jetty exercises.

References

Jetty 6.x Wiki
Jetty 7x, 8.x Wiki

Tuesday, June 22, 2010

Ternary Expressions Problem with JSF 1.2

This is a followup on my previous post that offered a workaround to this error message, as seen when working with JSF 1.2:

javax.el.ELException: Error Parsing: 
Caused by: com.sun.el.parser.ParseException: Encountered ":text"

In my first post, I failed to find the root cause, so I took the path of least resistance in the interest of the project schedule (using the heavy-handed approach of two separate panel groups, backing the condition out to the "rendered" attribute of each group). As it turns out, the root cause is lack of white space around the colon, as explained in this article from Oracle. That is, given an expression:

#{isThisTrue?doThis:doThat}

...and depending on what the deployment environment is, you'll either notice nothing or bump into this error message. When that first post was written, I was deploying to Glassfish 8.x - and I needed to use the workaround described above. Recently, I deployed to JBoss 4.x and later to 5.1, and here I did not notice anything - but then I redeployed to embedded Jetty 6.1.x, and the problem re-appeared. This is probably because JBoss shielded me from needing the magic combination of JSF, JSP, Facelets, JSTL and etc. to make things work, but vanilla Jetty does not.

In either event, you might have better luck than I with determining the dependency mix, including tweaking the web.xml to configure the correct expression factory, trying various permutations of Jetty's JSP libraries, and etc. (aka "time sink"); but, more simply, the fix is to change the problematic expression (well, actually it's a bug in JSF 1.2) to this:

#{isThisTrue?doThis : doThat}

This time, I googled for the right thing ("JSF 1.2 ternary parse exception") and, by now, the article from Oracle had finally been published (it wasn't there until after I originally bailed out with my workaround).

Monday, June 14, 2010

Blocking Sends When ActiveMQ Broker is Down

As it turns out, even though you may have specified useAsyncSend on an ActiveMQ broker, your sends will block if that broker goes down (or if the broker was not running when the client first attempts a send). Don't waste any cycles (like I did) tweaking the delivery mode, whether or not the session is transacted, or other connection settings - the "problem" will occur with use of the failover transport (i.e., my broker URL is failover://(tcp://localhost:61616)) . This transport will automatically attempt reconnection until success, but during these attempts, your client will block. This may not be the behavior you want; I'm not able just yet to give you a satisfying solution, but I can give you some hooks to come up with something on your own (and if so, please let me know!).

One so-called solution is of course to back off using the failover transport - e.g., use TCP instead. But this is heavy-handed and arguably overkill - the failover transport serves an excellent purpose in the face of broker failures, and I'm reluctant to let it go just to solve an exceptional condition. In either event, you'll receive something like a java.net.ConnectException with the TCP protocol when the broker is down; you can try-catch that and do whatever makes sense to you. But that's a questionable workaround. Instead, here's some advice from the ActiveMQ website:

If you use failover, and a broker dies at some point, your sends will block by default. Using TransportListener can help with this regard. It is best to set the Listener directly on the ActiveMQConnectionFactory so that it is in place before any request that may require an network hop. Additionally you can use timeout option which will cause your current send to fail after specified timeout. The following URL, for example

failover:(tcp://primary:61616)?timeout=3000

will cause send to fail after 3 seconds if the connection isn't established. The connection will not be killed, so you can try sending messages later at some point using the same connection (presumably some of your brokers will be available again). 

Now, I first tried the timeout parameter approach - as it turns out, your application will get an exception at the timeout expiration. But this introduces its own problems around performance - if you have thousands (or, even dozens) or messages, each one timing out after say 2-3 seconds, your throughput will be hurt badly. So, I bailed out on this approach and took a closer look at the TransportListener. Here's what I implemented in my publisher class:

    public void onCommand(Object command) { /* EMPTY */ }

    public void onException(IOException error)
    {    Logging.publisher.warn("Transport exception encountered", error);  }

    public void transportInterupted()
    {    Logging.publisher.warn("Transport interrupted! Is Broker down?");  }

    public void transportResumed()
    {    Logging.publisher.warn("Transport connectivity is restored");  }

My test program received callbacks for all of these methods, as expected, except for the onException. The ActiveMQ javadocs aren't much help here ("An unrecoverable exception has occured on the transport"). From this point, I tried combining the timeout parameter with the transport callback - e.g.

    public void transportInterupted()
    {    this.amqFactory.setSendTimeout(100); }

    public void transportResumed()
    {    this.amqFactory.setSendTimeout(-1); }

My strategy here is to minimize the timeout, and hence maximize the throughput in the face of dozens of sends, by reducing the timeout to a bare minimum (and then restoring it to the original timeout value when the broker comes back up). But, I had no success here; the timeout remained at the original value as set in the URL (which I consider to be too high in this context) even after transportInterupted (allegedly) changed it. I'm not willing to set my initial timeout to a low value either, since this may cause exceptions even under nominal operating conditions - not good.

From here, I leave it to you to find some clever use of the transport listener that can deal with a broker that goes down. Good luck, and let me know.


Friday, June 11, 2010

Preserve Unix Line Endings with Windows Intellij

Just a quick note: If you're editing shell scripts and the like on a Windows platform with Intellij (my version is 9.0), here's how I succeeded preserving Unix-style line endings. First, to more accurately describe the problem, a given file has been saved in the Windows IDE, then checked in to a Perforce repository, and then sync'd onto a Unix server. When editing that file with vi, the settings [dos] [noeol] appear - and I'd like to get rid of these. I do not want a DOS format, and I do want an end-of-line. Here are the steps I took:
  1. First, set the Intellij "Line separator" property to Unix - enter "Line separator" in the Settings search box, then choose the Code Style - General result. The Line separator property is towards the bottom of that panel. Yes, it does state "for new files", and not surprisingly I did not succeed simply editing an existing file - the DOS-style format remained after I saved and checked it in (Perforce). But, this does cover things going forward - at least this gets rid of the [dos] problem on new files - however, on the subsequent edit of the new file on Unix, the [noeol] is still there.
  2. To get rid of the [noeol] problem, I set one more thing in Intellij: in IDE Settings - search for "Ensure blank", which will bring up the optional checkbox for "Ensure blank line before end of file on Save". Check that option, and from here it appears that newly created files in the IDE will no longer have the [noeol] affliction. To address this problem on existing files, simply edit/save/check-in the file with the IDE. Probably, you could also just add a blank line at the end with vi on Unix - I didn't try this, but it sounds reasonable.
  3. To fix the [dos] problem on existing files, open the file in vi and type ":set ff=unix". Save, check in, and this problem should not reappear even after subsequent edits in the Windows IDE.
Don't forget, depending on what steps you take from above, that if you have set your CVS system to "revert unchanged" files (as I do with Perforce), that you may need to actually change something that the CVS will notice before it will check it in. Huh, maybe adding some comments...

Hope that helps. I'm sure I'll bump into this headache again, so I can also hope this helps me when that happens.