A Quick Look at Tellurium

Tellurium builds on top of Selenium to present a more object-oriented abstraction, potentially resulting in more maintainable test code, and perhaps even making it easier to construct the tests to begin with; both of these things would require more familiarity on my part to judge fairly.

While Tellurium provides an IDE (TrUMP), it is not used for the same purpose as the Selenium IDE - i.e. rather than click-and-record of sequential events, it facilitates click-and-generate of (Groovy) code snippets for a given UI component or hierarchical collection of components. These become the basis of so-called "UI Module" classes, to which the developer can add methods that access the UI component. Test classes - whether in TestNG, Junit, Groovy, etc. - then leverage that set of methods to write test cases. This approach encapsulates the complex lookup (XPath, etc.) needed around UI components with access methods around those components, and supports separating the concerns of UI component vs tests written against that component. This of course sets the stage for reuse of the UI components in different test classes. Since Tellurium provides a relative location lookup model, it insulates the test cases against changes to the UI (i.e. an absolute XPath expression could break if any of the enclosing path components changes, e.g. when the UI is laid out differently).

Good Things and Random Notes

Most notably, Tellurium provides an OO approach. This is good for quick construction of test code and, more importantly, ease of maintenance on that codebase over time.

As a convenience wrapper around Selenium, Tellurium starts up the Selenium server automatically when one runs a test case.

To use TrUMP, the Tellurium IDE, SSL must be turned off; this is however probably a good idea anyway for testing purposes. The IDE is used by turning Record on, clicking on one or more UI components (e.g. a form, a table, link, etc.), and then clicking on Generate in the IDE. This (ultimately) produces a set of snippets, with any components that are part of a hierarchy in the page represented as such. Each snippet presents all attributes in the given component; you can next proceed to customize things (include/exclude snippets, rename for better abstraction, etc.). Next, you'd export the snippets to a Groovy class that represents the given UI Module, and as mentioned add methods to support access by test classes.

One roadblock that took some time to solve was dealing with iframes; the browser instance became confused, losing track of its connection to Tellurium. To deal with this type of thing, set useMultiWindows = true in the TelluriumConfig.groovy file.

When things go wrong - i.e. unexpected failures in your test case - a screenshot is automatically saved to facilitate revisiting things in a post-mortem.

Criticism

As of this writing, TrUMP is not compatible with Firefox 3.5.2; I don't know yet about other 3.x releases.

The export to Groovy is a heavy-handed "save as file" approach; I would prefer just a quick export-to-window-in-TrUMP model so I could more quickly copy-and-paste into my IDE.

I had no luck using TrUMP to capture webpage elements that were generated by post-processing AJAX, i.e. links constructed dynamically by JavaScript based on the data returned. Nor did I succeed with TrUMP in capturing Dojo menus that appear from dropdown buttons (a typical Dojo approach). In both of these cases I had to do things manually, using various approaches like Selenium, FireBug "Inspect" (in particular leveraging the Search functionality in 3.5), and old-fashioned examination of the page source. The good news is that once a particular UI component's location has been fleshed out, that effort is reusable since the resulting UI module is reusable.

While Tellurium provides an OO approach - which IMO is a Good Thing - there are as such some (not unreasonable) assumptions around just what behaviors certain objects should have. For example, it's not uncommon in Dojo widgets to find onclick event handlers (moreless) attached to TD tags; but from an OO point of view, a TD should not be clickable. This appeared to be show-stopper until I (quite by accident) stumbled on an attribute that could be retrofitted to a UI module component description that instructs it to "respond" to click events (i.e. add respond:["click"] to the generated Groovy UI Module description).

The "quite by accident" serendipity described above should be better; in fairness, Tellurium is a nascent product, but I hope that soon some first-class JavaDoc and other reference documentation is provided (or, it could be that I just need to poke around more; but, again, this shouldn't take any effort at all to find - it "should" just jump out at me).

Conclusion

All in all, I'm giving a qualified two thumbs up. Tellurium out-of-the-gate shows great potential, and I personally prefer it to Selenium. It's moreless the kind of thing that I'd probably have ended up doing myself to get the level of abstraction I'd want; I'm grateful to the folks at Google to have done it for me.

There is still a great deal of facility I've yet to explore with this tool. It appears to have a fair amount of power, and I look forward to writing up more insights and tips around Tellurium.

Selenium IDE Tips

This is a quick cheatsheet for various Selenium (IDE-based) commands. There's nothing too jaw-dropping here; just a collection of tips that I've found useful.

---

General Usage

Here is a suggested development technique with caveats. There are lots of ways to accomplish the same thing; this is just my own approach:

1. Start by creating a directory that is named by the use case you're testing.
2. Create a test suite in that directory, naming it TestSuite.html. Using this convention will help distinguish the test suite from its test cases, all of which should use an HTML suffix.
3. Create test cases one by one, immediately give the test case a title via the Properties menu item (right click over the test case object), copy that name onto your clipboard before hitting OK, and then immediately save it to the use case directory by pasting that name into the file-save dialog and appending .html.
4. Do not expect to be able to re-use general purpose test cases from other use-case directories (e.g. a login sequence) - I've had to duplicate these in each use case that I use it in. This might be just me doing something wrong; I might need to revisit it to confirm. Obviously it sure would be nice if we could reuse things this way.
5. Do not expect to "Add Test Case" using an existing test case which you then alter, without altering the other uses of it within your test suite. Using my techniques here, one test case == one HTML file; modifying it will modify all instances of it within that use case.
6. Save often - ideally save test cases as soon you alter them, and save the test suite as soon as you add/delete test cases to/from it. If you change test suites, the IDE will not warn you that you have unsaved changes.

The following are some coding tips. Of course, consult the Selenium documentation for the definitive and exhaustive reference:

---

To confirm there's a DIV that contains an onClick handler with a given name and some template text of a given value:

<div onclick="doSomething()">foo</div>

...use this:

assertElementPresent
//div[@onclick[contains(., 'doSomething') ]][contains(., 'foo')]

The @onclick can be replaced with any attribute of the given DIV tag. The bracketed "contains" conditions can be chained, and are AND'd together.

---

To check/uncheck a checkbox in a table with a given ID that contains a cell with the given template text:

<table id="myID">
<tr>
    <td nowrap="nowrap">
        <input type="checkbox">check me</input>
    </td> 
</tr>
</table>

...use this:

click
//table[@id='myID']//td[contains(.,'check me')]//input

---

Avoid the annoying 30-second delay after clicking a link that use a JavaScript command as the HREF:

<a href="javascript:doSomething()">Do Something</a>

Set the timeout as needed:

setTimeout
1000

This will still get logged in the Selenium IDE as an ERROR, which unfortunately will result in the test being marked as failed. This of course makes it difficult to determine that your tests are in fact all green, since these failures show up as red. But I've just gotten into the habit of clearing the log before I start a test suite, and just ignoring the "[error] Timed out after 1000ms" log messages. If any other log messages are at ERROR level, these tell me there is something to look into; else, I consider my test suite to have passed.

---

To select an item from a menu:

select
menu
label=three

...use this:

select
menu
label=three

---

To enter a carriage-return character in an input field with a given ID:

keyPress
inputFieldID
13

So the number 13 is the HTML code for the carriage return character. Here's a reference of more character codes.

---

To click on a link with the given template text:

<a href="foobar">Click Me</a>

...use this:

click
//a[contains(text(),'Click Me')]

---

To consume an alert:

storeAlert

---