XPath is a complex query language that provides substantial benefits. It treats XML as a database, permitting queries as powerful as SQL SELECT. This post shows how to use a nifty visual explorer, XPath Checker, to write aggressive and accurate queries. Then we install these queries into test cases using assert_xpath.

This experiment uses Ruby on Rails and REXML. The techniques will port easily to most other web platforms.

XPath Checker

Whip out your Firefox web browser, and install XPath Checker from here:

https://addons.mozilla.org/en-US/firefox/addon/1095

It will help us reveal the inner secrets of a sample website, such as Google:

That Context Menu option produces this popup window:

The XPath Checker shows us where the edit field lives, at “/html/body/center/form/table/tbody/tr/td[2]/input[2]“, and it rendered a copy of the edit field for us.

(And note that Google still uses the CENTER tag - how quaint!)

XPath Checker generated a very long XPath. It’s too fragile for production code or tests, XPaths should be short and accurate, not long and fragile. An XPath should always return the same match, even if all the HTML around it changes. For example, if CSS zealots got their hands on Google’s front page, and changed that CENTER into a DIV, an XPath locating the entry field should still pass.

To fix this, we install Firebug, and use Inspect Element to locate a relevant characteristic of that edit field. We will use it to write a shorter, more accurate XPath:

That tells us how to write an XPath that’s unlikely to break (until we localize!):

The @title matches our input field’s title=’Google Search’ attribute.

(Read an XPath tutorial to learn how these fields work. Test cases that use assert_xpath often exploit these expressions to accomplish the traditional mission of assert_equal.)

Note that when you type “//input“, alone, into the XPath field, you get several more matches, including the search button itself. An XPath is a relational query, like an advanced Regular Expression, or a SQL SELECT statement. Each component of the query narrows its search results. Like a SQL SELECT statement, the goal of an XPath is a narrow match that only returns the results you need.

When you use an XPath in developer tests, they should work to describe the situation you need. A more specific query, “//form[ ‘f’ = @name ]//input[ ‘Google Search’ = @title ]“, also matches our INPUT field, and it enforces the program requirement that the INPUT field should live inside a FORM.

Retrofitting assert_xpath

Let’s write a test case that simulates developing Google’s front page. We use assert_tidy to convert the page into a REXML::Document, and use assert_xpath on our new query:

    def test_google
      require 'open-uri'
      google = open('http://www.google.com/').read
      assert_tidy google, :quiet
      # puts indent_xml  #  temporarily inspect the HTML we got
      assert_xpath '//form[ "f" = @name ]//input[ "Google Search" = @title ]'
    end

Note that most developer tests (”unit tests”) should never pull a web page across a wire. They should use a test pattern called “Mock the Server”, to internally calculate the page and then test it without serving it. Ruby on Rails makes this very easy, and web platforms that lock you into one vendor’s server make it very hard.

And note that Google’s homepage is not well-formed XHTML. assert_xpath requires that, so we use assert_tidy to clean up the source. Sometimes adding tests to legacy code is more important than fixing its broken tags. A new project should always write well-formed XHTML, and should not need assert_tidy.

After ensuring this case passes, we refactor it for readability. Statements should not do too many things (even in Ruby!!!), so we break up that assert_xpath into two distinct lines. assert_xpath allows us to extend a query by nesting it:

      assert_xpath '//form[ "f" = @name ]' do
        assert_xpath './/input[ "Google Search" = @title ]'
      end

The dot . character links the inner XPath to the outer one. Without it, the // notation would start the search over from the top of the document.

And assert_xpath permits : as an abbreviation for .//, so this is the final refactor:

      assert_xpath '//form[ "f" = @name ]' do
        assert_xpath :'input[ "Google Search" = @title ]'
      end

Now we upgrade to check we indeed have a submit button:

      assert_xpath '//form[ "f" = @name ]' do
        assert_xpath :'input[ "Google Search" = @title ]'
        assert_xpath :'input[ "submit"        = @type  ]'
      end

That shows how assert_xpath’s blocks are very handy, to batch together related contents. The blocks can nest together, matching how the source HTML Elements nest together.

When adding assert_xpath assertions to a project, you should alternate between adding developer tests and using XPath Checker to investigate that your expressions work the way you think they do. XPath’s power can lead you to write assertions that pass for the wrong reason, if you are not careful.

XPath Can See Around Corners

Suppose Google’s home page were very dynamic, and that various program systems could influence its input elements. We might feel we need an assertion showing that the Submit button was always the very next input element after the edit field:

      assert_xpath '//form[ "f" = @name ]' do
        assert_xpath './/input[ "Google Search" = @title ]/../input[ "submit" = @type ]'
      end

The “..” means to step back to the parent tag, and search inside it again.

That query only detects that the Submit tag is any sibling, not the next sibling. To figure out how to test this, use XPath Checker, and enter “.//input[ “Google Search” = @title ]/../input“. Note that we get 4 input fields. The extra criterion, “[ “submit” = @type ]“, would have simply picked one of them, without enforcing its location.

The “direction” of an XPath query is called its “axis”. The default axis, /, reaches from container to contained nodes. Fiddle with XPath Checker to see where other axes reach, such as “following-sibling::“.

The query “.//input[ “Google Search” = @title ]/following-sibling::input” will return the two submit buttons after the edit field:

To restrict our query to return only the very next input field after the edit field, add the [1] index:

When we select the node by index, not by attribute, we restrict everything down to the exact minimum that can pass our test:

         assert_xpath '//form[ "f" = @name ]' do
           assert_xpath './/input[ "Google Search" = @title ]' do
             assert_xpath 'following-sibling::input[ 1 = position() and "submit" = @type ]'
           end
         end

That test case can only pass if the very next input after the edit field is the submit button. And “1 = position()” is the explicit form of the “[1]” shortcut. Use position() when you need an “and” criterion.

Using XPath as relational queries helps us decouple test cases from production code, leading to fewer false negatives when we run each test batch.