Selenium RC and Perl

The Perl driver for SeRC complies with the standard TAP (Test Anything Protocol) format developed by the Perl team around 1992-93. Therefore it fits in well with the Perl test harness and related tools. In this protocol, each test script returns both a status code and a text message in STDOUT. The calling harness displays the messages and interprets the status code as the number of tests in each script that failed. It then summarizes those counts at the end of the run. See the Perl documentation for Test::Simple for an introduction to TAP.

For the Perl client, the Test::More modules must already be installed. This is standard on some Perl distributions, but not all. If you don't have it, use CPAN to install Bundle::Test. On WinXP I use the Camelbox Perl from Google, which does include it.

The newer selenium.pm module in the client driver files must then be copied in place of the one in the Perl library (C:\camelbox\site\lib\WWW\ on my machine). The new one is 91KB. The version it replaced was 70KB. If that module is not installed, use CPAN to install Test::Selenium first. (Why isn't the new version on CPAN?)

When tests are exported from the IDE as Perl scripts, they already have a set of 'use' statements to include various modules. I add the environment module, 'Env', to provide a means of passing parameters into the scripts. Perl will substitute these values into quoted strings as well as standard variables. (Note that strings delimited with single quotes are exempt from this substitution.) This allows me to pass in both the browser and URL to be used for each test run. There are a number of other parameters used in my own test suite.

use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
use Test::Exception;
use Env;

my $browser = $ENV{'SE_BROWSER'};
my $url = $ENV{'SE_SITE'};
my $build = $ENV{'SE_BUILD'};

# The SeRC server must already be running in another process.
my $sel = Test::WWW::Selenium->new( host => "localhost",
                                    port => 4444,
                                    browser => "*$browser",
                                    browser_url => "http://$url.example.com/" );

$sel->open_ok("http://$url.example.com/");
$sel->set_speed("1000");        # without this line, everything will error out.
$sel->wait_for_page_to_load_ok("10000"); # doesn't wait without the previous line

The 'no_plan' parameter behind 'use Test::More' says the number of tests is unknown. This is useful while developing the test script. But once the script is finalized, replace the '"no_plan'' with 'tests => 16', where the number indicates the actual number of tests in the script. Then the harness will warn you when the correct number of tests is not run.

I have written a batch file to set up the parameters and call the test scripts. All of the scripts are put into a Tests\ directory, so tests can be added or removed at will. The batch file template:

REM Run all test scripts in the Tests directory.

set SE_BROWSER=firefox
set SE_SITE=tst11.dev
set SE_BUILD=0037_00

perl -MTest::Harness -e "@ARGV= map glob, @ARGV   if  $^O =~ /^MSWin/; runtests @ARGV;" Tests/*.pl

Using glob like this, the test files will be sorted in lexical order before being run. If there are tests that need to be run in a specific order, you will need to prefix the file names with ordinal digits or letters.

Perl idioms I have implemented

I capture the basic test sequence using Selenium IDE, then export it to Perl and add a few additional pieces. Here are some of the test idioms I have identified.

Don't timeout

I have had problems with the driver not waiting for events to finish before trying to run the next test. I determined that it was possible to slow things down a little to make this problem go away. Every script now begins with these two lines.

$sel->open_ok("http://$site.example.com/");
$sel->set_speed("500");        # without this line, everything will error out.

I also discovered that the IDE does not always insert the wait_for_page_to_load delay after clicks on links. So I add that for each new page request. The ten second version looks like this.

$sel->wait_for_page_to_load_ok("10000");

Skipping a test

When there are fields that may or may not be present, it is possible to skip the tests that would fill them in. This technique uses a SKIP block to ignore the name fields if they are not present. The first parameter in the skip command is the reason to skip these tests. It is displayed as the output for those tests when skipped. The second parameter is the number of tests to skip. The rest of this line is evaluated by Perl. If it returns TRUE, the test is skipped. "unless" is a Perl idiom for "if not". You may have multiple SKIP blocks within a single script.

SKIP: {
   skip "Name fields not present", 2 unless $sel->is_element_present("id=theFName");
   $sel->type_ok("theFName", $patfname);
   $sel->type_ok("theLName", $patlname);
}

Check the page title

If you need to check to make sure the test has arrived at the correct page, test the text from the title. Be careful, if any of this is dynamic, like the site name, you need to leave it out of the test.

like($sel->get_title(), qr/Deposit Status/, "Deposit completed");

Checking text

A number of my pages contain configurable text. A two pass test can be used to insert text through the admin page, then test the site to verify the change was actually made. These lines can be used to validate the text in various contexts.

cmp_ok ($sel->get_text("css=div[id=footerNarrow]"), "=~", $build, "Correct build number");
cmp_ok ($sel->get_text("css=p[class=Success]"), "=~", "Your credit card was charged", "Successful");

The first line looks for a CSS DIV with the ID "footerNarrow" and retrieves the text from that div. The second parameter tells it to look for the third parameter within that text, in this case a string with the build number which comes from an environment variable (see above). The final parameter is the test name to be displayed in the output.

The second line looks for a paragraph with the CSS class "Success" and retrieves the text from it. It then looks for the string "Your credit card was charged" within that paragraph. Since both lines are variations of the OK test atom, they will return the normal TAP responses.

Checking tables

like($sel->get_table("id=totals.1.1"), qr/\$ 74.18/, "Charged total");
like($sel->get_table("id=totals.1.2"), qr/\$ 8.00/, "Fees total");
like($sel->get_table("id=totals.1.3"), qr/\$ 66.18/, "Deposit total");

It took me a while to figure out the tableLocator. You must have something unique to define a specific table. In my case I had three tables on one page, so I added an ID attribute to each one. i.e. <table id="totals">. This allowed me to define the #table_cell_address as above. Note that the last two parameters are offsets from the top left cell. In my page the top row was a header, but it still counted. Don't forget to escape the dollar signs or Perl will be looking for a variable to replace.

Terminate test run on a fatal condition

It is possible to truncate a test run on conditions that may be fatal. For example, it the site does not have the correct build of the code installed, I can truncate the entire test sequence with this check.

cmp_ok ($sel->get_text("css=div[id=footerNarrow]"), "=~", $build, "Correct build number")
      or BAIL_OUT("Build number doesn't match!");

This not only stops the current script, but will cascade up to the harness and prevent additional scripts from being run. It makes use of a specific side effect within Perl. The interpreter will only evaluate the right side of an 'or' condition if the left side is false. When it is true, the condition is already satisfied, so it drops through to the next line.