Posts tagged 'testing'

Frontend testing with phantomjs and casperjs

published on January 29, 2013.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

I am not usually fond of doing much frontend stuff, but I do like to dable in some javascript from time to time. Nothing fancy, no node.js, coffeescript and the likes for me. I still feel like making applications on the server side, and have the client just show things to the user. If needed some 3rd party javascript library or framework to make my life easier, and that’s about it.

For a while I was reluctant on writing any frontend tests, integration tests, or whatever you want to call them, because, y’know, refresh-click-click-click and the testing is done. Easy. Except when it’s not. From time to time some piece of user interface gets wild on javascript and it turns out after a couple of weeks, half a dozen of bugs reported, and hundreds of refreshes and thousands of clicks, that whole thing becomes tiresome. Thus, I decided to dabble my toes deeper in the waters of the javascript world and try writing some tests for all this.

What I found first is that there’s a lot of testing libraries out there for javascript. Won’t even try listing them all. When I set out for the hunt I knew what I wanted and needed. I wanted a tool that will help me automate all my refreshclicks, or at least to some extent. Tell the tool through my tests: “go to that page, check what and how is rendered, do some things with the UI, test again”. The less dependencies it has on other things, the better. I asked around among people who are bit more fluent in javascript than I am, what are they using and what would they recommend. Not too much to my surprise, everyone recommended a different thing, whatever fits their problem. So I ended up picking the tools that fit my problem.

The tools chosen

The first tool I picked was phantomjs. It’s a “headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.” (shamelessly copy-pasted from their website). When installing it either download a binary from their website, or compile the source on your own. If you’re on Ubuntu, do not install it via apt-get. It installs some very old version, 1.4 I think, and phantomjs will just scream at you something about some X servers. The binaries are pre-built for any and all systems, so just use them (not that I tried all of them, but I trust they all work).

To cut the story short, writing tests with just phantomjs is difficult. If at all possible. Because hey! It’s not a testing library. I think. This part is a bit blurry for me.

Enter casperjs! These two together look like a perfect match for doing automated, frontend tests full of javascript.

Even though last week on Friday I was being a bit frustrated and a bit more of a dick and said bad things about casperjs over twitter which I shouldn’t have. Sorry. Broken things on a Friday at 5PM can do that to a man.

Examples!

That’s why you’re here, to see some examples. And this example, which, btw, is on Github here, will have a simple login page with a form and a logged in page with some UI elements that you can click around and do stuff! And we’re going to test all that with phatnomjs and casperjs. Just like real life. Cookies included.

When writing tests with phantomjs and casperjs, all one need to do is think in steps. Same simple steps like in the refreshclick procedure. First, open page. Make sure we are on the correct page. Make sure all the elements are there. Click something. Make sure that thing is clicked. Fill in a field. Make sure the field is filled. And so on.

Let’s have a look at the first part of the tests.js file:

casper.start('http://localhost/frontend-testing', function () {
    this.test.assertUrlMatch(/login.php$/, 'Redirected to login page');
    this.test.assertExist("#login_form", 'Login form exists');
    this.fill('#login_form', {
        'email': 'email@example.com'
    }, false); // false means don't autosubmit the form
    this.test.assertField('email', 'email@example.com');
});
casper.thenClick('#login', function () {
    this.test.assertUrlMatch(/index.php$/, 'Redirected to index page after login');
});

The tests are surprisingly self-explanatory: start by opening up the home page. Assert that we are (redirected to) on the login page and that the login form exists. Then fill the email field of the login form with a given value. Assert that the field was indeed filled with that value. Once that’s done, then click on the login button, and assert that we end up on the index page.

Easy!

And practically that goes on in the entire test. Start, assert, then do this, assert, then do that, assert, done.

Testing ajax calls isn’t difficult either:

casper.thenClick('#do_ajax', function () {
    this.waitForResource('http://localhost/frontend-testing/ajax.php');
});
casper.then(function () {
    // Sometimes we need to wait a bit more for ajax requests ...
    this.wait(50);
});
casper.then(function () {
    this.test.assertTextExist('Just some ajax response.', 'Ajax request was made');
});

Do some action that triggers an ajax request, wait for that ajax request to finish and assert that something was done with the response from that ajax call. Of course, in real life examples you will have a bit more complicated setup, but hey… As for faking ajax requests I hear that can be done with this cool sinonjs library, but I haven’t managed to get that working, yet. Mostly because I didn’t need to fake any ajax calls.

The most voodoo-like thing in these tests is probably the evaluate() part. I don’t think I really know what that it is, but what I think it is, that, whenever you want to do something in the actual webpage, but from within the tests, you use evaluate().

For example, I had to use evaluate() to determine is the checkbox checked or not:

casper.thenClick('#enable_ajax', function () {
    // I could swear I had this one working
    // this.test.assertEquals(this.getElementAttribute('#enable_ajax', 'checked'), 'checked', 'Checkbox is checked');
    this.test.assertTrue(this.evaluate(function () { 
        return document.getElementById("enable_ajax").checked;
    }), 'Checkbox is checked');
});

Not sure how, but the this.getElementAttribute() way does actually work in my other tests. Honest. Not sure why it didn’t work in this example. Maybe some other factors not present here, affected my other tests? I don’t know.

Helpful bits

What I found extremely helpful while writing the tests is the this.debugHtml() and this.debugPage() casper functions. The former will dump the entire HTML to the terminal, and the latter will dump just the text of the entire page. Can be useful to figure out what’s going on.

The other helpful debugging function I used a lot is this.getElementAttribute(<selector>). It retrieves a bunch of helpful information on an element, which you can then dump to the terminal, to further figure out things:

casper.then(function () {
    require('utils').dump(this.getElementAttribute('#enable_ajax'));
});

Will result in an output like this:

{
    "attributes": {
        "id": "enable_ajax",
        "name": "enable_ajax",
        "type": "checkbox"
    },
    "height": 13,
    "html": "",
    "nodeName": "input",
    "tag": "<input type=\"checkbox\" name=\"enable_ajax\" id=\"enable_ajax\">",
    "text": "",
    "visible": true,
    "width": 13,
    "x": 12,
    "y": 58
}

I mean, it even gives the positions of the checkbox on the page! Super helpful.

casperjs supports CSS3 selectors and, which gives me much joy, XPath. Makes getting those pesky little elements covered in layers of divs and tables a walk in the park.

One thing though: you can’t do things like selecting multiple elements at once and then .each()‘em or something like that. You have to select elements one by one and do assertions on each of them. Even if the selector matches multiple elements, it will just return the first element, so you’ll probably end up using lots of :nth-child(n) selectors. But that’s what copy-paste was invented for.

That’s about it, I guess. Of course, there’s much more to both phantomjs and casperjs, but I found the documentations well written, so I believe it’s fairly easy to tackle even the testing of more complicated web pages with these libraries.

Happy testing!

Unit testing Zend Framework 2 modules

published on September 15, 2012.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

Porting this blog to Zend Framework 2, I decided to write some unit tests as well, while I’m at it. Not that the current code base doesn’t have unit tests, just it doesn’t have much of it… Anyway, I’d like to show how to get unit tests for modules up and running, as well how to throw in Mockery in the mix, as it can help us greatly with mocking out objects. Some of the parts shown here probably could be written in a more cleaner/nicer way, especially the autoloading bit, but so far it works for me.

The phpunit.xml file is rather simple:

<phpunit bootstrap='./bootstrap.php' colors='true'>
    <testsuite name='ZF2 Module Test Suite'>
        <directory>.</directory>
    </testsuite>
    <filter>
        <whitelist>
            <directory suffix='.php'>../src/</directory>
        </whitelist>
    </filter>
    <listeners>
        <listener class="\Mockery\Adapter\Phpunit\TestListener"
            file="Mockery/Adapter/Phpunit/TestListener.php"></listener>
    </listeners>
</phpunit>

The Mockery TestListener, as I found out the hard way, is needed for Mockery to work properly. You might add in some more stuff, like generating code coverage reports, and the like.

In the bootstrap.php we set up the autoloading for the modules, the ZF2 library, and Mockery:

<?php
putenv('ZF2_PATH=' . __DIR__ . '/../../../vendor/ZF2/library');
include_once __DIR__ . '/../../../init_autoloader.php';
set_include_path(implode(PATH_SEPARATOR, array(
    '.',
    __DIR__ . '/../src',
    __DIR__ . '/../../SomeRequiredModule/src',
    __DIR__ . '/../../../vendor',
    get_include_path(),
)));
spl_autoload_register(function($class) {
    $file = str_replace(array('\\', '_'), DIRECTORY_SEPARATOR, $class) . '.php';
    if (false === ($realpath = stream_resolve_include_path($file))) {
        return false;
    }
    include_once $realpath;
});
$loader = new \Mockery\Loader;
$loader->register();

It assumes that the currently tested module lives inside a ZF2 application. If not, you’ll probably need to adjust the paths accordingly. It also assumes that the Mockery files are in the vendor/ directory.

Testing the service layer

Don’t want to get into a fight about terminology, but the service layer for me, is the layer that lives between the controller layers and the database layers. It allows for keeping other layers clean of business logic, and easier testing. These services implement the Zend\ServiceManager\ServiceLocatorAwareInterface, which greatly simplifies unit testing, as it is easier to replace concrete objects with mocks.

Let’s assume that we have a “post” service, which we can use to get the recent posts. The post service itself does not interact with the databse, but calls an AbstractTableGateway which does all the database work. A test case for this post service, to avoid database calls, should mock the AbstractTableGateway, and use the ServiceManager to replace the concrete implementation with the mock object. An example test case for this post service could look something like this:

<?php
namespace BlogModule\Service;
use PHPUnit_Framework_TestCase as TestCase;
use Zend\ServiceManager\ServiceManager;
use Zend\Db\ResultSet\ResultSet;
use \Mockery as m;
class PostTest extends TestCase
{
    protected $postService;
    /**
    * @var Zend\ServiceManager\ServiceLocatorInterface
    */
    protected $serviceManager;
    public function setup()
    {
        $this->postService = new Post;
        $this->serviceManager = new ServiceManager;
        $this->postService->setServiceLocator($this->serviceManager);
    }
    public function testGetRecentPosts()
    {
        $mock = m::mock('Blog\Model\Table\Post');
        $this->serviceManager->setService('blogModelTablePost', $mock);
        $result = array(
            array(
                'id' => 1,
                'title' => 'Foo',
            ),
        );
        $resultSet = new ResultSet;
        $resultSet->initialize($result);
        $mock->shouldReceive('getRecentPosts')
            ->once()
            ->andReturn($resultSet);
        $posts = $this->postService->getRecentPosts();
        $this->assertSame($posts, $resultSet);
    }
}

On line 18 we set the service manager to be used with the post service, on line 22 we create a mock object, and on line 23 we set that mock object in the service manager. We set some expectations on the mock object - what method should be called, how many times and what should it return. Finally we call the actual method that is being tested on the post service and assert that the returned result is correct.

Testing the database layer

For testing the database layer, that is the AbstractTableGateway implementations, I use a little… trick. I don’t actually test what is returned from the database, but that the correct Sql\Select objects are being called, with the correct parameters in a correct order. This, in turn, means that I trust the underlying Zend\Db code that in the end, it will assemble the correct SQL queries, but I also don’t have to bother with setting up a test database, and also the tests run faster, as they don’t actually call the database. An example test case, continuing our example of getting recent posts:

<?php
namespace Blog\Model\Table;
use PHPUnit_Framework_TestCase as TestCase;
use Zend\ServiceManager\ServiceManager;
use Zend\Db\Sql\Select;
use \Mockery as m;
class PostTest extends TestCase
{
    protected $postTable;
    protected $select;
    protected $tableName = 'blog_posts';
    public function setup()
    {
        $adapter = $this->getAdapterMock();
        $this->postTable = new Post($adapter);
        $this->select = m::mock(new Select($this->tableName));
        $this->postTable->setSelect($this->select);
    }
    public function testGetRecentPosts()
    {
        $this->select->shouldReceive('from')
            ->once()
            ->with($this->tableName)
            ->andReturn($this->select);
        $this->select->shouldReceive('where')
            ->once()
            ->with(array('published = ?' => 1))
            ->andReturn($this->select);
        $this->select->shouldReceive('order')
            ->once()
            ->with('id DESC')
            ->andReturn($this->select);
        $this->select->shouldReceive('limit')
            ->once()
            ->with(10)
            ->andReturn($this->select);
        $this->postTable->getRecentPosts();
    }
}

Here we create a mock adapter (the getAdapterMock method can be seen in this gist), and use that mock adapter in our AbstractTableGateway implementation. We also create a mock Sql\Select object and we set expectations on that mock Select object. As I said, this way of testing might not be the best way out there, but it did help me catch a bug where I failed to add the where clause on the actual Select object. Yey for Mockery! Oh, and please do note that the adapter mock might not work in all cases, but again, so far it worked nicely for me.

One other thing would probably be interesting to show how to test, and that’s the action controllers, but I haven’t got around to write any controllers yet, so I’ll probably leave that for part 2. Or you could do it for homework.

Happy hackin’!

Robert Basic

Robert Basic

Software engineer, consultant, open source contributor.

Let's work together!

If you require outsourcing or consulting help on your projects, I'm available!

Robert Basic © 2008 — 2019
Get the feed