JavaScript Tests and Their Automation

Posted on by

This post is also available in: Russian

Test case efficiency is vital for large scale projects, when parts of applications vary in their behavior. Perhaps the most common case is when a large group of developers is working on the same or interfacing modules. This often results in unexpected changes in behavior of functions written by other programmers. Also, firefighting sometimes may result in inadvertent changes to critical parts of the application.

Testing of a Web application usually involves visual review of page elements and empirical assessment of the functional performance. In other words, a tester browses the application sections and makes actions on dynamic elements.

Over time, the project grows in functionality so that its testing becomes longer and more sophisticated. To automate the process, modular (unit) testing is used.

There are two approaches to build test cases:

  • Whitebox Testing – tests are based on the implemented functionality. It means that testing uses the same algorithms that are run by the system modules. This approach fails to ensure end-to-end system performance.
  • Blackbox Testing – test cases are based on specifications and system requirements. This way, you can verify the end results of your application, however you cannot troubleshoot small and occasional bugs.

What to Test

It could have seemed that you should test every feature you have implemented in your app. This is not entirely the case. It takes some time for the developer to write tests, so to optimize the development process write tests just for complex, critical features or that using results from other modules.  Use tests to cover controversial logic that can potentially cause errors. Also it is worth creating tests for those parts of the code that you plan to optimize, so that after optimization you can assess their performance.

Generally, it is essential to assess testing costs against your timeframes. Of course, if you’re not limited in time, you can afford covering each feature by tests.  But as a rule, development process runs under a hard time pressure, and the analyst or an expert developer should be able to discern where testing is necessary. Moreover, writing of tests increases the cost of the project.

Based on this, we can formulate three cases where unit testing is justified:

1)     When tests ensure quicker bug identification than common troubleshooting.

2)     When tests reduce debugging time

3)     When tests apply to frequently changed code.

Of the 3 major frontend components (HTML, CSS, JavaScript), perhaps only the JavaScript should be tested. CSS is tested but visually, when a developer/tester/customer browses the GUI in different browsers. HTML markup is tested in a similar way.

How to Test

While building your test cases, please adhere to the following guidelines:

  • Make your tests as simple as possible. In this case, the test result is more likely to indicate that very bug you are trying to reproduce.
  • Decompose tests run on large modules. This will make it easier to locate a specific bug .
  • Make your tests independent. The result of one test should in no case depend on the result of another one.
  • Test results should be fully repeatable and expectable. Each time you rerun your test, the result should be identical to the previous run.
  • For any application error, a separate test case should be created. This way you can make sure that a bug has really been fixed and would not emerge to the end users.

How to Test

For unit testing of JS code, several libraries exist. Perhaps the most commonly used library is QUnit. To run unit tests using this library, we should create a "sandbox", a simple HTML page hosting a test library, your tested code, and the tests themselves.

Test Functions:

(function() {
    window.stepen = function(int) {
        var result = 2;

        for (var i = 1; i< int; i ++) {
            result = result * 2;
        }

        return result;
    }

    window.returnFunc = function() {
        return 'ok';
    }
})();

Test Listing:

test('stepen()', function() {
    equal(stepen(2), 4, '2^2 - equal method');
    ok(stepen(3) === 8, '2^3 - ok method');
    deepEqual(stepen(5), 32, '2^5 - deepEqual method');
});

asyncTest('returnFunc()', function() {
    setTimeout(function() {
        equal(returnFunc(), 'ok', 'Async Func Test');
        start();
    }, 1000);
});

As you can see, QUnit supports three functions to validate code results against the expected performance:

  • ok() – considers the test successful if the return value = true
  • equal () – validates result against expectation
  • deepEqual () – validates result against expectation, including result type

Execution result:

As you can see, QUnit can test your code in multiple browsers at once.

There is a number of other unit test libraries. However, they share the same concept of building test cases, so you can easily move to another library, if needed.

Please Keep in Mind

Modern JS code is asynchronous. As a rule, test libraries support asynchronous tests. But if you are trying to test a feature that is sending a GET request to the backend and returning a response from it, you’ll have to stop the flow using stop(), run the tested feature and then re-start the flow using start() wrapped in setTimeout().  So you have to make for a certain time for the function to complete.  Please choose carefully the period length as, on the one hand, lengthy execution of a method may be specific or even necessary to an application or signify of its error.

Testing Backbone Applications

To test applications based on Backbone.js, let’s use the project described in our post "How to Develop Large-Scale JavaScript Based Applications."

Using unit tests, you can validate:

  • Your models and controllers
  • Data used in the models
  • Controller method execution (for this, they have to return a result)
  • Success of loading the views

Test code:

test('Backbone.js', function() {
    ok(sample, 'Namespace check');

    ok(sample.routers.app, 'Router check');
    ok(sample.core.pageManager.open('chat'), 'Page opening test
                 (Controller method call)')

    ok(sample.core.state, 'Model check');
    equal(sample.core.state.get('content'), 'sintel', 'Model data get test');

    stop();
    ok(function() {
        $.ajax({
            url: 'app/templates/about.tpl',
            dataType: 'text'
        }).done(function(data) {
                self.$el.html(data);
                return data;
            })
    }, 'Template loading check');

    setTimeout(function() {
        start();
    }, 1000);
});

Result of testing error handling:

Automating Test Execution

As a rule, application deployment is quite a frequent task in heavy development. Therefore, this operation is typically automated. We prefer to use Jenkins, a continuous integration tool. The idea is to combine Jenkins based deployment with automated tests.

QUnit tests are run in a browser. To bypass this, phantomjs is used to emulate the browser. Phantomjs developers have made a preset script to run QUnit tests, but we updated it a bit for better performance.

Test.js:

/**
 * Wait until the test condition is true or a timeout occurs.
 * Useful for waiting
 * on a server response or for a ui change (fadeIn, etc.) to occur.
 *
 * @param testFx javascript condition that evaluates to a boolean,
 * it can be passed in as a string (e.g.: "1 == 1" or
 * "$('#bar').is(':visible')" or
 * as a callback function.
 * @param onReady what to do when testFx condition is fulfilled,
 * it can be passed in as a string (e.g.: "1 == 1" or
 * "$('#bar').is(':visible')" or
 * as a callback function.
 * @param timeOutMillis the max amount of time to wait. If not
 * specified, 3 sec is used.
 */
function waitFor(testFx, onReady, timeOutMillis) {
    var maxtimeOutMillis = timeOutMillis ? timeOutMillis : 3001,
            //< Default Max Timout is 3s
        start = new Date().getTime(),
        condition = false,
        interval = setInterval(function() {
            if ( (new Date().getTime() - start < maxtimeOutMillis) &&
                   !condition ) {
                // If not time-out yet and condition not yet fulfilled
                condition = (typeof(testFx) === "string" ? eval(testFx)
                           : testFx());
                   //< defensive code
            } else {
                if(!condition) {
                    // If condition still not fulfilled
                    // (timeout but condition is 'false')
                    console.log("'waitFor()' timeout");
                    phantom.exit(1);
                } else {
                    // Condition fulfilled (timeout and/or condition is
                    //'true')
                    console.log("'waitFor()' finished in " +
                         (new Date().getTime() - start) + "ms.");
                    typeof(onReady) === "string" ? eval(onReady)
                         : onReady();
                      //< Do what it's supposed to do once the
                      // condition is fulfilled
                    clearInterval(interval); //< Stop this interval
                }
            }
        }, 100); // repeat check every 250ms };
};

if (phantom.args.length === 0 || phantom.args.length > 2)
    console.log('Usage: run-qunit.js URL');
    phantom.exit();
}

var page = new WebPage();

// Route "console.log()" calls from within the Page
// context to the main Phantom context (i.e. current "this")
page.onConsoleMessage = function(msg) {
    console.log(msg);
};

page.open(phantom.args[0], function(status){
    if (status !== "success") {
        console.log("Unable to access network");
        phantom.exit();
    } else {
        waitFor(function(){
            return page.evaluate(function(){
                var el = document.getElementById('qunit-testresult');
                if (el && el.innerText.match('completed')) {
                    return true;
                }
                return false;
            });
        }, function(){
            var failedNum = page.evaluate(function(){
                var el = document.getElementById('qunit-testresult');
                console.log(el.innerText);
                try {
                    return document.getElementsByClassName('fail')[0].
                                    innerHTML.length;
                } catch (e) { return 0; }
                return 10000;
            });
            phantom.exit((parseInt(failedNum, 10) > 0) ? 1 : 0);
        });
    }
});

To display results in the message console, add the logging function:

QUnit.log = function(test){
            if (!test.result) {
                console.log('- ' + test.message +
                 ' (expected:' + test.expected + '; actual: ' +test.actual);
                console.log('    ' + test.source);
            } else {
                console.log('+ ' + test.message);
            }
        }

Now let’s launch Jenkins, create a new project and make a new job in it:

Add a new build step with a shell command:

Now let’s enter the following command:

phantomjs test.js test.html

Now we just have to specify a build directory (i.e., that hosting a sandbox) and run the build:

Example of test failure:

Helpful links

Leave a Reply