Monday 24 February 2014

Creating an Eclipse UI to run AngularJS e2e tests using Karma

This post shows how I created a nice set of views in Eclipse to quickly see the execution result of AngularJS e2e (end-to-end) tests, without leaving the Eclipse UI.

The image below shows this UI in action, where:
  • The source code of the test is shown in the Eclipse Java editor
  • Just below is the console out of the Karma runner (which is detecting files changes)
  • On the top-right is the hooked browser (i.e. the one that will run the tests)
  • On the middle-right is the simple AngularJS Hello World page
  • On the bottom-right is the debug view of the hooked browser (which is what you get if you click on the Debug Button included on the top-right page)


Here are the tools and Eclipse Plugins used to create this UI:
The code used on this sample is based on my past attempts of using Karma (see A small AngularJS Jasmine test executed by KarmaJS ,  Running KarmaJS’s AngularJS example test/e2e/angular-scenario (on Chrome) and KarmaJS related posts)

Although the AngularJS and NodeJs Eclipse plugins provide nice helpers and views, they didn't provide (yet) the kind of UI that I was looking for. Namely they didn't support the use of KarmaJS to run AngularJS tests.

But since I now have the ability to quickly manipulate and create new Eclipse views without restarting Eclipse (using the Groovy REPL script environment ) it was not very hard to create the UI that I wanted (see Eclipse Plugin that allows the execution of REPL Groovy Scripts (in the current Eclipse Instance) and Fluent API for Eclipse+SWT).

Basically the brief was to:
  • Create new (Eclipse) view with a browser showing http://localhost:9879/__karma/ (the KarmaJs browser runner/hook)
  • Create new view with a browser showing http://localhost:9879/index.html (the AngularJS page)
  • Create new view with a browser showing http://localhost:9879/__karma/debug.html (debug view of Karma runner), with an toolbar button to refresh it.
Here is the gist with the code that creates this 3 Eclipse views:

Hopefully this Groovy script is easier to read (the idea of the Fluent API that I added to the Groovy REPL was allow the easy reading and understanding of scripts like this).

1) Setting up the environment

Now lets look at how this environment was setup:

We start with a simple AngularJS project test page that will just show the model-binding capabilities of AngularJS (yes I know that all those JS files should be in separate folders :) )



Then I opened up Groovy REPL script environment and wrote the script (shown above) that creates the multiple views:


Clicking on Execute will create 3 views:

a) the Karma Runner view:


b) the Website view:


c) the Karma Runner - debug view:


... which (as seen in the first screenshot of this post) looks like this:


Part of this UI (but not created using the Groovy script) is an Eclipse Console with the 'Karma Runner process' console out :


The AngularJS tests were executed when the Karma Runner view was opened, because the Karma process (configured via Karma.config.js) is set to wait for new browser's hooks/runners and (on connection or code changes) execute the configured tests.

2) Making changes to existing tests

 To see Karma in action, let's make a test fail :)

The current AngularJS page is basically just echoing/binding the contents of the Name TextBox into the H1 tag:


Here is the test that is currently being executed, which is an e2e test that runs on the browser (just like Selenium or Watin).


Hopefully this type of Fluent API is easy to read:

  • browser navigates to /index.html
  • value is inserted into the yourName input field (the TextBox)
  • the value of the binded element (.ng-binding) is checked to see if it matches

To make this test fail, let's change just the yourName value:


Immediately after saving the test with the changes shown above, Karma refreshes the hooked browsers in order to trigger the execution of the tests.

And as the image below will show, there is now one failed test:


In cases like this, the Karma Runner - debug is very useful since it will show more details on what happened, and where the test failed:


Just to confirm that the page is actually being loaded and the tests are happening in a real browser instance, if we add an Javascript alert to the current test:


... we will get an alert box in the  Karma Runner - debug (note that on the screenshot below we' are seeing the image of the final state of execution of the previous test (in this case the 'A small Angular Test')



 3) Hooking multiple browsers and only failing on Firefox

 In order to run the tests in multiple browsers all we need to do is to open the http://localhost:9879/__karma/ page (in the target browser), and a test execution will be triggered.

Note how on the image below:
  • Eclipse is in the background (containing the views previously created and showing the console out of the Karma test runner process)
  • There are 3 stand lone browser windows (Chrome, Safari and Firefox)
  • Firefox (top most window) shows the tests being executed (in green the test executed, in yellow the test being executed)


After execution, a quick look at the Karma runner shows that the modified test failed (as expected) on all browsers:


Just to make sure all is working as expected, let's create a test that will fail only in one browser.

For example Firefox is the only one that has defined the navigator.userAgent.contains Javascript function (so we can use this to detect Firefox, and create a test that will only fail if executed inside it):


After saving the changes, the tests will be executed on all 4 browsers, with the one test that failed being the one executed in Firefox:


Refreshing the Firefox debug runner, shows the failed test and assert:



4) Setup

 In addition to creating the views programatically, I also setup an Eclipse Run configurations for NodeJS and an External Tools Configuration for KarmaJS.

The NodeJS configuration was done here:


... where I created a Node Application runner that was pointed to server.js


... which contains a simple NodeJS web server (we could also had used NodeJS Express, but that would had added a lot of other 'stuff' to this project)


The KarmaJS config was done as an External Tools Configuration


... where I simply run the start.sh bash file


... which starts KarmaJS:



... using this simple karma.conf.js configuration


Note: I started the Karma process like this because there was an issue with KarmaJS finding the NodeJS executable from Eclipse (and at the time I didn't had time to debug why that was happening)

External References:
Posted by at
Labels: , ,