Choosing the End in End-to-end Tests for AngularJS App

There are many tutorials available that provide insights into the use of specific testing frameworks or guidelines on how to write end-to-end tests, but very few of these cover what the end point of these tests should be. Should the end be stubbed API calls or a fake API? Or should it be a real API, but with a special test database?

Every one of the various testing approaches that are available have their own drawbacks and choosing one for your project always involves some type of trade-off. Some tests are too brittle and fail randomly, while others are too time consuming to setup and maintain. I recently discovered an approach that is somewhat of a compromise but seems to require little effort to setup and has proven itself capable of finding real application bugs.

This article describes the end-to-end testing of single page applications (SPA) that are built on an AngularJS or other JS framework. The testing of SPA differs from testing traditional web applications because SPA consists of client-side app and back-end API. This gives more seams for mocking and more testing approaches to choose from. Let’s review which options are currently available for testing SPAs.

Stubbed HTTP Responses

In this scenario, the mocking framework is injected into application, which short-circuits HTTP calls. Instead of going to any remote back-end API, the client-side app receives canned JSON responses.

The main problem with this approach is that tests don’t go all the way to the back-end API. As such, this method doesn’t really allow end-to-end tests in the true sense and it won’t catch bugs when the API endpoint starts working differently.

Another problem is that it places an extra burden on developers to create and keep canned JSON responses up to date for every API endpoint.

Stubbed back-end API

This is a variation of the previous approach. Here HTTP requests go further, to the stubbed back-end API server, which replies with canned responses. However, this once again is not a real back-end API. Even completely different technology can be used to emulate real back-end API. For example, you can build real API with .NET and use node.js or 3rd party service to serve fake API responses.

There are number of services and tools available to enable such functionality. For example, http://apiary.io/ looks interesting as, besides serving as a fake API server, it can also generate nice documentation for the API.

The problems associated with this approach are the same as those described for the Stubbed HTTP Response method. Tests don’t cover the real back-end and developers are still required to create and maintain canned HTTP responses.

Special database for tests

Another potential approach is to make tests go all the way to the database through real back-end API. However, a special database is required specifically for UI tests, which is different from the main development database. This database is reset to a predefined state before each test run.

Such tests are much closer to reality as they test the same moving parts as the real user. Because tests go through the real back-end API they can catch integration problems on the client side app with back-end API.

I believe that the biggest problem with this method is that it requires extra effort to set up a special test database, get it to a predefined state before tests run and make sure its schema and data are up to date. For small databases this is not such a big deal, but maintaining a separate database for tests with hundreds of tables is time consuming.

Development database for tests

Now, I recently discovered a trade-off approach that is worth sharing. Just as described in the previous approach, this method involves tests that go all the way to the database using real back-end API. However, in this case, no special database is used. The same database that developers use on a daily basis is also used for the purposes of the end-to-end tests. This frees developers from the hassle of maintaining two databases.

The trick here is to limit test cases; for example, do not delete records and use clear naming conventions for the entities that are involved in the test cases. For example, records can be named “DO NOT TOUCH, USED IN TESTS,” so that developers can quickly see what they are and won’t mistakenly delete them. Tests will expect these records to always be there, their URLs will be known upfront, so tests can navigate to them directly.

Sure, there are some problems with this approach:

  • You can’t test every single scenario using this method, for example the deletion of records. But do you really need to? Maintaining tons of end-to-end tests is not fun and people hate broken UI tests. Besides, it is a better idea to have fewer end-to-end tests (only for critical paths) and more unit tests.
  • These tests are slower when compared with canned responses as they involve real databases. However, they also tend to find more real problems and are easier to setup.
  • Tests might fail because someone might delete the record that the test expects. However, this shouldn’t happen too often and it is also easy to fix.

As with all approaches to testing, some trade-offs can be expected and you should always carefully consider which approach is right for your project. I think that the benefits associated with the method proposed in this article, in terms of low ceremony setup and the capability to test the entire stack overweight shortcomings for many projects, make this approach a feasible and potentially valuable method.

  • Andrew Ganzz

    Hello Alexander,

    thank you for your article. For now I’m in a process of finding a best way of testing our SPA – as we have the outer DB which is maintained by other people and we need to find out whether there were some changes in API etc. So I assume that your proposed way is the one we will choose. As for deletion problem (deletion on the real db) – I would suggest just start test which creates test record first and then run the deletion test.

    Thanks,
    Andrew