In many companies, appplications quickly become a web of interconnected systems with dependencies upon each other and testing changes to a single application may be hard, as the behaviour of the application may depend on interaction with other interconnnected applications.

Ideally it would be nice to have a test complete environment with (dedicated) test versions of any other application impacting the application being tested - and have a coherent and applicable test-data set to use (but let’s leave test data for another post).

The purpose of this post is to outline the archetypes of test environments which typically exist and what they would be usable for. While the dedicated, isolated and interconnected test environment may be the dream, often some environments may be more sparse, but still useful for some testing.

---

I usually work with 5 archetype of environments. In theory they can be combined to some extend, but here let me focus on the pure versions.

A. Disconnected environment

You only have the application itself. None of the integration work, and no test dependent on any of the integrations can be executed.

Cheap and easy to setup, but often with severely limited in what can be tested, if the application have many integrations.

B. Stateless mocked integrations

A stateless mocked API integration is software which can do simple API simulations. Given a certain API call with some key parameter, the same result is always returned. While rare, some applications may offer a mock api to be used (including some cloud applications), but often you would need to construct the Mock server youself (and often using software like MockServer).

Having a mock server for integrations will allow for more testing.

If you’re working on a customer management system which does a credit check of the customer. A credit check API mock could offer replies like:

• A customer name starting with A would be credit ok.
• A customer name starting with B would be credit warning.
• A customer name starting with C would be reject credit.
• and anyother first letter in customer name would give no response.

Having these mocked cases, would allow you to slightly more testing - including doing test cases requiring the credit check and testing the various possible replies from the now mocked credit check as the replies are predictable - and thus the expected test outcome too.

C. Stateful mocked environment

A slightly more complicated version of the mocked API integrations are the stateful mocks. With these the mock server has memory injected so that the replies from the mock server can be impacted by previous calls to the mock server. The change in replies may also be triggered by other things such as time or calling (test trigger APIs) wich would cause the reply in the next call to change approriately. This would allow for “flow based testing”.

There are multiple software tool which can help create stateful Mocks. One example of this is WireMock.

One simple example where stateful mocking may be useful would be a “package shipping API” where useful calls could be create shipping label and then query state of shipping (using label ID) with multiple states over time - and eventually having the mocked API telling the package was delivered or returned to sender.

D. Integrated environments (with connections to test-version of all connected applications)

The fully integrated environment is often what many companies have. One test version and one production version of all applications connected. Any test is potentially possible, but often the difficulty is the challenge that all applications may change or be broken in the test environment, as all use the same (“global”) environment to do their testing - it also often create challenges on test data, as every test often contaminate test data used by another application.

In some cases a company may have multiple complete coherent (but still global) environments - Typically TEST environment, User Acceptance Envrinment, PreProduction environment or similar. The idea beening that the closer the release pipeline comes to production the more stable the environments are and the more robust the tests can be.

E. Sharded data Integrated environments

If it isn’t possible to establish enough integrated test environments, the shared data integrated environment is a way to try (if possible) to create virtual test environments within a single environment by sharding data.

In a CRM system such sharding may be constructed like this:

• All customers with zipcodes 1000-2000 are in “environment 1”
• All customers with zipcodes 2000-3000 are in “environment 2” … and so on.

This can work for some systems, but it does require that the data can be kept in isolated batches which in practical terms often becomes really hard.

---

Test environments are hard…

While it is possible to build integrated test environments it’s often expensive and complicated.

In practical terms in big companies there’s often only the option to have one big test environment used by everyone.

To compensate the developers of one application often need to use mocking to allow them to scope the dail testing to only be within their application and have “predicable replies” from the integrations needed once deployed to a production environment.