SOA Testing is Hard

  • 2017-04-02

You gotta climb the highest mountain to master the hill
You gotta climb over your ego to master your will

Logic, Never Been

Since my first post on Bluebook I've been thinking of clear ways to explain what's behind the idea and how I think software testing should be performed. This is what we're going to talk about in the first follow up on Bluebook project.

What does it mean to test software?

Testing software at some levels is easy, and at others it's hard. Software testing can be partitioned into four categories.

  • Automated technology facing tests.
  • Automated business facing tests.
  • Manual technology facing tests.
  • Manual business facing tests.

Automated technology facing tests

Every engineer is familiar with these types of tests. These tests include simple unit tests, service integration tests and even tests at the system level. We write these tests to ensure that our software is behaving correctly at various levels of the software stack under different types of conditions.

Automated business facing tests

When we're talking about business facing tests we want to understand whether the product that we're offering is behaving as expected. This is what we call functional testing. These tests don't focus as much on single service integration tests, but they are placing more weight on end-to-end testing, which usually starts at the UI level, whatever the definition of UI is.

Manual technology facing tests

I've seen these types of tests mostly done by platform and infrastructure teams. Manual technology testing includes things such as capacity planning, security and penetration testing.

Manual business facing tests

Manual business facing tests usually revolve around interactions with the product. It includes things such as usability testing or A/B testing. The purpose of these tests is to optimize user satisfaction and various revenue channels.

Where does Bluebook fit in?

The idea of Bluebook is to solve the problems that are relevant to the above automation categories within SOA niche. The hypothesis I have is that SOA is not going anywhere, and microservices are going to become even more popular. When you have microservices running on your infrastructure you need an easy way to write integration and functional tests. These tests should be run on your pre-production environments before the changes are released, and then run periodically on production environment to ensure that everything is operating as expected. We also want to be notified when tests are failing.

There's actually nothing new about what I said. The trickiest part is about writing and managing these tests. Currently there's only Runscope that's very close in terms of functionality and ease of use that I'm looking for. Any other setup that I've seen involved some crazy build pipeline for running tests.

How will Bluebook solve SOA testing?

By offering entire package for service testing. Test management, execution, service virtualization, monitoring and alerting must be part of one offering. I'm a believer in fully assembled products. From the small research that I've done there's not many products that do that.

This is how I'm going to change the way services are gonna be tested:

  1. There's going to be a new language for writing service tests.
  2. Once foundation is strong we will have central control for managing and executing tests.
  3. Monitoring and alerting will follow.

The above 3 goals are massive, but we don't have to finish all of them before offering something to the users. I'm going to explore each of these goals in more detail when it's the right time to do so. Language that's specific to service testing is crucial. It provides lock in opportunities and gives immediate tools to the engineers. Make these tools useful and easy to use and you have your initial user base which will serve as foundation for further market penetration.

Hey there! I'm working on a new goal management software. Check out Simple OKR — The easiest way to manage OKRs.

Simple OKR

I'm working on a new web app for managing goals with objectives and key results. Check it out it's awesome.

Simple OKR — The easiest way to manage OKRs

Bluebook - API Testing for Developers

API, end-to-end, and integration testing made simple.

Try Now


Subscribe to stay up to date with the latest content:

Hut for macOS

Design and prototype web APIs and services.

Download Now