Today’s Approach to Testing Has No Future – Part I

So here we have a reality in which new technologies, trends and software are becoming more and more complex, which of course means that the work of a tester is much more demanding. And if we add the ubiquitous agile approach in which the tester creates the product along with the rest of the team, and thus must have sufficiently broad programming skills (or rather, we would like it to be so), it gets really hard.

Let’s not kid ourselves – the role of software testers is still underestimated. Despite efforts to build awareness of the essence of testing, there are still companies that would prefer to avoid the costs involved. Some people may think this is a “prehistoric” approach, but we can still come across the following opinions:

• tests do not bring measurable benefits,
• tests make programmers lazy,
• what’s the point of testing, if the developer should program everything as he is supposed to?

This is obviously a short-sighted approach, as in reality these arguments can be reduced to two aspects, namely money and a desire to reduce costs. In addition, unfortunately, the testing process itself is designed in such a way that it does not actually provide enough added value to the project. And this is not necessarily the fault of the testers themselves.

Testing – how it looks in practice

Let’s take a look at a typical project. We have certain needs in terms of performing tests in the course of a given project, yet they are not fixed, but are often highly varied and subject to change over time. Of course, at the beginning of the project, we try to anticipate and manage such different eventualities appropriately, but we must always assume that debt will arise sooner or later, resulting from the simple dissemination of possibilities in relation to the resulting needs.

Let’s assume that we delegate two people to our abovementioned project. What does it mean? That we have two testers who have a given amount of time, specific knowledge and experience. Even if they are very experienced testers who are able to perform various types of tests such as manual testing, automation, performance tests, safety tests, API testing, static testing, we can be sure that they are not specialists in every area. So they are able to provide us with a defined quality. In other words, it may turn out that we have very good manual testers, who are just learning how to carry out performance tests, so the quality of these tests will be somewhat uncertain.

This means that, having finite resources, we will never be able to provide the highest quality tests in the context of all needs which emerge – even if our testers give their best efforts and work or study after hours. And what about when the work of testers is no longer needed? After all, we still pay for the chairs occupied by the testers – which of course means that resources are squandered.

Figure 1. Needs with regards to the delivered tests. The dotted line indicates the work done, the shaded area the values that cover the needs of the project, and the unshaded area represents the areas not covered by testing.

Testing tools

We must also consider the issue of tools. The fact that we have specific tools at our disposal does not mean that they are the best. Typically, the choice of tools resembles the way we choose the bakery in which we buy our bread rolls every day. The best rolls are not actually the best ones in the whole city, but those that were good enough to stop us looking around for another bakery. Bread rolls are not always our priority, just as testing is not a priority for many software companies. Because they do not specialize in testing, they rarely have knowledge of the latest, optimal solutions, and it is difficult to require them to have a wide range of such solutions at their command.

Specialization

Priority and specialization are the key words in this case, because software development companies will not usually treat testing as the core of their activity. Rather, programming is. So maybe such a company should look for a suitable technology partner, for whom testing is the priority?

When working in a system where our testers have to meet very different requirements, we will always have problems with appropriate test coverage. Such a system not only provides inefficient tests, but also produces employees who are “masters of nothing”, testers who can do “a little of this, a little of that”. The problem is that even with the best of intentions and predispositions, these testers face the “average ability” trap.

So let’s stop kidding ourselves. Good quality automated tests will not be performed by a tester who deals with them only sporadically, because every day they deal with manual tests, or sometimes performance tests. Automated testing requires a specialized tester who will devote more time not only to preparing testing scripts, but also to maintaining them, refactoring, etc. On the other hand, maintaining a dedicated tester in the team to perform tests every four months is not financially viable.

Safety and efficiency are also good examples. We do these tests every so often, so having a full-time specialist tester is hard to justify, at least from an economic point of view. So what if we want our tests to be performed by an experienced specialist who will deliver the highest quality tests? The solution to this problem seems to be services: internal or external.

To be continued…