Articles | February 24, 2023

Test automation – common myths

There are several myths regarding the implementation, maintenance, costs and types of tests that can be automated. Discussions on test automation have kept coming up during multiple IT conferences and meetups. The number of tools, and thereby the possibilities of using them, is increasing; however vague knowledge of test automation does not help to popularize such tests and therefore to deliver products of the highest quality. In the article, I shall examine automated testing and the common misunderstandings around automated tests to clear up doubts in this area.

Test automation – common myths

What is test automation? How does it work?

Before I discuss the test automation process further, a quick reminder: automated testing is software testing by executing a set of test cases using dedicated tools. Now, let’s see how such tests are created.

Firstly, we specify areas requiring automated testing, focusing on relevant issues from a business perspective. We set up the goal and terms depending on the tested system, bearing in mind one of the principles of testing: “Testing is context-dependent”.

Once the goal is set up, we proceed to the technical aspects. We choose a testing tool, and create the testing environment and testing scripts. Having run tests, we analyze the results and verify whether we have managed to obtain business value as assumed earlier on.

Deterministic automated tests work in a quite simple way. First of all, we put the application in the initial state to test a desired action, for example how the application acts while being used by, say, 1000 users. The aim of the verification is to compare the assumed results with the actual state.

Read also: Software testing: do we really need more tests?

Types of automated tests

There are several different types of automated tests. To get the most out of it, you need to know the key differences between the two. Tests are divided into functional and non-functional.

nearshore 2023.11.xx graphic 1 2

Functional testing

Is a quality assurance process and a type of black box testing that bases its test cases on the specifications of the software component under test.

  • Unit testing: Unit tests are the first level of testing and form the basis for other more extensive types of testing. These are usually done manually by the developers.
  • Integration testing: This type allows for layered testing. This type allows for layered testing. Generally, it’s about integrating 1 component with 2. For example, API with a database. This is often at the code level. 1st class with 2nd class (if developers do it).
  • System testing: It’s a type of functional test that only covers the most crucial features of the software to ensure that it could be further tested.
  • Regression testing: A type of change-related testing to detect whether defects have been introduced or uncovered in unchanged areas of the software.
  • User Acceptance Testing: Software test, the purpose of which is not to detect errors, but only to obtain a formal confirmation of the implementation of the software of appropriate quality.

Non-functional testing

Is testing an application or system against non-functional requirements: how the system works, not the specific behaviors of that system.

  • Documentation testing: includes the plans, results, and testing of a system or system component. It includes test case specifications, test plans, test procedures, test reports, and test logs.
  • Installation testing: Also known as Implementation testing and it is done to check whether the software is properly installed in the new system with its prerequisites and necessary data and is working as expected or not.
  • Performance testing: The practice of evaluating how a system performs in terms of responsiveness and stability under a particular workload. Performance tests are typically executed to examine speed, robustness, reliability, and application size.
  • Reliability testing: An important software testing technique that is performed by the team to ensure that the software is performing and functioning consistently in each environmental condition as well as in a specified period.
  • Security testing: Type of Software Testing that uncovers vulnerabilities, threats, risks in a software application and prevents malicious attacks from intruders.

Automation testing and myths

1. Each test can be automated

According to one of the widespread myths, each test can be automated. Let’s think it over: is it really possible? As per the International Software Testing Qualifications Board – the software testing certification organization – the testing process is infinite, which is why we cannot forecast and test everything.

Test automation is still incomparable to efficiency tests, volume testing, and stress testing. Automated testing has its limitations, though. For example, it cannot be used for non-functional tests. These are related to our personal perception of esthetics, legibility and transparency.

In the abovementioned area we are unable to define the initial requirements and the expected final results. Some of the aspects, for example the appearance and transparency of the application, are perceptible only to the human eye. The same thing applies to accessibility testing, which aims to determine whether users with disabilities can use the system with ease.

All in all, knowing that some of the tests cannot be automated, we understand that it is impossible to entirely replace manual testing with automated testing – as per the second common myth. There will always be areas requiring manual testing.

2. Automated tests can replace manual testing

This misunderstanding brings to mind a common myth from the past, that one day television would entirely replace radio. Nowadays, as we can see, radio, television and the Internet coexist and complement each other. It works similarly when it comes to automated and manual tests.
The first ones are advantageous due to their repeatability. With appropriately implemented tests we can be sure that every test will be run in the same, replicable manner.

 Additionally, contrary to human nature, automatons do not need coffee, the weekend or sleep, so they can work perpetually. On the other hand, humans are better when it comes to intelligence, intuition and imagination, which are essential to running an effective test of the application.

The majority of tests related to functional and non-functional requirements can be performed manually. There will always be a need for manual tests, calling for the engagement and knowledge of a tester – namely usability tests, accessibility tests and security tests.

Test automation supports manual testing, but cannot replace exploratory testing or the technique of error guessing, wherein the tester’s knowledge in the context of the product and tools is essential.

3. Automated tests help to catch a lot of defects

Contrary to popular opinion, automated tests will not help to detect a lot of defects. One can ask: “How can that be? Isn’t that what we run them for?” Of course, it is one of the goals we are trying to attain; however, automated tests are quality-oriented, not quantity-oriented. To give an example, automated tests are useful in catching functionalities’ change-related defects in the event that software code has been amended. These are called regression tests and are used to ensure that changes implemented to either software, configuration or the environment will not impact the developed product itself.

Automated regression tests can successfully replace manual testing carried out by entire development teams, which – depending on the software’s complexity – would normally take hours, days or even weeks. Our trust increases with every single automated test. Even though they do not catch a lot of defects, they allow us to save time and money, enabling us to focus on the quality of the product instead.

4. Each automatable case should be automated

Knowing that not all tests can be automated and that there are areas in which we will still need manual testing – is it worth automating every automatable testing case?

There are situations in which swift software testing is required and engaging additional resources, tools or funds (which is often related to test automation) is not an option. Let’s paraphrase a quote by Roger Needham, Microsoft’s Research Laboratory Director, to clear up doubts: automation is a process of replacing something that works with something that almost works, but is quicker and cheaper.
This reminds us of the two principal ways of defining whether we should automate tests or not:

  • automated tests should be faster than manual verification
  • the cost of programming tests cannot exceed the income gained thanks to test automation

From a business perspective, the cost of tools and time needed for the implementation of automated tests in comparison with manual testing should be the decisive factor.
 This should facilitate the decision-making process in terms of automating testing cases.

Read more: The test pyramid – what, when and how to test software to get profits

Benefits of automation testing

Automated tests have many benefits. Application testing is a process. Every time the code is modified, it needs to be changed and retested in different modifications or systems. In such a situation, manual testing would be extremely time-consuming and thereby costly. With test automation, you can run tests multiple times at no extra cost. 

Test automation also allows for continuous testing and a quick check of the app. This allows you to avoid bottlenecks at the end of the software development process. Test also automation helps make your life easier. Reporting in manual tests is uncomfortable and thanks to test automation, reporting is smooth.

Test automation tools

So how to run the tests in the most effective way? Here is a list of a few best practices for test automation, as well as the tools and automation frameworks that will help you achieve success.

To start with, determine which test cases should be automated first. Then select an automation tool for testing. If you have already defined your needs, you can proceed and choose the appropriate test solution, for example:


Despite many myths related to test automation, automated tests are gaining greater understanding and popularity. Myths related to automated tests keep coming up, which proves that a lot still needs to be done to clarify doubts. I hope that I have managed to clear up some of them and that the article will help to define further discussion on automated tests. This is priceless help for testers in the process of delivering a high-quality product.

Technical leader and software tester, specializes in automated testing. Her adventure with testing began quite unexpectedly – on the production line of the Slovak car factory. Then she entered the world of embedded software testing and fell in love with the diverse world of mobile and web applications.

Exclusive Content Awaits!

Dive deep into our special resources and insights. Subscribe to our newsletter now and stay ahead of the curve.

Information on the processing of personal data

Exclusive Content Awaits!

Dive deep into our special resources and insights. Subscribe to our newsletter now and stay ahead of the curve.

Information on the processing of personal data

Subscribe to our newsletter to unlock this file

Dive deep into our special resources and insights. Subscribe now and stay ahead of the curve – Exclusive Content Awaits

Information on the processing of personal data

Almost There!

We’ve sent a verification email to your address. Please click on the confirmation link inside to enjoy our latest updates.


If there is no message in your inbox within 5 minutes then also check your *spam* folder.

Already Part of the Crew!

Looks like you’re already subscribed to our newsletter. Stay tuned for the latest updates!

Oops, Something Went Wrong!

We encountered an unexpected error while processing your request. Please try again later or contact our support team for assistance.

    Get notified about new articles

    Be a part of something more than just newsletter

    I hereby agree that Inetum Polska Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as: my full name, e-mail address, telephone number and Skype ID/name for commercial purposes.
    I hereby agree that Inetum Polska Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as: my full name, e-mail address and telephone number for marketing purposes.

    Read more

    Just one click away!

    We've sent you an email containing a confirmation link. Please open your inbox and finalize your subscription there to receive your e-book copy.

    Note: If you don't see that email in your inbox shortly, check your spam folder.