Automation is an increasingly valued approach to software testing. It not only brings about time and cost savings in projects, but in the long run it allows us to deliver better quality products to a customer. When deciding on automation, it is worth carefully considering what tools to use, including test frameworks. In this article you will find an overview of the most important types of test automation frameworks. I will also take a closer look at the not yet widely known tool Atata, based on Selenium WebDriver, and compare its capabilities with those offered by pure Selenium.
When should we automate software tests and why?
What is Automation?
Before we focus on the topic of the Atata test automation framework, I would like to briefly remind you what software test automation is in general and why it is worth taking an interest in it. There are a number of myths around this topic.
Testing is a very important part of the software development process. One of the approaches to this aspect is automation, i.e., the use of special tools and frameworks that enable the programming of tests and automatic execution without further human intervention.
Benefits of Automation
Automation of these processes can bring countless benefits, including some of the major ones – time and cost savings. In addition, automatic tests are more repeatable and more accurate than manual tests, which translates to reliability and effectiveness.
Scenarios and test cases and automation
It is worth considering test automation in your project due to the option of conducting tests with complex scenarios and in many environments at the same time, which could be a challenge when manually implementing a test scenario on a daily basis. It is possible to perform tests on different platforms, in different hardware and network configurations, which would be difficult or even impossible to achieve in the case of manual tests.
Creating test cases in an automated form ensures their repeatability, and thus increases their effectiveness when it comes to detecting defects. This is why automation is particularly useful in testing the critical functionalities of the app. However, it is worth remembering that the test reality is not black and white , and like everything, it has its pros and cons. What are the downsides of automation?
The Dark Side of Automation
Test automation can be a fantastic solution in many projects, but it is not a universal solution. In some cases, manual testing can be a much better, more effective option, or even the only one available. Therefore, before deciding to automate tests, it is worth carefully analyzing the specifics of the project, the needs, and any available tools and frameworks.
You also need to bear in mind the time that must be invested in the creation of automated test cases. This requires many hours of work, configuration, and troubleshooting that you do not have to deal with during manual software testing. This is a process that can develop very slowly at first, and credible results are often visible only after putting a lot of work into the project. There is also a need to hire an automation tester who will be able to design the entire system so that the tests are stable and in line with good programming practices.
Types of test automation frameworks
Automation frameworks are tools that make this task easier by providing solutions to many common problems. Depending on the needs and specifics of the project, we can choose from different types of frameworks. Below is a brief description of some of them.
Library Architecture Framework
The Library Architecture Framework (LAF) is a popular software test automation technique. As part of this methodology, libraries with ready-made test functions that can be called in the test code are used. LAF is often used in cases where tests are complicated and require multiple stages, as well as in situations where tests must work on different operating systems and web browsers.
This type of framework is particularly useful for low-level tests, such as unit tests.
Module Based Testing Framework (MBTF)
This is a type of software test automation framework that is based on a modular approach to test design. In this model, tests are designed in a modular way, and each module covers one functionality or test cases related to one application module. In MBTF, test modules are designed in such a way as to be universal and can be used repeatedly for various applications and systems. Once you’ve created a set of modules, you can use them in different projects without having to rewrite the code.
Linear Automation Framework
Linear Automation Framework is a type of test automation framework that is based on a linear approach. At LAF, tests are designed to reflect business processes or application usage scenarios. This means that they are performed in a sequential manner. In this model, tests are designed as sets of steps, and each step of the test is associated with a user action or result that is expected in the application. The advantage of the Linear Automation Framework is that the tests are closer to the actual use cases, which can help us to find more realistic defects in the application, such as those that the user would come across.
Hybrid Test Automation Framework
This is a kind of framework that combines the features of all the above-mentioned approaches. This enables the design and execution of tests in a flexible way, tailored to the needs of the project. In HTAF, tests are created in a modular way, similar to the Module Based Testing Framework, but with more flexibility and access to a wide range of test tools. This framework allows the integration of various tools and technologies, such as Selenium WebDriver, TestNG, Appium, JMeter, etc., which allows us to automate tests in various environments, such as web, mobile, desktop applications and many others.
Read also: End to end testing with Cypress
Atata automation framework
Atata is a full-featured test automation framework that is based on Selenium WebDriver and designed for web applications. It was written in C# and offers a wide range of functionalities that make it easier to automate web application tests.
Atata – Advantages
- Availability – The Atata framework is fully open source and available on the GitHub platform under the Apache License 2.0.
- Transparency and ease of use – one of the key elements of the Atata framework is the use of the Fluent Page Object Pattern. This pattern allows you to write code that is easy to read and understand. The built-in login system makes it easy to track and troubleshoot problems during testing.
- Triggers – what is more, the framework has a unique trigger functionality that allows you to run custom code when specific events occur.
- Low Entrance Threshold – Atata is designed to be easy to use and maintain. The framework provides a simple and intuitive syntax for defining and using the objects on the page. This requires a minimum amount of code, which positively affects readability and facilitates the introduction of further testers to the project.
- Ready-made components – the framework also provides a set of ready-to-use components to facilitate work. These components include controls, pages, and layouts. Atata provides a wide range of built-in controls that can be used to interact with web elements on the site. Controls include buttons, text boxes, check boxes, radio buttons, and many, many more.
- Data-driven testing – Atata also supports data-driven testing, allowing testers to create tests using the Data-Driven Testing function. With this feature, developers can run the same test scenario with different datasets.
- Parallel testing – the framework also supports parallel testing. With parallel testing, testers can run multiple tests at the same time, which can significantly reduce the time taken for testing. This is especially useful for large web applications that require extensive quality control.
Atata – disadvantages
Despite its many advantages, the Atata framework also has some drawbacks.
- Limited use – one of the main drawbacks is that the framework usage is limited to websites only. This is an important limitation, for example, for companies developing software that also design other types of applications in addition to websites. In this case, you may need to use another test automation tool.
- Available only for C# – this framework is only created for the C# language, which makes it unavailable to many programmers who specialize in other popular programming languages.
- Small community – another disadvantage, compared to other test automation tools, may be the small community surrounding it. This framework is not as well-known as other test tools which have been available on the market for years. We need to take into account the fact that there are fewer online resources and documentation, as well as support materials available to users. Fortunately, the user community is highly engaged and ready to help. It is also worth noting that contact with the creators of the framework is extremely simple, and communication is highly efficient.
However, despite these disadvantages, the Atata framework is still one of the best tools for automating C# web tests. Its numerous advantages make it popular among testers who are looking for a simple and intuitive way to automate.
Atata vs Selenium
Let me start by saying that the question “Atata or Selenium?” is not quite appropriate in the context of the issue we are discussing. In particular, comparing Selenium to an overlay, which in fact Atata is, may seem baseless. After all, Atata is nothing more than a framework covering the database with its functionalities, which provides us with the necessary basic tools.
Thanks to this we get a “nicer” version of Selenium, which is more user-friendly and faster to implement. Nevertheless, I decided to lay out the important differences in the table below. It is worth being aware of the benefits of using such an overlay for the project. After all, when planning work, choosing a framework is not easy; the more options there are, the harder it is to make a decision even within frameworks that rely on other tools which are already available.
One of the biggest advantages of the Atata framework is its useful login system which allows you to track and diagnose problems during testing. Atata also offers unique features such as a trigger mechanism that allows you to execute custom code in response to specific events during testing.
Atata also offers an intuitive syntax for defining and using page objects, making it easy to program and maintain tests. It supports the use of the Page Object Pattern model, which is a huge advantage for fans of this pattern. This makes the structure clear and easier to understand.
The framework has a built-in set of ready-made components such as controls, pages and layouts, which speeds up the test automation process. Below I have provided an example of what Atata offers – it not only contains basic elements of pages such as text fields, but also differentiates fields for e-mail addresses, passwords, numbers or files. Furthermore, it supports choice lists, clickable items, and many, many more. A full list of specifications and capabilities can always be found in the very clear documentation at https://atata. io/components/.
Despite its enormous capabilities, however, it sometimes turns out that we lack a component or it is impossible to use a ready-made solution offered by Atata. Luckily, the developers haven’t removed the option to access a pure version of Selenium, and we can search for our elements or apply specific actions to them with no issues. Below is an example of a WebDriver call.
Fans of reports and tables will also find a useful function here. Atata offers a built-in tool to take screenshots according to the selected pattern, e.g., after performing each step. If we connect this function with a library for creating reports, such as ExtendedReports, we will get a very pleasant environment for generating reports in the form of a .html file. Once the tests are run, we have an immediate view of the results along with statistics, marked errors, tags and graphs. At first glance, we can say that IDE itself can also offer such data but the form of compiling them in this library is much more readable and easy on the eye. After small changes to adapt to the needs of the project, we get a report that can be presented almost effortlessly during a business meeting.
The issue of navigating between application pages during one session was also intuitively solved. In the example below, you can see that using the Go function, It (go to) or He (stay on the page) is sufficient. Just upload the appropriate page address according to the method – there is no need to open and close the session properly (as with Selenium) without an overlay. This contributes to significant time savings when configuring your environment.
Unfortunately, this framework was created only for the C# programming language, which is a significant limitation when compared to Selenium, which also works with Java and Python.
Which test automation framework is best for my project?
The choice of test automation framework depends on many factors, such as project type, business requirements, developer preferences, and resources available. Here are some steps that can help you choose the best framework for your project:
- Define your requirements
Before deciding to use a given framework to automate tests, you need to know what your requirements are. For example, if your project involves the development of a web application, then a framework that supports user interface testing might be the best choice. There is also no point in choosing a technology that supports other types of applications as well at the expense of the one that interests us.
2. Find the right tools
Consult developers and testers who have experience in test automation and ask them for recommendations on tools. Collect information about different frameworks, their functionalities, limitations and social resources.
3. Test the frameworks
Test selected frameworks in practice. Prepare a set of tests representative of your project and implement them in each of the frameworks. Analyze which frameworks are the most intuitive and easy to use.
4. Check the integration
If the developed application uses external tools, make sure that the framework you choose integrates with them. For example, if you use a Continuous Integration platform, make sure the framework you choose has a plugin for it.
5. Pay attention to the availability of documentation and community support
Choose a framework that has good documentation and a wide community of users. This will help to solve the technical problems you may encounter when working with the framework.
To sum up, the choice of a test automation framework depends on many factors and there is no clear answer to the question of which one will work best in your project. Currently, there are many types of frameworks available on the market, so you need to explore different options and choose the one that best suits your requirements and project specifics. I hope that with this article I have also managed to detail the capabilities of the not as yet well-known tool Atata.
Although Selenium is one of the more popular and widely appreciated tools, it is always worth knowing about other options. Atata is constantly being developed by the developer community, which means that anyone can contribute to advancements by reporting bugs, making improvements or writing documentation. This way, Atata becomes a better tool.