What Is Intelligent Test Automation & Analytics? Definition + Examples

In my latest book, Accelerating Software Quality: Machine Learning and Artificial Intelligence in DevOps, I wrote about innovative ways that testers are applying AI and ML to become more efficient, more productive, and smarter. This broader movement has a name: Intelligent Test Automation.

Keep reading for my working definition, reasons why it is such a big deal, and some real-world applications.

What Is Intelligent Test Automation (ITA)?

Intelligent test automation (ITA) is a method for software testing that uses AI algorithms to help author scripts, make automation more stable, and generate analytics for more efficient debugging and decision making, among others.

Why Teams Adopt Intelligent Test Automation

Less Busywork

Software testing keeps teams busy. From writing scripts to maintaining scripts to determining root causes of test failure. Manual testing on multiple devices is the enemy of swift, agile development.

With intelligent test automation, teams can offload much of the time-consuming work as AI and/or ML supports them and works on their behalf. Test teams execute many functional tests on multiple devices in hours - not days. That means more focus on the work they enjoy (which just so happens to be the work that matters most).  With faster feedback, developers can fix bugs and return code quickly to the testing team.

Smarter Decisions

Data is only as good as the analysis that follows it. Testing teams create unbelievable amounts of test results. Making sense of it is a job all its own. Intelligent test automation and analytics find ways to reduces the glut of data into actionable insights.

Faster Releases

Today’s software and apps must be released quickly and continuously in the face of increasingly complex and fragmented processes — and devices. When your testing feedback is accelerated, so are your results. With intelligent test automation and analytics, manual processes are swapped out with autonomous ones. Teams can move faster, decide faster, and release faster.

[Watch On-Demand: Expert sessions on AI & ML in continuous testing, code quality, and DevOps productivity.]

The Most Common Challenges of Test Automation

However, test automation does have some challenges, especially when it is first adopted. 

•    Complex test creation leads to a high percentage of manual testing.
•    Reporting noise like false negatives results in irrelevant test feedback.
•    Untrusted test automation leads to a high percentage of manual testing.
•    Execute parallel testing while scaling requires significant IT investment.
•    Managing an in-house lab is costly and difficult to keep up to date.

These common challenges are easily overcome with the right automation strategy and the right automation tools.

[Read Our Guide: Best Practices to Solving the Top Digital Test-Automation-At-Scale Challenges.]

Real-World Examples of Intelligent Test Automation

Now that you have a high-level understanding of what intelligent test automation is and can do, let’s cover a few examples.

Intelligent Test Authoring

When it comes to authoring test automation, what if AI could help?

In a product like 21 Labs, AI can learn an application — specifically the functionality and navigation of each screen — based on the tests that have been created. It allows teams to add assertions, change test data, reorganize the flow, and run in minutes. Of course, you can customize as necessary.

Such tools are based on standard test frameworks, like Appium and Selenium, making them very generic to author across platforms and dev frameworks.

AI in 21 Labs, a scriptless mobile testing technology, learns an app's functionality and can pre-build tests.

Intelligent Test Maintenance

Simple changes, like moving a button to a different side of the screen, can be great for UX. But such changes can be the biggest productivity drains for anyone on the QA side of the house. Those broken tests must be fixed. No fun.

One of the big game-changers for teams has been the self-healing capabilities that AI has introduced. In technology like Perfecto Scriptless, an algorithm automatically overcomes over 90% of the changes (like the one I mentioned above) in an app. The test does not break from a simple change. Testers can work on new tests instead of fixing old ones.

AI in Perfecto Scriptless, above, prevents automation from breaking by learning your application and attributing weights to the most important elements.
 

Intelligent Test Automation Analytics

As I mentioned earlier, teams — especially those conducting continuous testing or testing at a large scale — generate mountains of test results. This burdens anyone in charge of analysis, anyone who is required to allocate and prioritize, for example, what should be dropped into Jira.

In a product such as Perfecto, ML makes the whole analysis aspect much more intelligent. The reporting engine is able to detect and classify errors and then separate the false negatives (cause by pop-ups, element not found, server down, and more) from the real failures that need to be fixed.

You can literally and figuratively turn the volume down on data noise and make analysis much more productive.

Intelligent test automation analytics, like that pictured above in Perfecto, can filter out false negatives.
 

Bottom Line

Intelligent test automation is impacting testing. It is making quality easier to scale and achieve.

At the same time, it does have limits.

While AI and ML have made substantive improvements to testing, there are still many quality activities that must be addressed and solved with nothing less than the human mind. Autonomous processes still need human oversight, for example.

But if there’s one key point I hope to make with this blog it is this: Intelligent test automation lets testers be more efficient and strategic about the things that matter.

See How AI Transforms Test Analytics in Perfecto

Perfecto's cloud-based Intelligent Test Automation platform uses AI to improve test creation and analytics.

Further Resources

• [WEBINAR] Advancing the State of the Art in AI & Testing
• [BLOG] AI in Software Testing
• [BOOK] Accelerating Software Quality: AI & ML in the Age of DevOps