Does your functional automation really add value?
We all know that automation is one of the key enablers for those on the CI-CD journey.
Most teams are:
talking about its benefits
talking about tooling
In your experience, or in your current project:
How long does it take for tests to run and generate reports?
In most cases, the product-under-test is available on multiple platforms – ex: Android & iOS Native, and on Web. In such cases, for the same scenario that needs to be automated, is the test implemented once for all
platforms, or once per platform?
How easy is it to debug and get to the root cause of failures?
How long does it take to update an existing test?
How long does it take to add a new test?
Do your tests run automatically via CI on a new build, or do you need to “trigger” the same?
What is the test passing percentage?
Do you “rerun” the failing tests to see if this was an intermittent issue?
Is there control on the level of parallel execution and switch to sequential execution based on context?
How clean & DRY is the code?
In my experience, unfortunately most of the functional automation that is built is:
does not run fast enough
gives inconsistent feedback, hence unreliable
Hence, for the amount of effort invested in implementing automation, are you really getting the value from this activity?
In this talk, we will discuss these challenges and why it would lead to poor ROI of automation. More importantly, we will discuss the following techniques to make automation valuable:
know the objective for the automation framework
establish criteria for tests to be automated
design your framework with proper abstraction layers
develop using appropriate design patterns
Slides: here | Talk video: here