The role of test automation has changed over the years. Successful test automation still hinges on:
- How quickly it adapts to change
- How fast it runs
- How accurate the results are
- How reliable the results are
All new systems are under constant change. Even older systems may be subject to evolution as the demands of the users change.
A suite of automated tests needs to be able to be easily updated. Any change that needs to be made should only have to occur for the area that has changed. A successful automation suite will isolate functional areas so that only the functional areas affected by a change need to be updated. Successful suites will also be updatable without too much effort and in a very short time.
Test results are needed as quickly as possible.
The trade off against speed is reliability - especially when the suite is for UI validation.
A test is not a test that does not perform a validation. A test is not a test if it does not perform a useful validation. A test is not a test if it won't report a problem if there is something functionally incorrect.
When a test runs, the errors it reports should be due to the functional workings of the application and not something like a timing error in the automation suite. False-positive results from an automation suite will result in the suite being abandoned.