What is a test script? There are many definitions, but we'll focus on the one that seems to be the most common in the field and the most practical for today's QA professionals:
A test script is code that can be run automatically to perform a test on a user interface. The code will typically do the following one or more times:
(1) Identify input elements in the UI, (2) Navigate to the required UI component, wait and verify that input elements show up, (3) Simulate user input, (4) Identify output elements, (5) Wait and verify that output elements display the result, (6) Read the result from the output elements, (7) Assert that output value is equal to expected value, and (8) Write the result of the test to a log.
For example, if you want to automatically test the login function on a website, your test script might do the following:
You'll notice we added a broken line after step #3, to separate between two important parts of the test script:
Using these basic elements of GUI automation, simulation and validation, you can test even very complex multistep operations. In complex test scripts these elements will repeat themselves several times, each time for a different part of the user's workflow.
Now that we understand what a test script is, there are a few questions remaining:
We'll cover the answers to these questions in the sections below.
There are a few different ways to create automated testing scripts: record/playback, keyword/datadriven scripting, and directly writing code.
You are probably familiar with tools like HP Quick Test Professional (now called UFT), which help QA professionals create automated tests. With these tools, testers can perform a set of user actions on an application or website, “record” those user actions, and generate a test script that can “play back” or repeat those operations automatically.
QTP and similar systems identify objects in the user interface being tested, and enable testers to view the hierarchy of objects in the application under test. It is possible to edit and modify tests by selecting from these objects and specifying user actions that should be simulated on them.
It's important to note:
Record/playback systems do generate test scripts behind the scenes but they are typically written in simple scripting languages like VBScript. Advanced users can (and typically will) go in and manipulate the code directly to finetune how a test behaves.
Some testing tools make it possible to define a set of “keywords” which specify user actions. Testers can specify these keywords and a script will be generated that performs the desired actions on the system under test.
For example, the keyword “login user” might designate that the script should navigate to the login screen and enter certain user credentials. A developer has to write code behind the scenes that implements this functionality, and then a tester can provide a table like this:
Or lines of text like this:
Open Browser To Login Page Input Username david Input password 16485 Submit Credentials
And a script will be generated that knows how to navigate to the login screen, type in the data provided (username “david” and password “16485”) and click Submit.
Here too there is a distinction between the simulation and validation parts of the test script:
A variant of this approach, called data-driven testing, is when the same test needs to be repeated numerous times with different data values or different user operations. For example, in an eCommerce system you might want to perform a purchase numerous times, each time selecting a different product, different shipping options, etc, and observe the result of every test. The tester can provide a table indicating which operations will be performed, and which data should be input in each iteration of the test, and this helps to specify a complex set of tests with very little effort.
The advantage of keyword/data driven testing is that the code that simulates user operations is not repeated again and again in every test script, rather it is defined in one place, and a tester can use it as a building block in multiple test scripts. This make it easier to define test scripts without needing to write code, and also reduces the maintenance required for the test scripts as the application under test changes.
A well-known keyword-driven testing tool is the open source Robot Framework. Many other test automation frameworks provide some keyword-driven or data-driven testing functionality.
Below is an example of how a login operation is specified using keywords in Robot Framework:
A few common frameworks that allow you to write test scripts in almost any programming language are Selenium (for web applications), Appium (for mobile), and Microsoft Coded UI (for Windows applications).
These and similar frameworks provide special commands you can use in a test script to perform operations on the application under test, such as identifying screen elements, selecting options from menus, clicking, typing, etc.
Here too there is a distinction between simulation and validation:
By the way, most test automation frameworks, including Selenium and Microsoft Coded UI, also provide an option for record/playback scripting, like in HP QTP. The difference is that the primary way of working with these frameworks is directly writing the test script code, and that they use “real” programming languages and not simplified scripting languages.
The record/playback functionality is often seen as a way to start for beginners, but the expectation is that eventually the advanced user will write the code directly without relying on record/playback.
If you read the previous section, it should be clear that the answer depends on the way you chose to create the test script:
Test scripts are run by automation frameworks (sometimes called “test harnesses”). We provided several examples of such frameworks above HP QTP, Selenium, Appium, and Robot Framework. These kinds of frameworks know how to take a script written according to their specifications load up a user interface, actually simulate the user actions in the interface, and report on the result.
The test scripts you write will have to conform to the test framework you are using. Typically you can't produce a “generic” test script and run it on any test automation system, nor can you easily port your tests from one system to another.
Here is how you will actually run the tests:
Once you get into automated testing with scripts, you quickly realize that authoring and maintaining scripts is very time consuming. No matter which method or tool you use to build your scripts, you are likely to run into the following vicious cycle:
At the end of the day, you will find that your test coverage is limited by the time you have to author and maintain your scripts (or by your access to development resources to help you build your scripts).
QA professionals are measured on test coverage our main goal is to test as many important features as possible to reveal as many defects as possible. Automated testing was supposed to help with that by testing more in less time. But most QA teams find that they are able to test only a small fraction of the functionality with their automated testing scripts.
Think about it if you could make test scripts much simpler and easier to maintain, you could build many more test scripts in less time, and have enough time to maintain them. You'd have higher test coverage, discover more defects automatically and make everyone happy!
How do you simplify your test scripts? Let's go back to our original distinction between two parts of the test script:
It's always possible to simplify your script by doing less validation but that's a cop out. If you already have a test script running, not checking a few more parts of the interface under test is a shame. But the more you validate, the bigger your script gets and the more time is needed to maintain it.
That's where we come in. We at Applitools built Applitools Eyes, a practical tool that does visual testing which handles the validation part of a test much more easily than you can do with ordinary scripts. A test script can do only the simulation part, and then call the Applitools API to perform the validation part with no code (eliminating steps 4 through 8 in all your test scripts).
Here's how Applitools Eyes works in a nutshell:
Visual UI testing is a different approach that has two major benefits:
Visual UI testing can prevent the vicious cycle you can get much more test coverage without adding many more lines of code to your test scripts, and without increasing the time to author and maintain your scripts.
Most importantly, with visual UI testing you will deal with much less code and focus on what you do best stamping out bugs. Give it a try for free!