For this example, we will be looking at Coverage Test setup on Android in pre-production (staging). The focus of this test cycle is to run pre-production regression.

Note: this is not a template or the only way to set up a test of this type; this is merely a detailed example.

First, we will select the Test Type we will be running (in this case, Coverage).

Next, we need to specify the Test Environment, which is the environment we want testers to evaluate.

Then, we need to specify the particulars of accessing this test.

After adding an environment, we need to select which bugs are in scope for this test.

Note: Since this is a Coverage Test, all levels of severity bugs will be reported.

Different test types include different severity levels in the test scope. Sometimes the bug severities you define internally may vary from test IO's default guidelines. To make sure our views on severity are in line, here’s how we classify bugs and issues. 

Critical - Preventing a function of the app or website, causes a potential loss of income for the company running the app or website - i.e. an app crash or "not able to login."

High - Serious impact on user experience but doesn’t prevent the function of the app or website.

Low - Minimal impact on user experience.

Visual - The user can accomplish a task, but the interface looks wrong, typically because of responsive design, CSS, HTML, or layout framework problems.

Content - Bugs relate to missing data, images, or broken links.

Note: Non-Functional Bugs are classified as low-severity bugs by definition.

Next, we will select which Features we are testing. These are the Features we want to get cover in this test cycle. Depending on the Test Type (Coverage, in this case), you may have the option to select a variety of bug types that are in-scope for this test as well as a variety of features. At least one Feature should be in-scope to be able to run a test.

First, select the “Add feature” option.

Then, fill out the necessary input required to add the new feature

For more in-depth information, best practices, and tips on creating Features, check out this article.
​

Here, we are concerned with “pre-release regression,” particularly in terms of international coverage, so we checked all the Features that are relevant to our changes and have enabled Functional, Content, and Visual bugs.

Moving down the page, we continue with the test setup.

First, we need to state the Goal of the test. This section can make the difference between a good and a great test cycle!

Note: This section should be your guide for our testers. Think about the desired results of the test and what information the testers need in order to discover these types of issues. Common examples include the release of a new feature, a sanity test for critical functional issues only, changes to a common workflow, etc.
​
It’s also common to include a brief description of your company and product so testers have some context about the software they’re testing. In other words, keep in mind that test instructions are the only way for you to communicate your needs and expectations to a crowd of testers who might have never had experience with your site/platform/app before. 

Next, we state what/if any functionalities are Out of scope. This section should explicitly state any Features or functionality out of scope for this test. It’s also common to:

Then, we state any Additional requirements. This section should be used to provide any additional information testers should know to achieve the desired results. Examples include:

We have no Attachments for this test cycle and will skip this step.

Next, we will need to specify “Where to test” (on which devices) and the “Bug language.”

When selecting “Where to test,” we will be choosing which devices our testers will use to run our test. We need to keep in mind our audience and their devices. Since we’re testing for Web and Mobile Web, we’ll include desktops, smartphones, and tablets.

Note: You can define device scope by editing device type, device model, OS, and/or browsers. Please keep in mind that any “unconventional” or outdated device, OS, and browser combinations may lead to irrelevant device specific bugs. 

Note: Bug report language is defaulted to English.

Lastly, we select our desired test date and start time. In this case, we are running a single Coverage test, which is set to begin at 7:00pm Wednesday, 6th March, delivering results by 7:00pm the following day.

Note: Due to our Team Lead review and invitation process, the earliest a test can start is the next full hour +1 (e.g. current time: 9:40am / test time: 11:00am, current time: 5:05pm / test time: 7:00pm). Duration ranges vary based on Test Type, from two hours (Rapid Test) up to 48 hours.

Click Create Test.
This will essentially save the test in "draft." At this point:

•You should review all test information and make any adjustments as needed before you submit the test and tester/Team Leader invitations go out.
•If you need to leave and return to it, can find your test on the "All Tests" page in the same state. 

To edit the Goal, Out of Scope, Additional Requirement fields before submitting follow the appropriate steps below:

To edit the Features before submitting follow the appropriate steps below:

Once all information has been confirmed you can Submit the test.