For this example, we will be looking at Rapid Test setup on Web/Mobile Web in production (live). The focus of this test cycle is to check recently implemented hotfixes.

Note: Rapid tests are our fastest turnaround tests and are available to our Professional and Elite customers.

Note: this is not a template or the only way to set up a test of this type; this is merely a detailed example.

Step 1

First, we will select the Test Type we will be running (in this case, Rapid).

Next, we need to specify the Test Environment, which is the environment that we want the testers to evaluate.

Then, we need to specify the particulars of accessing this test.

First, we input the Title of the test. In this case, we are running the test on “product filtering” in a production (live) environment, so the we’ve named the test “Product filtering, PROD.”
It’s a website, so we’ve input the appropriate URL and made sure “Url,” not “Upload,” is selected.
Since it’s a live environment, we do not need a username or password to access staging and instead are using the test IO proxy (include “Use test IO Proxy” in “Further instructions”) to access the live site. It’s important that we whitelist the given IP.

Note: Since this is a Rapid Test, only critical severity bugs will be reported.

Closer Look: Bug Severities

Different test types include different severity levels in the test scope. Sometimes the bug severities you define internally may vary from test IO's default guidelines. To make sure our views on severity are in line, here’s how we classify bugs and issues.

Functional Bugs

Critical - Preventing a function of the app or website, causes a potential loss of income for the company running the app or website - i.e. an app crash or "not able to login."

High - Serious impact on user experience but doesn’t prevent the function of the app or website.

Low - Minimal impact on user experience.

Non-Functional Bugs

Usability - suggested improvements to existing features and functions that would make the product easier and more intuitive to use.

Visual - The user can accomplish a task, but the interface looks wrong, typically because of responsive design, CSS, HTML, or layout framework problems.

Content - Bugs relate to missing data, images, or broken links.

Note: Non-Functional Bugs are classified as low-severity bugs by definition.

Step 2

Next, we will select which Features we are testing. These are the features we want to cover in this test cycle. Depending on the Test Type (Rapid, in this case), you may have the option to select a variety of bug types that are in-scope for this test as well as a variety of features. At least one Feature should be in-scope to be able to run a test.

Since we are only concerned with “product filtering” for this rapid test, we won’t select any default features but will create our own using “Add feature.”

The name is that of the functionality we are testing, “Product filtering.”
Describe where on the site the feature can be found.
Define the expected behavior (the correct functionality) of this feature.

Note: Make sure that you’ve checked the Feature box for Features you wish to include in your test (for Rapid Tests, there is a limit of four Features per test cycle).

Closer Look: Features

For more in-depth information, best practices, and tips on creating Features, check out this article.

Step 3

Moving down the page, we continue with the test setup.

First, we need to state the Goal of the test. This section can make the difference between a good and a great test cycle!

Note: This section should be your guide for our testers. Think about the desired results of the test and what information the testers need in order to discover these types of issues. Common examples include the release of a new feature, a sanity test for critical functional issues only, changes to a common workflow, etc.

It’s also common to include a brief description of your company and product so testers have some context about the software they’re testing. In other words, keep in mind that test instructions are the only way for you to communicate your needs and expectations to a crowd of testers who might have never had experience with your site/platform/app before.

“The purpose of this test is to ensure that all combinations of product filtering work correctly: Price, Location, Size, Brand, etc. Product filters can be found on the landing page and any page while shopping. Please test on all pages where the filters exist.”

Next, we state what/if any functionalities are Out of scope. This section should explicitly state any Features or functionality that are out of scope for this test. It’s also common to:

State undesired functional severities or bug types in this sections
Inform testers not to take particular steps in the flow (make final purchases, contact support, send out applications etc.)
Share known issues in a given Feature

“This is a live site, do not place any orders. Only test product filtering. All other areas of the site are out of scope. Some features are not available on mobile web. These features are out of scope.”

Then, we state any Additional requirements. This section should be used to provide any additional information testers should know to achieve the desired results. Examples include:

Test accounts
Test payment information
Any special requirements to the environment access/bug report format/device usage/access settings

“Please clear cache between filtering attempts. If on mobile web, include browser version at end of 'Actual Results' section.”

We have no Attachments for this test cycle and will skip this step.

Step 4

Next, we will need to specify “Where to test” (on which devices) and the “Bug language."

When selecting “Where to test,” we will be choosing which devices our testers will use to run our test. We need to keep in mind our audience and their devices. Since we’re testing for Web and Mobile Web, we’ll include desktops, smartphones, and tablets.

Note: You can define device scope by editing device type, device model, OS, and/or browsers. Please keep in mind that any “unconventional” or outdated device, OS, and browser combinations may lead to irrelevant device-specific bugs.

When choosing the Bug language, please keep in mind that testers get invited based on the report language; therefore, the instructions/Features and bug report languages should be consistent.

Note: Bug report language is defaulted to English.

Step 5

Lastly, we select our desired test date and start time. In this case, we are running a single Rapid test, which is set to begin at 4:00pm Tuesday, 5th March, delivering results by 6:00pm the same day.

Note: Due to our Team Lead review and invitation process, the earliest a test can start is the next full hour +1 (e.g. current time: 9:40am / test time: 11:00am, current time: 5:05pm / test time: 7:00pm). Duration ranges vary based on Test Type, from two hours (Rapid Test) up to 48 hours.

Click Submit Test.

If you need to edit the features before submitting follow the appropriate steps below:

Open the Manage Features page in a new tab
Make the change to the Feature in the new tab
Click Submit