Prior to engaging with your Customer Success Manager during your Kickoff Call, it will strongly benefit your team and ability to begin testing immediately if you come prepared with answers to the following questions:
Who on your team will take the lead in communicating with test IO?
- You should know who will be your team's designated Business Partner (focused on business goals and payments) and Operational Partner (focuses on the technical implementation and use of test IO's platform).
Are responsibilities divided by product, environment, or other?
- Will your team segment its members' accountabilities by the product that is being tested (web/mobile web or mobile), the environment (staging or live), or something else?
- Responsibilities might entail monitoring the test IO Dashboard, reviewing, and exporting test results.
Note: Understanding questions (1) and (2) helps test IO understand your team's work capacity, release cadence, and testing processes. This understanding leads to decisions that will help craft tests that yield the proper quality and quantity of feedback.
What are your current challenges?
- Where in the development lifecycle is your teaming currently getting stuck? Are there particular components or parts of your in-house development or QA processes that are bottlenecking efficient issue-triage efforts?
Note: Understanding your challenges better informs test IO of the context in which you are facing current quality challenges.
What does a successful partnership with test IO look like?
- Are there particular results, metrics, or trackable indicators that you are looking to improve that we should know about? What are your expectations surrounding communication? What about your ideal timeline for reaching these goals?
Is there anything else we should know about your team, your product, or your expectations?
It will also help to start thinking about answers for the below test-related questions:
What are examples of high value bugs?
- High-value bugs are those that severely impact the expected functions of your product, including bugs that inhibit user experience or bugs that would prevent a user from performing core revenue-generating flows.
- Example (1) We have known issues around connecting social media apps and need some help surfacing these.
- Example (2) Anything preventing the user from checking out and successfully completing a purchase.
How do you develop your software?
- What programs do you use? What does your SDLC look like? How often do you release builds: daily, weekly, monthly?
How do you find bugs now, and how do you triage them? How do you deal internally with feedback?
- How does your team perform its QA duties currently? Do the developers perform their own checks prior to release, is it through customer feedback, or do you have a designated QA resource in-house?
- How are findings shared? Do you keep track of bugs in a bug tracker? How are they ranked and distributed for triage.
- Once bugs are received, who decides whether they become a ticket, who assigns, and prioritizes them? Will bugs be delivered right away or with the next release? How fast can you digest test results and action bugs?
How would you define a successful test? Do you have specific criteria for accepting bugs?
- Are there certain criteria you evaluate when trying to determine if something is a legitimate bug for you? (i.e. high priority device requirements, bugs that you would prioritize, or gaps between automated and manual testing.)
- This will help us establish a baseline of what you define as valuable findings.
Any upcoming product or feature release?
- Do you have any product or feature releases coming up in the near future? If so, how will they impact your testing processes and/or focus.
Can you think of any examples of important bugs that your customers have found in production that you want us to find for you in testing?
- Oftentimes, test IO customers are looking to get ahead of customer complaints/requests by providing results in a pre-production environment.
- Example (1) Testers discover bugs that we haven't been able to reproduce on our own with a wide variety of devices around completing the checkout process.
- Example (2) When customers are continually submitting questions about “How do I make a return?” or “How do I perform x?," that’s a good indication that the UI is not intuitive.
What bugtracker do you use? Can it be made accessible to external users?