3 Questions to Ask Early in the Software Quality Assurance Process

Congratulations—all your hard work has paid off and your boss tells you that you’ll be a subject matter expert (SME) on an upcoming IT project. What do you do now?

While product owners and architects are busy drawing diagrams, planning architectural layout and building a backlog, there is one area that often gets overlooked in the beginning: How to prepare for software quality on a project.

Here are three questions that you need to answer early in the software quality assurance process, so you can avoid delays after the project starts:

1. How many environments are needed?

Since there is a cost associated with environments—and there can be as many as six environments on a software project—you need to figure out how many environments you need very early on.  Here are the six different environment types to consider:

• Development environment: This allows developers to run unit tests against their code to ensure it works before the builds move to the next environment.
• QA or test environment: This allows testers to test the features and functionality of what they’re developing to ensure they meet the acceptance criteria of the story.
• User Acceptance Test environment (UAT): This allows the product owners and SMEs to test and ensure the application looks and behaves as intended.
• Staging environment: This typically mirrors what the production environment looks like, so teams can troubleshoot production issues and conduct testing for hot fixes or maintenance builds.
• Load & performance environment: This also mirrors the production environment and is the environment where you can execute performance tests, load tests and stress tests to ensure the architecture and code can handle real-world traffic.
• Production environment: This is the final resting place where the application lives for the public—or the entire company if it’s an internal application.

At a minimum, you must have development, QA, staging and production environments.  But if the application has 500 or more concurrent users, you should also perform load & performance; if the IT project is global and load and performance is not in the plan, consider raising your hand to discuss this with the team in more detail.

2. How should we set up test accounts?

This one is a sleeper and usually doesn’t come up until it’s too late, but it’s imperative to know how many different access levels you need for your application. Why? Because testing is time-consuming—to fully test the functionality, testers will need to log in under each persona and verify access levels are enforced. The more you have, the longer it will take.

Additionally, if load & performance testing is required, then you’ll need hundreds or thousands of test accounts, which takes up even more time and usually needs to go through a security and compliance department.

Remember—if the application is using multi-factor authentication (MFA), then the test accounts need to be set up so MFA is not applied to those accounts. Why? Because If MFA is active for the test accounts, then you can’t automate the login process.

Finally, remember to ask if the production active directory is being used for the lower environments—like dev, QA, UAT and load and performance—because a stress test with hundreds or thousands of test users could affect the performance of production users trying to access the application.

3. How do we outline the different testing elements during the quality assurance process?

Often times, projects start before teams have truly had time to think through all the different testing elements for the project.  Sure, there are smoke tests, integration tests, regression tests and user acceptance tests—but what about load and performance, security and compliance tests, failover testing, compatibility testing and mobile devices? You’ll need to collaborate with your team and identify where these tests will occur, who is responsible for the execution, and how the results will be presented to the project team. Here are some considerations:

• Load and performance requires special tooling and skillset—you may not have either within your company, so you may need to consider outsourcing or hiring for these skills.
• Security and compliance is such an important, time-consuming and massive undertaking that many projects will outsource this to third-party companies that specialize in this type of testing.
• Failover testing is crucial because it shows you how the application behaves when the network goes down or a data center goes offline. But failover testing also requires a broad team: network administrators, DevOps groups and, in some cases, data centers.
• Mobile devices need to be procured if executives intend to access the application from their phones or tablets. I recommend testing on physical devices rather than simulators, as I’ve seen tests pass using simulators but fail on the real device. This means you likely need to purchase additional tools and ensure the project team has mobile developers.

While there are several other questions to consider in regards to your software quality assurance process for a project, these three must be answered to avoid excess costs and delays.

Quality is not just executing tests on a project; it encompasses planning, execution and communication. You have to ask the right questions and open up the dialogue to improve efficiencies on the final delivery of the product. Have the mindset that everyone is responsible for delivering a quality product—not just the project’s QA Team.

If you’re unsure where to start, contact AgileThought to learn how our experts can help you develop—and maintain—a successful software quality assurance process.