Software Engineering Institute | Carnegie Mellon University

SEI Insights


SEI Architecture Technology User Network (SATURN) News and Updates

SATURN 2015: QA to AQ: Shifting from Quality Assurance to Agile Quality (Session Notes)

Posted on by in

Rebecca Wirfs-Brock, Wirfs-Brock Associates, and Joseph Yoder, The Refactory, Inc.

How do you make quality happen? Budget time for quality discussions and quality testing. During envisioning and requirements gathering, identify core qualities. The core goal of agile and lean was not just to go faster, but to get rid of waste. Quality can be a result of those processes, but you need to engineer for quality by architecting for quality and then testing for it. You'll also need to determine appropriate times when qualities can be tested and delivered.

For quality, much depends on whose eyes you're looking through. The developer has a different and sometimes competing perspective with the user, as do the engineer and the artist. So who are your system's stakeholders? They are not just users, but also executives, sponsors, developers, database administrators, business process experts, and corporate compliance. The traditional system development lifecycle includes phases from initiation to disposition. Where does system quality fit in? The requirements analysis and integration-and-test phases were typically when a team considered quality. Testing for functionality and testing for quality measure different aspects of a system, and often testing and QA occur right before launch, positioning QA as a gatekeeper. That's better than not doing anything, but it's not the most effective use of QA. Agile approaches carry certain design values. Among them are design simplicity, communication, continuous improvement, teamwork, trust, satisfying stakeholder needs, and building quality software. Myths about agile approaches include the following: System qualities can easily be added into the evolutionary architecture and design; we can easily adapt to new requirements; we can change the system fast; and we don't need to worry about performance, scalability, security, or usability until the functionality is working.

  • Gregor Hohpe offers: Premature optimization is the root of all evil.
  • Yoder replies: Premature abstraction is the second root of all evil.

Design is all about tradeoffs. For example, usability and security often have orthogonal qualities. Performance vs. small memory is another important tradeoff. The quality of balancing the "good enough" is enough a key factor in a company's success in building a system. To do this, an organization needs to become agile at quality. Quality is not an act, it's a habit, and "habit" is not just doing something over and over again. Habit is something you do that you don't have to think about because it is embedded in your mindset. What makes good agile teams? They communicate well, have cross-functional skills, focus on stakeholders' needs, incrementally deliver working software, adapt to change, collaborate, and self-organize. Cross-functionality doesn't mean that everyone is a generalist; it means that QA folks understand a little about coding, or it means that the team as a whole covers cross-functional skills. QA teams understand testing well and how to specify and validate system qualities. They certify the functionality of the application based on the contract and requirements. And many work independently from the software teams. This can mean that they are included late in the project, with little communication between them and the developers. In an agile quality team, architects and QA will work closely together during development, and they will engage in QA activities much sooner. They will also work closely with the business, product owner, and developers to understand, define, and validate the quality requirements. But breaking down the walls necessary to do this can be difficult: there are physical barriers, cultural differences, background differences, communication and language differences, and time constraints. The Us and Them mentality has to go. One way to embed QA into the team is to "Pair with a Quality Advocate." Build quality into your project rhythms. Some decisions are too important to leave until The Last Responsible Moment. So choose the Most Responsible Moment based on your tolerance for risk, ability to test incrementally, and ability to scale. Techniques for reducing risk on larger projects and programs include grooming and vetting project roadmaps and timelines, Agile Landing Zones, architecture spikes, risk reduction backlogs, and set-based design. Test-driven development, which Yoder also calls test-first-development, should include quality concerns from Iteration 0. Building quality in does not mean that all the quality work occurs up front, then test-test-test at the end; you break it into chunks, do a little bit of modeling, then some coding, test, verify, refactor, and continue. Yoder and Wirfs-Brock run through a few quality scenarios and then discuss "fold-out qualities," or finding even more qualities. These include quality-related acceptance criteria that can be attached to specific functional user stories. QA and architects should consider these stories together. "You can't test for warm and fuzzy," but the user experience is important. Turn the warm fuzzies into a testable usability scenario and agree on target values. Criteria are not simply pass/fail. Have a landing zone of minimal, target, and outstanding. You can still go into production if you meet the minimal criteria. Features don't make a viable system. Sprinting forever doesn't get you to the finish line. A viable system is accomplished by focusing on features and paying attention to system qualities. And a sustainable architecture is livable for both its users and developers.

About the Author

Bill Pollak

Contact Bill Pollak
Visit the SEI Digital Library for other publications by Bill
View other blog posts by Bill Pollak



We welcome comments with a wide range of opinions and views. To keep the conversation focused on topic, we reserve the right to moderate comments.

Add a Comment


Type the characters you see in the picture above.