Agile projects with incremental development lifecycles are showing greater promise in enabling organizations to rapidly field software compared to waterfall projects. There is a lack of clarity, however, regarding the factors that constitute and contribute to success of Agile projects. A team of researchers from Carnegie Mellon University's Software Engineering Institute, including Ipek Ozkaya, Robert Nord, and myself, interviewed project teams with incremental development lifecycles from five government and commercial organizations. This blog posting summarizes the findings from this study to understand key success and failure factors for rapid fielding on their projects.

A key area in our interviews explored how Agile projects deal with the pressure to rapidly deliver high-value capability while maintaining project speed (delivering functionality to the users quickly) and product stability (providing reliable and flexible product architecture). For example, due to schedule pressure, we often see a pattern of high initial velocity (the amount of product backlog effort a team can handle in one sprint) for weeks or months, followed by a slowing of velocity due to stability issues.

Business stakeholders often find these changes in velocity disruptive since the rate of capability delivery slows while the team addresses stability problems. We found that when experienced practitioners were faced with these challenges, they did not apply Agile practices in a silo. Instead they combined practices--Agile, architecture, or other--in creative ways to respond quickly to unanticipated stability problems, as described below.

Balancing Speed and Stability

The ability to balance speed and stability involves achieving and preserving a software development state that enables teams to deliver releases that stakeholders value at a tempo that makes sense for their business. The desired software development state is different for each organization. This state is one in which architecture (often in the form of platforms and application frameworks), supporting tool environments, practices, processes, and team structures exist to support efficient and sustainable development of features. The entire organization--including development teams, management, and other stakeholders--must have visibility into the desired state, so that they neither over-optimize the supporting development infrastructure nor quit working on it.

In organizations that operate in highly regulated environments, such as defense, avionics, financial services, and health care, software development teams often interact with system engineering, deployment, and quality assurance teams that may be operating under different tempos. One challenge that software development teams face is that these competing forces often result in a significant slowdown in delivery following a high initial velocity. When confronted with this slowdown, the organizations that we interviewed applied a variety of tactics and practices to get back to the desired state. For example they often applied Agile practices in combination with other practices, especially architecture practices, to rapidly field projects.

A New View of Agile

Many software developers steadfastly maintain that Agile requires small, co-located teams, downplays architectures, and provides no documentation. The reality is that organizations--especially those faced with the challenge of rapidly fielding software systems in highly-regulated environments--have been applying varied architecture practices that build on the foundations of Scrum and Extreme Programming (XP).

The approaches revealed by our analysis of the interviews we conducted fall into three categories:

• When the project was going well, teams described using foundational Agile practices, such as Scrum status meetings, Scrum collaborative management style, small dedicated teams and limited scope, continuous integration, test-driven development, and so on to enable rapid development.
• When teams described a problem, they would explain how they often combine Agile practices with architecture and other practices ranging from management to engineering to get back to desired state. Examples of these combined practices include release planning with architectural considerations, prototyping with a quality attribute focus, release planning with external dependence management, test driven development with quality attribute focus, and technical debt monitoring with quality attribute focus. Development projects incur technical debt when shortcuts (intended or otherwise) lead to degraded quality.
• We also collected some examples of inhibiting factors that prevented developers from rapidly delivering software products. These included slow business decision-making processes, limitations in measuring architectural technical debt, over-dependency on architecture for knowledge, and stability-related efforts that are not entirely visible to the business.

Below are a few examples of Agile architecture practices that enable speed and stability that emerged from our interviews:

• Release planning with architecture considerations. This practice extends the feature release planning process by adding architectural information to the feature description document prior to release prioritization.
• Prototyping with a quality attribute focus. This practice extends the prototyping/demo user feedback practice commonly integrated into Scrum to include a focus prototyping on quality attributes, such as performance or security.
• Roadmap/Vision with External Dependency Management. This practice incorporates external dependency analysis into the roadmap planning process to reduce the risk of being blind-sided by unanticipated external changes.
• Test-Driven Development with Quality Attribute Focus. This practice merges test-driven practices, such as automated test-driven development and continuous integration, with a focus on runtime qualities, such as performance, scalability, and security.
• Technical Debt Monitoring with Quality Attribute Focus. This practice merges the notion of tracking technical debt with a focus on quality attributes (such as performance, security, reliability) that go beyond defects and functional correctness.

More examples of combined practices can be found in the paper A Study of Enabling Factors for Rapid Fielding: Combined Practices to Balance Tension Between Speed and Stability that I co-authored on this research with Ipek Ozkaya and Robert Nord.

The combined practices listed above allow experienced developers to address the problem of velocity slowdown resulting from stability issues with minimal disruption to capability delivery. Over time, however, acceptance of this approach has evolved. The initial release of Scrum advocated that the practices must be applied exactly as described in the Scrum handbook (if this was not done the project was considered to have "Scrum But") syndrome, so the outcome would be questionable.

In a recent blog posting, Ken Schwaber--one of the originators of Scrum--amended the initial Scrum doctrine by saying he would like to change the mindset of "Scrum But" to "Scrum And." He explained that "Scrum And" characterizes an organization that is on a path of continuous improvement in software development beyond the basic use of Scrum. In highly regulated environments, such as defense, avionics, financial services, and health care, organizations must often employ "Scrum And" approaches to balance speed and stability in an effort to meet budget, quality and timeline expectations.

Inhibiting Factors

Through our interviews with organizations, we also identified several factors that prevented development teams from rapidly delivering a software product. Many of these inhibiting factors were the result of incorrect or inconsistent applications of Agile or architecture practices, including (but not limited to) the following:

• a desire for features that limits requirements analysis or stability-related work
• slow business decision, feedback, or review-response time
• problems that resulted from challenges with external dependency management
• stability-related effort that was not entirely visible
• limitations in measuring architectural technical debt
• inadequate analysis, design, or proof-of-concept
• inconsistent testing practices and/or deficiency in quality attribute focus
• poor testing consistency

Concluding Thoughts

While agile architecture practices can help organizations assure the stability of fielded systems , it is important to understand the root causes of the inability to deliver at the expected pace and management of the tension between speed and stability. Organizations must also make problems more visible to developers, management, and stakeholders. When considering whether to combine Agile and architecture practices, organizations should consider the following questions:

• Are we delivering software to our customer at an expected pace?
• Are we aware of problems that are cropping up as a result of losing focus on architecting when Agile adoption activities become the primary focus?
• Does our technical roadmap address short-term and long-term issues?
• Does the team of software developers have skills that would enable them to successfully implement Agile and architecture?
• Do we have the visibility into not only the project management of the system, but also the quality expected from the system?

We hope that by codifying and sharing the practices exemplified above, other organizations can learn to apply these approaches to contend with the often conflicting demands of rapidly delivering software that is reliable, stable, and flexible in a rapidly changing environment.

Future posts in this series will explore other hybrid Agile approaches identified by the organizations that we interviewed including prototyping with a quality attribute focus. If you have experience applying a hybrid Agile architecture approach in your organization, please share your story with us in the comments section below.

Additional Resources

This article is an excerpted version of an article that appeared in the June 2013 issue of the Cutter IT Journal. For more information, please visit
https://www.cutter.com/journals/cutter-business-technology-journal

To read more about the SEI's research in architectural technical debt, please visit
https://www.sei.cmu.edu/research-capabilities/all-work/display.cfm?customel_datapageid_4050=6520

To read more about the review of architecture-centric risk factors, please read the article "Architecture for Large Scale Agile Architecture Development: A Risk-Driven Approach," which was published in the May/June edition of Crosstalk by Mike Gagliardi, Mike, Robert Nord, and Ipek Ozkaya, please visit
http://www.crosstalkonline.org/issues/mayjune-2013.html

To read the article "A Study of Enabling Factors for Rapid Fielding: Combined Practices to Balance Tension Between Speed and Stability," by Stephany Bellomo, Robert Nord, and Ipek Ozkaya, please visit
https://resources.sei.cmu.edu/library/