This post is the third and final installment in a three-part series that explains how Nedbank, one of the largest banks in South Africa, is rolling out the SEI's Team Software Process (TSP) throughout its IT organization. In the first post of this series, I examined how Nedbank addressed issues of quality and productivity among its software engineering teams using TSP at the individual and team level. In the second post, I discussed how the SEI worked with Nedbank to address challenges with expanding and scaling the use of TSP at an organizational level. In this post, I first explore challenges common to many organizations seeking to improve performance and become more agile and conclude by demonstrating how SEI researchers addressed these challenges in the TSP rollout at Nedbank.

In a 10-year retrospective on agile methods, Jeff Sutherland, co-creator of the Scrum agile method, listed some of the major challenges and impediments for organizations that adopt agile methods, including

• demanding technical excellence
• promoting individual change
• leading organizational change
• organizing knowledge
• improving education

We've encountered and addressed many of these challenges in our work with Nedbank, which is one of several large companies to successfully pilot TSP and undertake an organizational rollout. For a TSP project (or any project using a new process) to succeed, management and other stakeholders must often change their behavior to support the process; they all must use the process in a way appropriate for their roles. Their behavior must support the empowerment of the TSP project team. Conversely, if the stakeholders do not behave appropriately, they can undermine the empowerment of the team and its ability to complete the project successfully.

Demanding Technical Excellence

In our experience, the key to success is to train management on how to ask for technical excellence and to know it when they see it. Technical people often know the details far better than their managers. Managers must be provided training and guidance on how to set the right goals for these empowered teams and to track the right measures. For Nedbank, the most important outcome of technical excellence is low rates of fielded severity 1 defects, that is, defects that stop the system. With TSP, this quality goal is not just a matter just for quality assurance (QA), but for the development teams to manage.

In an organization with self-managed teams based on TSP principles and practices, the teams and individuals manage the technical work. The teams define their process, estimate their work, track progress, and collect data on all defects. Status reports do not include detailed data, but summaries. Managers who oversee technical excellence need to hold teams accountable, but must avoid micromanaging the details of the technical work. This trust, while sometimes hard for management to grant, is essential to team empowerment and commitment.

At Nedbank, senior management reviews a laundry list of visible outcomes for the project including adherence to committed delivery date, cost estimation accuracy, defects in QA and production, and accuracy of status reports. The teams are thus held accountable for project outcomes, managing their work, and keeping management informed. This accountability eliminates the fear of the data being misused, empowers the teams to make decisions, and aligns incentives and goals. To expand TSP use beyond early adopters, the organization must make expectations explicit and non-threatening. Project scorecards must focus on publically visible outcomes rather than on detailed process data. TSP includes training for management to help with this transition. This training is just one aspect of managing organizational change.

Promoting Individual Change and Leading Organizational Change

Attempts to change organizational behavior often fail. At the SEI, we address this difficulty by providing change-management training to TSP coaches. This training is effective at the individual and team level, but more is needed on an organization-wide scale.To address this gap, we coach the executive team on change management and the logistics of rollout and sustainment. The organization needs to build these new ways of working into its processes for

• training
• project planning
• project evaluation
• personnel evaluations

Nedbank is fortunate to have an executive who understands the need for change and the difficulties surrounding it, and has provided resources to ease the transition.

Organizing Knowledge and Improving Education

Education must be tailored to the target audience. TSP provides specific training for coaches, instructors, developers, non-developer team members, team leads, and executive management. This training, which also addresses another challenge referenced by Sutherland, helped all involved at Nedbank understand the principles behind the change and their role in making new organizational practices a success. Through the Center of Excellence (COE) presented in my second blog posting, Nedbank assures that training is performed and that everyone involved is prepared to work on a TSP project.

The function of organizing knowledge remains a work in progress as the COE collects project data and experiences. Perhaps the greatest barrier to collection of TSP data at the organizational level is the lack of enterprise tools. We are teaming with two large partners to develop the next generation of TSP-data tools at the enterprise level, but they are still a work in progress.

Agile development organizations must also secure sponsorship, build capability, identify and train change agents, develop organizational support for the initiative, track progress, and highlight success to senior management. They must not associate long-term success with any single project, but rather with demonstrated and sustained improvement over many projects and many people. Nedbank, which is at the leading edge of TSP implementation, started down this path with a rollout strategy that included funding and time for

• training executives, project leads, coaches, and team members
• setting expectations about how projects will be planned and conducted
• gathering, reporting, and analyzing specific project data

The SEI worked with the COE to help Nedbank pilot TSP and train instructors and coaches. TSP coaches continue to help Nedbank plan and implement its organizational rollout.

Results: An Organization Changed

Nedbank now begins all new software projects with TSP. This effort is led by the TSP COE, which estimates needs and supports teams with coaching and training. In addition to providing operational support, the Nedbank COE collects data to track organizational progress and provides relevant planning data for its teams. Relevant data includes ranges of project schedule and cost variances, scope growth, component size, and effort estimation accuracy, normalized defect levels in QA and user-acceptance test, schedule and cost ratios of development, testing, maintenance, cost of specific activities such as peer inspection, and cost of rework. Nedbank uses this data at the organization level to

• Assess the overall cost efficiency of work. The benefits in schedule predictability, quality, and overall cost containment are made explicit and real.
• Plan projects. The use of historic data of comparable projects means the planning parameters are no longer just guesses. We have data from similar projects at Nedbank to suggest how results might be affected by the duration of front-end or back-end projects, the duration in testing, the calendar time that resources must be committed to support testing, and changes in the number of staff.
• Improve performance. Technical excellence and quality are economic decisions. Improving cost, quality, or schedule performance requires change, (e.g., taking time to plan, training for inspections, and measuring quality early). It's important to know what changes will occur and what those changes will cost.

Only by understanding the current process can the organization set realistic goals. By combining realistic goals and a data-driven knowledge of the process, empowered teams can make specific changes to the way they do work and evaluate the results. Simple outcome measures include severity defects in production, schedule overruns, and cost per change. To improve performance, Nedbank measured how much effort and time were required for testing and how much effort was devoted to defect fixes. With reliable data consistently defined in projects across the organization, Nedbank is building performance benchmarks and can now begin to make data-based cost benefit analysis for process changes.

When Nedbank introduced detailed project planning and tracking, the use of TSP made it possible to see how these changes altered time spent in specific activities and the associated results on schedule performance, cost containment, and quality at delivery. In the context of a defined process, a few simple measures--direct effort, defects, schedule, and size--transformed their ability to see and manage their software releases.

Looking Ahead

We are continuing to work with Nedbank (and other organizations) to share our TSP research. Nedbank's COE currently focuses on growing capacity during the rollout. The process is arduous and will take several years to complete, which can lead to frustration. Shortcuts, such as incomplete training, lack of coaching support, or stakeholders who have not yet fully bought into the new approach, will degrade results and likely create resistance. When the rollout is complete, the emphasis will shift to sustainment. As shown in the COE's pilot video, Nedbank is already seeing benefits from these changes.

If you're interested in learning more about TSP, please consider attending the upcoming TSP Symposium in St. Petersburg, Florida, USA.

Additional Resources

For more information about TSP, please visit
www.sei.cmu.edu/tsp

For more information about the 2012 TSP Symposium, please visit
www.sei.cmu.edu/tspsymposium/2012/

To read the SEI technical report Deploying TSP on a National Scale: An Experience Report from Pilot Projects in Mexico, please visit
www.sei.cmu.edu/library/abstracts/reports/09tr011.cfm

To read the Crosstalk article A Distributed Multi-Company Software Project by Bill Nichols, Anita Carleton, & Watts Humphrey, please visit
www.crosstalkonline.org/storage/issue-archives/2009/200905/200905-Nichols.pdf

To read the SEI book Leadership, Teamwork, and Trust: Building a Competitive Software Capability by James Over and Watts Humphrey, please visit www.sei.cmu.edu/library/abstracts/books/0321624505.cfm

To read the SEI book Coaching Development Teams by Watts Humphrey, please visit
www.sei.cmu.edu/library/abstracts/books/201731134.cfm

To read the SEI book PSP: A Self-Improvement Process for Engineers by Watts Humphrey please visit
www.sei.cmu.edu/library/abstracts/books/0321305493.cfm