Prioritization identifies the sequence in which requirements should be addressed and allows end users and stakeholders to evaluate and provide feedback on the most valuable features of the evolving system. In Agile software development, requirements and desires are expressed as items in the product backlog. All development-related activities are drawn from the backlog. For small Agile products, there will typically be a single backlog. For large-scale Agile development efforts using the Scaled Agile Framework (SAFe), there may be multiple levels of interconnected backlogs containing thousands of entries of varying size and scope. In this blog post, I will present a new method to prioritize an Agile backlog using mission-based parameters.

Whether small-scale or large-scale, Agile development emphasizes responsiveness to change, including changes in priority. Consequently, Agile backlog owners prioritize and re-prioritize their backlogs at regularly scheduled intervals and whenever new items are added to the backlog. While numerous prioritization schemes exist, SAFe prescribes the use of Weighted Shortest Job First (WSJF). While this technique has merit, in my experience as a SAFe Program Consultant (SPC) for major DoD software-intensive programs I have found drawbacks with WSJF, which include a monetary-based focus more appropriate to industry than to DoD; the time investment required to maintain up-to-date priorities; and the innate subjectivity of a relative ranking scheme.

On several of the DoD projects on which I work, I have piloted a new approach that overcomes these WSJF shortcomings. It utilizes objective (rather than subjective), mission-focused (rather than monetary-based) criteria and allows ongoing re-prioritization to be conducted with minimal overhead.

Monetary Focus.The WSJF method focuses on minimizing the cost of delay, which aligns with the first principle of SAFe: Take an economic view.

WSJF works ideally for companies that develop commercial products, have a financial motivation, a desire to corner the market, and/or a goal to be the first to market. Return on investment is a key input in deciding whether to continue developing a system or cancelling it.

WSJF fails to account for the motivations of the government, where profit, market share, or the focus of a single user community often does not define success, and where time to market involves an entirely different path. Major difficulties arise when a government program attempts to align its prioritization with the contrasting motivations underlying commercial enterprise.

Time Investment. WSJF requires that all backlog items be listed and evaluated against each other in four categories: user-business value, time criticality, risk reductions and/or opportunity enablement, and job duration or size (Figure 1).

While this method works for a small backlog, it is not practical for the large-scale systems of systems under development within the government. A relatively small backlog of only 100 items requires 400 comparisons. For reference, a medium-sized program that I am currently supporting has 178 portfolio epics and 467 program epics decomposed, so far, into more than 200 capabilities and more than 900 features currently in their backlogs.

To further complicate this analysis, the WSJF prioritization must be redone every time a new item is added or the smallest item is removed from the backlog. Due to the time required for proper WSJF prioritization, most programs I have worked with ignored the existing requirements and focused only on the new ones. Thus, they would take these limited results and squeeze them into the existing priority list. Within a very short time, the backlog was no longer functionally prioritized.

Prioritization by Committee. The next largest obstacle faced by government organizations trying to prioritize their backlog is prioritization by committee. Of all the government programs I have coached, not a single program, solution, or portfolio backlog was managed by one person. Over time, the members in a group will stabilize, but some groups must contend with new members each time they meet to groom and prioritize the backlog(s).

Members will have differing opinions based on personalities, technical background, and previous experience as users of the system. While the diversities of opinion that a committee brings may be extremely valuable, any discrepancy between comparisons during prioritization are discussed and settled before moving onto the next backlog item. These discussions become longer and tend to veer off topic more often as the number of members in the committee grows.

But the largest hindrance to proper prioritization using a committee is the highest paid person's opinion (HiPPO) attitude. By arbitrarily overruling the other members based on their grade/rank, the ranking member makes the backlog a self-portrait rather than reflecting what is best for the program.

After trying unsuccessfully to fit our square features into WSJF's round holes, I came up with another approach. What if we could focus on mission instead of financial criteria and replace WSJF with a mission-based prioritization formula?

Instead of arguing over hundreds of values, what if committee members had to agree only to weighting factors of several mission-impacting parameters one time? Once that weighting was done, prioritization would require only answering the question, "is the factor applicable or not?"

Based on the projects I've applied this approach to, the outcomes were better than anticipated. Prioritization discussions were no longer marked by arguments but instead focused on learning and sharing knowledge. Assessments were based on facts, not opinions. The HiPPO even changed his attitude when the sergeant was able to easily justify his answers and quickly became a valuable member of the backlog-grooming team.

After the initial pilot of this method, I tried it with several other programs with similar results. Interestingly, while designed for features, this same method works for prioritizing epics, capabilities, or stories (as long as you rank only similar items).

Mission parameters can be changed to reflect the domain or environment of each program. The initial list I began with was compiled with assistance from experienced SEI Agile coaches and covers most of the concerns of the programs that we have encountered. These mission parameters include

• Provides strategic or tactical advantage
• Foundational (other features depend on this feature)
• Replaces deficient functionality
• Supplies missing functionality in existing system
• High sponsorship / visibility
• Fulfills direct user request
• Required to field
• Hardens security
• Time critical / urgent
• Previously de-prioritized

I created an Excel spreadsheet that handled the calculations and prioritized the list. The pilot program had a group of 19 responsible for the backlog. This group consisted of product owners, product managers, engineers, a Release Train Engineer (RTE), business owners, an architect, and a program manager.

Once they weighted the parameters (about two hours), they were able to prioritize 55 features in approximately one hour. The first three features took 30 minutes to allow the group to become familiar with the new process and their definitions of the parameters.

The resulting prioritization list generated by the tool is a starting point. If any feature's ranking does not make sense, it is appropriate to override the algorithm and adjust the priority value.

Unlike WSJF, mission-based prioritization does not require re-assessing every requirement when updating the backlog unless a parameter (not its weight) is changed. Maintaining the backlog's prioritization is very lightweight. I encourage program offices to address the backlog at a regularly scheduled meeting. If there are no changes, move on to the next agenda item; otherwise spend five minutes to maintain the backlog.

After the program increment (PI) planning is completed, archiving the current spreadsheet will maintain a record of the justification of why one item was selected over another. This record allows completed items to be removed from the tool, but they will not be lost. It is important to note that the resulting prioritization list is by no means set in stone; if any feature's ranking doesn't make sense, move it to the appropriate place.

The mission-based prioritization tool is available in two versions:

• a no-frills version that accounts for restrictions on government computers. The results must be manually sorted.
• an alternate version of the tool includes a small amount of Visual Basic code that creates a tab containing the sorted prioritized list and a tab containing features grouped by the mission parameters.

The process to use this tool is simple. Steps 1,2,3 are done only the first time the tool is used.

1. Review parameters for applicability to your program. Add, remove, or modify any of the parameters and agree on the definitions to fit the program's context.
2. Apply relative weight factors and calibrate. This step involves ranking the importance of the parameter to the goals of the program. The easiest method I found for groups is dot voting. Enter the number of dots in the weight column on the Legend tab.

Be careful if any parameter receives a significantly larger vote, because that will skew all development toward that parameter. When this happens, I have the team continue and assess the final prioritized list. If it does not accurately reflect the program's objectives, I give them each one more dot to vote for anything except the top parameter. We update the weights in the Legend tab. The priorities automatically update, and this new list is now aligned with the group's expectations.

1. Define difficulty and uncertainty. Two factors serve as a proxy for job size in the WSJF formula. They are the difficulty-to-implement and uncertainty parameters. The members need to define and agree to these and document their definition on the Legend tab.

1. Optional: size penalty. A later addition to the tool was a prioritization penalty applied when a feature was too large. By enabling this penalty, the team encouraged smaller features in the backlog. This had the added benefit of more features that do not continue into the next program increment. If you do not wish to use this feature, skip this step.
   1. Select "Y" in the "add large feature penalty" field.
      b. Enter the number of iterations within your standard program increment.
      c. Decide when to apply the penalty. The default is any feature larger than 45 percent of the total program increment.

1. Prioritize the backlog.
   1. Add the items to be prioritized to column A on the Mission Based Priority tab.
      b. For each feature, determine whether each parameter applies to that individual feature. If it does, place a "Y" in the cell; if not leave blank. (This is significantly different from the WSJF method where features are compared to each other.)
      c. Rank the difficulty to implement and the uncertainty from 1 to 5.
      d. Enter the number of iterations (sprints) it will take to implement. Even without the large feature penalty enabled, documenting the size encourages breaking down features into smaller pieces.

If you use this mission-based prioritization tool, please reach out to me at info@sei.cmu.edu. I am especially interested in your experience using the tool along with any parameters that did not apply to your program, any you added or changed, and what domain you are working in.