As recent news attests, the rise of sociotechnical ecosystems (STE)--which, we define as a software system that engages a large and geographically-distributed community in a shared pursuit--allows us to work in a mind space and a data space that extends beyond anything that we could have imagined 20 or 30 years ago. STEs present opportunities for tackling problems that could not have even been approached previously because the needed experts and data are spread across multiple locations and distance.

Since STEs can be complex and have many diverse stakeholders, a key challenge faced by those responsible for establishing and sustaining them is eliciting requirements to inform their development efforts. Yet stakeholders often have requirements that they are not aware of, so they do not specify them. Uncovering these unstated requirements can be hard and is not well-supported by traditional approaches to requirements elicitation. This blog post describes initial results of an effort by researchers at the Carnegie Mellon University Software Engineering Institute--in addition to myself, the team included Nancy Mead, Robert Stoddard, and Mary Beth Chrissis--aimed at developing an approach for determining the unstated needs of stakeholders typical of large, diverse programs and especially STEs.

Foundations of Our Work

Apple founder Steve Jobs famously said "But in the end, for something this complicated, it's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them." What Steve was saying is that there are often critical product features that end users are not even aware they need, and that an insightful designer will figure out that need and offer it.

Therefore, "unstated requirements" can remain unstated not because users hold back or forget them, but rather because they are unaware of them. Users may have sensed something was wrong with the current software system in particular situations, but did not explicitly recognize it as a problem that could or should be addressed. Perhaps with additional information or after reflection, such users might come to realize and then express what it is that occurred that could have been prevented or mitigated.

Unstated user and stakeholder needs can be a source for innovative system features and key architectural drivers and therefore can fundamentally impact the design, implementation, performance, and evolution of complex software systems. For example, the new iPhone 6 sports 64-bit computing on the new A8 chip (as did the previous generation iPhone 5S with its A7 chip) which might lead observers to ask, Why 64 bits? One reason might be: with the amount of random access memory (RAM) that exists in smartphones, we are approaching the maximum RAM storage possible for 32-bit computing, which is four gigabytes. By introducing 64-bit processors now, Apple may be positioning the broader iPhone ecosystem for 64-bit processing, setting the stage for an explosion of memory-intensive healthcare and home automation apps. Such forethought illustrates an appreciation for subtle longer-term needs that are best addressed early in the product (and in this case, product-line) lifecycle.

Prior research conducted by a team on which I participated foresaw an emerging need for requirements elicitation involving multiple and diverse parties proactively engaged in an early and ongoing conversations about the health of the ecosystem as a whole and for greater visibility into the goals, activities, and capabilities of various ecosystem participants. From such discussions, preventable problems can be identified and globally-sound and efficient solutions can be jointly-developed.

Our research team thus decided to initially focus on this research question:

Could existing requirements elicitation methods that address unstated needs be adapted to work across a geographically distributed community of users, stakeholders, and software developers?

A Review of Existing Methods

Early in the project, then, our team conducted a literature review to determine which requirements elicitation methods

• had been investigated
• addressed unstated needs
• had potential for automation and analytic tool support

Most current requirements elicitation methods are specification-driven and do not address unstated needs. Those that do either do so directly by only targeting a small number of stakeholders and a small number of features (e.g., as in prototyping and simulation) or indirectly by analyzing user scenarios for a range of distinct user types (e.g., persona-based approaches).

While other methods have some unique benefits, the team selected the KJ method, because of its systematic approach to eliciting and organizing user experience data with the goal of identifying potentially innovative new features and quality attributes. The KJ method is named after Jiro Kawakita, who originally conceived it, and consists of the following steps:

Step 1: Evaluate existing knowledge of requirements
Step 2: Design open-ended probing questions
Step 3: Conduct KJ interviews
Step 4: Analyze output to form context need/activity statements
Step 5: Conduct KJ affinitization
Step 6: Identify unstated needs and candidate innovative solutions
Step 7: Conduct Kano Analysis to determine must-be's, satisfiers, delighters

KJ is typically applied in co-located settings in a time-boxed fashion. Our research focused on enhancing KJ so that it works in a distributed environment--one in which not only users but requirements analysts and involved technologists are at different locations.

Challenges

Several challenges are common to requirements elicitation methods and not necessarily unique to our work. During requirements elicitation, underrepresenting a key stakeholder does more than simply omit the immediate impact that the stakeholder could offer. That stakeholder may be focused on a cross-cutting concern (e.g., security, dependability, usability, etc.) that has important consequences for other stakeholders and end users. Stakeholders can include mid-stream suppliers to the product or system being developed and/or smaller niche solution providers--it is easy to overlook stakeholders that can be the source for innovative requirements.

Another, more formidable, challenge that we encountered is that many requirements elicitation methods tend to be applied in co-located, time-boxed team settings, providing only one or two days to identify and analyze use cases or quality attribute scenarios. Such requirements elicitation methods tend to proceed in a lock-stepped, sequential fashion limited by the number of attendees that can be in one location, the range of perspectives that can be brought to bear and thus the comprehensiveness of the requirements analysis is limited. These sacrifices in scope and attention can result in an incomplete identification of key system requirements and priorities. For example, incompleteness can manifest itself as a lack of attention to critical security requirements because software developers easily overlook user motivations and limitations; trends in markets; or the emergence of new technologies that may negatively impact the system and its users and other stakeholders.

There is also the concept of externalities, which occurs when stakeholders take a local action driven by a local perception of utility (value proposition) and fail to consider the extra expense that they cause other stakeholders who are not involved in the dialogue. It is important to step outside of that mind trap and consider the broader issues: Who is gaining? Who is losing? What is the real cost to society as a whole?

Failing to consider these broader issues can result in a compromised system where privacy can be lost and data breaches can occur, as seen from some of the deep dives undertaken as part of another recent research project.

Finally, teams often suffer from group think, in which members fail to explore alternative or dissenting viewpoints to avoid creating conflict within the group. Without adopting a data-driven approach to requirements elicitation, requirements analysts can suffer from group think regarding what they perceive to be the primary needs of users and other stakeholders.

Current Status and Looking Ahead

In recent months, we have completed the definition for our method, which we call "KJ+" and developed training and initial versions of the tools that support use of the method. KJ+ tackles each of the challenges noted: it can be applied in a distributed fashion and over a longer period of time, and thus is more inclusive than co-located, timeboxed approaches would be. Our method also proves to be more scalable than current approaches for the same reason. KJ+ inherits KJ's focus on the unstated and unobvious, and because it is data-driven, it can mitigate the effects of group think.

In coming months, we plan to further pilot the method and compare its performance to results obtained using more traditional requirements elicitation approaches. Longer-term, we hope to have the opportunity to further research how we might augment or streamline the first few steps of KJ+ by automating analyses of user and developer discussion forums to identify more subtle and longer-term problems being encountered and thereby identify opportunities to improve the health of the STE. Such an approach might also support a more thorough discussion of externalities associated with particular solutions before they get broadly deployed.

We welcome your feedback on our research. Please leave comments in the feedback section below.

Additional Resources

Read the paper The KJ Method: A Technique for Analyzing Data Derived from Japanese Ethnology by Raymond Scupin.

Read the paper A Persona-Based Approach to Exploring Architecturally Significant Requirements by Jane Cleland-Huang, Adam Czauderna, and Ed Keenan.

Learn more about KJ+, obtain a copy of the full-day tutorial presentation, Eliciting Unstated Requirements, which was presented in August 2014 at the IEEE International Requirements Engineering Conference.