Dealing with Wicked Problems, Written by John Slack, M.B.A., PMP

The International Institute of Business Analysis (IIBA) has identified a number of different requirement types. The most recent type to be identified is the transition requirement. It is a requirement type that, like many good insights, was obvious once identified. Many project managers and business analysts had used the concept without formally declaring the concept. A transition requirement is a capability that the solution must have in order to facilitate transition from the current state of the enterprise to the desired future state, but that will not be needed once that transition is complete. This recognizes that it is difficult to find a solution implementation that doesn’t have some requirements that exist to allow a successful bridging from the current state to the future state. The more complex the solution, the more transition requirements typically need to be addressed.

This also opens up a discussion about discovery that takes place during implementation. Many of our commonly used project lifecycles assume that the project is complete when the solution is implemented. That is not really the case in many of today’s projects. When the solution is implemented we can rightfully claim that phase has reached a successful conclusion; however, it is more and more likely that there has been significant discovery during implementation. Rather than regarding this situation as frustrating, you can actually regard it as a success. This becomes more obvious when we add a categorization of problem types. The initial problem types addressed by IT starting in the 1960s could be called “tame” problems. Tame problems can be classified as Simple or Complex. A problem is Simple when both the problem and the solution are well understood. There are several lifecycles that are well-suited to Simple problems. They would include the waterfall approach, the modified waterfall, and the incremental approach. A problem is Complex when the problem is understood but the solution is not. The various iterative project lifecycles are appropriate for these Complex problems; however, there is an additional problem type which supplies new insights.

One of the major challenges facing the IT community today is that more and more of the stakeholder requests are asking us to address problems that can be classified as “wicked.” When addressing these types of problems, any solution that is initially put in place should be regarded as addressing transitional requirements. We need to put a solution in place in order to confirm our understanding of the problem.

What is a Wicked Problem?

J. Rittel and M. Webber originally introduced the concept of wicked problems as common in the area of social planning. Later, C. West Churchmen popularized the concept in 1967 in an article in the magazine, Management Science. The concept addresses problems with incomplete, contradictory and changing requirements. The problem is not understood and the solution is not understood. The problem includes high degrees of uncertainty, risk, and social complexity; in addition, the solution to a wicked problem typically uncovers an even more complex problem.

The concept is still evolving and there is difficulty in getting a consistent definition of what a wicked problem is. The Nautilus Institute applies the following definition: “Wicked problems are complex problems that change when you apply a solution.”

A wicked problem has several characteristics according to computer scientist Jeff Conklin. They are:

  1. The problem is not understood until after formulation of a solution.
  2. Stakeholders have radically different world views and different frames for understanding the problem.
  3. Constraints and resources to solve the problem change over time.
  4. The problem is never solved.

Why is this important to me?

In many ways IT is a victim of its own success. Despite their grumbling, the stakeholders intensely value what IT has done for them and want more, but they are asking for the work to be done in more complex domains.  One of the reasons that this concept is of interest to the IT community is because most of the “tame” problems have been solved. A “tame” problem would have the opposite characteristics from those identified above as characteristics of the “wicked” problem. These have been described as:

  1. The clear definition of the problem also unveils the solution. The solution is determined according to criteria revealing the degree of effect. The solution goal may be achieved fully in a single project or require several projects; however, the success of each project is measurably true or false.
  2. The causes of a problem are determined primarily by experts using scientific data (e.g., clinical trials).
  3. The task is complete when the problem is solved.
  4. The problem is like other problems for which there are scientifically based protocols that guide the choice of solution(s).

This sounds like the way that we typically approach our projects and that viewpoint is fine for most projects. For example, when IT is asked to add some functionality to the book keeping application, both the problem and the solution are well understood. The book keeping application is an example of a Simple problem. Let me emphasize that Simple does not mean easy!

It gets a bit more demanding when you are asked to apply Long Tail concepts (the economic phenomenon where products that are of interest to only small communities, and thus result in low demand and low sales volume, can collectively result in a large aggregate market) to the company’s web site. Long Tail applications are what allow organizations such as Amazon® to make recommendations that are focused on your particular tastes. The problem is well understood; however, the solution is not well understood. The project is addressing a Complex problem. These Complex problems are initially very challenging, but the problem becomes tamer as we gain more experience.

As we move further in we face problems where the solution uncovers a new problem. That doesn’t mean that the organization is not making progress. The initial solution may be a major step forward; however, it is a step on a journey not the end of the journey. An example given to small groups in a recent seminar was “Solve the problem of terrorism.” Before identifying the solution, you need to get agreement what “terrorism” is. That seems simple until you try to get a group to agree. Once you get agreement on the terminology, the nature of the problem needs to be re-visited.

How do we address Wicked problems?

Wicked problems require expert facilitation in order to establish a goal or set of goals for the project effort. There are several approaches that you can apply against a wicked problem. Horst Rittel developed the Issue-Based Information System (IBIS) technique in the 1980s. The emphasis is to produce a visual that captures the comments of the various stakeholders and illustrates the relationships between those comments. The IBIS method only has three elements:

  1. Issues (or questions): these are issues that need to be addressed.
  2. Positions (or ideas): these are responses to questions. Typically the set of ideas that respond to an issue represents the spectrum of perspectives on the issue.
  3. Arguments: these can be Pros (arguments supporting an issue) or Cons (arguments against an issue). The complete set of arguments that respond to an idea represents the multiplicity of viewpoints on it.

There have been several software implementations of the technique since its inception. One product that supports IBIS mapping is called Compendium.

Another approach which uses the Ibis technique is called Dialogue Mapping™. This was developed by Jeff Conklin.

The three elements are ideas, questions and arguments. Any meeting comment is classified as one of the three elements. The symbols are replied to people’s comments in the meeting. The symbols include “?”, “+”, “-“, and the light-bulb. As you might infer, the light-bulb represents an idea. The question mark indicates a question about a comment. The Argument is either for or against. The plus sign indicates support for a comment while the minus sign indicates an objection to a comment. You are not restricted to the four basic symbols as you can see. Other symbols can be incorporated as needed.

How does this affect solution definition and development?

When we face an opportunity that has many of the characteristics of a wicked problem, we should expect to do an awful lot of requirements analysis and problem definition in association with prototyping and agile development.

Another critical component of managing this type of effort is to clearly set the expectations of the various stakeholders. If the stakeholder’s primary experience with solution development has been in the area of tame problems they have an expectation that a best solution will emerge and they will become very frustrated when no best solution is identified. The project manager and business analysts must spend time educating the stakeholders about the nature of the problem. The key stakeholders and other project participants need to understand that the initial efforts to identify a solution will arrive at a solution which is:

  1. Acceptable to the various parties, but clearly not an ideal and
  2. An initial solution with expectation that implementation will generate new requirements.

Wicked problems clearly have life cycles. As the organization implements and uses a solution, the organization will develop a different understanding of the problem being addressed. This results from the nature of wicked problems. As said earlier, when working in the domain of wicked problems, any solution will either uncover a new problem or clarify our understanding of the original problem. In either case, the solution which was installed was really an interim or transitional solution.

Any solution has transitional requirements. When we are discussing tame problems, the transitional requirements address needs that must be addressed as the organization is moving from the old solution to the new solution. As the problems that we are addressing become less easily defined, there will be transition requirements that allow the solution to be useful as we get a greater experience in the problem domain and then the functionality is replaced based upon that new understanding. When we come to the wicked problems, the solution itself may be regarded as a transitional state. The solution was developed based upon a view of the problem trying to incorporate a set of incomplete and contradictory requirements from a wide set of stakeholders. As the solution is used, the stakeholders become more able to express requirements.

The concept of a wicked problem might seem frustrating, but we commonly face wicked problems in real life. My favorite example of a problem that has multiple stakeholders with incomplete and contradictory requirements and expectations is rearing a child.

We have all listened to parents discussing their child’s future as if it was a tame a problem. One amusing situation is where the parent has bought a little sweatshirt from their favorite University and the shirt indicates the planned graduation date from the graduate school. None of us are surprised years later when the parents are trying to understand what they did that generated a situation with their child that is so at variance with their plan.

In one case, the child exhibited intelligence and a great deal of natural artistic skill very early. The parents insured that the child got the best possible training in dance, painting, and music. They made sure that the adolescent girl was in the best possible school and getting the artistic opportunities they thought she needed. This went on for years and the parents had high expectations for their daughter’s future in theater or cinema. Today, the parents still do not understand what made their artistic daughter to choose a different University than the parents had planned and why she chose to become a lawyer.

Based on our discussion, what can we conclude?

The key to wicked problems seems to be to design the solution assuming that substantial amounts of it will have to be replaced. Another way of looking at the solution is that it is a prototype. The actual use of the prototype allows a better understanding of what is actually desired. This discovery is then used to develop the next instantiation of a solution. A solution was successful when the stakeholders felt it gave value and moved their understanding of the problem forward so they could better describe what the next version of a solution needed to be.

Research

  1. Hass, Kathleen; Wessels, Don; Brennan, Kevin “Getting It Right: Business Requirement Analysis Tools and Techniques” Management Concepts, 2007
  2.  Hossenlopp, Rosemary; Hass, Kathleen B.”Unearthing Business Requirements: Elicitation Tools andTechniques (Business Analysis Essential Library)” Springer, 2010
  3.  Carkenord, Barbara “Seven Steps to Mastering Business Analysis” J Ross, 2008
  4. Podeswa, Howard “The Business Analyst’s Handbook” Course Technology PTR 2008
  5. Paul, Debra; Yeates, Donald; Cadle, James “Business Analysis” British Informatics Society Ltd, 2010
  6. Hass, Kathleen B.; Horst, Richard Vander; Ziemski, Kimi; Lindbergh, Lori “From Analyst to Leader: Elevating the Role of the Business Analyst (Business Analysis Essential Library)” Management Concepts, Inc., 2007
  7. Brennan, Kevin “A Guide to the Business Analysis Body of Knowledge® (BABOK® Guide)” International Institute of Business Analysis, 2009
  8. Marriot, Alexander “All About Business Analysts!” Kindle Edition