The Importance of Naivete, Written by John Slack, MBA PMP
How does the knowledge of subject matter experts affect use cases?
In order to develop a use case that is truly useful, we need subject matter experts. One of the key elements of a good use case is that it tells a story that can be analyzed. There is, however, a significant danger in having subject matter experts telling the story. By definition, a subject matter expert is telling their version of the story based upon their context. There are however a set of subject matter experts that are very informative to include. These subject matter experts include:
- Wizards (very knowledgeable practitioners)
- Average Practitioners
- New Practitioners
- 8 Year-Old Children
- Frustrated Customers
Obviously, anyone can look at that list and dispute whether or not several of its members are really subject matter experts. The first three on the list are labels typically used by Alan Cooper, author of books on user interface design such as “About Face 3: The Essentials of Interaction Design.” Alan makes a very valid point when he states that, in our effort to find those who can tell us the story accurately, we sometimes get a very skewed view. I have used the fourth category, the eight-year-old child, as a perspective test. I will address each of these sources in turn.
What is the danger in talking to those who understand the process?
Wizards are those who understand the business process better than anyone else. They are the ones whom everyone turns to when they are stumped. This stakeholder is invaluable for us to gain a thorough understanding of what the process and the tools supporting the process can really do.
The Wizard can easily do things in the process that the rest of the population finds extremely difficult, if not impossible. This gives the analysis team a much more thorough understanding of what is available. There are, however biases associated with the Wizard. Any effort to improve the process runs into the Wizard’s bias in two ways.
- The primary bias is a loss of status. The Wizard has a lot of their personal status on the job based upon their ability to do things that others cannot; they are the people to whom others turn to for help. Major changes to the existing process could eliminate the value that these Wizards built up over the course of years. This is not to imply that these highly knowledgeable people are holding back or consciously editing their input. They simply do not have a big stake in eliminating components of the process that they understand far more than anyone else.
- The Wizard may not perceive a feature as needed since they don’t need it. This leads to the second area of bias on the part of the Wizard. The Wizard will desire a set of features; however, their desires may seem quite unique. This leads to one of the strengths of the Wizard’s inclusion in the research.
Since the Wizard is the one sought when a process user is in trouble, they have unique insights into the exception cases. People don’t come to them asking about the routine. They tend to deal with the weird and offbeat. If the average practitioners only encounter an exception of a given type once a year, there is little likelihood that they will remember it. The Wizard typically will remember it since he or she is the individual that the average practitioners turned to when they encountered this rare exception.
Most systems have a number of exceptions that occur infrequently. The Wizard is going to hold a great deal of knowledge about these exceptions in the use case. Their reaction when walked through the use case supplied by the typical system user would be to comment that it was accurate but incomplete.
What does the average user bring to the use case?
It should come as no surprise that the average practitioners bring the viewpoint of the day-to-day. The average practitioner will tend to describe the most common flows including the most common exceptions. They will have a number of insights into the features that would make a significant impact on the business since that is the bulk of the business processing.
Once again, the researchers need to be aware that the average practitioners have an investment in the existing business system similar to the Wizard. While the Wizard’s investment relates to an in-depth knowledge of the exceptions to the system, the average practitioners have an investment in their knowledge of the way things currently work. Throughout human history, people have heard the refrain: “We have always done it that way.”
The average practitioner lives in the realm of “this is the way we do it” and may lean toward opportunities that build upon their existing knowledge while overlooking opportunities that would make that knowledge obsolete.
What insights can be obtained from the raw recruit?
The new practitioner also supplies unique insights. While the Wizard has the advantage of an in-depth understanding of the system, the new practitioner has the advantage of not being invested in the existing system. The typical new practitioner has been through training in the fundamentals of the existing system, but their knowledge is still quite malleable. They will often see problems and/or opportunities that the Wizards and the average practitioners are no longer even aware of. The issues seen by the raw recruit may be perceived as “the normal learning curve” rather than problems or opportunities.
There is obviously a danger in having the Wizards, the average practitioners, and the raw recruits in the same session. The raw recruits may be hesitant to bring their insights forward since they fear appearing incompetent in front of those more skilled. On the other hand, the Wizards may tend to override everyone else with their opinions since they consider themselves to be the most knowledgeable people in the room.
This will clearly be a facilitation challenge; however, it can work quite well if the rules are clearly established and adhered to.
What is the advantage of having someone who admittedly does not already understand the system?
Another category that was listed earlier was the “eight-year-old child.” This is not a serious recommendation that we bring children into these sessions.
It is however, a serious suggestion that someone be included who is ignorant of the process and willing to ask questions from that basis of naïveté or willing to really play that role.
It can be an enlightening experience to try to answer the questions of an eight-year-old. Every parent has gone through encounters where the child is asking questions and pointing out that our adult answers do not really answer the question.
Years ago, my son became interested in scouting. He started out in Cub Scouts and went all the way through the scouting program. When he started, the local scouting organization needed volunteers to work with these small groups of young children. I volunteered to do so and discovered just how much effort it is to address their questions and expectations. A child in the eight to 10-year-old range has no reservation about saying that he does not understand your answer. He will keep probing until he gets an answer that he can understand, or the adult terminates the conversation in frustration.
Six Sigma and ISO 9001 have an approach that they call the “why technique.” They recognized that the real answer to most questions about problems needed to go three levels deep. This might require going to seven levels when the problems are quite complex. There is a tremendous hesitation in most individuals to keep asking for clarification.
This need for clarification applies at virtually every level of our analysis. For example, most organizations have a mission statement and/or a vision statement. Many of these statements are long and convoluted; yet, they remain unchallenged.
Here is an example of a mission statement that seems to be clear: “organize the world’s information and make it universally accessible and useful.” An eight-year-old might need some clarification about the terms used; however, he or she would understand the mission when they understood the words.
It is not difficult to make things complex. It requires a great deal of effort to make ideas clear. Any analytic effort needs people who are willing to ask for clarity; whether that clarity requires three questions or seven.
What is the advantage of having someone who is frustrated with the system?
The final category listed was the “frustrated customer.” More than 50 years ago, Ishikawa said that you could not start to define required quality until you could find a customer who was unhappy with your product.
The tremendous advantage of the “frustrated customer” is that they can be very specific in what they consider to be the deficiencies of the system.
For example, in the case of one company, they felt that their telephone support operations were performing well. The surveys which customers took after completing their transactions indicated that people were happy.
When the company did a survey of customers about their telephone support system they received different results. It turned out that their telephone support system only allowed incoming calls and the telephone support specialists could not call a customer back when a call was dropped. The existing system tended to lose customers when they were put on hold.
The customers’ perception was that either the telephone support specialists had deliberately hung up on them or that they did not care enough to call the customer back. Typically, the customer called back, was connected to a different telephone support representative, and completed their transaction. If they chose to take the survey, they evaluated the call they had just completed rather than the call that had frustrated them.
Once the frustrated customers were asked about the source of their frustration, the problem became clear. The “frustrated customer” can be an invaluable source of ideas.
Based on our discussions, it is evident that a wealth of knowledge regarding our system can be acquired by obtaining the opinions of system end-users with all levels of expertise, as well as the opinions of those completely unfamiliar with a system. We can consider all of the categories of end-users to be “Subject Matter Experts” even when they may not be a “Wizard.”
Viewpoints from various subject matter experts are necessary to create the most complete analysis of the subject and to overcome the various biases built into each expert’s respective perspective that would otherwise be overlooked.
- Bittner, Kurt. “Use Case Modeling” Addison-Wesley Professional, 2002.
- Rosenberg, Doug; Stephens, Matt. “Use Case Driven Object Modeling with UML.” Apress, 2007.
- Cockburn, Alistair. “Writing Effective Use Cases.” Addison-Wesley Professional, 2000