Arthour:Gabe Sawhney
Edited by Christine Keene
Problem
Expert roundtables are a tool for quickly gathering opinions and perspectives from well-informed people. However, the problem with roundtables is that they can produce conflicting information. More influential, aggressive or senior roundtable members can inadvertently exert undue influence on the result. It can also be difficult to get the desired participants together at the same place and time.
Solution
Using the Delphi method, expert opinions are solicited in order to achieve a consensus. Participants are given individual questionnaires by a facilitator, in a series of rounds. With each round, the questions and answers are focused, taking into account the ideas and opinions expressed in the previous round. Answers are anonymous, which eliminates “bandwagon” and effect. It also makes it easier for participants to revise their opinion with each round as justification is not required, nor are they judged on it. Communication between the facilitator and participants is often done over the Internet.
Background
The Delphi method was developed in the 50s/60s by the RAND Corporation, originally to forecast defense threats and technologies.
Use When
The Delphi method is typically used during problem framing. It is applied when advice or informed opinion in a specialized domain is required, but the specific question is subjective (i.e., “Whatʼs the best way to get to the airport?” instead of “Which bus goes to the airport?”).
Use For
Forecasting, searching for solutions to complex problems and validation of a proposed course of action are some of the uses for the Delphi Method. Access to subject matter experts is required. This method is valuable method when participants know each other and/or the influences of professional or social factors want to be avoided. Engagement of tacit knowledge is afforded, as participants are not required to provide justification for responses. In addition, participants don’t need to be in the same place; however, they will generally need access to the Internet.
Process
1) Demographics – Participants are selected based on research goals. Typically, no less than 12 participants are involved.
2) Question – The facilitator develops the first questionnaire and sends it to the participants. The format of the questionnaire can vary: questions can be yes/no, multiple-choice, open-ended, etc.
3) Responses – Participants answer the questionnaire. They are free to research their answer but, to ensure anonymity, cannot consult with anyone else.
4) Analysis – The facilitator analyzes the responses and provides statistics to the participants—for example, “75% of participants answered ʻYesʼ to Question 1”—along with some or all of the written comments. Along with this, the facilitator sends a new questionnaire, which may include the same questions or slightly revised questions. Even if it includes the same questions as round 1, the responses received may be different, since the participants will reply to responses in the previous round.
5) Consensus – This process continues until a consensus emerges or the facilitator decides to stop. If there is no clear consensus, the facilitator can use a formula to make a guess. For example, if a question asks for a ranking from 0 to 5, the formula could be:
6) Considerations – Delphi results may not be particularly accurate, especially for foresighting. The method is highly dependent on the insight provided by the participants, the participants selected and the ability to craft well-designed questions and effectively analyze results. Delphi is oriented toward consensus, but consensus should not be confused with certainty.
Variations
There are lots of small variations that can be made to Delphi. Larger variations include The Cooke Method, in which participantsʼ opinions are weighted by the relevancy of their expertise to the question — in an attempt to differentiate consensus and certainty. Delphi has also been combined with card sorts, for use in project resource estimation, and in information architecture.
References
Delphi Method. (n.d.) Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Delphi_method
Linstone, H., & Turoff, M. (1975). The Delphi Method: Techniques and Application. Addison-Wesley. Available online: http://www.is.njit.edu/pubs/ delphibook/