Using Narratives to Make Sense of High Uncertainty
18/09/20 13:44
This was originally published in June 2020 on Britten Coyne Partners' Strategic Risk Blog
The arrival of the COVID19 pandemic has made the distinction between risk and uncertainty painfully clear. In the case of the former, the range of possible future outcomes is known, as are their probabilities, and potential impact.
In the case of uncertainty, some or all of these are unknown.
We have many tools available to help us make good decisions in the face of risk. While the crises occasionally remind us that these tools aren’t perfect (e.g., Long Term Capital Management in 1998, the housing collapse in 2008, and COVID19 in 2020), most of the time they serve us well.
But that is not true when we confront highly uncertain systems and situations.
To be sure, our first reaction is to try to adopt our risk tools to help us in these situations. In place of probabilities based on the historical frequencies at which common events (e.g., car accidents) occur, we take the Bayesian approach, and substitute probabilities signifying our degree of belief that historically rare or unprecedented events will occur in the future.
The four years I spent on the Good Judgment Project reminded me once again of the benefits (e.g., superior forecast accuracy) that arise from the disciplined application of Bayesian methodology.
Yet over a forty-year career of making decisions in the face of uncertainty, I’ve also seen that Bayesian methods can sometimes creates a false – and dangerous – sense of security about the precision of our knowledge, leading to overconfidence and poor decisions.
What we need is broader mix of methods for analyzing and making good decisions under conditions of uncertainty (and ignorance, or “unknown unknowns”), not just risk.
With this in mind, I was excited to read a new paper, “The Role Of Narrative In Collaborative Reasoning And Intelligence Analysis: A Case
Study”, by Saletta et al, which significantly adds to our understanding of the role of narratives and how we construct them to help us make sense of highly uncertain situations
At Britten Coyne Partners, we have written a lot about the important, if too little understood, role that narrative plays in both individual and collective sensemaking and decision making under uncertainty.
Researchers have found that when uncertainty rises, evolution has primed human beings to become much more prone to conformity and to rely more on imitating what others are doing (so-called "social learning" or “social copying”).
Paradoxically, as uncertainty increases, people are more likely to become attracted to a smaller (not larger) number of competing narratives that explain the past and present and sometimes predict possible future outcomes.
In other words, as uncertainty increases the conventional wisdom grows stronger, even as it is becoming more fragile and downside risks are rising.
Today's hyperconnected socio-technical systems — including financial markets — are therefore more vulnerable than ever before to small changes in information that trigger feelings (especially fear) and behavior that spread quickly, and are then further amplified by algorithms of various types. The increasing result is sudden, non-linear change.
In recent years, the role of narratives in economic cycle and financial markets has increasingly been a subject of academic inquiry (e.g., “Narrative Economics”, by Bob Shiller; “Constructing Conviction through Action and Narrative: How Money Managers Manage Uncertainty and the Consequences for Financial Market Functioning”, by Chong and Tuckett; and “News and Narratives in Financial Systems: Exploiting Big Data for Systemic Risk Assessment”, by Nyman et al).
While there are many definitions of “narrative” (and synonyms for it, like “analytical line”, and sometimes “mental models”), most have some common elements, including descriptions of context and key characters, actions and events that move the narrative forward through space and time, and causal links to outcomes and various types of consequences (e.g., cognitive, affective, and/or physical).
Saletta and his co-authors take our understanding of narrative from the macro to the micro level, and describe how, “individuals and teams use narrative to solve the kinds of complex problems organizations and intelligence agencies face daily.”
They “observed that team members generated “micro-narratives”, which provided a means for testing, assessing and weighing alternative hypotheses through mental simulation in the context of collaborative reasoning…
“Micro-narratives are not fully developed narratives; [instead] they are incomplete stories or scenarios…that emerge in an unstructured manner in the course of a team’s collaborative reasoning and problem solving...
[Micronarratives] “serve as basic units that individuals and teams can debate, deliberate upon, and discuss in an iterative process in which micro-narratives are generated and weighed against each other for plausibility with regard to evidence, general knowledge about the world, and fit with other micro-narratives. They can then be organized and assembled into a larger, more developed narrative…
The authors document that the intelligence analysts they studied “ran mental simulations to reason about evidence in a complex problem... [and] test and evaluate many diverse, interacting and often competing micro-narratives that were generated collaboratively with other team members...They tested the plausibility of these micro-narratives against each other, what they knew, and their best estimates (or guesses) for what they didn’t know”…
“In a non-linear and iterative process, the analysts used the insights developed in the process of generating and evaluating micro-narratives to develop a macro-level narrative.”
The authors conclude that, “narrative thought processes play an important role in complex collaborative problem-solving and reasoning with evidence…This is contrary to a widespread perception that narrative thinking is fundamentally distinct from formal, logical reasoning.”
This is also a very accurate description of the team forecasting process that I experienced during my years on the Good Judgment Project.
At Britten Coyne Partners, we also stress that this basic process of collaborative sensemaking in highly uncertain systems and situations can be further enhanced through the use of three complementary processes.
The first is structuring sensemaking processes around the three critical questions first described by Mica Endsley twenty-five years ago:
(1) What are the key elements (e.g., characters, events, etc.) in the system or situation you are assessing, over the time horizon you are using?
(2) What are the most important ways in which these elements are related to each other (e.g., causal connections and positive feedback loops that give rise to non-linear effects)?
(3) Given the interaction of the critical trends and uncertainties you have identified, how could the system/situation evolve in the future, either on its own or in response to actions you and/or other players could take?
The second is using explicit processes to offset what Daniel Kahneman has called the WYSIATI phenomenon (“What You See Is All There Is”), or our natural tendency to reach conclusions only on the basis of the information that is readily at hand. As Sherlock Holmes highlighted in “The Hound of the Baskervilles”, it is often the dog that doesn’t bark that provides the most important clue.
In “Superforecasting”, Professor Phil Tetlock showed how the WYSIATI problem can be overcome (and forecast accuracy improved) by combining the information at hand with longer term, “base rate” or “reference case” data.
Another approach is Gary Klein’s “pre-mortem” technique. Tell your team to assume it is some point in the future and their assessment or forecast has turned out to be wrong. Ask them to write down the information they missed or misinterpreted, including important information that was absent.
A final technique is to have your team assume that their original evaluation of the evidence was wrong, and to generate alternative assessments of what it could mean.
The third process is Marvin Cohen’s critiquing method, which focuses on finding and resolving three problems that reduce the reliability of macro narratives (e.g., more formal scenarios). Incomplete narratives are either missing key narrative elements or represent them using assumptions rather than direct evidence. The second problem is the use of assumptions to explain away conflicts between available evidence. And the third problem is the use of doubtful assumptions that have weak evidential support.
In the post-COVID19 world, the ability to make sense of unprecedented uncertainty and maintain ongoing situation awareness under rapidly evolving conditions will be a hallmark of high performance teams.
Unfortunately, this is not a capability that has been developed in the course of many leaders’ previous training and experience. Mastering new tools and processes, like the use of narrative, is critical to organization’s future success.
These and other processes and tools for making good decisions in the face of unprecedented uncertainty are covered in our new online course, leading to a Certified Competence in Strategic Risk Governance and Management. You can learn more about it at the Strategic Risk Institute LLC (an affiliated of Britten Coyne Partners and Index Investor LLC).
A 50% discount on the launch price is offered for the first 100 subscribers. Enter the code 50PERCENTPOUND (for payment in pounds) or 50PERCENTDOLLAR (for payment in dollars) on the payment page.
The arrival of the COVID19 pandemic has made the distinction between risk and uncertainty painfully clear. In the case of the former, the range of possible future outcomes is known, as are their probabilities, and potential impact.
In the case of uncertainty, some or all of these are unknown.
We have many tools available to help us make good decisions in the face of risk. While the crises occasionally remind us that these tools aren’t perfect (e.g., Long Term Capital Management in 1998, the housing collapse in 2008, and COVID19 in 2020), most of the time they serve us well.
But that is not true when we confront highly uncertain systems and situations.
To be sure, our first reaction is to try to adopt our risk tools to help us in these situations. In place of probabilities based on the historical frequencies at which common events (e.g., car accidents) occur, we take the Bayesian approach, and substitute probabilities signifying our degree of belief that historically rare or unprecedented events will occur in the future.
The four years I spent on the Good Judgment Project reminded me once again of the benefits (e.g., superior forecast accuracy) that arise from the disciplined application of Bayesian methodology.
Yet over a forty-year career of making decisions in the face of uncertainty, I’ve also seen that Bayesian methods can sometimes creates a false – and dangerous – sense of security about the precision of our knowledge, leading to overconfidence and poor decisions.
What we need is broader mix of methods for analyzing and making good decisions under conditions of uncertainty (and ignorance, or “unknown unknowns”), not just risk.
With this in mind, I was excited to read a new paper, “The Role Of Narrative In Collaborative Reasoning And Intelligence Analysis: A Case
Study”, by Saletta et al, which significantly adds to our understanding of the role of narratives and how we construct them to help us make sense of highly uncertain situations
At Britten Coyne Partners, we have written a lot about the important, if too little understood, role that narrative plays in both individual and collective sensemaking and decision making under uncertainty.
Researchers have found that when uncertainty rises, evolution has primed human beings to become much more prone to conformity and to rely more on imitating what others are doing (so-called "social learning" or “social copying”).
Paradoxically, as uncertainty increases, people are more likely to become attracted to a smaller (not larger) number of competing narratives that explain the past and present and sometimes predict possible future outcomes.
In other words, as uncertainty increases the conventional wisdom grows stronger, even as it is becoming more fragile and downside risks are rising.
Today's hyperconnected socio-technical systems — including financial markets — are therefore more vulnerable than ever before to small changes in information that trigger feelings (especially fear) and behavior that spread quickly, and are then further amplified by algorithms of various types. The increasing result is sudden, non-linear change.
In recent years, the role of narratives in economic cycle and financial markets has increasingly been a subject of academic inquiry (e.g., “Narrative Economics”, by Bob Shiller; “Constructing Conviction through Action and Narrative: How Money Managers Manage Uncertainty and the Consequences for Financial Market Functioning”, by Chong and Tuckett; and “News and Narratives in Financial Systems: Exploiting Big Data for Systemic Risk Assessment”, by Nyman et al).
While there are many definitions of “narrative” (and synonyms for it, like “analytical line”, and sometimes “mental models”), most have some common elements, including descriptions of context and key characters, actions and events that move the narrative forward through space and time, and causal links to outcomes and various types of consequences (e.g., cognitive, affective, and/or physical).
Saletta and his co-authors take our understanding of narrative from the macro to the micro level, and describe how, “individuals and teams use narrative to solve the kinds of complex problems organizations and intelligence agencies face daily.”
They “observed that team members generated “micro-narratives”, which provided a means for testing, assessing and weighing alternative hypotheses through mental simulation in the context of collaborative reasoning…
“Micro-narratives are not fully developed narratives; [instead] they are incomplete stories or scenarios…that emerge in an unstructured manner in the course of a team’s collaborative reasoning and problem solving...
[Micronarratives] “serve as basic units that individuals and teams can debate, deliberate upon, and discuss in an iterative process in which micro-narratives are generated and weighed against each other for plausibility with regard to evidence, general knowledge about the world, and fit with other micro-narratives. They can then be organized and assembled into a larger, more developed narrative…
The authors document that the intelligence analysts they studied “ran mental simulations to reason about evidence in a complex problem... [and] test and evaluate many diverse, interacting and often competing micro-narratives that were generated collaboratively with other team members...They tested the plausibility of these micro-narratives against each other, what they knew, and their best estimates (or guesses) for what they didn’t know”…
“In a non-linear and iterative process, the analysts used the insights developed in the process of generating and evaluating micro-narratives to develop a macro-level narrative.”
The authors conclude that, “narrative thought processes play an important role in complex collaborative problem-solving and reasoning with evidence…This is contrary to a widespread perception that narrative thinking is fundamentally distinct from formal, logical reasoning.”
This is also a very accurate description of the team forecasting process that I experienced during my years on the Good Judgment Project.
At Britten Coyne Partners, we also stress that this basic process of collaborative sensemaking in highly uncertain systems and situations can be further enhanced through the use of three complementary processes.
The first is structuring sensemaking processes around the three critical questions first described by Mica Endsley twenty-five years ago:
(1) What are the key elements (e.g., characters, events, etc.) in the system or situation you are assessing, over the time horizon you are using?
(2) What are the most important ways in which these elements are related to each other (e.g., causal connections and positive feedback loops that give rise to non-linear effects)?
(3) Given the interaction of the critical trends and uncertainties you have identified, how could the system/situation evolve in the future, either on its own or in response to actions you and/or other players could take?
The second is using explicit processes to offset what Daniel Kahneman has called the WYSIATI phenomenon (“What You See Is All There Is”), or our natural tendency to reach conclusions only on the basis of the information that is readily at hand. As Sherlock Holmes highlighted in “The Hound of the Baskervilles”, it is often the dog that doesn’t bark that provides the most important clue.
In “Superforecasting”, Professor Phil Tetlock showed how the WYSIATI problem can be overcome (and forecast accuracy improved) by combining the information at hand with longer term, “base rate” or “reference case” data.
Another approach is Gary Klein’s “pre-mortem” technique. Tell your team to assume it is some point in the future and their assessment or forecast has turned out to be wrong. Ask them to write down the information they missed or misinterpreted, including important information that was absent.
A final technique is to have your team assume that their original evaluation of the evidence was wrong, and to generate alternative assessments of what it could mean.
The third process is Marvin Cohen’s critiquing method, which focuses on finding and resolving three problems that reduce the reliability of macro narratives (e.g., more formal scenarios). Incomplete narratives are either missing key narrative elements or represent them using assumptions rather than direct evidence. The second problem is the use of assumptions to explain away conflicts between available evidence. And the third problem is the use of doubtful assumptions that have weak evidential support.
In the post-COVID19 world, the ability to make sense of unprecedented uncertainty and maintain ongoing situation awareness under rapidly evolving conditions will be a hallmark of high performance teams.
Unfortunately, this is not a capability that has been developed in the course of many leaders’ previous training and experience. Mastering new tools and processes, like the use of narrative, is critical to organization’s future success.
These and other processes and tools for making good decisions in the face of unprecedented uncertainty are covered in our new online course, leading to a Certified Competence in Strategic Risk Governance and Management. You can learn more about it at the Strategic Risk Institute LLC (an affiliated of Britten Coyne Partners and Index Investor LLC).
A 50% discount on the launch price is offered for the first 100 subscribers. Enter the code 50PERCENTPOUND (for payment in pounds) or 50PERCENTDOLLAR (for payment in dollars) on the payment page.