Analytical Lessons
Key Takeaways from Richard Heuer’s ‘Psychology of Intelligence Analysis’
Psychology of Intelligence Analysis, originally published in 1999, compiles and builds on a collection of articles written by Richard Heuer between 1978 and 1986 for internal use by the CIA’s intelligence analysts. Heuer was a 45-year CIA veteran who was an early student of of Kahneman and Tversky’s work on cognitive psychology and developed many of the techniques still used by CIA analysts today.
Heuer preached ‘challenging, refining and challenging again’ and advised these practices be established and encouraged at the organizational level. At Voss we try to adhere to this with our iterative research process. The book has become required reading at Voss and we highly recommend it to any investment analyst. Below we compile some of what we believe are the key takeaways from Psychology of Intelligence Analysis and list some actionable insights.
Perceptions and Expectations
Analysts invariably approach a problem with a set of assumptions about how events typically transpire in that area, both consciously and subconsciously. This mindset is inescapable. We tend to perceive what we expect to perceive.
Patterns of expectation become so deeply embedded that they continue to influence our perceptions even when we’re alerted to them. Simply trying to be objective does not ensure accurate perception.
Events consistent with expectations are perceived and processed easily, while events that contradict prevailing expectations tend to be ignored or distorted in perception.
The circumstances under which analysis is generally conducted is a recipe for inaccurate perception— highly ambiguous situations, using information processed incrementally, and under pressure for quick judgment.
Mindsets tend to be quick to form but resistant to change. Once events have been perceived one way, there is a natural resistance to other perspectives.
The amount of information necessary to change your mind is typically considerably greater than the amount of information you required to make the initial interpretation. The initial interpretation is maintained until the contradiction becomes so obvious that it forces itself upon our consciousness.
Impressions often remain even after the evidence on which they are based has been totally discredited.
One of the more difficult but valuable analysis feats is to take a familiar dataset and reorganize it visually or mentally to perceive it from a different perspective. Analysts must understand the situation as it appears to each of the opposing forces, and constantly shift back and forth from one perspective to the other as they try to fathom how each side interprets an ongoing series of interactions.
Actionable Insight:
Try to suspend judgment for as long as possible when new information is received. Aim to ensure that our mind remains open to all alternative interpretations.
Even once a conclusion has been reached, be sure to periodically re-evaluate your perception from the ground up (re-underwrite the position) in order to avoid thesis creep.
Attempt to achieve objectivity by making any basic assumptions and reasoning as explicit as possible so that they can be challenged by others.
Understanding How Your Mind Works
The pattern of relationships among data in you mind is called a schema. When a chess master develops an accurate schema in a static domain like chess with set rules that never change, there is no need to change it. An investment analyst however must cope with a rapidly changing world.
When analysts connect similar data points frequently within their mind, they form paths that makes it easier to take that route in the future. Once analysts have become accustomed to thinking about a problem one way, these reliable mental pathways get activated repeatedly and strengthened. This serves to facilitate the quick retrieval of information and quick interpretations which is often useful. However, these same pathways can also become the mental ruts that make it difficult to reorganize information so as to see it from a different perspective.
If information does not fit into what analysts know, or think they know, they will have a greater difficulty processing it. Without an appropriate “category” for something in our minds, we are unlikely to perceive it, store it in memory, or be able to easily retrieve it later. This “hardening” of existing categories in our minds is a common analytical weakness.
New ideas often result from the association of old elements in new combinations. When that linkage is made, the light dawns. This ability to bring previously unrelated information and ideas together in meaningful ways is what makes an open-minded, imaginative, and creative analyst.
Actionable Insight:
Avoid “hardening the categories” in your mind. Don’t just tolerate ambiguity but embrace it, allow information to flow freely in your mind in search of non-traditional associations. By allowing known information to be reorganized and combined with new information analysts can stimulate previously unconsidered relationships.
We often get asked by potential LPs about our idea generation process. While we would like to say we have some highly systematic process that just churns out ideas regularly, the truth is much less mechanical. Instead, a more common source of investment ideas at Voss comes from continuously working to build as wide a knowledge base as possible and letting new incremental information spark previously unconsidered connections.
Doing The Analysis
When dealing with a new or unfamiliar subject, an analyst should first begin with a non-selective accumulation of information. This is a process of absorbing information, not analyzing it. At some point in this absorption phase an apparent pattern may emerge organically.
It can help to to decompose and externalize the problem.
Decompose meaning breaking a problem down into its component parts. Get it into some simplified form that shows the main variables, parameters, and elements of the problem (i.e. company or industry) and how they relate to each other.
Externalize in terms getting the problem out of your head, either verbally or onto some visible form that you can work with (paper or a whiteboard). Physically writing things down activates different parts of your brain and can stimulate new associations.
Note the absence of evidence as well as its presence. One’s attention tends to focus on what is reported rather than what is not reported, “out of sight, out of mind”. Consider whether the absence of information is normal or is itself an indicator of unusual activity or inactivity. People remember occurrences more readily than non-occurrences, but the latter can be just as significant to the analysis.
The first ideas that come to mind will be those that are most common or usual. It is necessary to run through these conventional ideas before arriving at original or differentiated ones.
Check original source materials rather than relying on someone else’s interpretation.
Assumptions are fine as long as they are made explicit in your analysis and you analyze the sensitivity of your conclusions to those assumptions. Analysts should try to disprove their assumptions rather than confirm them.
One kind of assumption an analyst should always recognize and question is mirror-imaging—filling gaps in your own knowledge by assuming that the other side is likely to act in a certain way because that is how you would act under similar circumstances.
If an analyst cannot think of anything that would cause a change of mind, his or her mindset may be so deeply entrenched that the analyst cannot see the conflicting evidence. Relevant information is sometimes discounted, misinterpreted, ignored, rationalized, rejected, or overlooked because it fails to fit the prevailing narrative.
Actionable Insight:
Similar to the way immersion is the best way to learn a new language, we find immersion can also be the best way to get up to speed on a subject enough to effectively analyze it. If we’re approaching an area we aren’t overly familiar with we’ll often “immerse” ourselves in the subject matter. Depending on the company or industry this can typically include some combination of reading industry magazines and white papers, posting on relevant message boards, lining up expert calls, downloading dozens of podcasts on the subject, watching countless hours of YouTube videos, attending industry conferences, visiting factories or stores, and doing demos/buying the company’s products.
Constantly write things down to better visualize the problem and stimulate new associations between the factors.
Seek the input of others if you can’t think of anything that would disprove your assumptions.
Don’t assume that the other side is likely to act as you would. This can be particularly dangerous on the short side.
Emphasize procedures that expose and elaborate alternative points of view.
Stimulate creative thinking by tapping unusual sources of data, asking new questions, and applying unusual analytic methods.
When Drawing Conclusions
Analytical conclusions should always be regarded as tentative.
A surprise or two, however small, may be the first clue that your understanding of what is happening requires some adjustment.
When an event happens we often offer up a set of causal connections to explain it. Because of a need to impose order on our environment, we seek and often believe we find causes for what are actually accidental or random phenomena. If no pattern is apparent, our first thought is that we lack understanding, not that we are dealing with random phenomena that have no purpose or reason. Events will almost never be perceived intuitively as being random; one can find an apparent pattern in almost any set of data or create a coherent narrative from any set of events.
We tend to assume that causes are similar to their effects in the sense that important or large effects must have large causes. This isn’t always true.
We often inaccurately judge the probability of an event by the ease with which they can imagine similar events or the number of such events that they can easily remember.
Analysts are often reluctant to devote time to studying things they do not believe will happen.
Actionable Insight:
Always specify in advance things to look for that would cause you to change your mind. This will make it more difficult for you to rationalize such developments if they occur. It’s useful to keep a record of unexpected events and think hard about what they might mean, not disregard them or simply explain them away.
Identify any alternative conclusions that were considered and justify why they were deemed less likely. Is the downside probability significant enough to warrant further analysis?
Try not to deem recent events as more likely simply because they are recent.
It’s a useful exercise to imagine an outcome that you do not expect to occur has occurred e.g. a stock your long goes to zero. Put yourself into the future and look back to explain how this could have happened. What events could have led to this? Basically do a post-mortem, write the obituary of the company.
Reviewing Past Analyses
When an analysis turns out to be wrong, it is often because of key assumptions that went unchallenged and proved invalid.
Analysts interested in improving their own performance need to evaluate their past estimates in the light of subsequent developments.
With the benefit of hindsight, factors previously considered relevant may become irrelevant, and factors previously thought to have little relevance may be seen as determinative.
When events occur, people tend to overestimate the extent to which they had were expected to occur. Conversely, when events do not occur, we tend to underestimate the probability we had previously assigned to their occurrence.
It is difficult to accurately reconstruct our previous thinking. Once an event has occurred, it is impossible to erase from our mind the knowledge of that event and reconstruct what our thought process was ahead of that.
Analysts rarely appear—or allow themselves to appear— very surprised by the course of events they are following.
In reconstructing the past, there is a tendency toward determinism, toward thinking that what occurred was inevitable under the circumstances and therefore predictable.
Actionable Insight:
Thorough post-mortem analyses should be conducted on all significant analytical failures and successes.
Journaling can help crystalize your thinking in the moment and be used to refer back to.
"Analysts should be self-conscious about their reasoning processes. They should think about how they make judgments and reach conclusions, not just about the judgments and conclusions themselves.”