The book “aims to help intelligence analysts achieve a higher level of performance. It shows how people make judgements based on incomplete and ambiguous information, and it offers simple tools and concepts for improving analytical skills.” If it’s good enough for the CIA, it’s likely to be useful for the average investor.
You can download a copy of the CIA Book for free at their website here..
I've included some of the key points below .. the first quotes are from the CIA Book and following are quotes in italics from the Investment Masters.
Be a Generalist
CIA: "To the extent that an experienced specialist may be among the last to see what is really happening when events take a new and unexpected turn. When faced with a major paradigm shift, analysts who know the most about a subject have the most to unlearn."
Bruce Berkowitz: “We’re generalists, but we need to find the non-Wall Street people who have lived and breathed and suffered in the industries and business we’re now looking at.”
Have a Multi-disciplinary mindset
CIA: "If analysts’ understanding of events is greatly influenced by the mind-set or mental model through which they perceive those events, should there not be more research to explore and document the impact of different mental models?"
Charlie Munger: “For some odd reason, I had an early and extreme multi-disciplinary cast of mind. I couldn’t stand reaching for a small idea in my own discipline when there was a big idea right over the fence in somebody else’s discipline. So I just grabbed in all directions for the big ideas that would really work. Nobody taught me to do that; I was just born with that yen.”
Focus on Collecting Information that Matters
CIA: "The reaction of the Intelligence Community to many problems is to collect more information, even though analysts in many cases already have more information than they can digest. What analysts need is more truly useful information to help them make good decisions. Or they need a more accurate mental model and better analytical tools to help them sort through, make sense of, and get the most out of the available ambiguous and conflicting information."
David Dreman: “Investment experts continue to be convinced that their major problems could have been handled if only those extra few necessary facts had been available. They thus tend to overload themselves with information, which usually does not improve their decisions but only makes them more confident and more vulnerable to serious errors.”
Seek out Other Information
CIA: "Relying only on information that is automatically delivered to you will probably not solve all your analytical problems. To do the job right, it will probably be necessary to look elsewhere and dig for more information."
Phil Fisher: "Reading the printed financial records about a company is never enough to justify an investment. One of the major steps in prudent investment must be to find out about a company's affairs from those who have some direct familiarity with them." Phil Fisher
Julian Robertson: "I think the main thing is management, getting good management, and checking with their competitors, checking with their compatriots, their suppliers, and finding out, really, if they are good."
Ray Dalio: “I dealt with my not knowing by either continuing to gather information until I reached a point I could be confident or by eliminating my exposure to risks of not knowing.”
Test your Investment Ideas
CIA: "Objectivity is achieved by making basic assumptions and reasoning as explicit as possible so that they can be challenged by others and analysts can, themselves, examine their validity."
Ray Dalio: “I stress tested my opinions by having the smartest people I could find challenge them so I could find out where I was wrong.”
Charles Koch: "I have a lot of ideas. Most of them are terrible. But what saved me – well, to the extent I’ve been saved – is that… I want to get people with the best knowledge and insights in each one of those key aspects and get a challenge from them."
Remain Open Minded
CIA: "People form impressions on the basis of very little information, but once formed, they do not reject or change them unless they obtain rather solid evidence. Analysts might seek to limit the adverse impact of this tendency by suspending judgment for as long as possible as new information is being received."
Seth Klarman: "We strive to eliminate biases in our decision making that could cause us to reject new information or cling to erroneous beliefs."
Bruce Berkowitz: "Facts change, we change."
Beware Of Commitment/Confirmation Bias
CIA: "Once an observer thinks he or she knows what is happening, this perception tends to resist change. New data received incrementally can be fit easily into an analyst’s previous image. This perceptual bias is reinforced by organizational pressures favouring consistent interpretation; once the analyst is committed in writing, both the analyst and the organization have a vested interest in maintaining the original assessment."
Warren Buffett: “What the human being is best at doing is interpreting all new information so that prior conclusions remain intact.”
Todd Combs: "I never liked talking to my limited partners about ideas I had, because you become somewhat wedded to it, it's harder to change your mind over time, you become pre-committed to your positions and so forth. That's always been my stance."
Analysts Improve with Experience
CIA: "Substantive knowledge and analytical experience determine the store of memories and schemata the analyst draws upon to generate and evaluate hypotheses. The key is not a simple ability to recall facts, but the ability to recall patterns that relate facts to each other and to broader concepts—and to employ procedures that facilitate this process."
Ken Shubin Stein: "This is an accretive business. The longer you do it, the more you learn, the better you get at it because you see more things. We see more cycles, we see more industries, we learn more business models. We learn how more business models fail. And all of us in business tend to get better as we get older."
Glenn Greenberg: “I have been in the business since 1973, so I have been looking at companies for a long time. There are a lot of things in my head. There are a number of different models of the kinds of business or situations that can work. It may be the local monopoly concept, the low-cost commodity producer concept, the consolidated industry that has come down to a few competitors, a basic essential service that isn’t going to stop growing, or an industry that may be growing too slowly to attract any competition. So, there are a lot of different models.”
Deal with Change
CIA: "The intelligence analyst, however, must cope with a rapidly changing world."
John Burbank: "Markets change radically, every five years that I've seen. Markets aren't nearly as good at discounting the future as people think."
Stanley Druckenmiller: "Probably one of my greatest assets over the last 30 years is that I’m open-minded and I can change my mind very quickly."
Be Creative
CIA - "New ideas result from the association of old elements in new combinations. Previously remote elements of thought suddenly become associated in a new and useful combination. When the linkage is made, the light dawns. This ability to bring previously unrelated information and ideas together in meaningful ways is what marks the open-minded, imaginative, creative analyst."
CIA: "Creativity is required to question things that have long been taken for granted."
CIA: "Creativity, in the sense of new and useful ideas, is at least as important in intelligence analysis as in any other human endeavour."
Leon Levy: "If intelligence is the ability to integrate, creativity is the ability to integrate information from seemingly unconnected sources, and a measure of both abilities is necessary for long-term success in the markets."
Seth Klarman: “We put great emphasis on a consistent investment process that demands enormous creativity, energetic sourcing, outside-the-box thinking, intellectual honesty, and vibrant debate.”
Consider Alternate Hypothesis
CIA: "The simultaneous evaluation of multiple, competing hypotheses permits a more systematic and objective analysis than is possible when an analyst focuses on a single, most-likely explanation or estimate. The simultaneous evaluation of multiple, competing hypotheses entails far greater cognitive strain than examining a single, most-likely hypothesis. Retaining multiple hypotheses in working memory and noting how each item of evidence fits into each hypothesis add up to a formidable cognitive task."
CIA: "Research on hypothesis generation suggests that performance on this task is woefully inadequate. When faced with an analytical problem, people are either unable or simply do not take the time to identify the full range of potential answers. Analytical performance might be significantly enhanced by more deliberate attention to this stage of the analytical process."
CIA: "Exploring alternative hypotheses that have not been seriously considered before often leads an analyst into unexpected and unfamiliar territory."
George Soros: “The financial markets generally are unpredictable. So that one has to have different scenarios... The idea that you can actually predict what's going to happen contradicts my way of looking at the market."
Paul Singer: "What actually happens in markets is never the only scenario that could have taken place. Elliot's portfolio has been designed to maintain stability in a range of different outcomes, some more difficult than what actually occurs at various times in the ebb and flow of markets. Being set up for the broadest scope of market scenarios has been one of the principal reasons for Elliot's high level of stability in almost every period of adversity for the past 38 1/2 years."
Seek to Disprove Hypothesis Not Confirm Them
CIA: "Focus on developing arguments against each hypothesis rather than trying to confirm hypotheses."
Charlie Munger: "Constantly take your own assumptions and try to disprove them."
Be Alert to Surprises
CIA: "A surprise or two, however small, may be the first clue that your understanding of what is happening requires some adjustment, is at best incomplete, or may be quite wrong."
Leon Levy: "Investors also have to be alert to changes in the market that could change their original assumptions."
Warren Buffett: "Charlie and I believe that when you find information that contradicts your existing beliefs, you've got a special obligation to look at it - and quickly."
Seek out Individuals who Disagree
CIA: "Analysts should try to identify alternative models, conceptual frameworks, or interpretations of the data by seeking out individuals who disagree with them rather than those who agree. Most people do not do that very often. It is much more comfortable to talk with people in one’s own office who share the same basic mind-set."
Bill Ackman: “One of the best ways to get confidence in an idea is to find a smart person who has the opposing view and listen to all their arguments. If they have a case that you haven’t considered, then you should get out. But they can also help give you more conviction.”
Think Backwards
CIA: "One technique for exploring new ground is thinking backwards. As an intellectual exercise, start with an assumption that some event you did not expect has actually occurred. Then, put yourself into the future, looking back to explain how this could have happened."
CIA: "Thinking backwards changes the focus from whether something might happen to how it might happen. Putting yourself into the future creates a different perspective that keeps you from getting anchored in the present. Analysts will often find, to their surprise, that they can construct a quite plausible scenario for an event they had previously thought unlikely. Thinking backwards is particularly helpful for events that have a low probability but very serious consequences should they occur."
Leon Levy: "One of the virtues of envisioning the present from a different time is that it underscores the important role of the intangibles, such as mood and psychology, that govern the way we perceive and interpret the supposedly hard numbers on which investors base their decisions. My attempt to imagine the present as it would look from a different time helps me sort the real from the illusions that blind us to what is before our eyes."
Charlie Munger: “Invert, always invert” Jacobi said. He knew that it is in the nature of things that many hard problems are best solved when they are addressed backwards.”
Appoint A Devils Advocate
CIA: "A devil’s advocate is someone who defends a minority point of view. He or she may not necessarily agree with that view, but may choose or be assigned to represent it as strenuously as possible. The goal is to expose conflicting interpretations and show how alternative assumptions and images make the world look different."
Lee Ainslie: "I play devil's advocate and make sure the level of analysis has been complete and thorough and that all the relevant resources have been brought to bear."
Watch out for Unexpected events
CIA: "Analysts should keep a record of unexpected events and think hard about what they might mean, not disregard them or explain them away."
Bill Nygren: "Throughout the time we hold a stock, the analysts will challenge each other as to whether or not our sell target correctly incorporates all the new information we’ve seen subsequent to our purchase."
Leon Levy: "Investors have to be alert to changes in the market that could change their original assumptions."
Keep Questioning
CIA: "A questioning attitude is a prerequisite to a successful search for new ideas. Any analyst who is confident that he or she already knows the answer, and that this answer has not changed recently, is unlikely to produce innovative or imaginative work."
CIA: "If you find yourself thinking you already know the answer, ask yourself what would cause you to change your mind; then look for that information."
Chris Mittleman: “If you allow yourself to start thinking you’ve got it all figured out, that’s probably the beginning of the end.”
Consider the Interactions Between the Variables
CIA: "The number of possible relationships between variables grows geometrically as the number of variables increases."