Congress has an 11% approval rating but an 85% reelection rate. How does that happen?

It's actually simple. Most people cannot stand Congress as a whole but like and approve of their local Congressman.

In lots of areas of life, people's basic psychology is to be optimistic about their own decisions but pessimistic about other people's choices. This isn't surprising. We need optimism about our own choices to feel good about our future prospects, but it has to be offset with pessimism over other people's choices to protect us from conflicting priorities of other groups.

Harvard psychologist Max Bazerman has another example of this in practice. When analyzing other people's home renovation plans, most people estimate the project will run between 25% and 50% over budget. But when it comes to their own projects, people estimate renovations be completed nearly on time and at budget.

Daniel Kahneman calls this the "inside view vs. outside view." He tells a story about writing a psychology textbook with a group of colleagues.

The psychologists were confident they could finish the textbook in two years, and didn't think about the possibility of failing. When they reviewed the performance of other academics, they realized it typically takes at least seven years to complete a textbook, and 40% of those who begin the process eventually quit.

Kahneman and his colleagues quietly ignored these statistics, since believing them would deter them from them continuing. No one was willing to pursue the project under those odds.

They completed the book in eight years later. It was never used.

Kahneman says this was not only embarrassing but "one of the most instructive experiences of my professional life." It taught him a couple lessons.

1. There are two types of forecasting. One is the inside view, which forecasts specifically on the case at hand, like your own renovation or your own textbook project. The other is the outside view, which forecasts based on relevant examples of other people undertaking a similar endeavor.

2. There's a common planning fallacy. People cling to the inside view, which makes them systematically overconfident about their own predictions and decisions. The outside view should, according to Kahneman, at least become a benchmark forecast for what you're pursuing. It's more likely to be accurate, but we often fight it and ignore it because we're more optimistic about ourselves than others.

It's hard to understate how relevant this framework is to investing.

The average investor isn't very good, but the average investor thinks they, themselves, are sensational.

That's hardly an overstatement. A decade ago, two economists asked a group of investors how they performed as investors. They then checked those self-assessed returns with the investors' brokerage statements to reconcile the difference -- which, not surpassing, was astronomical. The average investor overestimated his returns by 11 percentage points per year. For perspective, Warren Buffett became the best investor of all time by outperforming the market by the same amount, 11 percentage points per year. Less surprising but just as important is that the average investor considered himself above average.

The root of so many investing problems is that people think errors, bad behavior, mistakes, and overconfidence applies to other people, but not themselves. Almost all of us think this way. Kahneman, who has spent much of his career showing how bad people are at predicting outcomes, once wrote: "Despite 45 years of work in the field, I am still inclined to make over-confident predictions."

Kahneman came to our office a few years ago. I asked how people can make better financial decisions in a world where we're so tempted to go astray. He mentioned three things: 

  • Talk to many people.
  • Talk to people you disagree with. 
  • Talk to people who are in different emotional states than you are.

Talking to different people is the only way to see how other people who are navigating the same waters you are have fared. Investing tends to be a solo activity – just you, some data, and a brokerage account – which makes it easy to fall prey to the inside view. Knowing how others have done is a good benchmark for knowing how you're likely to do. That's not to say no one can outperform the masses. But until you know where other people screw up, you'll never know where your own risks likely reside. 

Talking to people who disagree with you is one of the hardest but most important skills an investor can have. Confirmation bias is one of the great scourges of investing. A prerequisite to having a strong opinion about anything in markets is the ability to state the opposing side's view as well as they can. Most people can't because they're tied to their own view of how the world works, which is always a sliver of how the world actually works.

Finally, managing money is always emotional. Successful investing isn't about numbers and balance sheets; it's about dopamine and serotonin. One of the best ways to counter how much are brains work against us is to run our financial decisions by someone who's not in the same emotional state we are. That means not your spouse, your kids, or your co-workers if you own your employer's stock. It can be a financial advisor, but I've learned a ton from financially knowledgeable friends who can say, "Have you ever thought of this ...?" when I tell them how handle my money. They're able to view the situation more objectively than I ever could.  

For more:

Performance vs. outcomes

Why does pessimism sound so smart?

Things I'm pretty sure about

Hard truths for investors to wrap their heads around

How the investing industry could change  

Contact Morgan Housel at Motley Fool has a disclosure policy.