Napoleon's definition of a military genius was, "The man who can do the average thing when all those around him are going crazy." It's the same in investing.

Building wealth over a lifetime doesn't require a lifetime of superior skill. It requires pretty mediocre skills -- basic arithmetic and a grasp of investing fundamentals -- practiced consistently throughout your entire lifetime, especially during times of mania and panic. 

Most financial advice is about today -- what should I do now, what stocks look good now. But the vast majority of the time, I've come to realize, today isn't that important. It's just buying and waiting. Most of what matters as a long-term investor is how you behave during the 1% of the time everyone else is losing their cool.

To demonstrate my meaning, I used Yale economist Robert Shiller's market data going back to 1900 and created three hypothetical investors. Each has saved $1 a month, every month, since 1900.

The first is Betty. She doesn't know anything about investing, so she dollar-cost averages, investing $1 in the S&P 500 every month, rain or shine.

Sue, a CNBC addict, invests $1 a month into the S&P, but tries to protect her wealth by saving cash when the economy is in recession, deploying her built-up hoard back into the market only after the economy officially exits a recession.

Bill, a mutual fund manager whose only incentive is to look right in the short run, invests $1 a month, but stops investing in stocks six months after a recession begins, and only puts his money back into the market six months after a recession ends.

After 113 years of investing, who's won? Boring Betty takes it by a mile:

What fascinates me about these numbers is that of the 1,353 months since 1900, only 321 of them have been altered for Sue and Bill, because that's the number of months the economy has been in recession. If you dig further into the numbers, you'll see that a much smaller percentage of months make up the bulk of the difference in returns. Ten percent of months explains two-thirds of the difference between dollar-cost averaging and sitting out during recessions. Five percent explains more than half. According to analyst Eddy Elfenbein, over the last 20 years, the 24 best days accounted for the entire gain in the S&P 500. "The other 99.5% of the time has been a net loss."

Every one of the market's top days took place during periods of sheer terror. Of the 20 best market days of all time, 17 were during the Great Depression, one was a few days after the crash of 1987, and two were during the depths of the 2008 financial crisis. Missing these days devastated your long-term returns. And most investors who missed them were those who sold out, or stopped buying, after stocks crashed and everyone around them were busy panicking. Those who try to avoid losses consistently end up missing even larger gains. That's why Boring Betty, who is average all of the time, ends up above average overall.

A pilot once described his job as "hours and hours of boredom punctuated by moments of sheer terror." I think of investing the same way. Your success as an investor will be determined by how you respond to punctuated moments of terror, not the years and years spent on cruise control.

Check back every Tuesday and Friday for Morgan Housel's columns on finance and economics.