Since 1964, Berkshire Hathaway (BRK.A -0.10%) (BRK.B -0.09%) has produced 3,787,464% total returns for investors. That's not a typo – and it means that a $10,000 investment in Berkshire Hathaway at that time would be worth about $379 million today. Some of Berkshire's early investors have become very rich.

There are plenty of reasons why Warren Buffett has produced such incredible returns for investors since taking control of Berkshire Hathaway in the mid-1960s.

For example, Buffett figured out how to use the excess capital held by insurance companies to generate outsized returns. He has also made several successful opportunistic investments along the way that have worked out very well, such as the financial crisis-era investment in Bank of America (BAC -0.05%) that has increased roughly fivefold in value in just over a decade.

However, there is one big factor in Berkshire's outperformance that is often overlooked by investors – the importance of outperforming the market during the bad times.

Berkshire's track record in down markets

One of the most notable Warren Buffett quotes is "Rule number one: Never lose money." And while this is obviously not an achievable rule (Berkshire's investment record certainly isn't a flawless one), it does guide Buffett to think about preserving capital in tough times more than the average investor. One big part of Buffett's investment process is to find businesses set up to make money no matter what, and it has played a big role in how Berkshire performs when the market gets ugly.

Consider this track record. Including 2022, the S&P 500 has produced negative total returns in 13 separate years. Berkshire has outperformed the benchmark index in 11 of them. In seven of the market's down years, Berkshire has produced positive total returns. And this was true in 2022 – the S&P 500 had a negative 18.1% total return, while Berkshire gained 4%.

Outperforming in the bad times is more important than you may think

There's a solid argument to be made that outperforming in the bad years is more important than outperformance during the good times. To be sure, Berkshire's stellar long-term performance is due to a combination of both types of outperformance to some degree, but not as much as you might think. During the two-year stretch from 2019 through 2020, for example, the S&P 500 produced a fantastic total return of 56%, while Berkshire managed just 13% in that time. The same can be said in several other intervals when the overall stock market performed exceptionally well.

But outperforming in the tough times puts the long-term mathematics in Berkshire's favor. Think of it this way – if the S&P 500 loses one third of its value (33%), it would need a 50% rise just to get back to even. On the other hand, if you can avoid such a steep loss in the first place, you can outperform the market with much less of a gain when things rebound.

Thinking defensively can be one of your best weapons

It's a common misconception that investing in ways that protect your portfolio during volatile times is an inherently "boring" way to build a portfolio and can't outperform the market over the long run. But Warren Buffett and Berkshire Hathaway's track record show exactly why this isn't true. By focusing on businesses that can thrive no matter what the stock market or economy is doing, it can help set you up for long-term wealth creation, while still allowing you to sleep well at night.