Most people know Warren Buffett as one of the richest men in the world, who took a relatively small sum of money and turned it into one of the largest companies in the United States, Berkshire Hathaway (BRK.B -0.69%) (BRK.A -0.76%).

When most people try to copy Buffett's style, they tend to focus on how he finds the hidden upside-potential in stocks, but that's not the most important thing. The best thing you can do for your portfolio is to think defensively, that is, to pick and choose companies that will do well when the rest of the market is doing poorly.

Why is it so important?
Stocks tend to go up over time. In fact, they have outperformed essentially every other asset class over any long period of time, so why do we care so much about the bad years?

Simply put, negative years have more of an impact on long-term performance. If your portfolio loses 20% in a year, you would need your stocks to gain 25% the next year just to break even.

To illustrate this point, consider two scenarios.

First, let's consider an aggressive investor whose portfolio returns an average of 20% during the years the market rises, but loses 15% during "down" years. Then, consider a defensive investor who makes 15% during positive years, but only loses 5% during negative ones. For simplicity's sake, let's assume we have two positive years for every one bad year.

As you can see, the defensive portfolio beats our aggressive example by about 30% over a 30-year timeframe.

The proof is in Berkshire's performance
You might be surprised to hear Berkshire has actually underperformed the S&P in four out of the last five years. In fact, the only year Berkshire beat the S&P, 2011, was the only year the market didn't post double-digit gains.

Generally, when the market shoots to the upside, the riskier and more volatile stocks tend to benefit the most. Well, that is exactly the type of investment Berkshire doesn't make. However, consider a year like 2008, when the market was in panic-mode. The S&P lost 37% of its value during that year, but Berkshire Hathaway's book value per share only dropped 9.6%.

In other words, the S&P needed to gain 59% to make up for its 2008 losses, which took the next four years to do. Berkshire, on the other hand, only needed an 11% rise to get back to even, which it achieved and more the very next year.

To further illustrate just how rock-solid Berkshire's portfolio is, here is perhaps the most impressive fact about the company's performance.

The S&P has had 11 down years out of the past 50. Berkshire's portfolio beat the S&P in each and every single one of those years. There has not been one single down year where the company's book value per share fell more than the broader market. That is the kind of performance that builds wealth.

The lesson to learn
The most successful investors know creating wealth isn't about finding the next big thing or taking lots of risks to possibly produce gains. In baseball terms, a hitter with 300 singles is much more valuable to his team than one with just a few home runs, and the same applies to investing.

As Buffett said to shareholders at this year's annual meeting, "In up years, we'll underperform, we'll outperform in down years, and over any cycle, we'll outperform."