This article was updated on April 30, 2018, and originally published on Feb. 28, 2017.

On one of the first pages of every Berkshire Hathaway (BRK.A 0.36%) (BRK.B 0.21%) annual report is a table that compares the company's performance to the change in the S&P 500 for each year since 1965, the year Warren Buffett took control of Berkshire.

It's on this page where one can observe firsthand the power of compounding returns. Over the past 50 years, the compound annual gain in Berkshire's book value per share was 19.1%. The gain on its stock came in slightly above this, with a compound annual return of 20.9%. The latter translates into a total gain of over two million percent.

Based on these figures, a $10,000 investment in Berkshire Hathaway stock in 1965 would be worth $240 million as of the end of 2017. After adjusting for inflation, this means that Warren Buffett took enough money to buy a mid-level BMW and turned it into generational wealth for early shareholders. 

Warren Buffett

Warren Buffett, the chairman and CEO of Berkshire Hathaway. Image source: The Motley Fool.

Another thing you can do with the data from this page of Berkshire's annual report is to track the trend in the company's performance. 

There are a number of ways you can do this, but what I've found most insightful is to chart the trend in Berkshire's annual stock performance compared to the S&P 500. As an example, this is Berkshire's alpha, for its excess return over the broader market through 2016:

A line graph of Berkshire Hathaway's performance compared to the S&P 500.

Data source: Berkshire Hathaway's 2016 annual report. Chart by author.

This shows that Berkshire's best days were in the late 1970s, a tumultuous decade thanks to wildly fluctuating energy prices and rapidly accelerating inflation. Since then, Berkshire's performance relative to the S&P 500 has steadily narrowed. The rolling five-year average of Berkshire's alpha dipped into negative territory in three out of the past five years.

There are a lot of things one can point to for causing this drop. From 1999 through 2001, for instance, Berkshire was absorbing billions of dollars' worth of unexpected losses that it inherited in its 1998 acquisition of General Re, one of the world's leading reinsurers. These losses weighed on the rate of change in Berkshire's book value per share for years, which would be reflected in its stock price.

Additionally, Berkshire issued 272,000 shares to make the acquisition, increasing Berkshire's outstanding share count by 21.8%. It was this decision to dilute then-existing shareholders that led Buffett to quip in his annual shareholder letter: "Today, I would rather prep for a colonoscopy than issue Berkshire shares."

Then, of course, came the financial crisis, which caused Berkshire's book value per share to drop for only the second time since Buffett took over in 1965. The only other time was in 2001, when Berkshire's General Re was flooded with reinsurance claims triggered by the terrorist attacks in September of that year.

Also working against Berkshire is its growing size, a point Buffett has pointed out in the past. It has the fifth largest market capitalization on the S&P 500, behind only Apple, AlphabetMicrosoft, and Facebook. It also owns a vast assortment of subsidiaries, from home builders to car insurers to restaurants.

The net result is that Berkshire's annual returns theoretically should, and increasingly are, mirroring that of the economy and thus the broader market. Buffett has said this would happen. He's obviously right.