Most investors would consider Berkshire Hathaway (BRK.A -0.18%) (BRK.B 0.03%) a no-brainer stock to own.

After all, the conglomerate has trounced the market over its history, returning a compound annual gain of 20.3% from 1965 to 2019, more than double the annual return of the S&P 500, at 10% with dividends included.

Based on that math, a dollar invested in Berkshire in 1965 would be worth 140 times more today than a dollar put into the broad market index. In fact, investing just $100 in Berkshire Hathaway in its first year would have made you a millionaire today, as that would now be worth roughly $2.7 million.

In addition to that track record, Berkshire Hathaway is run by Warren Buffett, generally regarded as the greatest investor in history. Buffett, who has run Berkshire as both a holding company and a conglomerate with wholly owned subsidiaries, deserves most of the credit for his company's success, as his value investing approach has delivered stratospheric returns to early investors. 

However, I don't think Berkshire is a smart buy today, and personally, I don't plan to ever own it. There are a number of reasons why. 

Berkshire Hathaway CEO Warren Buffett

Image source: The Motley Fool.

About that track record

While it's true that Berkshire Hathaway has crushed the S&P 500 over its history, in the recent past, Buffett hasn't kept up. Berkshire, which has historically avoided high-growth stock and tech stocks, performed poorly over the last year, missing out on the broad recovery that most investors enjoyed. Over the last year, Berkshire shares are up just 3%, compared to a total return of 18% for the S&P 500.

Worse, Berkshire has underperformed the broad market over long periods. The chart below shows how the broad market has beaten Berkshire by a wide margin over the last decade.

BRK.B Chart

BRK.B data by YCharts. Note: Two sets of figures correspond to Class A and Class B shares of Berkshire.

You would have to go back 20 years to find a time frame going up to today in which Berkshire has beaten the S&P 500. While the stock was once a juggernaut, the results over the last two decades speak for themselves. It's been no better than a broad-market index fund. 

There are a number of reasons for that. It's become harder for Buffett to find market-beating investments as Berkshire has gotten bigger, and the emergence of the tech sector as the engine for the stock market's growth has caught the company flat-footed as Buffett has historically avoided tech stocks. Additionally, a number of Berkshire's biggest moves over the last decade have flopped, such as a big investment in IBM, the merger of Kraft Heinz, and a busted deal with Occidental Petroleum. As a result, Berkshire is no longer the market-thumping stock it once was.

Buffett won't be around forever

Warren Buffett is 90 years old. Berkshire Hathaway's biggest competitive advantage is him, and his partner Charlie Munger, who is 97. They built the company and have guided it through its lifetime. Every year, tens of thousands of shareholders flock to Omaha for Berkshire's annual shareholders' meeting and to see Buffett speak.

But Buffett is clearly in the sunset of his career. The Berkshire chief has not yet made clear his succession plan, but has given greater responsibilities to deputies who have led investment in tech stocks like Apple and the recent IPO Snowflake. Buffett himself expressed his confidence in the company's prospects even when he is no longer in charge, saying in his most recent shareholder letter that Berkshire owned a diverse array of businesses with high returns on invested capital, and crediting his highly skilled and loyal management team. 

However, Buffett himself plans to sell his stake in the company over the 12 to 15 years after his passing, creating a weight on the stock as he owns 15% of shares outstanding. More importantly, there's a premium baked into Berkshire because of Buffett's reputation. Once he is no longer running the company, the stock is likely to take a hit, and the investment thesis for many investors will be more difficult without him at the helm.

What's its economic moat?

Buffett's focus as an investor has been finding profitable businesses with economic moats that are difficult to disrupt. He prefers boring cash-generating companies to riskier high-growth companies. But as technology improves, the pace of innovation and disruption is speeding up, and some of its key subsidiaries are starting to look vulnerable. For instance, Buffett himself even noted that the innovation of self-driving cars would present problems for auto insurance subsidiary GEICO, a business built on car accidents and human error. While it's true that many of the companies Berkshire owns outright are in industries that seem timeless, like utilities, insurance, or railroads, I can't see what the company's competitive advantage is aside from Buffett.

The conglomerate structure is often unwieldy, and many of the industrial powerhouses of the past, like General Electric, United Technologies, and Honeywell, have broken up or spun off entities in recent years. The collection of businesses Buffett has accumulated may be enduring, but the business is barely growing faster than inflation. In 2019, revenue increased just 2.7%, and in 2018 growth was slightly faster at 3.3%. Through the first three quarters of 2020, its revenue fell 4.3% and profits were slashed by about 50%, showing that many of its business are vulnerable to a disruptive event like the pandemic. On another note, it's rare for a business growing as slowly as Berkshire to not pay a dividend, meaning investors don't get growth or income.

Investors may argue that Berkshire's chosen subsidiaries or its capital allocation methods give it an economic moat, but that doesn't seem to count for much when there's almost no growth. And if investors are looking for purely capital allocation, it's hard to beat the record at ARK Invest in recent years. 

Ultimately, Berkshire, despite its reputation, is a stock that's only matched the S&P 500 for the last 20 years, and it will eventually lose Buffett's stewardship. There are better places to look in the market for the stocks that promise more growth, have disruptive potential, and have more focused missions.