It seems to be a rite of passage for any investors worth their salt around here to buy into Berkshire Hathaway (BRK.A 0.64%) (BRK.B 0.54%). However, here I am, 25 years into my journey as a member of the Motley Fool family, and I have never heeded the call of the Omaha wild. 

I have never owned Berkshire Hathaway, and that's probably not going to change anytime soon. I have nothing but love and respect for the empire that Warren Buffett and Charlie Munger have built, but I can't see myself as a shareholder. It's just hard for me to get excited about the future of Berkshire Hathaway, and that's the only thing that matters when it comes to putting new investing dollars to work.

BERKSHIRE HATHAWAY CEO WARREN BUFFETT AT HIS COMPANY'S ANNUAL SHAREHOLDER MEETING.

Warren Buffett. Image source: The Motley Fool.

He went Hathaway

One of the biggest risks of owning Berkshire Hathaway in recent years is that it was run by a pair of octogenarians. In three months, that will no longer be true: Buffett turns 90 this summer; Munger is already 96. 

I don't have a problem with nonagenarians. I trust the minds of Berkshire's two iconic gatekeepers to be as sharp and crisp as (and, let's face it, superior to) that of any hot-shot stock picker half their age. But mortality is always a concern when considering an investment in Berkshire Hathaway. These two brilliant dudes aren't getting any younger, and there's a good chance that your planned investment in Berkshire Hathaway is going to outlive one of them, if not both. 

It's naive to think that Berkshire Hathaway stock isn't going to take a hit when one of them is gone. No matter how much faith you may have in the team of top lieutenants whom they have assembled, it's hard to fathom the market giving them the benefit of the doubt. There's a premium priced into Berkshire Hathaway stock for Buffett and Munger that will take time for their replacements to earn.

When Buffett wrote in this year's annual report for investors to look at the forest and forget the trees, he was referring to the importance of not assessing Berkshire Hathaway in terms of its individual investments. But when it comes to leadership, there is no doubt that Buffett and Munger are the two trees. That is the forest. 

And I realize it may sound like heresy to point this out, but even Buffett and Munger haven't been at their best lately. Berkshire Hathaway's compounded return since 1965 may have nearly doubled the market, but it's a different story these days, with the stock trailing the S&P 500 over the one-, three-, and five-year periods.

Berkshire Hathaway's stock rose just 11% last year, as the S&P 500 with dividends appreciated by 31.5%. This year is playing out even worse for Berkshire investors, as the stock's 25% slide through Friday's close is more than double the damage being dealt to the general market. If you invested $100 in Berkshire Hathaway at the end of 2014, you would have $112 today. That same $100 bill in the S&P 500 would be worth $154.

Berkshire Hathaway has had twice as many down years over the past decade as the S&P 500, and it's hard to get excited about where the company is right now. If you see the conglomerate's business concentrations in insurance, energy, and railroads, it's easy to see why investors are steering clear of Berkshire Hathaway in the new normal. The stock portfolio itself is also a collection of meandering investments.

Apple (AAPL -0.57%) joins financial services giants and a stalling beverage giant as the five largest holdings accounting for a whopping 69% of Berkshire Hathaway's once celebrated stock portfolio. Buffett would go on to dump his stakes in four fading airlines last month. Apple has appreciated to the point of being Berkshire Hathaway's largest holding, and it's the lone bright spot among the top holdings. Aren't you better off just buying Apple instead of Berkshire Hathaway with all of the unnecessary baggage?

Berkshire had more than $137 billion in cash and short-term investments at the end of March, and given the portfolio's track record lately, it's probably a good thing that the money wasn't put to work. I'm not sure if there's much of a bullish argument for Berkshire Hathaway that doesn't lean on its legacy. This doesn't seem to be one of the market's safe stocks anymore, and we can't party like it's 1965.