When it comes to investing, Berkshire Hathaway (BRK.A -0.28%) (BRK.B -0.68%) CEO Warren Buffett is in a class of his own. Though there is no shortage of Wall Street analysts, pundits, and money managers that make their rounds on the major television networks, it's the Oracle of Omaha that's doubled up the average annualized total return of the broad-based S&P 500 (^GSPC 0.02%) over a 58-year stretch (19.8% vs. 9.9%).

Warren Buffett's ability to run circles around Wall Street is what helped grow the fanfare associated with Berkshire Hathaway's annual meetings from a couple dozen people in 1973 to more than 30,000 on an annual basis. Investors and Berkshire Hathaway shareholders eagerly await his nuggets of wisdom on the U.S. economy and investing to guide their own strategies.

Warren Buffett at Berkshire Hathaway's annual shareholder meeting.

Berkshire Hathaway CEO Warren Buffett. Image source: The Motley Fool.

But while Warren Buffett is an unwavering long-term optimist when it comes to America and the stock market, his actions and his words don't always align in the short term.

Warren Buffett's secret portfolio offers an ominous warning to Wall Street

The vast majority of investors tracking Buffett's investment activity pay close attention to Berkshire Hathaway's Form 13Fs, which are required quarterly filings with the Securities and Exchange Commission for money managers with at least $100 million in assets under management. However, you might be surprised to learn that Berkshire's 13Fs fail to tell the complete story.

In 1998, Buffett's company acquired reinsurance giant General Re in a deal valued at $22 billion. Though the reinsurance operations were the crown jewel of this acquisition, General Re also owned a specialty investment company known as New England Asset Management (NEAM). When Berkshire acquired Gen Re, NEAM came with it.

New England Asset Management is a large enough investment company that it's required to file a quarterly 13F. Although the Oracle of Omaha isn't overseeing the investment activity in NEAM's portfolio, these holdings are, ultimately, part of Berkshire Hathaway. Thus, New England Asset Management is, effectively, Warren Buffett's secret portfolio.

As of the end of 2022, Buffett's hidden portfolio contained approximately $5.4 billion in invested assets. But as of March 31, 2023, NEAM held just $671.6 million worth of invested assets. In other words, Warren Buffett's secret portfolio dumped 88% of its invested assets during the first quarter, which is a very clear warning to Wall Street.

Collectively, Apple (AAPL 1.27%), Chevron (CVX 0.44%), Bank of America (BAC -0.13%), and HP (HPQ 1.55%) accounted for 86% of New England Asset Management's investment portfolio on Dec. 31, 2022. During the first quarter, NEAM respectively sold 99% of its Apple stake, 98% of its Chevron and Bank of America positions, and completely exited its nearly 16.5-million-share HP stake. Collectively, Apple, Chevron, and BofA now comprise just 2.7% of NEAM's invested assets. 

Buffett's 29 billion reasons to be cautious

Some of you might be thinking this substantial reduction of invested assets by Buffett's secret portfolio is much ado about nothing since Warren Buffett isn't actively overseeing it. Though you may be correct, the Oracle of Omaha himself is also sending mixed signals.

In addition to Berkshire Hathaway's 13F filings, investors can get a clearer feel for what the Oracle of Omaha and his investing lieutenants (Ted Weschler and Todd Combs) have been up to by examining the company's equity cash flows in Berkshire's quarterly operating results.

For example, during the fourth quarter, Buffett and his team purchased $1.68 billion in equities and completed $16.32 billion in equity sales. That works out $14.64 billion in net-equity sales between the start of October and end of December.

The story was similar during the March-ended quarter. We saw the Oracle of Omaha and his lieutenants do a bit more purchasing ($2.87 billion), but they, ultimately, sold $13.28 billion in equity securities. As a whole, this equates to $10.41 billion in net-equity sales during the first three months of 2023. And I'm still not done.

Roughly one month ago, when Berkshire Hathaway held its annual shareholder meeting, the Warren Buffett divulged that his company had "sold net some stock [in April], which produced maybe $4 billion." Collectively, Berkshire Hathaway has sold a net of $29 billion worth of stock in a seven-month period. That's not particularly encouraging.

S&P 500 Shiller CAPE Ratio Chart

S&P 500 Shiller CAPE Ratio data by YCharts.

If I had to pinpoint the single biggest factor behind this selling, I'd suggest it's stock valuations.

Warren Buffett and his team are uncompromising when it comes to buying great companies at a fair price -- emphasis on the word "fair." But according to the Shiller price-to-earnings (P/E) ratio (also known as the cyclically adjusted price-to-earnings ratio, or CAPE ratio) for the S&P 500, stocks aren't cheap. The Shiller P/E ratio closed out May at 29.3, which is well above its average of 17, when back-tested to 1870. 

More importantly, bad things have historically happened when the S&P Shiller P/E surpasses 30. In the five previous instances where this occurred, the Dow Jones Industrial Average or S&P 500 eventually went on to lose at least 20% of their value. For context, the Shiller P/E topped 30 for a sixth time in February 2023.

A dollar sign rising up from a financial newspaper that's displaying visible stock quotes and charts.

Image source: Getty Images.

Don't bet against America or stocks, but be mindful of history

The selling in both Warren Buffett's secret portfolio and the nearly $338 billion investment portfolio the Oracle of Omaha and his lieutenants oversee look to be clear indicators that most stock valuations aren't appealing. While that's a pretty apparent warning to Wall Street, it's important to be mindful of history.

Over the past 73 years, the S&P 500 has undergone 39 double-digit corrections. Excluding the current bear market, all 38 previous declines in the S&P 500 were eventually fully recouped by a bull market rally. Though corrections are perfectly natural, if not commonplace, so is the idea of investors being rewarded for their patience.

Even though he'll never come out and say it, Warren Buffett isn't big on market timing. He's a much bigger fan of time in the market than trying to time when to invest in the market. Considering that the S&P 500 has averaged a 9.9% annualized total return, including dividends, since the mid-1960s, it's hard to argue with this logic. Despite bear markets, crashes, and panics, S&P 500 index investors have been doubling their money, on average, every seven years and change for nearly six decades, with reinvestment.

Furthermore, a quick look at U.S. gross domestic product shows a very well-defined trend that moves up and to the right. Even though recessions are a normal part of the economic cycle, the Oracle of Omaha is well-aware that periods of expansion last considerably longer than downturns. By acquiring and investing in predominantly cyclical businesses, Buffett and his team have set Berkshire Hathaway up to thrive during these long bull markets.

Although short-lived tough periods are inevitable for Wall Street, being optimistic is a winning strategy for long-term minded investors.