There are a lot of smart people on Wall Street. However, few manage to draw investors' attention quite like the Oracle of Omaha, Warren Buffett. Since becoming the CEO of Berkshire Hathaway (BRK.A -0.76%) (BRK.B -0.69%) in 1965, his company's Class A shares (BRK.A) have increased by an aggregate of more than 4,300,000%, which works out to nearly a 20% annualized return spanning almost six decades.

Given how consistently Warren Buffett and his investing lieutenants, Ted Weschler and Todd Combs, have outperformed the benchmark S&P 500 (^GSPC 1.02%), investors often follow in his footsteps to generate sizable long-term gains.

Warren Buffett at Berkshire Hathaway's annual shareholder meeting.

Berkshire Hathaway CEO Warren Buffett. Image source: The Motley Fool.

Warren Buffett's has a "secret" $690 million investment portfolio

Riding Buffett's coattails can be done by tracking Berkshire Hathaway's quarterly Form 13F filings with the Securities and Exchange Commission. A 13F is a required filing for money managers with at least $100 million in assets under management that provides everyday investors with a snapshot of the stocks and exchange-traded funds (ETFs) Wall Street's smartest minds have been buying and selling.

For example, Berkshire Hathaway's most recent 13F, which was released a little more than a week ago, showed that Buffett and his investing lieutenants purchased five stocks during the June-ended quarter. Based on quarter-ending market values, Berkshire's brightest minds put more than $800 million to work in a trio of homebuilders -- D.R. Horton, NVR, and Lennar (Berkshire purchased the B shares, LEN.B). Additionally, Buffett and his team continued to add to Berkshire's stakes in Occidental Petroleum and Capital One Financial.

While 13Fs can be a smart way to track what Warren Buffett and his team are buying and selling, they surprisingly fail to tell the full story.

A quarter of a century ago, Berkshire Hathaway completed its acquisition of General Re. Though the crown jewel of this deal was General Re's reinsurance operations, it also owned a specialty investment company known as New England Asset Management (NEAM). When Berkshire closed the deal to buy General Re, it also became the owner of NEAM.

Although NEAM's invested assets pale in comparison to that of parent Berkshire Hathaway -- $350 billion in invested assets for Berkshire, compared to $690 million for New England Asset Management -- what NEAM owns is, nevertheless, part of Berkshire Hathaway. Even though Buffett doesn't oversee NEAM's investment portfolio in the same manner that he does with Berkshire Hathaway's investments, New England Asset Management is, effectively, Warren Buffett's secret portfolio.

A businessperson holding a stopwatch behind ascending stacks of gold coins.

Image source: Getty Images.

This top holding in Buffett's secret portfolio is a smart buy for everyday investors

As of June 30, 2023, Buffett's secret portfolio held 101 positions in a variety of securities, including common stock, preferred stock, and ETFs. However, the No. 1 holding in this "hidden" portfolio is an investment the Oracle of Omaha has previously urged most people make.

Coming into 2023, Buffett's secret portfolio looked somewhat similar to Berkshire Hathaway's core investment portfolio. In no particular order, Apple, Bank of America, Chevron, and HP, made up the lion's share of invested assets. But during the first quarter, NEAM's investment team jettisoned these positions. HP is no longer a holding, while BofA, Chevron, and Apple have been reduced by 98% or 99%.

Taking the place of these stalwarts as the largest holding in Warren Buffett's secret portfolio is the SPDR S&P 500 ETF Trust (SPY 0.95%), which attempts to mirror the performance of the broad-based S&P 500. This S&P 500 tracking index accounts for almost 16.4% of NEAM's invested assets, as of June 30.

During Berkshire Hathaway's 2020 annual shareholder meeting, which took place during the early stages of the COVID-19 pandemic, Buffett urged investors not to be against America, and to own cross-sections of the American economy over long periods. The easiest way to do that is to purchase an all-encompassing ETF that mirrors the performance of the S&P 500. In Buffett's words:

In my view, for most people, the best thing to do is own the S&P 500 Index Fund... You're dealing with something fundamentally advantageous, in my view, in owning stocks. I will bet on America the rest of my life.

Depending on your investment timeline, the track record of the S&P 500 is unbeatable.

Each year, market analytics company Crestmont Research updates an extensive dataset that's calculated the rolling 20-year returns of the S&P 500 all the way to 1900.  Though the S&P didn't come into existence until 1923, and hasn't held its trademark 500 companies prior to 1957, Crestmont was able to accurately trace its performance back to 1900 given that its components could be found in other major indexes at the time.

What Crestmont's dataset shows is that if an investor were to have purchased an S&P 500 tracking index and held that position for 20 years, they would have generated a positive total return, including dividends paid, 104 out of 104 times. Every single rolling 20-year end period from 1919 through 2022 produced a positive annualized total return. In other words, timing the market has mattered far, far less than time in the market.

Keep in mind that the SPDR S&P 500 ETF Trust isn't the only S&P 500 tracking index to choose from. Call me partial, but the Vanguard S&P 500 ETF (VOO 1.00%) is, arguably, a better choice. Though both ETFs aim to mirror the performance of the S&P 500, the 0.03% net expense ratio of the Vanguard S&P 500 ETF is lower than the 0.0945% net expense ratio for the SPDR S&P 500 ETF Trust.  If you're investing big bucks over the long run, this six-basis-point difference in net expense ratio can add up.

An S&P 500 tracking index may not be the edgiest of investments, but it gets the job done for patient investors.