The S&P 500 (^GSPC -0.38%) is again on its way to a strong annual return. The artificial intelligence (AI) boom has helped the index climb a wall of worry that would likely have derailed many past bull markets. After all, the U.S. remains in conflict with Iran, consumers are struggling with high costs, and long-duration interest rates have been edging higher.

AI has sparked one of the largest corporate spending booms in history, with major tech companies investing an unfathomable amount of money in AI infrastructure and seeing strong returns. At the same time, AI investments are helping companies across industries reduce costs and improve efficiency, boosting corporate earnings.

However, while the market remains in bull mode, a valuation red flag from the dot-com era has just resurfaced.

Image source: Getty Images

For three consecutive months, the S&P 500's cyclically adjusted price-to-earnings (CAPE) ratio has remained above 40. This metric was devised by famed economist Robert Shiller in 1988 to help smooth out earnings cyclicality and is based on a 10-year average of inflation-adjusted earnings.

To put that into context, the CAPE ratio has only hovered above 40 one other time in history, and that was right before the tech bubble burst in 2000. That 40 score is about 50% above its 20-year historical average and well above its historical baseline of around 17.

Meanwhile, history shows that the S&P 500 has never had a positive three-year return after the CAPE index finished a month above 40.

Why history may not repeat itself

Despite history indicating that the market could be in for a rough ride over the next three years, there is no guarantee this will happen.

First, the sample size of the CAPE hitting 40, is very small. Second, the S&P 500 is much different today than it was 10 years ago, but the CAPE ratio gives the 2016 S&P 500 index as much weight as it does today's S&P 500.

A decade ago, the benchmark index had far more exposure to lower-margin, asset-heavy industrial and energy companies. Today, the index is heavily weighted toward high-margin tech companies that generate tremendous operating cash flow and have fortress balance sheets.

At the same time, AI looks like a huge technological game changer that is quickly moving the needle, while the technology innovation curve is also accelerating. While the internet was a massive breakthrough, it took telecom and web infrastructure companies over a decade to generate a return on their investment. Amazon recently said it expects to break even on its AI infrastructure investments within two to three years, while Space Exploration Technologies (SpaceX) has said it will achieve payback within a year. At the same time, AI is helping corporate America save costs and become more efficient, driving profit growth.

Most large tech companies that currently dominate the S&P 500's top holdings aren't expensive based on future earnings expectations. Meanwhile, if hyperscalers (owners of large data centers) can continue to show strong returns on their AI infrastructure investments and enterprise customers continue to use AI to help drive operational efficiencies, corporate earnings growth should continue to outpace earlier years. The CAPE ratio does not capture what could be a major fundamental shift in the market driven by AI.

How investors should play it

A CAPE ratio of 40 isn't an automatic sign that you should dump your stock holdings and start hiding cash under your mattress. However, it is a warning signal, and if AI infrastructure spending slows dramatically in the coming years, you can be sure the S&P 500 will take a hit, given its current tech-heavy makeup.

That said, there is no need to panic. I'd personally follow Warren Buffett's current strategy and keep some cash on the sidelines in case the market does see a major pullback. Otherwise, I'd keep dollar-cost averaging into top exchange-traded funds (ETFs), such as the Vanguard S&P 500 ETF (VOO -0.38%), because generally, trying to time a market crash is a foolhardy endeavor.