Over the long term, you'd struggle to find a better wealth creator than Wall Street. The annualized returns of housing, gold, oil, and bonds simply don't stack up to what Wall Street can bring to the table for investors over long periods.

But things get a lot dicier when attempting to predict directional moves in the broader market over shorter periods. Since this decade began, the widely followed Dow Jones Industrial Average (^DJI 0.40%), broad-based S&P 500 (^GSPC 1.02%), and innovation-driven Nasdaq Composite (^IXIC 2.02%), have fluctuated between bull and bear markets.

Although all three major stock indexes were seemingly unstoppable during the first half of 2023, investor sentiment has shifted in a big way over the past three months. The S&P 500 and Nasdaq Composite are 10.3% and 11.9% below their year-to-date closing highs, as of Oct. 27, 2023, with the Dow down 9%. This means the broadest barometers of Wall Street's health are undergoing a correction.

A bear figurine set atop newspaper clippings of a plunging stock chart and a declining quarterly bar chart.

Image source: Getty Images.

The all-important question is: Will stocks bounce from correction levels, or will the S&P 500 and Nasdaq Composite head even lower? According to one closely watched valuation indicator, the answer is pretty clear.

This valuation tool has a knack for predicting the future

Before digging in, I want to make it clear that there's no such thing as an economic datapoint or predictive tool that can, with 100% accuracy, forecast short-term directional moves in the major indexes. There are, however, indicators with phenomenal track records and uncanny correlations to directional moves in the market. The S&P 500 Shiller price-to-earnings (P/E) ratio just happens to be one of them.

The Shiller P/E ratio, which is also commonly referred to as the cyclically adjusted price-to-earnings ratio, or CAPE ratio, is based on average inflation-adjusted earnings from the previous 10 years. Looking at 10 years' worth of corporate earnings softens the effect of potentially wild fluctuations in profits over the course of a single year (such as during the initial stages of the COVID-19 pandemic).

Although the S&P 500 didn't expand to 500 components until 1957, and the Shiller P/E ratio wasn't widely introduced until the late 1980s, it's a valuation tool that can, nevertheless, be back-tested all the way to 1870. 

S&P 500 Shiller CAPE Ratio Chart

S&P 500 Shiller CAPE Ratio data by YCharts.

As you can see from the chart above, the Shiller P/E has been elevated for much of the past 30 years. This is likely a function of the internet democratizing access to news and trading platforms for everyday investors, which has increased their willingness to take risks and accept higher valuation premiums. Additionally, historically low lending rates throughout the 2010s led to corporate earnings growth and higher stock valuations.

At a closing value of 28.14 on Oct. 27, the S&P 500's Shiller P/E is well above its average of 17.06 over the past 153 years. However, there's a far bigger historic correlation that investors should note.

Since 1870, there have only been six instances where the S&P 500's Shiller P/E has surpassed 30. The five previous instances (1929, 1997-2001, Q1 2018-Q3 2018, Q4 2019-Q1 2020, and Q3 2020-Q2 2022) were all, eventually, followed by at least a 20% decline in the benchmark S&P 500. The Shiller P/E ratio has been above 30 throughout most of 2023, representing the sixth such instance of extended valuations.

Keep in mind that the Shiller P/E isn't a timing tool. If stocks are pricey, they can remain so for weeks, months, or even years, as they did from 1997 through most of 2001. But what the Shiller P/E ratio does demonstrate is that stocks can't maintain extended valuations in perpetuity. Even though the Shiller P/E is now marginally below 30, history would suggest the Dow, S&P 500, and Nasdaq Composite have further to fall.

A smiling person looking out a window while holding a financial newspaper.

Image source: Getty Images.

Time in the market beats timing the market

In addition to the Shiller P/E ratio, multiple money-based indicators signal that the U.S. economy is set to weaken in the coming quarters. Recessions have historically been bad news for the stock market -- at least over short periods.

Despite these near-term concerns, history conclusively shows that time in the market always trumps trying to time the market.

As much as investors might dislike stock market corrections, bear markets, and crashes, these are all perfectly normal, and inevitable, aspects of the long-term investing cycle. Even though we can't predict with any sustained accuracy precisely when downturns will occur, how long they'll last, or where the ultimate bottom will be for the Dow, S&P 500, and/or Nasdaq, we do know that every major downturn in history (save for the 2022 bear market) was eventually cleared away by a bull market rally. If anything, corrections are blessings in disguise for long-term-minded investors.

Statistically, being an optimist makes all the sense in the world, as well. In June, wealth management company Bespoke Investment Group unveiled its comprehensive analysis that compared the average length of bull and bear markets for the S&P 500 dating back to the start of the Great Depression. As you might imagine, bull markets have handily outpaced bear markets for nearly a century.

Over the past 94 years, the typical S&P 500 bear market has lasted 286 calendar days, which works out to around 9.5 months. That compares to the average S&P 500 bull market, which has endured 1,011 calendar days. There's little reason to fight a trend that clearly favors optimism over the long run.

Time is also undefeated in the return column.

Recently, Bank of America Global Research released a report that compared the probability of negative returns, relative to the S&P 500's total returns, when back-tested to 1929. BofA's report showed a perfect correlation between time in the market and the likelihood of negative returns.

For instance, an investor who, hypothetically, held an S&P 500 tracking index for one day, one quarter, or one year had a 46%, 32%, and 25% respective probability of losing money since 1929. But hold that position for 20 years, and it's always generated a positive total return.

No matter what the U.S. economy and Wall Street have in store for investors in the coming weeks and months, a mountain of historical data shows that brighter days are ahead for stocks and investors.