If there's one thing new investors should understand, and tenured investors constantly need to be reminded of, it's that Wall Street can be unpredictable. Although it's proved to be a money machine for patient investors over numerous decades, the moves stocks make over shorter time frames are difficult to forecast.

While there are no foolproof indicators or metrics that can concretely tell us which direction stocks will head next, there are an abundance of tools we can use as investors to make our lives easier. One such valuation tool, which has been back-tested all the way to 1870, offers a rather ominous warning for Wall Street and investors.

A twenty dollar bill paper airplane that's crashed and crumpled into the business section of a newspaper.

Image source: Getty Images.

The Shiller P/E ratio has a history of forecasting stock market downturns

The valuation metric I'm talking about is the Shiller price-to-earnings (P/E) ratio, which is also known as the cyclically adjusted price-to-earnings ratio, or CAPE ratio. Whereas a traditional P/E ratio takes a company's share price and divides it into trailing-12-month earnings, the Shiller P/E ratio is based on average inflation-adjusted earnings from the previous 10 years. Looking at inflation-adjusted earnings over 10 years can help smooth out the wild fluctuations occasionally witnessed in short-term corporate earnings.

Before I go any further, let's state the obvious: A lofty valuation, by itself, is generally a poor reason to sell a stock or run for the hills. Companies that are industry leaders, have virtually impenetrable moats, or are on the cutting edge of innovation within their specific sectors or industries, may rightly command a premium valuation.

Nevertheless, the Shiller P/E ratio, which is tethered to the benchmark S&P 500 (^GSPC 1.02%), tells quite the tale over the long run.

Although the Shiller P/E ratio was only introduced roughly three decades ago, it's been back-tested all the way to 1870. As you can see in the following 153-year chart, the Shiller P/E has been markedly higher over the past 30 years than in the preceding 123 years. I'd attribute this to historic monetary policy easing since the end of the Great Recession, as well as ease of access to information. With one click, everyday investors can access income statements, balance sheets, management commentary, and breaking news. These factors have led to an increased willingness by investors to take risks.

S&P 500 Shiller CAPE Ratio Chart

S&P 500 Shiller CAPE Ratio data by YCharts.

However, valuation expansion has its limits. Regardless of the innovations or moats backing up the companies leading the S&P 500, Nasdaq Composite (^IXIC 2.02%), and Dow Jones Industrial Average (^DJI 0.40%) higher, a line in the sand can be drawn where things have historically broken. That line is a Shiller P/E ratio of 30.

Since 1870, there have only been six instances where the S&P 500's Shiller P/E ratio has surpassed and held above 30. Looking back at the previous five instances, the S&P 500 or broader market eventually went on to lose at least 20% of their value.

  • 1929: Following the Black Tuesday crash in October 1929, the iconic Dow Jones Industrial Average would lose as much as 89% of its value.
  • 1997-2001: During this extended period, the Shiller P/E would hit its record high of a little north of 44. When the dot-com bubble burst, the S&P 500 shed just shy of half its value.
  • Q1 2018-Q3 2018: Approximately 17 years after the dot-com bubble, the S&P Shiller P/E found itself above 30 again. During the fourth quarter of 2018, the index declined by 20%.
  • Q4 2019-Q1 2020: The Shiller P/E ratio was north of 30, once again, leading into the COVID-19 crash, which wiped a maximum of 34% off the S&P 500 in 33 calendar days.
  • Q3 2020-Q2 2022: The fifth incident occurred after the federal government propped consumers up with stimulus checks due to the pandemic. After the Shiller P/E ratio briefly topped 40 in January 2022, the S&P 500 went on to lose as much as 28% (from peak to trough) last year.

The sixth such instance of the Shiller P/E ratio for the S&P 500 surpassing 30 occurred in February 2023. As of May 17, 2023, it was still hovering above 29. 

Although valuations can stay extended for some time -- i.e., the Shiller P/E ratio isn't capable of forecasting when a downturn will begin -- the implication based on prior history is that the S&P 500 could eventually lose 20% or more of its value.

To add to the above, the Shiller P/E has a history of bottoming out around 22 over the past quarter of a century (with the exception of the financial crisis). This, too, would suggest an eventual decline of at least 20% awaits the broad-based index.

A businessperson closely reading a financial newspaper.

Image source: Getty Images.

Don't forget about Wall Street's closest thing to a guarantee

While the Shiller P/E ratio for the S&P 500 has an impressive track record under certain parameters, there are a few other datapoints long-term investors should keep in mind that don't require set parameters for success.

For instance, data from sell-side consultancy firm Yardeni Research shows that there have been 39 double-digit percentage declines in the S&P 500 over the past 73 years.  Aside from corrections and bear markets being more common than you might have realized, the important takeaway is that every single one of these declines, save for the current bear market, was eventually wiped away by a bull market rally. Though we'll never know ahead of time how long a correction will last or how steep the decline will be, history has conclusively shown that the Dow Jones, S&P 500, and Nasdaq Composite always, eventually, find their way to new highs.

To build on this point, I used Yardeni's data to tabulate how many calendar days the S&P 500 spent in correction, as compared to a bull market, between the start of 1950 and May 2022. For every one calendar day spent in correction, Wall Street has historically enjoyed 2.6 calendar days of expansion.

^SPX Chart

^SPX data by YCharts.

Want additional evidence that it pays to be an optimist? Look no further than my favorite report to reference, which is from market analytics company Crestmont Research.

What Crestmont did was analyze the rolling 20-year total returns, including dividends paid, of the S&P 500 and back-tested its data to 1900.  In other words, it allowed investors to see what a hypothetical investor would have earned on an average annualized basis if they'd purchased an S&P 500 tracking index and held that position for 20 years. This gave Crestmont 104 ending years of data to work with (1919 through 2022).

This buy-and-hold-for-20-years strategy produced winners in all 104 end years. It literally didn't matter when you put your money to work on Wall Street. As long as you remained invested for 20 years, you generated a positive total return that, in virtually all instances, crushed the prevailing rate of inflation.

Despite the ominous signs being presented by the Shiller P/E ratio, long-term-minded investors are in good shape.