A Modest Plea for a Better Index

For most investors, the stock market, as they know it, is  the Dow Jones (INDEX: ^DJI  ) and the S&P 500 (INDEX: ^GSPC  ) . For the media, the two represent everything. How's the market doing? Better cite the Dow or the S&P.

It's unfortunate, really. Both indexes that we've come to obsess over have flaws. The Dow weights its components by share price, giving IBM (NYSE: IBM  ) 22 times the weight of Bank of America (NYSE: BAC  ) . No one I know has ever rationalized this practice. The S&P is weighted by market cap, which makes more sense, but often skews it toward the market's most overvalued companies.

Yet, both share a more glaring defect: They don't include dividends.

Let's cut to the chase. There's no better way to put it than this:

Source: Yale, author's calculations.

We fixate over how much the market indexes have gone up or down in the last month, year, or decade. But since the S&P 500 was created in 1957, one-third of average annual returns have come from dividends. Assuming dividends are reinvested, rising market prices on shares purchased with dividend proceeds means 82% of the market's cumulative return is the result of reinvested dividends. Why would we ignore that? It's the equivalent of measuring how much your child has grown while ignoring everything from the neck down.

It wasn't always this way. Classic investment books written in the early 20th century emphasize dividends as the driver of stock returns. The 1931 book Only Yesterday provides the most telling example. When the author, Frederick Lewis Allen, writes about stock prices, he includes the annual dividend payment in parenthesis next to the share price -- "American Can ($2) $77." That was the standard notation of the day.

Attitudes toward dividends changed in a big way starting around the 1980s. As trading costs plunged after brokerage deregulation in 1975, investors, for the first time, could legitimately become short-term traders. As time horizons shrank, so did the importance of dividends, because their value lies in long-term compounding. By 2000, dividends were a sideshow. One Standard & Poor's study on dividend payments that year begins by admitting, "Though currently not a hot topic.…"

But ignoring dividends, when measuring market returns, leads to of all kinds of misconceptions. Like the idea that the market suffered a lost decade from 2000 to 2010, when it actually returned 20% after dividends. Or the notion that the market was flat from 1968 to 1982, when it nearly doubled, with dividends.  

Here's what's baffling. Both the Dow and the S&P 500 already adjust their respective indexes when a company pays a special, one-time dividend. The price of each index is determined by a "divisior" that divides against the components' share prices (for the Dow), or market cap (for the S&P). After a special dividend is paid, the divisor changes. For example, Microsoft (Nasdaq: MSFT  ) paid a one-time special $3 per-share dividend in 2004, which caused the Dow's divisor to fall, effectively adjusting the index for the dividend payment. (You can see a list of divisor changes here.)

But those changes only apply to one-time dividends. Normal quarterly dividends aren't adjusted for. Why not? I contacted both Dow Jones and Standard & Poor's, and neither could provide a good explanation. S&P stated that special dividends cause a stock's price to fall after being paid out, which needs to be adjusted for. Yet the same is true for normal dividends. The analyst I spoke with argued that normal dividends are expected by the market and, thus, are already priced into shares, while special dividends are "surprises," affecting the share price. But that's a big, th eoretical assumption. Not all special dividends are surprises, and not all quarterly dividends are expected, particularly in size. If indexes adjust for special dividends, there's no excuse to not adjust for normal ones, too.

Ironically, Standard & Poor's calculates a "total return" index that adjusts for dividend payments. But few pay attention to it, particularly in the media. In the last decade, the S&P 500 Total Return Index has been mentioned in news articles fewer than 100 times, according to Google.

Will we ever get to a point when the media uses dividend-adjusted indexes when reporting the nightly news? I don't think so. The unadjusted numbers are too familiar to be changed. But that doesn't have to stop you. Whenever you're looking at how the market has performed, or how much your portfolio has returned, never, ever, forget to include dividends. Give your returns the credit they deserve.

Fool contributor Morgan Housel owns shares of Microsoft and B of A preferred. Follow him on Twitter @TMFHousel. The Motley Fool owns shares of Bank of America, Microsoft, and International Business Machines. Motley Fool newsletter services have recommended buying shares of Microsoft. Motley Fool newsletter services have recommended creating a synthetic covered call position in Microsoft. Motley Fool newsletter services have recommended creating a synthetic long position in International Business Machines. The Motley Fool has a disclosure policy. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. Try any of our Foolish newsletter services free for 30 days.


Read/Post Comments (19) | Recommend This Article (57)

Comments from our Foolish Readers

Help us keep this a respectfully Foolish area! This is a place for our readers to discuss, debate, and learn more about the Foolish investing topic you read about above. Help us keep it clean and safe. If you believe a comment is abusive or otherwise violates our Fool's Rules, please report it via the Report this Comment Report this Comment icon found on every comment.

  • Report this Comment On September 21, 2012, at 10:45 AM, LMedico wrote:

    Why not 'start at home'? Since the S&P does provide a Total Return metric, why not edit The Motley Fool Website to track that one alongside the usual trio of Dow, S&P, and NASDAQ? Why not encourage your fellow Fool authors to go to the Total Return first when referencing the market? How about using the Total Metric in figuring out CAP scores?

    After all, since one of the ideology of the website is long term investing, then the Total Return of the S&P index, including dividends, would be the most appropriate to use on this site.

  • Report this Comment On September 21, 2012, at 2:22 PM, seattle1115 wrote:

    Amazing - the YTD difference between the plain vanilla S&P 500 and the TR version amounts to nearly 200 basis points. That's huge, even in the relatively short term.

  • Report this Comment On September 21, 2012, at 8:24 PM, SuntanIronMan wrote:

    @LMedico

    CAPS uses the SPY, the SPDR S&P 500 ETF, that tracks the price and the yield of the S&P 500. Dividends are adjusted into the score four times a year (when the SPY pays them out).

  • Report this Comment On September 22, 2012, at 10:18 AM, Lucaskasan wrote:

    The blue line shows that mutual funds are even worse than we thought. We already knew that most of them cannot beat the SP 500 if only because of fees. Presumably your mutual fund is reinvesting dividends (or not). Perhaps the reason reinvested dividends are not counted is because investors do not necessarily reinvest them. Or maybe they use their dividends to buy an entirely different stock. Dividends are deposited into the investor's account as settled funds available for trading unless the investor has director otherwise, usually through a 401(k) which generally hold only mutual funds. Now we have come full circle and have to wonder why mutual funds, even index funds do not perform better in comparison to the red-line SP 500.

  • Report this Comment On September 22, 2012, at 12:17 PM, HectorLemans wrote:

    Partly because of the dividend issue and because you can't actually buy the S&P500 index, I always compare my investments to the Vanguard Total Stock Market ETF (VTI). It trades just like a stock and you can include dividends in your return calculations.

    https://personal.vanguard.com/us/funds/snapshot?FundId=0970&...

  • Report this Comment On September 22, 2012, at 12:55 PM, scs660 wrote:

    Does the Stock Advisor Tracker factor in dividends too?

  • Report this Comment On September 22, 2012, at 5:29 PM, SuntanIronMan wrote:

    @scs660

    Here is the methodology Stock Advisor and Rule Breakers use:

    "Total average returns are the average of all individual stock recommendations (active and sold) and the average of the S&P 500 Total Return Index, starting from the end of the day we make each recommendation. Both the stock and benchmark returns include reinvested dividends."

  • Report this Comment On September 22, 2012, at 10:38 PM, neamakri wrote:

    I invest in dividends. Every 2-3 months I take my collected dividends and buy another chunk of stock. The DOW and the S&P 500 have absolutely no meaning for me.

    Why does it mean anything to you? It only tells you what happened in the past!

    Even if you owned all 500 stocks in the S&P, it would still mean nothing. If you're a day trader you don't need to know these indexes. If you're a buy-and-hold investor, you still don't need these indexes.

    The indexes don't even provide any jobs; a well-constructed spreadsheet with live feeds can calculate these indexes for you. All I can see is that they are a way for lazy journalists to "report" on the stock market. So what.

    I hope that we Fools follow our own stocks, then buy or sell based on value. To paraphrase Blazing Saddles "We don't need no stinking Indexes."

  • Report this Comment On September 24, 2012, at 9:49 AM, LMedico wrote:

    @WhichStocksWork

    Thanks, I was not aware. I haven't used CAPS yet, as I don't consider myself knowledgable at Stock Picking or valuations - yet. But that's why I am here, to learn.

  • Report this Comment On September 24, 2012, at 12:48 PM, akutach wrote:

    So, make a new one!

    One for MF in which you can systematically work in contributions from small, micro, and even more skewing (dividend/capital return) players such as REITS and BDCs and MLPs.

    And even a very strained suggested acronym:

    Motley Index of Dividend-Adjusted Stocks (MIDAS)

    (I'll accept a dinner with Morgan as compensation for the naming rights!)

    While one does not NEED indexes, they are useful tools for objectivity: Are all my efforts on equity selection wasted because I'm not even beating a low-cost index-tracking ETF?

  • Report this Comment On September 24, 2012, at 1:20 PM, pondee619 wrote:

    " No one I know has ever rationalized this practice" The Dow Jones Industrial AVERAGE is just that, An AVERAGE of the PRICES of the stocks contained therein.

    There is your "rationalization". It is the definition of AVERAGE.

  • Report this Comment On September 24, 2012, at 1:30 PM, TMFMorgan wrote:

    ^ But share prices don't indicate anything about the size, value, or relevance of a company. Why should IBM have 22x the weight of BofA when its market cap is only 2x the size?

  • Report this Comment On September 24, 2012, at 2:28 PM, TMFCrocoStimpy wrote:

    @pondee619

    I think the non-nonsensical nature of calculating an average in this way is that, given IBM has 22x the current weighting of BAC, if BAC chose to do a 1-for-10 reverse split then IBM would suddenly have only a 2.2x weight relative to BAC, even though nothing of value for the companies had changed.

    The market cap weighting scheme for the S&P500 has a little more rationalization too it, in that it is similar to starting with all companies given an equal weight at some arbitrary time in the past, and then their relative influence grows/shrinks as they themselves grow/shrink. It would be like having a portfolio that starts equal weighted and receives no future capital injections, so the value moves in proportion to the size of each company at any given time. However, Morgan's primary thesis still stands - not accounting for the dividends in this total value discards a large component of the long term growth.

    -Xander

  • Report this Comment On September 24, 2012, at 2:52 PM, TMFGalagan wrote:

    @Morgan -

    The flip side of the argument is that for those who spend rather than reinvest dividends, a dividend-adjusted index paints a misleadingly rosy picture of the market. Retirees would wonder how the market would supposedly be at new highs when their non-reinvested portfolio balances had actually fallen.

    I agree with your point that having both is valuable, but I wouldn't replace one with the other.

    best,

    dan (TMF Galagan)

  • Report this Comment On September 24, 2012, at 3:22 PM, pondee619 wrote:

    Morgan:

    Yep, that;s right, but to calculate it any other way would not make it an AVERAGE. The Dow Jones Industrial AVERAGE is, and always was, an AVERAGE of its member's prices. Your statement; "No one I know has ever rationalized this practice" is foolish. It was, and still is, rationalized as being an AVERAGE of the member's prices. Perhaps no one has ever rationalized its use in gauging the market to your satisfaction. But it is a perfectly correct and rational AVERAGE of the stock prices therein. The Dow Jones Industrail AVERAGE tells you what the AVERAGE price of its members stocks would be accouting for splits, and changes in the members. What each of us do with that information is up to us.

    Xander- how would YOU calculate an AVERAGE? Other than add up the members and divide by the number of members? Any other calculation would lead to something other than an AVERAGE. BTW what is a"non-nonsensical nature"?

    So, the Dow Jones Industrial Average is flawed. This has been pointed out for as long as I have been investing (early 70's). It is not, however, so flawed to lose its meaning and relevance.

    Dividends matter. Duh. Thanks

  • Report this Comment On September 24, 2012, at 3:32 PM, TMFMorgan wrote:

    Dan,

    In that case, an index that simply added dividends back into the index price rather than reinvesting them would be appropriate, right?

  • Report this Comment On September 24, 2012, at 4:05 PM, TMFGalagan wrote:

    @Morgan,

    Yeah, that'd work, although the computations would get really tricky because you'd have to calculate changes to the index excluding the div amount. I'd almost rather see a separate running total of the cumulative dividends since whenever. Just like your historical reference about how stock quotes used to include the dividend right there, a running total of divs next to an index value would separate it out nicely and show just how valuable the divs were compared to the overall index.

    best,

    dan

  • Report this Comment On September 24, 2012, at 4:57 PM, TMFCrocoStimpy wrote:

    @pondee619

    "BTW what is a"non-nonsensical nature"?"

    I would have to say a typo :)

    Fixating on the word AVERAGE, and assuming that this is (and was originally intended to be) a SIMPLE AVERAGE is the crux of the question here. There is no doubt that a simple average is used to calculate the DJIA, but if you step back to when the index was constructed the question is what were they trying to measure? If the index was there to gauge the general movement of the stock market, then what are the conditions that would have allowed a simple average to do so? It seems likely (but this is speculation on my part) that the number of shares of companies didn't have as much fluidity as they do today - splits, secondary (tertiary, quaternary, internal options grants, etc) offerings were not anywhere near as prevalent. Thus the number of shares would stay reasonably constant, so the use of share price was a fairly good indicator of market value. In essence, I'd think that a simple average of prices in the early years of the DJIA would have reflected something very similar to a market cap weighted average, more like we see with the S&P500 today.

    So, that was a rather long answer to how I might calculate an average, which is that I would weight it by the relevant factors that reflect the underlying information I'm interested in tracking. Market Cap is one method since it shows a proportional move relative to the capital within the market, but isn't always the best measure. Regardless of what one chooses to use as a weighting measure, the one thing that the index should be invariant to is changes in value(s) that have no impact on investors such as splits, which is why I shy away from considering a simple price average to convey as much useful info as other measures.

    Foolishly,

    -Xander

  • Report this Comment On September 24, 2012, at 10:55 PM, TerryHogan wrote:

    I think one of the problems is the fixation on short-term results. I mean really the Dow and the S & P are there to provide news anchors and radio hosts something to say about "the markets". And most people (however foolishly) are only interested in hearing what happened that day - in that respect I'd say the S&P does a pretty good job. For long-term results I'd agree that including dividends makes more sense. But think about the vested interests. Who wants to be the first mover on this one. The first mutual/hedge fund to start comparing themselves to the S&P with dividends reinvested will look like an even crappier investment than one who compares itself to the plain old S&P.

Add your comment.

Sponsored Links

Leaked: Apple's Next Smart Device
(Warning, it may shock you)
The secret is out... experts are predicting 458 million of these types of devices will be sold per year. 1 hyper-growth company stands to rake in maximum profit - and it's NOT Apple. Show me Apple's new smart gizmo!

DocumentId: 2025772, ~/Articles/ArticleHandler.aspx, 10/24/2014 8:13:26 AM

Report This Comment

Use this area to report a comment that you believe is in violation of the community guidelines. Our team will review the entry and take any appropriate action.

Sending report...


Advertisement