My name is Tim Hanson, and my colleague Frank Thomas and I are members of the Fool's Investing Intelligence team  -- a cross-functional group of Fools that aims to provide our organization with context and insights around our most important business asset: our investing ideas and performance.

graphic that reads Inside Foolish Indexes, Tim Hanson and Frank Thomas, Motley Fool Investing Intelligence

To do this, we are building the technology platforms, including an application you may have heard of called Fool IQ, that we need to identify our best ideas across The Motley Fool, LLC; accurately track our results; provide feedback to our analysts about what they are good and not so good at; and automate as many of our investing processes as we can. And we are also using those systems to build what we are here to talk about today : Motley Fool stock indexes. We currently publish one index to the world, which is the Motley Fool 100, which I hope you've heard about.

So first I just want to define some terms. What is a stock market index?

A stock market index is a way to describe and measure a section of the stock market

Does anybody know the first index ever created?

It was the Dow Jones Transportation Average, which was first published by journalist Charles Dow in 1884 to track the health of the country's transportation sector, which at that time consisted mostly of railroads and was a major part of the U.S. economy. Dow followed that up two years later with the now better-known and more diversified Dow Jones Industrial Average, which was designed to be a proxy for the health of the American stock market overall. Interestingly, Dow's indexes were price-weighted, which meant that the weight of a stock in the index was determined by the ratio of its stock price to the sum of all of the stocks in the index, which is kind of quirky given that a company's stock price is probably the least meaningful thing about it.

But Dow's indexes have lasted for well more than a century now, and one can really describe the history of the country in how they have risen and fallen over time.

That brings me to why we have indexes. Today stock market indexes are used to compress information. That means they provide a common reference point for people to talk about some part of the stock market -- usually with regards to returns. Even though thousands of stocks have thousands of different returns, we all know how the market is doing every day because of an index.

Stock market indexes are used to compress information, track returns, benchmark performance, and create index-linked investment products

Now, incredibly, there are more indexes in the U.S. than publicly traded stocks  -- about six thousand to four thousand  -- tracking all manner of market niches, which is another use of stock market indexes. If you want to know how hydroelectic companies in Nepal are doing, for example, you can just pull up the NEPSE Hydropower index. (They're down about 15% year to date, so not a good year for Nepalese hydropower.)

From there we use indexes to benchmark performance. That includes both whether an investing approach is doing better or worse than a relevant index and also whether an approach is taking more or less risk than that relevant index.

And finally, what mostly explains the recent proliferation of indexes is that they can be used to create index-linked investment products such as ETFs, which seek to match an index's composition and performance as closely as possible, before fees. Thanks to the convenience and relatively low cost of these products, they have become a very popular investment choice for Americans, with almost $700 billion of assets flowing into index-based investing strategies in 2017 alone.

Now, to do all of these things successfully, indexes need to be constructed in very specific ways. They must be transparent, meaning that anyone can replicate an index from publicly available information; investable, meaning that index constituents must all meet certain liquidity requirements; and systematic, meaning that the methodology is rules-based and incorporates no subjective judgment. 

Stock market indexes must be transparent, investable, and systematic

So that's a short summary of the history, theory, and application of stock market indexes. And while it's pretty straightforward, when you finally get into building these things, there are a lot of choices to make and lots of room for creativity -- which brings me to the fun stuff: Foolish indexes. 

Given that background, the first question you might have on this topic is: What took us so long?

After all, the Foolish universe is a unique and interesting section of the stock market that was begging to be more conveniently measured and described. What's more, given our firm's somewhat idiosyncratic approach to investing, we need a holistic way to track and benchmark our performance. Finally, as you may know, our mission is to help the world invest better. With the world more and more expressing its desire for convenient, index-linked investing strategies, it made sense that there was an opportunity in that space in order to fulfill our mission.

It's for these reasons that we began our work on index development. Getting started, we knew we had lots of good stock ideas to fill up an index and that it was just a matter of putting them together in a transparent, investable, and systematic way.

And those were three of the principles that underpinned our project, but there was also a fourth. That was that a Foolish stock market index also had to be Foolish. That's an easy thing to say, but what does it mean? Sitting down together, we began to unpack the word.

First, given the heritage of The Motley Fool, we thought any methodology we chose would need to have demonstrated market-beating historical performance. That was non-negotiable for us.

A Foolish stock market index must be transparent, investable, systematic, and Foolish

Second, we said a Foolish index had to be inclusive of all of our Motley investing approaches  -- and 100% allocated to stocks we like and 0% allocated to stocks we don't. 

Third, it shouldn't try to be something it's not. That is to say it needed to have the right amount of exposure to the companies, sectors, and trends that have defined The Motley Fool universe over the past 25 years  -- the innovative technology companies and rule-breaking consumer brands that we've recommended for decades. 

Fourth, all else being equal, it should be low-turnover  -- we are buy-to-hold investors. 

Fifth, we thought it needed to include enough stocks that it would meet the diversification standard that we recommend to our members.

Finally, we thought the index had to be simple and straightforward. After all, indexes were new to us. This had to be something we could easily explain not only to our fellow Fools at the office here, but to members like you and to the broader investing industry.

With those as guiding principles, Frank took the lead on aggregating and sanitizing our historical data and backtesting the performance of several different potential approaches. 

Immediately, we faced challenges. For example, what stocks do we like? That sounds like a simple enough question, right? But as members you know that we clearly like some stocks more than others. We have best buys now, stocks on hold, stocks in the penalty box, stocks as partial sells, stocks as small positions, stocks as large positions, starter stocks, buy recommendations with bearish options strategies attached to them, and so on and so forth. 

For our first index we decided to first test a simple approach: We'd include any stock that had had a binary and active buy recommendation in one of our non-real-money services. 

But we also knew that our company likes to move fast. What would happen, we wondered, if an advisor changed, or if one of those services went away? We might still like those stocks even if they were no longer published buy recommendations. Since we wanted our index to be low-turnover, this was a potential problem. 

Here's where Fool IQ came in handy. As a database that aggregates and ranks all of our investing ideas, it would persist even if certain services didn't. But it had also been around a lot less time than Stock Advisor. Would it be fair to knit the two universes together? We decided we'd have to test that.

What about the number of stocks? Would it be better to have more or less? Again, that was something we thought we could test.

And what about allocations? There are a lot of different approaches to this problem in the index world. Dow did price weighting. Today, market cap weighting is probably the most common approach  -- that's how the S&P 500 is constructed. But there's also equal weighting and fundamental weighting , each of which has pros and cons. Since we wanted to keep it simple, we thought market cap weighting and equal weighting were our two most likely candidates, so we tested to see whether one offered an advantage over the other or not.

So this was a lot of work for poor Frank. Lots of questions. But he did a great job, and some of the outputs of his good research, which led to the creation of The Motley Fool 100 in its current form, I'll share with you today.

First on performance. Here we got lucky because we had such good source material to work with. Our first crack at it  -- a simple market cap weighted index of the 100 largest stocks we liked, rebalanced quarterly  -- yielded this handsome historical graph, which we then verified with an independent index calculation agent. 

Chart showing the back-tested performance of the Fool 100 Index versus the S&P 500

Past performance is no guarantee of future results. Performance information is provided by Solactive AG, the index calculation agent ("Solactive"). Solactive is not affiliated with The Motley Fool and, while we believe this information to be reliable, we make no warranties, express or implied, regarding its accuracy for any particular purpose. This performance information does not relate to any asset management products based on this index. No allowance has been made for trading costs or management fees which would reduce investment performance. Actual results may differ. Indexes are not managed investment products and, as such, cannot be invested in directly. Returns represent simulated performance based on rules used in the creation of the index, are not a guarantee of future performance, and are not indicative of any specific investment. This material is not intended as an offer or a solicitation for the purchase and/or sale of any security or financial instrument, nor is it advice or a recommendation to enter into any transaction. All performance presented prior to the index inception date is backtested performance. Backtested performance is not actual performance, but is hypothetical and was derived from the retroactive application of the index methodology with the benefit of hindsight. The Motley Fool, LLC (TMF) constructs, publishes, and licenses indexes and does not offer or provide investment advice or offer or sell any securities, commodities or derivative instruments or products, and nothing in this presentation should be taken as constituting financial or investment advice. Unlike its affiliates, TMF is not registered with the SEC and is not permitted to provide investment advice. Neither TMF nor its respective directors, officers, employees or affiliates make any representation regarding the advisability of investing in any security or instrument. A decision to invest in any such security or instrument should not be made in reliance on any information in this presentation. Inclusion of a security in an index is not a recommendation to buy, sell, or hold that security.

Having that performance in our pocket gave us permission to test around the margin with regards to some of the questions I raised earlier. Market cap weighting versus equal weighting was a question we were particularly interested in.

To be honest, my bias was against market cap weighting because that always seemed to me like a daft way to do things, particularly as it pertained to index performance. Why should an index have more exposure to a company simply because that company was big?

But as it turns out, there are a few benefits to market cap weighting that I was missing.

Chart showing the performance of the Fool 100 Index when it is weighted by market capitalization and when it is equally weighted

Past performance is no guarantee of future results. Performance information is provided by Solactive AG, the index calculation agent ("Solactive"). Solactive is not affiliated with The Motley Fool and, while we believe this information to be reliable, we make no warranties, express or implied, regarding its accuracy for any particular purpose. This performance information does not relate to any asset management products based on this index. No allowance has been made for trading costs or management fees which would reduce investment performance. Actual results may differ. Indexes are not managed investment products and, as such, cannot be invested in directly. Returns represent simulated performance based on rules used in the creation of the index, are not a guarantee of future performance, and are not indicative of any specific investment. This material is not intended as an offer or a solicitation for the purchase and/or sale of any security or financial instrument, nor is it advice or a recommendation to enter into any transaction. All performance presented prior to the index inception date is backtested performance. Backtested performance is not actual performance, but is hypothetical and was derived from the retroactive application of the index methodology with the benefit of hindsight. The Motley Fool, LLC (TMF) constructs, publishes, and licenses indexes and does not offer or provide investment advice or offer or sell any securities, commodities or derivative instruments or products, and nothing in this presentation should be taken as constituting financial or investment advice. Unlike its affiliates, TMF is not registered with the SEC and is not permitted to provide investment advice. Neither TMF nor its respective directors, officers, employees or affiliates make any representation regarding the advisability of investing in any security or instrument. A decision to invest in any such security or instrument should not be made in reliance on any information in this presentation. Inclusion of a security in an index is not a recommendation to buy, sell, or hold that security.

First, as you can see here, an equal weight approach actually underperformed our market cap weighting approach over the period of the backtest. One reason for that  -- and I underappreciated this about market cap weighting  -- is that this approach always has you adding to your winners, whereas an equal weight approach has you selling your winners to buy your losers. Looking at it that way, I now view market cap weighting as a very Foolish  -- capital "F"  -- approach to index construction. We want people adding to their winners. Our CIO Andy Cross is fond of telling people to water the flowers and pull the weeds.

The other reason to favor market cap weighting has to do with turnover. In addition to showing less return, the equal weight approach showed significantly more turnover, which made not going that direction a pretty easy decision.

Chart showing the Fool 100 Index's turnover based on whether it's weighted by market capitalization or equally weighted

Past performance is no guarantee of future results. Performance information is provided by Solactive AG, the index calculation agent ("Solactive"). Solactive is not affiliated with The Motley Fool and, while we believe this information to be reliable, we make no warranties, express or implied, regarding its accuracy for any particular purpose. This performance information does not relate to any asset management products based on this index. No allowance has been made for trading costs or management fees which would reduce investment performance. Actual results may differ. Indexes are not managed investment products and, as such, cannot be invested in directly. Returns represent simulated performance based on rules used in the creation of the index, are not a guarantee of future performance, and are not indicative of any specific investment. This material is not intended as an offer or a solicitation for the purchase and/or sale of any security or financial instrument, nor is it advice or a recommendation to enter into any transaction. All performance presented prior to the index inception date is backtested performance. Backtested performance is not actual performance, but is hypothetical and was derived from the retroactive application of the index methodology with the benefit of hindsight. The Motley Fool, LLC (TMF) constructs, publishes, and licenses indexes and does not offer or provide investment advice or offer or sell any securities, commodities or derivative instruments or products, and nothing in this presentation should be taken as constituting financial or investment advice. Unlike its affiliates, TMF is not registered with the SEC and is not permitted to provide investment advice. Neither TMF nor its respective directors, officers, employees or affiliates make any representation regarding the advisability of investing in any security or instrument. A decision to invest in any such security or instrument should not be made in reliance on any information in this presentation. Inclusion of a security in an index is not a recommendation to buy, sell, or hold that security.

Similarly, you can see how the data made us feel pretty good about using both our services and our Fool IQ universe to build the eligible universe for the Fool 100. Comparing the returns, they're basically identical. 

Chart showing the performance of the Fool 100 Index with and without Fool IQ

Past performance is no guarantee of future results. Performance information is provided by Solactive AG, the index calculation agent ("Solactive"). Solactive is not affiliated with The Motley Fool and, while we believe this information to be reliable, we make no warranties, express or implied, regarding its accuracy for any particular purpose. This performance information does not relate to any asset management products based on this index. No allowance has been made for trading costs or management fees which would reduce investment performance. Actual results may differ. Indexes are not managed investment products and, as such, cannot be invested in directly. Returns represent simulated performance based on rules used in the creation of the index, are not a guarantee of future performance, and are not indicative of any specific investment. This material is not intended as an offer or a solicitation for the purchase and/or sale of any security or financial instrument, nor is it advice or a recommendation to enter into any transaction. All performance presented prior to the index inception date is backtested performance. Backtested performance is not actual performance, but is hypothetical and was derived from the retroactive application of the index methodology with the benefit of hindsight. The Motley Fool, LLC (TMF) constructs, publishes, and licenses indexes and does not offer or provide investment advice or offer or sell any securities, commodities or derivative instruments or products, and nothing in this presentation should be taken as constituting financial or investment advice. Unlike its affiliates, TMF is not registered with the SEC and is not permitted to provide investment advice. Neither TMF nor its respective directors, officers, employees or affiliates make any representation regarding the advisability of investing in any security or instrument. A decision to invest in any such security or instrument should not be made in reliance on any information in this presentation. Inclusion of a security in an index is not a recommendation to buy, sell, or hold that security.

And that turned out to be a serendipitous decision given the recent merging of Inside Value, Income Investor, and Hidden Gems into Market Pass. Had we not felt comfortable using Fool IQ to build our Fool 100 constituent universe, those events would have led to quite a bit of turnover in index. Fortunately, though, the two data sets played nicely together, which makes sense. We should like the stocks we like regardless of the place we say we like them.

So that's how we backtested and made a few of the key decisions around the Fool 100, which is now live and tracking the performance of the 100 largest buy recommendations in The Motley Fool's universe. You can see the daily return up on Fool.com.

Now what's next? Well, we've gotten a lot of positive feedback about the Fool 100 and heard from members that they'd find other indexes useful, such as a Fool high growth or Fool biotech index.

But rather than try to build a series of one-off indexes, we've opted instead to try to create a holistic Total Fool Index, which could then enable us to create any number of subindexes, with the hope being that if Total Fool is good, anything that we derive from Total Fool  -- such as Fool Small Cap or Fool Biotech  -- is likely to be reasonably good as well. The advantage here for us is scalability and the potential to launch multiple new indexes at the cost of slowing the development time of the very next index. 

In theory that would look a little bit like this, with one index of everything we cover then divisible into useful parts based on factors such as size or valuation or sector.

Image showing potential Fool subindexes

But as you can see with this example, we have a challenge when it comes to developing the more granular indexes. Specifically, we don't have opinions on enough companies to build a diversified and investable index when we get down to something as specific as small and midcap biotech stocks. There are only 17 stocks we like in the Fool IQ universe that meet those criteria. 

That has us back and wrestling with the fundamental question of what we like. We have fewer services now than before, and our services are becoming increasingly flexible. That's great for our members, but it makes life a little more difficult for us. What's more, we'd prefer to build Total Fool entirely on top of our most persistent source of information: our Fool IQ database. 

So how can we increase the scope and scale of that database without compromising on quality?

Fool IQ, for those of you who don't know, is a software application that our analysts around the world use to tell us about the stocks they follow. Within it, we ask our individual analysts to build portfolios composed of the stocks they cover and to apply position weights to each idea that add up to a total portfolio allocation of 100 or less. These weights represent their individual convictions, allowing us to differentiate their highest-conviction or favorite ideas from ideas they feel less strongly about and from negative-conviction ideas, or stocks they might sell.

We then pass this data into an aggregated list, making a series of small adjustments to build the most informed view possible, using a weighted up/down voting methodology that incorporates information from each of the analysts covering a stock.

Specifically, we adjust each opinion based on the size of the analysts' workload, how their picks have performed in the past, and what the Fool's investing group, as a whole, thinks of similar companies.

Applying these adjustments to the raw conviction results in an adjusted conviction that is a real number we use to generate a ranked list. We also separate the list into high, positive, neutral, and negative groups, with "high" being ideas are analysts are overweight in, "positive" approximating equal-weight ideas, and "neutral" and "negative" representing ideas they would not buy today. 

Here is the result of some interesting work that Frank did to measure the persistence of our ratings from quarter to quarter. 

chart and table showing persistence of Fool IQ ratings

This is a Markov chain, which illustrates how a complex system moves between discrete states. It answers questions such as: If a stock is high conviction, what is the likelihood that it stays high conviction? Or if a stock is uncovered, what is the likelihood that it enters our universe as high conviction?

As you can see, our high conviction universe is fairly stable: 80% of high conviction ideas stay high conviction. The same is true of positive conviction ideas  -- 80% of them stay positive. It makes sense that our analysts' opinions rarely change (we're long-term, business-focused, buy-to-hold investors) and that when they do, those changes appear very deliberate.

Yet 20% quarterly turnover would be unacceptable to us from an investment standpoint. 

But when Frank dug deeper, he found that more than half of that turnover isn't caused by an analyst changing his or her conviction on a stock. Rather, it's tied to changes in the threshold for determining high conviction, which is currently defined as the top quintile of IQ stocks, ranked by group conviction. That threshold can move with the smallest changes in groupwide opinion, and it can push borderline stocks out of the high-conviction universe. That dynamic has driven much of the movement between the high-conviction and positive-conviction states during the last two to three years.

Long story short, we're working on an alternative methodology that would preserve the quality of our database and reduce quarterly turnover to less than 5%  -- which is more acceptable as the foundation of a Total Fool Index.

Another point is that if we're covering 800 companies, that means that we're not covering some 4,000 companies. These aren't necessarily companies we don't like, but companies we don't know if we like or not. If we were able to help our analysts better prioritize which companies they might like or , better yet , predict which companies they would like and where they would rate on our leaderboard, it would help us grow the scope of our leaderboard in powerful ways.

And we're also at work on those two things. One of our projects is a recommender tool that would let our individual analysts know about stocks they don't follow that they might like to follow -- and be good at following based on the things they already follow and their performance across certain types of stocks. In other words, it would be something similar to the affinity algorithms that companies like Netflix use to recommend personalized content for you.

Additionally, we've begun preliminary work with a machine learning company to see if we can feed their system our data, have it learn what we look for in stocks we like, and then predict which stocks we're likely to like in sectors such as biotech, or markets such as Japan, where we currently don't have much coverage. Knowing that would provide us with much more raw material that we could use to construct more and more granular indexes.

Those outputs, however, probably won't come for quite some time. There's a lot of work to be done and a lot of data for us to collect and analyze. We have a lot of ideas in that space and a heritage in finding great small caps, so we're optimistic that our research and backtesting will yield something worthwhile.

That's the inside look at our Foolish indexes. If you're interested in learning more about the Fool 100 and what's in it, you can get all of that information at fool100.com.

The Motley Fool (TMF) constructs, publishes, and licenses indices and does not offer or provide investment advice or offer or sell any securities, commodities or derivative instruments or products, and nothing on this website should be taken as constituting financial or investment advice. Unlike its affiliates, TMF is not registered with the SEC and is not permitted to provide investment advice. Neither TMF nor its respective directors, officers, employees or affiliates make any representation regarding the advisability of investing in any security or instrument. A decision to invest in any such security or instrument should not be made in reliance on any information on this website. Inclusion of a security in an index is not a recommendation to buy, sell or hold that security.

Motley Fool Asset Management, LLC and Motley Fool Wealth Management, LLC are affiliates of The Motley Fool ("TMF"), and all investment decisions for investment products and client portfolios are made independently by the asset managers at MFAM and MFWM. Both MFAM and MFWM are investment advisors registered with the SEC.