Last month, Capital Performance Group, a consulting company specializing in the banking industry, analyzed the nation's banks to determine which characteristics are shared by the top performers.

Unfortunately, the analysis missed the mark. More specifically, I believe it misses on two significant tenets of successful banking; tenets that bank investors must understand to successfully navigate today's financial services industry.

What the analysis did and didn't do
The rankings sorted U.S. banks by their average return on average equity figures for the three-year period from 2012 to 2014.  They divided the results into a community bank group of institutions with less than $2 billion in total assets, a mid-sized group with assets between $2 billion and $10 billion, and a large bank group with assets north of $10 billion.

Their analysis categorized "high performing" banks, like BofI Holding and Bank of the Ozarks, as banks ranking in the top 10% of the more than 5,000 banks considered. Those two specific banks had average returns on equity of 17.5% and 15.8%, respectively.

The overall highest ranking went to Fremont Bancorp, which reported a staggering average return on equity of 24.6%.

With this in mind, the first and most significant problem with the analysis was Capital Performance Group's process of selecting the top-performing banks.

This selection criteria ignores any measure of credit quality, and there's no consideration for a bank's efficiency. Stock performance is ignored as well.

I agree that earnings are a huge differentiator between banks, but I disagree that earnings on equity matters more than on assets; Warren Buffett once said that "earnings are key to evaluating banks, and you earn on assets." When Buffett zigs, you shouldn't zag.

That said, I understand the appeal of a clean and simple selection process: one metric that at least indirectly speaks to profits, expense management, and performance relative to the bank's capital structure. In that light, any number of metrics would suffice, return on average equity among them.

But there's a bigger problem that that matters more than whatever single metric you want to use. 

Just three years of history? Really?
Any bank can do exceptionally well over a three-year period of economic growth, a bull market, and an expanding credit market. In fact, I'd argue that the very best banks likely won't stand out during these easy, good times.

The best banks do well not just when times are good, but also when the economy hiccups, when the credit cycle contracts, when seemingly good loans suddenly go bad. Three years of history will never capture that quality in a bank, but it's the most important quality.

That failure leads the analysis to the wrong conclusions
Capital Performance Group concluded, based on its analysis, that the top banks separate themselves with revenue growth, scale, loan growth, and core deposits. Of those four, I will not argue against core deposits.

History has shown that the rest, important as they are, are simply not the most critical factors impacting bank success.

To be clear: rapid revenue, loan, and asset growth are not necessarily bad things. Sometimes they're a great thing. My point is that it is grossly inaccurate to say these are the qualities driving great bank performance, particularly over such a short time period.

Let's look at a quick example to illustrate the point. The chart below illustrates revenue growth among five financial institutions for 2004 through 2006 -- an arbitrarily chosen three-year time period.

Bank of America and JPMorgan Chase led the group in revenue growth, while US Bancorp and Citigroup brought up the rear. 

BAC Revenue (Annual) Chart

BAC Revenue (Annual) data by YCharts.

Looking at the subsequent three years, it is painfully obvious that revenue growth had absolutely nothing to do with how well these financial stocks performed. Some high-growth stocks nearly failed (others did fail), and others managed relatively well. The same is true for the slow growers: there were winners and losers.

Why? Because risk management, credit quality, and efficiency are far more important to bank success over the long term. 

Financial Institution Rev Growth '04-'06 Stock Performance '07-'10
Bank of America 87.44% -75%
Wells Fargo 25.72% -13.3%
Citigroup 11.47% -91.4%
US Bancorp 8.68% -25.4%
American International Group 39.46% -96%
JPMorgan Chase 84.74% -11.8

Using data from the FDIC's Quarterly Banking Profile, you can quickly confirm that limited example above holds true across the industry.

From the first quarter of 1984 to the first quarter of this year, the strongest correlation between returns on equity and other bank metrics is with the loan loss provision. The correlation between return on assets and the loan loss provission is actually even stronger.

The real top-performing banks, those that succeed in all phases of the credit cycle, are the banks that consistently make high-quality loans. Profits and growth follow naturally.

Banking is an industry of boom and bust
Banking is an unusual business in that the product a bank sells must be returned for the business to succeed. When a bank makes a loan, that loan must be paid back. It's just that simple. 

If loans aren't repaid, then it just doesn't matter how many loans that bank makes or how much it charges for those loans.

Yes, return on equity is key. So is growth. But both of those financial measures are lagging indicators. Successful banks, at their core, put prudent risk management and customer service before everything else. Risk management protects the bank and customer focus creates products and relationships that earn, in both good times and bad.

If you really want to find the best banks, don't focus on return on equity over just three years. Look at returns on equity and on assets throughout the last few credit cycles. The banks that do well over that time horizon are the banks worth applauding.