Credit card companies make for great investments so long as they avoid the urge to grow at the expense of credit quality.

It's important to keep in mind that credit card loans are the riskiest types of loans a lender can make. In this year's stress test, for example, the Federal Reserve estimated that 13% of all credit card loans would default in a "severely adverse" economic downturn akin to the financial crisis. As you can see in the chart below, this is dramatically worse than any other loan category.

Given this, it should come as no surprise that credit card companies were projected by the Fed to experience the largest loan loss rates among their financial industry peers if the economy turned south. Discover Financial (DFS 2.02%) was projected to lose 12.2% of its loan portfolio, Capital One (COF 0.48%) came in at 10.8%, and American Express (AXP 2.54%) followed closely behind at 9.2%.

By contrast, traditional lenders were projected to lose roughly half as much across a diverse array of loan portfolios. According to the Fed, US Bancorp and Wells Fargo would lose somewhere along the lines of 6.5% and 5.8% of their total loan portfolios, respectively, given the same economic scenario.

This doesn't mean investors should avoid credit card companies altogether. Doing so would be a mistake, as companies like American Express have handily outperformed the market over the long-run. Since the mid-1990s, for instance, American Express's shares have returned a total of 1,250%. That's leaps and bounds ahead of the KBW Bank Index, which has experienced a decline over the same period.

AXP Total Return Price Chart

What it does mean, however, is that investors need to be careful about which credit card companies to invest in. The trick is to identify the ones that have a history of surviving, and even thriving, throughout past economic downturns. This serves as potent evidence that they'll be able to do so again.

There are any number of ways to do this, however, the most straightforward approach is to examine the earnings of credit card companies throughout the financial crisis. By doing so, one clear winner stands out: American Express.

Among the major credit card companies that aren't simply payment processors like Visa and MasterCard, American Express was the only one to avoid losing money even at the nadir of the crisis. Discover Financial reported a $104 million net loss in the first quarter of 2010, and Capital One lost $1.4 billion in the fourth quarter of 2008.

These performances speak volumes about the risk management acumen of the three companies, and American Express in particular. They also reveal why Warren Buffett has long been an American Express shareholder. In short, any company that can generate double-digit yields on a loan portfolio when times are good without giving too much of that back when the credit cycle turns is a company that's worthy of investors' respect.