If you carry a balance on your credit card, then you know how oppressive the interest rate can be -- in many instances, exceeding 20%. But why is this? How is it possible banks can get away with charging such usurious rates?

In short, they have no other choice. Check out the following chart, which compares how much the Federal Reserve believes a typical bank will lose on different types of loan portfolios if and when the economy turns sour.

As you can see, the loss rate on credit card loans (15.2%) is more than twice the average (6.9%). It's three times the rate of loss from a portfolio of first-lien mortgages (5.7%). And it's almost twice that of the third riskiest category, commercial real estate loans (8.4%).

The damage that can be wrought by a portfolio of credit cards is illustrated by Citigroup's (C -0.32%) performance during last week's stress tests.

According to the Fed, if the economy were to become "severely adverse," Citigroup would have to write off $24.8 billion in credit card loans. That equates to nearly 45% of Citigroup's estimated loan losses under the most extreme of the stress test's two hypothetical scenarios.

And similar situations would prevail at other large credit card issuers. JPMorgan Chase (JPM 0.49%) would stand to lose $14.4 billion, while Bank of America (BAC -0.13%) would be on the hook for $13.7 billion.

Bank of America CEO Brian Moynihan addressed this in an interview with CNBC at the end of last year:

At the high point [before the financial crisis] we probably had $250 billion in credit card related balances. We now have $100 billion. When we had $250 billion, everyone said what a great business; you are making this much money. We proceeded to charge off tens of billions of dollars of charge-off receipts.

One can assume from all of this that credit card loans fail more frequently. This shouldn't be a surprise, right? If you've ever applied for a mortgage and a credit card, then you know how much easier it is to get the latter. The implication being, many people who probably shouldn't qualify for a credit card are getting them nonetheless.

On top of this, when credit cards do go into default, the loss to the bank is much more extreme than, say, when a mortgage defaults. This is referred to as loss severity, and it follows from the fact credit card loans aren't secured. If they fail, unlike a mortgage or car loan, there isn't collateral that can be confiscated in lieu of payments.

The takeaway here is twofold. First, the decision to charge such high rates for credit cards balances is justified by the data. If you don't want to accrue the cost, don't carry a balance. Second, banks with large credit card portfolios have different risk profiles than those that don't. We saw this in the last recession, with respect to both Bank of America and Citigroup's performances, and we'll see it in the next downturn.