Investors learned during the financial crisis of 2008-2009 that investing in bank stocks isn't for the faint of heart. But despite the unusual risk of bank stocks, the best banks have some of the widest profit margins of any business, regardless of industry.

Banks are risky
The first thing prospective bank investors need to appreciate is that banks are risky investments. Since the modern era of banking began with the National Banking Acts of 1863 and 1864, more than 17,000 individual lenders have failed, while thousands of others were snatched from the jaws of defeat by better-heeled competitors.

The main reason banks are so prone to failure is that they're highly leveraged: The average bank holds assets equal to 10 times shareholders' equity. When it comes to leverage, JPMorgan Chase (NYSE:JPM) and Wells Fargo (NYSE:WFC), two of the best stocks for investing in banks, tower over companies such as Apple, Costco, McDonald's, and Procter & Gamble, among others. The net result is that even a 10% decline in the value of a typical bank's assets can render it insolvent.

A second reason banks are unusually prone to failure stems from their tendency to borrow money at low short-term interest rates from depositors and institutional lenders in the money market, and then lend those very same funds out at higher long-term rates to prospective homebuyers, credit card holders, and businesses.

Known as "interest-rate arbitrage," this business model is lucrative when there's a large spread between short- and long-term rates. But when this spread tightens or inverts, a bank's profitability can drop sharply. In the early 1980s, for instance, the short-term benchmark federal funds rate topped out above 19%, while the long-term benchmark rate on 10-year Treasuries failed to eclipse 16%. That triggered a series of crises in which 1,848 banks and savings and loans associations went out of business between 1981 and 1991.

The use of short-term funds to acquire long-term assets is also problematic if a bank's creditors lose confidence in its ability to remain solvent. In that case, as we've seen time and again over the years -- from Continental Illinois (the first "too big to fail" bank) in 1984 to Washington Mutual (the largest bank failure in history) in 2008 -- depositors withdraw their funds en masse. This pulls the proverbial rug out from under a bank, forcing it to liquidate assets at fire-sale prices to satisfy the redemption requests. Known as a "bank run," this is the proximate cause of most bank failures.

Banks are also highly profitable
Now that you're sufficiently scared of bank stocks, let me tell you why the best bank stocks can nevertheless make rewarding investments. In short, thanks to the incredible difficulty of running a great bank, as well as the high capital and regulatory requirements associated with the industry, there are large barriers to entry. And with large barriers to entry come wide profit margins.

The following chart compares profit margins (net income as a percentage of revenue) across leading companies in a variety of industries. At the bottom are discount retailers such as Costco and Wal-Mart, which compensate for their low margins by selling a high volume of goods and services. On the other end of the spectrum are companies such as Apple, Google, and Facebook, all of which make a lot of money, though none of them matches the profit margins of lenders JPMorgan and Wells Fargo.

Just to be clear, JPMorgan and Wells Fargo aren't representative of the broader bank industry. Most banks -- particularly lenders such as Bank of America and Citigroup that are still tending to wounds sustained in the financial crisis -- earn a fraction of the money these two do. But JPMorgan and Wells Fargo not only survived the crisis, but even thrived as a result of it, both effectively doubling in size thanks to bargain-basement acquisitions of ne'er-do-well competitors -- JPMorgan bought Bear Stearns and Washington Mutual, while Wells Fargo acquired then-fourth-largest U.S. lender Wachovia.

Consequently, the goal of a shrewd investor shouldn't be simply to own bank stocks, but rather to own only the best. Here you can take a cue from Warren Buffett, whose Berkshire Hathaway has sizable stakes in Wells Fargo, U.S. Bancorp, and M&T Bank, among others (and Buffett himself is said to own shares of JPMorgan in his personal portfolio). All three of these banks have produced phenomenal returns over the past three decades, and there's every reason to believe they'll continue to do so in the future.