A lot of investors shy away from financials. Companies that make money lending to others, whether consumers or businesses, are inherently risky, as you can never be certain how the loans will perform. Those who suffered from bank failures during the past spring know these risks all too well. With underwater loans and fleeing depositors, several sizable banks saw their shareholders get wiped out in short order.

The short-lived banking panic has led to an investor exodus from the sector in 2023. Many bank stocks are now down 30% or more while the broader market has soared this year. But clearly, not all banks are headed to zero, creating some fantastic buying opportunities for contrarian investors. Ally Financial (ALLY 0.41%) looks to be one of these bank stocks. Here's why the automotive lender can make you a fortune in dividends if you buy now and hold for the long haul. 

Major headwinds (that will subside)

At its core, Ally Financial has a simple business model: It takes in money from depositors, pays interest to keep them around, and lends these deposits out to people who want to buy cars. It earns a spread -- known as net interest margin (NIM) -- on the interest it pays versus the interest it receives, which is how it makes a profit.

Like other banks this year, Ally has faced pressure on the deposit side of its business, as the Federal Reserve has gone into its fastest interest rate-raising cycle in history. The situation has forced Ally to rapidly increase the interest it pays to depositors. In the third quarter, it paid more than 4% in interest to its depositors, compared with just 1.58% a year ago. The interest it earns on its loans is repricing higher, but will take multiple years to recover, given that it still holds loans made in 2020 and 2021, when the Federal Reserve had interest rates near zero.

Long story short, the interest it pays all depositors immediately reprices whenever it decides to raise rates, while the interest it earns from its loans takes multiple years to cycle through. This situation has compressed Ally's NIM in recent quarters, leading to lower net earnings. In the second quarter, Ally posted a pretax income of $228 million, versus $417 million a year ago.

The good news is, it looks as if these headwinds are about to turn a corner. Officials at the Federal Reserve say they're unlikely to raise rates much further, if at all, with inflation getting under control. As Ally's loan book gets repriced, this situation will eventually lead to NIM expansion, which management expects to happen within the next year or two. The next few quarters could still look rough, but Ally has been able to weather the storm. It's still generating a profit, is as well capitalized as any bank out there, and has access to tens of billions in liquidity if the automotive market goes into a downturn.

Great track record of deposit growth

The lifeblood of a consumer bank is its deposits. Ally has done quite well growing its customer numbers and overall deposits in the past decade-plus, giving it more firepower to make automotive loans. It currently has 3 million bank deposit customers and has grown its customer count for 58 consecutive quarters. With a proven playbook of running a digital-only bank with lower overhead costs than the traditional players, Ally is able to offer higher interest rates (read: 4% right now) while still generating a profit.

I see no reason this trend can't continue over the next 10 years, and I  wouldn't be surprised to see Ally double its customer count over the next decade or so. Doing so will allow it to expand its loan book and grow its earnings. 

ALLY Dividend Per Share (TTM) Chart

ALLY Dividend Per Share (TTM) data by YCharts

Where is the dividend headed?

Ally has been a consistent dividend payer, with a dividend per share of $1.20 at the moment. That's approaching a dividend yield of 5% with the share price around $24. Even with earnings in a cyclical downturn, Ally's earnings per share (EPS) are still well above its dividend payout, at $3.66 over the past 12 months. Yes, that's a dirt cheap price-to-earnings ratio (P/E) of 6.6, which shows how pessimistic people are on banks right now.

This spread of Ally's EPS and dividend payout means a few things. First, it shows that its dividend is sustainable and at minimal risk of being cut. Second, it means that Ally can build up its capital base over the next few years, allowing it to eventually repurchase some of its outstanding shares. In the past five years, it has shrunk its share count by around 27%, but has paused its buyback program as it waits for these headwinds to subside. 

Lastly, and most importantly, it indicates that Ally will have plenty of room to grow its dividend per share over the next few years and beyond. With an already attractive-looking yield of 5%, I think Ally stock is a fantastic buy right now.