The recent banking panic caught a lot of investors off guard. The sector is going through its biggest shake-up since the 2008 financial crisis after the Federal Reserve implemented one of its quickest rate-hiking cycles ever.

However, some investors saw this coming and have sidestepped the recent troubles that hit banking stocks. You might not be surprised to hear that legendary investor Warren Buffett was among this select group.

The Oracle of Omaha has sold a lot of his bank holdings from his portfolio at Berkshire Hathaway (BRK.B 0.44%). Aside from its monster $30 billion position in Bank of America -- a company that is likely benefiting from this industry turmoil -- Berkshire has greatly reduced its exposure to the troubled sector. 

But there is another bank stock that Berkshire hasn't exited: Ally Financial (ALLY 2.50%). Even though it makes up less than 1% of its portfolio, Berkshire owns close to 10% of Ally's common stock outstanding and is the company's largest shareholder. What potential do Buffett and the team at Berkshire see in this online bank and automotive lender? Let's investigate.

Ally Financial: Sticky consumer deposits and automotive loans

Ally Financial was originally the financial subsidiary of General Motors, but was spun off as its own separate company after the 2008 financial crisis. The company runs a fairly simple business model by taking in consumer deposits and then lending out the money to other people through car loans; the spread between the interest it pays on deposits and the interest it charges on its auto loans makes up its net interest margin.

Since becoming its own banking entity, Ally has consistently expanded its customer base by offering convenient branchless banking and paying higher interest rates than big banks, such as Bank of America. It has grown its total customers for a whopping 56 consecutive quarters, closing out the first quarter with 2.8 million depositors.

A growing level of deposits gives Ally more funds it can lend out in automotive loans, which will increase its earnings power as long as it doesn't make bad loans to car buyers.

Historically, the company has been pretty good at making automotive loans, earning positive net income every year since 2014, even during the height of the COVID-19 crisis. 

ALLY Net Income (TTM) Chart

 Data source: YCharts ALLY net income (TTM)

The Fed is a known risk, but also an opportunity

There are two risks facing banks right now. One is the potential of fleeing depositors, which crushed Silicon Valley Bank and First Republic in recent months. The second is a tightening of net interest margins as the Fed raises the baseline interest rate for the entire economy.

Rising interest rates force banks to pay higher rates to depositors, which can narrow net interest margins (and consequentially hurt profitability). These two risks are the likely culprits for why Ally's share price is down 35% in the past year.

Ally seems to be doing fine on both these fronts. Its depositors have not pulled their money from the bank. In fact, the company is actually growing its deposits, adding $813 million just in the first quarter.

The company is feeling some pain on the net interest side of things as it has increased the interest rate it pays depositors, while it is still earning interest on automotive loans made in 2021 and early 2022 at low rates.

Luckily for Ally, its automotive portfolio turns over every two to three years, which gives it much more agility in changing the interest rates it earns on its loan book compared to a bank that makes 10-year real estate loans, for example.

The company has minimal exposure to real estate, which is where a lot of the pain in the banking sector is being felt. Last quarter, Ally's net interest margin was 3.54% compared to 3.95% in the same period a year prior.

Despite facing major headwinds with these interest rate hikes, Ally still has a positive net interest margin and has generated more than $1 billion in earnings over the past 12 months. Once the Fed stops this hiking cycle and this headwind dissipates, we should see Ally's net interest margin start to widen, which again would be positive for its long-term earnings power.

Why does Buffett like the stock?

Buffett and the Berkshire team probably see an opportunity in Ally Financial for a few reasons, all of which are general characteristics he looks for in his investments. One is that the stock is cheap. At a market cap of $7.9 billion, shares trade at a dirt cheap price-to-earnings ratio (P/E) of 6 based on Ally's trailing net income of around $1.4 billion.

Another reason is that the company has a great track record of consistently growing its deposits. Along with its current depressed net income margin, this should help Ally report higher net income, which will reduce its P/E even further.

Third is Ally's capital returns strategy. The company pays a healthy dividend (currently yielding 4.5%) and uses the rest of its excess capital to repurchase its common stock. Over the past five years, Ally's shares outstanding have been reduced by 30%.

Warren Buffett's partner, Charlie Munger, says that looking for cannibalistic companies (a euphemism for heavy repurchasers) is one of the keys to finding great stocks for your portfolio. Add this to the mix, and you can see why Buffett and Berkshire Hathaway are the largest shareholders of Ally Financial.