Investor enthusiasm seems to be elevated these days. With both the S&P 500 and the Nasdaq Composite Index in record territory, it's hard not to be optimistic.

Some businesses are doing very well for shareholders amid the bull market. Look at Ally Financial (ALLY 0.41%). The digital bank stock has soared 46% since the start of 2023, although it's been a volatile ride. Investors are hoping that the good times can continue rolling.

Should you buy Ally Financial stock right now? Let's look at bull and bear arguments.

Reasons to be bullish

Ally Financial is known first and foremost for its customer service. The online-only financial institution boasts an impressive retention rate of 97%, indicating how wonderful of a job it does taking care of account holders and their various needs. Even better, 299,000 customers now use multiple products that the company offers, nearly triple the amount at the end of 2019.

Looking at Ally's deposits proves this is a consumer favorite. As of Dec. 31, the business had retail deposits of $142.3 billion, which was up 3.3% year over year. I view this as an extremely positive development, particularly in light of the regional banking crisis that rattled the industry almost a year ago. Customers are showing with their savings that they trust this bank.

Ally benefits from a growing, low-cost, and sticky source of funding. If interest rates decrease this year, as the central bank is planning on doing, Ally's net interest margin could get a boost. That's because it can offer lower rates on its deposits.

Some investors are focused on receiving income from the stocks that they own. Ally has been paying dividends since 2016, with the current yield at a respectable 3.4%. What's encouraging is the fact that the quarterly payout went from $0.08 at the start about eight years ago to $0.30 today. This highlights management's confidence in Ally's financial strength.

The share buybacks are also noteworthy. Since the start of 2019, Ally has repurchased almost 30% of its outstanding stock, a move that helps to increase earnings per share.

Success with customers and an attractive capital allocation strategy might be key reasons that Warren Buffett-led Berkshire Hathaway owns a 9.6% stake in the digital bank. The Oracle of Omaha is an expert when it comes to analyzing financial stocks, giving Ally an invaluable stamp of approval.

Reasons to be bearish

A quick glance at Ally's loan book will raise some eyebrows. The company's primary lending product is retail auto loans and leases, which make up 50% of the interest-bearing assets. That's not surprising, given that Ally started as the financing subsidiary of GM. Of course, the business has started to diversify into other products, but car loans reign supreme.

Not only that, but 43% of Ally's auto loans in 2023 were originated by GM and Stellantis dealerships. There's not only product concentration but a heavy reliance on two main automakers.

I view this as a critical risk factor. Like its peers in the banking industry, Ally is a cyclical operation, overly exposed to changes in interest rates and the macroeconomic environment. But in this case, the company is also reliant on robust conditions in the auto industry, introducing other variables to the mix, most notably supply chain dynamics that can impact the price of cars.

Even more alarming, if we look at the recent past, it's strikingly clear that Ally has been a below-average investment. In the last five years, shares have returned 60%, with dividends included. During the same time stretch, an investment in JPMorgan, or even an S&P 500 index fund, would've nearly doubled your money.

The disappointing track record, along with the other bearish points, is a worrying sign that forces me to pass on Ally Financial stock right now.