As of the time of this writing, the S&P 500 is down 1.74% year to date in 2015. Not a stellar year by any means, but certainly not the worst. Some stocks did well, some did average, and others, like the three bank stocks we will discuss here, did terribly in 2015.

The bottom of the barrel
Using a survey of 63 of the largest U.S. banks, three stand out as the worst banks of 2015. Each struggled for different reasons, but the end result is the same.

Company Name

YTD Stock % Change (as of Dec. 21, 2015

First Bancorp (NYSE:FBP)

-41.9%

SLM Corporation (NASDAQ:SLM)

-38%

Ally Financial (NYSE:ALLY)

-22.4%

For context beyond the performance of the broad S&P 500, we can look to the KBW Bank Index, which measures the performance of 24 of the largest U.S. bank stocks. That index is down 2.43% over the same period, worse than the broader markets but still far superior to our three worst bank stocks.

Out of the 63 large U.S. banks used to determine this year's worst bank stocks, the average percentage change year to date was 0.49%.

By whatever standard you use, it was a tough year for shareholders of First Bancorp, SLM Corp, and Ally Financial.

FBP Chart

FBP data by YCharts.

SLM Corporation, a.k.a. Sallie Mae
SLM Corp, the the specialized student loan bank was the second worst performer of 2015. In 2014, the bank spun off its loan servicing division into a separate public company, Navient Corp. (NASDAQ:NAVI). Navient has come under regulatory fire this year for its practices in collecting some of the student loans it services. Its stock is down 45.4% year to date as I write.

SLM's bear market run started in June as investors ignored the company's strong 8%-plus loan origination growth to that point in the year and the 38% increase in diluted earnings per share. They were likely worried that SLM Corp would be subject to the same regulatory pressures (and potentially similar financial repercussions) as Navient, despite being indemnified from any consequences after the spinoff.  

A turnaround for SLM Corp.'s stock in 2015 will depend on both the larger regulatory environment as well as the company's ability to get its costs under control to drive better earnings performance. One factor the company can control, while the other will be determined by regulators and the government.

A tough July for First Bancorp
First Bancorp is a Puerto Rican bank whose fate has been closely tied to the island's troubled economy for years now. The stock is down over 98% over the past 10 years, but has been on a roller-coast ride of highs and lows over its more recent history. The chart below gives a picture of the ups and downs over the past five years.

FBP Chart

FBP data by YCharts.

The Puerto Rican economy in general is the core of First Bancorp's problem. Investors have lost faith in the island's ability to find any growth and even pay its own government debts.

A perfect example came on June 30 of this year, when comments from the Puerto Rican governor, Alejandro Garcia Padilla, sent shares of First Bancorp plummeting. Padilla said that the government's debts "are not payable." First Bancorp shares were down nearly 15% by the end of that same day.

Because the bank relies on the individuals and businesses of Puerto Rico to originate loans, any turnaround story for this beleaguered bank starts with a turnaround in the Puerto Rican economy. For the bank to succeed, the island economy must have both demand for loans as well as borrowers with the ability to repay the debts. Given that the debt problems start at the government level itself, Puerto Rico remains a tough environment for any bank.

Ally Financial has been steadily falling since July, but spiked sharply lower to end the year
After a strong start to the year with impressive earnings and lots of optimism, Ally Financial's stock ran into a brick wall to end the year.

Ally is a major player in the auto loan market, a space that can be quite profitable when times are good but can be equally risky if loan repayment falters. Ally's management warned that charge-offs and delinquencies would increase in the fourth quarter, a sign of increasing problem loans.

It also faced a major headwind this year as General Motors ended its partnership deal with the bank and moved the financing of vehicles sold at GM dealerships to its own in-house lending unit. Ally's done an admiral job overcoming this change, but losing that business clearly hurt the company's top and bottom lines this year. The lender's third-quarter earnings declined to $0.47 per share versus $0.74 in last year's Q3.

Going forward, however, Ally's prospects will be much more closely linked to loan quality and new loan volume. Thanks to increased volume in other sales channels, Ally has already made up its lost revenue from the GM partnership. The company's core business looks to be strong heading into 2016, with improvements in efficiency, growth, and the prospect of getting regulatory approval to return capital to shareholders in 2016. Investors may have punished the stock in 2015, but of the three stocks here, it looks to me to have the best shot of turning things around next year.

Jay Jenkins has no position in any stocks mentioned. The Motley Fool recommends General Motors. Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.