Image source: iStock/Thinkstock.

It's a tough time to be a bank. With interest rates still near historic lows, banks are earning significantly less from their loan and securities portfolios than they have in the past. This weighs on their profitability, which in turn weighs on their stock valuations.

The hope for the past few years has been that interest rates could soon be about to climb. This belief has been fueled by the Federal Reserve, which has done its best to convince consumers that the economy has recovered enough to handle higher rates.

But aside from a single 25-basis-point boost in December, the Fed has since taken a more conservative stance toward further rate increases. Indeed, it was this change in stance that has largely been responsible for the dismal performance of bank stocks over the past 12 months.

Since this time last year, the KBW Bank Index, which tracks 24 of the nation's leading publicly traded banks, has fallen by more than 10%. Meanwhile, the S&P 500 has climbed by 5% over the same stretch.

^SPX Chart

^SPX data by YCharts.

The problem now is that it's looking increasingly likely that we could be in for a long period of low rates. How long? That's impossible to say. However, there's precedent that suggests it could last for decades.

You can see this in the chart below, which shows the bank industry's worst nightmare. The chart plots Japan's discount rate, a key short-term interest rate benchmark in the Asian country.

Data source: Federal Reserve Bank of St. Louis. Chart by author.

In the mid-1990s, a real estate bubble burst in Japan, much like the one that popped in the United States two decades later. Interest rates in Japan have stayed low ever since, even dipping into negative territory this year.

It would be bad news for banks if interest rates here in the United States follow a similar path. And it would be particularly bad for banks that have positioned their balance sheets in anticipation of rising rates.

Bank of America serves as a case in point. Over the past few years, the North Carolina-based bank has stuffed its balance sheet full of mortgage-backed securities, which tend to generate much more income when interest rates climb.

The net result is that a mere 100-basis-point increase in short- and long-term rates would translate into $7.5 billion in additional net interest income for Bank of America.

Many other banks are in the same boat, though few of them to the same degree. For instance, JPMorgan Chase's net interest income would grow by $3 billion from a similar 100-basis-point boost in rates. And while Wells Fargo has shifted its strategy in anticipation of lower for longer rates, it too would benefit from tighter monetary policy, as the California-based bank explains in its latest quarterly filing:

In general, the Company is positioned to benefit from higher interest rates. Currently, our profile is such that net interest income will benefit from higher interest rates as our assets reprice faster and to a greater degree than our liabilities.

The point is that higher interest rates will go a long way toward helping banks earn the type of profits investors expect. But if Japan's experience is any indication, banks could be hoping and praying for this for years, if not decades.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.