There's a war over debit card fees. Retailers say they're too high. Banks say they help fund the high cost of administering a checking account. Last week, a judge sided with retailers, ruling that fees should fall, perhaps by as much as half.

In response, investors sent shares of major banks and card processors -- namely, Visa (V 0.07%) and MasterCard (MA 0.07%) -- down on the news.

Here's how debit card fees will drive banking stocks.

The interchange fee disease
Each time you swipe a credit or debit card, the bank that issued your card charges a processing fee to debit your account. This interchange fee is gravy for the issuing bank, since it can earn a small fee on millions of transactions every day.

Interchange fees are also an indirect boon for Visa and MasterCard. In exchange for access to their card network, the card processors add their own fees on top of interchange fees. When interchange fees are high, banks have an incentive to issue more debit cards. More debit cards mean more revenue potential for the payment networks, too.

When the Fed took aim at interchange fees, it didn't take direct aim at Visa or MasterCard. Interchange fee revenue flows to the issuing bank. If you bank at Wells Fargo, your bank would receive the maximum interchange fee of $0.24 for each transaction. These fees add up to big money. In its most recent quarter, Wells Fargo earned 4% of its total income from card fees. Another 6% came from deposit service charges.

Source: Visa annual report.

The effects of lower interchange fees were already mostly priced into banking stocks. Major retail banks added monthly service charges to many accounts to counter lower revenue from interchange transactions last year.

Who loses from lower fees
New cuts to interchange fees may start affecting the processors, a group that largely escaped lost revenue when the Fed slashed the maximum fee.

If interchange fees dip further, banks may hold back on issuing debit cards altogether, especially to accounts with low balances, which are less profitable for banks. That puts a strain on the business model of Visa and MasterCard, which rely on the frequent use of millions of debit cards to earn network fees on each transaction.

More importantly, banks may seek financial incentives from card processors to make up for a lower maximum debit card fee. That could crush margins for processors as their share of each payment dwindles to keep banks interested in debit card programs.

Finally, each time the Fed brings fees back in the spotlight, it shines a light on Visa and MasterCard's powerful duopoly. The interchange fee is only half the equation; regulators may set their sights on Visa and MasterCard's slice of processing fees to drive down retailers' costs of payment processing.

In the past year, Visa generated roughly 25% of its revenue from debit cards in the United States. MasterCard, which is more established globally and a small player in signature debit cards, owes 12% of its revenue to American debit cards, according to an analyst from Macquarie Securities.

If the Fed follows the suggestion of the ruling judge to limit interchange fees to $0.12 per transaction, Visa will shoulder the most risk of falling interchange fee revenue. Investors who want geographic and product diversification would be better suited to consider MasterCard in such a scenario.

When the Fed goes back to the drawing board on fees, it will be a make-or-break moment for card processing companies. Investors should carefully watch as the interchange fee debacle unfolds. If the Fed shoots for significantly lower fees for banks, the debit card may become less convenient or accessible for millions of fee-generating Visa and MasterCard customers.