Losing Costco (COST 0.17%) was quite the blow. Costco members made up 8% of spending on American Express' (AXP 0.07%) network in 2014, of which 70% was in other stores where AmEx collected a full fee. Even worse, about 20% of the company's loan book came from Costco cards.

But while these issues will challenge American Express on the revenue side of the equation, AmEx does have some room for improvement when it comes to minimizing expenses. Recently, the company has taken to its banking units to lower its cost of funds.

American Express has FDIC-insured banks known as Centurion Bank and American Express Bank, which would rank among some of the largest banks even as standalone companies. Its banks hold $49.3 billion in customer deposits, up 12% in just nine months. Of these, 57% are traditional savings account balances.

When it comes to taking deposits, American Express isn't all that much different from the likes of Ally Financial. Because of limited features and a lack of branches, American Express pays dearly for deposit accounts. The average deposit costs it roughly 1% per year.

While a full percentage point of interest may seem expensive, it pales in comparison with the company's long-term borrowings. When American Express taps the debt markets, it pays 2.25% per year, on average. (Its "long-term" debt isn't really that long in duration.)

Recent growth in the bank isn't accidental. Since the end of 2014, the fastest-growing deposits are "third-party" deposits, another phrase for brokered deposits. Costlier than savings accounts, brokered deposits are still less expensive than debt, thanks to FDIC insurance. Take notice that deposit growth kicked into overdrive in the first quarter of 2015, when news broke of Costco's decision.

Deposits now make up 37.1% of the company's liabilities, up from 31.9% at the end of 2014. Long-term debt is trending down, now making up 36.6% of total liabilities, versus 41.9% at year-end 2014.

Of course, American Express can't solve all its woes with financial engineering. Eliminating the entirety of its $48.7 billion of long-term debt and replacing it with deposits -- a Herculean task no one should expect -- would save it about $600 million in annual pre-tax interest expense. It made three times that amount in the third quarter of 2015.

There are some risks to accumulating deposits to offset long-term debt. Should rates rise, rate-sensitive deposits will expose the company to higher borrowing costs it might have otherwise locked in with long-term debt financing. Its borrowers won't necessarily pay a higher rate to offset higher deposit interest rates, given the competitive landscape for credit card balances.

But it is interesting to see American Express springing to action to save a few million dollars here and there. It needs it. Its long-standing deal with Costco comes to an end in March.