Given Warren Buffett's impressive track record as an investor, Berkshire Hathaway's $368 billion public equities portfolio is a valuable source of potential investment ideas, especially the larger holdings. Near the top of that list, investors will notice that Bank of America (BAC -0.21%) and American Express (AXP -0.62%) make up a combined 18.1% of the portfolio.

Buffett and Berkshire have owned both of these financial stocks for quite some time -- but are either of them worth your investment dollars right now?

Bank of America

With total assets of $3.2 trillion (as of Dec. 31), Bank of America is one of the biggest financial services institutions in the U.S. Its massive scale and strong brand recognition help it attract low-cost deposits, a sticky source of funding that helps fuel loan growth.

Buffett has said that his favorite investment holding period is forever. There might be no other industry where investors can find these types of businesses as readily as in the banking industry. That's because banks are fundamental to the functioning of an economy, facilitating the movement of capital. Their essential services will always be needed.

Bank of America's business has hit a bit of a rough patch due to macroeconomic uncertainty and higher interest rates. Net interest income declined 5% in the fourth quarter, as Bank of America started paying higher yields on its deposits that weren't entirely offset by higher rates on its loan products.

Shares trade right around book value right now, which indicates that they might be fairly valued. But this valuation is notably more expensive than the stock carried just three months ago.

American Express

American Express is similar to part of Bank of America's business in that it approves borrowers for credit cards, lends its customers money, and earns interest and fees in doing so. But it's also like Visa and Mastercard in that it operates the underlying payments network through which its transactions are processed. This means the business benefits from a strong brand, particularly among affluent customers, as well as network effects.

Amex has lately been posting much better financials than Bank of America. Its revenue (net of interest expense) and diluted earnings per share both rose by 14% in 2023, with double-digit percentage gains expected for this year as well. Cross-border spending activity is fueling its growth.

What makes American Express stand out among other card issuers is that its net charge-off rate is typically the lowest among its peers. Because it targets a higher-income customer group and makes money from a spending-centric model, the company is resilient in the face of economic cycles.

Its share price has doubled in the last five years, but the stock trades at a price-to-earnings ratio of about 19 today, which is in line with its trailing five-year average.

What stock should you buy?

It would be understandable for investors to want to follow in Buffett's footsteps by investing in both of these businesses. After all, the Oracle of Omaha is certainly an expert when it comes to analyzing companies in the financial services sector. But it's best to think about stock picks critically and independently.

I fully acknowledge Bank of America's dominance and competitive advantages in its industry. And I understand that the business has made it through numerous recessionary periods in the past. However, I'm just not a fan of cyclical companies. Moreover, Bank of America provides largely commoditized products and services. This makes me not want to own its shares.

On the other hand, I am a fan of American Express. While it can be impacted by the swings of the broader economy due to its lending operations, the fact that it also operates a lucrative payments network intrigues me. And this is one of the world's strongest and most valuable brands, which somewhat differentiates it within its industry.

Of these two top Buffett stocks, I view American Express as the better one to own over the next five years.