Image source: iStock/Thinkstock.

There's one guy who isn't sweating bullets about Brexit's impact on bank stocks: Warren Buffett.

The billionaire chairman and CEO of Berkshire Hathaway (BRK.A -0.59%) (BRK.B -0.74%) has consistently used times like these to pick up shares of the very same bank stocks that everyone else runs screaming from when a downturn in the economy sends them tumbling.

Take this selection from an article announcing Buffett's original investment in Wells Fargo (WFC 0.89%) in 1990 (emphasis mine):

Analysts said the purchase of Wells Fargo stock by Mr. Buffett was a sign of his confidence in the company even though its stock has lost almost half its value this year as bank stocks generally have lost favor.

Here are a couple of paragraphs from an article published two days later, underscoring how concerned investors in banks were at the time (pay particular attention to the second paragraph):

Almost by definition, the best time to buy is also the most difficult time to make the decision. It is when things look bleakest, when almost everyone is bearish, that prices hit bottom. For the entire stock market, that time may not have arrived yet. But in bank securities, it is to be fervently hoped that a bottom is near, if not at hand, and that Mr. Buffett, who built his reputation by buying insurance and newspaper stocks at a time when few wanted them, will be proved right again.

The alternative conclusion is that the banking industry will go the way of the savings and loans, becoming a Government ward. If that is to happen, it is hard to be bullish about anything.

And here's a selection of what Buffett himself had to say in that year's letter to Berkshire Hathaway's shareholders:

Our purchases of Wells Fargo in 1990 were helped by a chaotic market in bank stocks. The disarray was appropriate: Month by month the foolish loan decisions of once well-regarded banks were put on public display. As one huge loss after another was unveiled -- often on the heels of managerial assurances that all was well -- investors understandably concluded that no bank's numbers were to be trusted. Aided by their flight from bank stocks, we purchased our 10% interest in Wells Fargo for $290 million, less than five times after-tax earnings, and less than three times pre-tax earnings.

Fast-forward to today, and the chart below shows how Berkshire Hathaway's investment in Wells Fargo has since performed compared to the S&P 500:

^SPX Chart

^SPX data by YCharts.

Buffett essentially did the same thing two decades later, in 2011, by investing $5 billion of Berkshire Hathaway's money in Bank of America (BAC 0.45%) near the post-crisis trough in the bank's share price.

Here's how an article published the month before Buffett's investment described Bank of America's predicament:

 It is a company that faces enormous legal risks. Critics claim the CEO is not the right man for the job. Its stock price has tanked.

Here's what Buffett had to say at the time:

Bank of America is a strong, well-led company, and I called Brian to tell him I wanted to invest in it. I am impressed with the profit-generating abilities of this franchise, and that they are acting aggressively to put their challenges behind them. Bank of America is focused on their customers and on serving them well. That's what customers want, and that's the company's strategy.

And here's a chart that puts the timing of Buffett's bet on Bank of America in perspective (remember, he bought the shares in the middle of 2011):

BAC Chart

BAC data by YCharts.

It's times like these that real investors make money by investing in companies with great franchises and then allowing the law of compounding returns to work its magic in the decades ahead. As Buffett says: "Investors pay a high price for a cheery consensus."