The stock market has rebounded dramatically since the COVID-19 pandemic shut down the U.S. economy in March. In fact, despite plunging by about 40% at its low point, the S&P 500 is actually up by 2.5% for the year.

However, not all stocks have rebounded like this. Here are three in particular that most investors seem pessimistic about, but I'm loading up on.

Person at desk looking at papers.

Image source: Getty Images.

Iconic assets at a discount

One of the worst-performing real estate investment trusts, or REITs, in the entire stock market this year has been New York City office REIT Empire State Realty Trust (NYSE:ESRT). In addition to the iconic Empire State Building, the company owns a portfolio of mostly office properties in the New York City metro area.

It's not difficult to see why the company has underperformed. New York City office real estate was grappling with oversupply and the effects of property tax reform even before the pandemic. The struggles of coworking office companies like WeWork (which Empire State has no exposure to) added to investor worries.

New York was the U.S. epicenter of the pandemic earlier this year. Many urban office tenants are still having their employees work entirely remotely and many experts think there will be mass vacancies in office buildings when the pandemic is over. To make matters worse, the company had just completed a multiyear renovation of the Empire State Building observatory before the pandemic forced its shutdown. Talk about bad timing. 

However, I believe the fears about office real estate are overblown, especially in New York City. Facebook (NASDAQ:FB) just signed a lease for 730,000 square feet of Manhattan office space, and other companies are actively expanding their physical footprint in cities as well. Recent surveys have indicated that most people want to work in offices, at least on a part-time basis. There's no stock I've added more shares of during the pandemic than Empire State Realty Trust, and it's one that I feel will have a very bright future once the pandemic is in the rearview mirror.

One of the best banks has been among the worst performers

Banks have been one of the worst-performing types of businesses during the pandemic, and consumer-focused banks like U.S. Bancorp (NYSE:USB) have performed worst of all. The short explanation is that there's a tremendous amount of uncertainty when it comes to pandemic-related loan losses, and consumer banks don't have investment banking divisions (which do better during volatile markets) to help offset losses.

That said, there is absolutely no reason why U.S. Bancorp should be underperforming the S&P 500 by 40 percentage points this year. The bank has an excellent track record of responsible lending and avoiding risky financial instruments -- in fact, while just about every other bank was holding tons of risky derivative securities in the mid-2000s, U.S. Bancorp focused on its tried-and-true lending operations instead, and remained profitable throughout the financial crisis. While the bank might take a short-term coronavirus hit to its revenue, there is no reason for patient long-term investors to worry about the bank's health. Now could be a great chance to get into this historically expensive bank stock at a rare discount.

Physical retail isn't dead

I've said several times that the supposed demise of physical retail has been overblown. While brick-and-mortar retail certainly needs to make some changes to adapt to the modern economy, the thought that people won't want to leave the house to shop simply because e-commerce exists is absurd to me. Since the pandemic started, I've bought shares of several retail real estate companies, but Tanger Factory Outlet Centers (NYSE:SKT) seems like the best out-of-favor bargain in the space. Even after a recent pop, Tanger is still 56% lower than where it started the year.

Some of Tanger's tenants have gone bankrupt as the result of the pandemic, and there could be more pain to come. But outlet space is relatively easy to repurpose for new tenants, and with the desire to add more "experiential" retail components to its malls, this could actually be a long-term positive for the company.

Recent numbers show that the worst is over for Tanger. The company collected 85% of its billed rent in August, which implies that substantially all tenants who haven't declared bankruptcy are paying as agreed. Customer traffic is now at nearly 90% of its year-ago levels. Most importantly, Tanger is now cash flow positive and has more than $600 million in liquidity (more than its entire market cap).

I love these stocks, but expect some volatility in the short term

As an investor who plans to hold these stocks for the next decade at a minimum, I'm very optimistic about the future of these three companies. However, it's important to mention that I have no delusions that the path to great long-term performance will be a smooth one. All three of these are likely to be quite volatile, especially as the COVID-19 pandemic is an ongoing situation, and there's no way to know what the economic effects (if any) on these businesses and the U.S. economy as a whole will be.

So, if you're a patient investor with a relatively high appetite for risk, these stocks could be worth a look. I've bought shares of all three since the pandemic hit, and I'm confident that I'll look back years from now in disbelief at the bargain prices I paid for them.