I'm no doomsday prophet, nor am I pessimistic about the future -- especially when it comes to Amazon (AMZN 0.23%) stock. In fact, few investors would lose as much of their portfolio as I would if the company's stock fell: it currently accounts for 21% of my family's real-life stock holdings.

I didn't plan for it to be this way. The shares we bought over seven years ago have just kept growing and growing and growing. I have no intention to sell those shares now, but when you're in the situation I'm in, it's enormously valuable to investigate the bear thesis. So that's what I'll be doing today.

Businessman standing on falling diagram and peering into the future

Image source: Getty Images

Amazon stock's valuation is sky-high

The first and most obvious reason that Amazon's stock could fall is because it is ridiculously expensive by almost any traditional metric. While Wal-Mart (WMT 0.09%) is far from the same company, it holds a commensurate level of significance globally. For that reason, let's compare the valuation of the two to give you an idea of just how expensive Amazon's stock is.

Company P/E P/FCF P/S PEG Ratio
Amazon 295 78 3.5 13.8
Wal-Mart 22 15 0.6 3.8

Data sources: Yahoo! Finance, E*Trade. P/E given using non-GAAP earnings.

Depending on the metric you choose, Amazon is anywhere between four and thirteen times more expensive than Wal-Mart. You won't find me trying to make an argument that Amazon shouldn't have a premium price -- but that's not the point. The point is that this is an expensive stock with high expectations baked in.

If there's any trouble on the horizon (more on that below), the stock could react disproportionally to the news. That'a a reality shareholders need to understand.

A sinking tide lowers all ships

There's no doubt that e-commerce still has a long way to go. In the United States, e-commerce still makes up less than 10% of all retail sales. 

Census Bureau chart of e-commerce statistics

Data and image source: U.S. Census Bureau

But that doesn't change the fact that if there's a slowdown in the economy, Amazon shareholders will feel the pain. Middle-to-upper-middle class Americans make up the majority of Amazon Prime members. According to an analyst note from 2016, nearly 70% of American households making over $112,000 pay for the service. And all Prime members combined spend over twice as much on Amazon as non-Prime members.

If the economy falters, or the stock market enters a recession, the purse strings of these customers could tighten quickly. And even if Amazon maintains its market-leading position through a downturn -- which would be great news to uber-long-term investors -- that doesn't mean the stock won't suffer (see reason one above).

Indeed, during the Great Recession, Amazon's stock lost 61% of its value from peak to trough. A similar downturn would bring the stock all the way back to $450 per share. 

I'm not saying I think that's going to happen, but it's worth considering if the company occupies a huge part of your portfolio.

Competition is starting to encroach on the e-commerce king

For a long time, Wal-Mart's e-commerce ambitions had been left for dead. But momentum started shifting ever so slightly with the company's acquisition of Jet.com. That shift has reached a tipping point, as Wal-Mart's e-commerce division -- while growing from a much smaller base -- is starting to get people talking.

But that's not all. Fulfillment by Amazon (FBA) has become an important part of Amazon's burgeoning network effect. Because no one else has the same scale as Amazon, and because the website draws in so many shoppers, third-party operators are paying to list their products on the platform and have Amazon take care of the logistics.

While that may seem like a good deal, those third-party operators are worried: they are becoming overly reliant on Amazon, are forced to compete on ever-more crowded screens, and have little control over their branding.

That's where start-ups like Flexe come in. The company has created a marketplace for on-demand warehousing and fulfillment services that are cheaper than a company trying to build out its own fulfillment network on its own, and don't rely so heavily on a possible competitor. 

It remains to be seen if such efforts will stumble on a chink in Amazon's armor. But it is the type of movement that, if it gains momentum, could narrow the moat surrounding Amazon's business model.

I'm staying put...for now

Amazon has almost always had a sky-high valuation. And a market downturn doesn't scare me, since I plan on holding these shares until I retire 30 years from now. Further, while Wal-Mart and Flexe represent formidable competition, I'm not convinced they have the stamina to forgo e-commerce profitability to compete with Amazon over the long-haul. For these reasons, I'm not selling shares.

In fact, the biggest reason I might finally part ways with some Amazon shares is more personal: at over 20% of my holdings, it's imprudent to let it get much bigger. But if you understand the risks involved, and have a more diversified portfolio than I do, there's no reason to get some of your own skin in the game with Amazon.