Gold has fallen roughly 45% from the highs it reached less than five years ago. It's been a painful drop that's made some market watchers question the value of gold to investors. But not every investor is willing to say, "This time is different." Here are some of the billionaires bucking the trend with contrarian bets that gold will be going up again.

From Soros with love
One big name that's been shifting into gold is Stanley Druckenmiller. Don't feel bad if you don't know that name -- he stepped out of the money business in 2010. Now he just manages money for himself. But he's got so much money that he still has to report large stakes to the SEC. And as of the end of the third quarter, he was one of the top 10 holders of SPDR Gold Trust (GLD 0.40%).

Gold Bars. Image source: Newmont Mining.

So, why should you care about some rich guy who doesn't manage money for anyone but himself? His pedigree. Druckenmiller was the chief strategist for George Soros, a name you've probably heard of. If Soros trusted Druckenmiller with his billions, he's probably someone worth listening to. It's worth noting that Soros was known for taking big-picture bets that worked out in big ways; Druckenmiller helped make those calls.

And, while Druckenmiller shut his own hedge fund, Duquesne Capital Management, at the turn of the decade, that actually makes his gold bet all the more interesting. Druckenmiller is putting a fairly large chunk of his $4 billion or so fortune into his gold bet through his personal investment vehicle, Duquesne Family Office. His roughly 2.9 million shares of SPDR Gold Trust are worth around $300 million dollars.

Not giving up
Another big name that's still got a stake in the barbarous metal is John Paulson and his hedge fund group Paulson & Co. According to regulatory fillings, the 9.2 million or so shares of SPDR Gold Trust his company owns are worth around $900 million at recent prices. That's a lot of money.

The thing about Paulson is that one of his best trades came out of the precious metals space. Indeed, he moved aggressively into gold in 2009 and 2010, just as the metal was heating up. He reportedly made as much as $5 billion on that single trade. But that's not the only big-picture call he's made that's worked out -- he also bet against housing in 2007, earning a reported $4 billion profit.

To be fair, he doesn't have as large a stake in gold right now as he did in 2010. But a nearly billion-dollar position isn't chump change, either. What's interesting here is that in late August, Paulson called gold fairly valued. It's only kept falling since then.

Moreover, he has noted that gold has a place in portfolios as insurance against the unexpected. With the Fed starting to raise interest rates, the risk of unintended consequences looks like it just started to heat up. Keep an eye on Paulson's gold bet...if he starts to up his stake, you'll want to take notice.

Do you understand history?
That sentiment is backed up by Ray Dalio, founder of Bridgewater Associates. He's been quoted as saying, "If you don't own gold, you know neither history nor economics." Basically, gold is the insurance play, which is what Paulson has been saying. Dalio, for his part, appears to be worried about what's going on in the world's markets today, recently telling Bloomberg that he believes the global economy is in what could best be described as a fragile position.

Unlike the two billionaires above, however, Dalio isn't getting his exposure to gold through the SPDR Gold Trust, he's investing directly in gold miners Newmont Mining (NEM 2.15%), Barrick Gold (GOLD 1.67%), and Goldcorp (GG). While none of these positions are large in their own right, for Dalio at least, the fact that he's keeping them around speaks volumes. And taken together, they start to add up.

Maybe gold isn't all bad
Druckenmiller's gold bet is pretty large, making a statement that he's seeing something the rest of the market doesn't. That's worth a deeper dive since his work history includes helping one of the world's most famous hedge fund managers invest. But even if you don't think a bet as large as Druckenmiller's makes sense, having a gold "insurance policy" might still be a good idea as the Fed shifts the United States toward a new economic reality -- one that will have major ramifications for global markets. Dalio and Paulson would likely fall into this camp right now.

If either of these two billionaire investors starts to really kick up the gold bet, though, you'll want to pay far more attention to a metal that some, perhaps too many, consider an investment relic.