Wireless security researcher VirnetX (AMEX: VHC) is perhaps the most polarizing stock on the market. Some investors love it for the promise of owning patents in the LTE 4G security space. Others wonder if the patents are as important as VirnetX wants them to be. From the latter point of view, the stock looks tremendously expensive.

Where will the company go in 2012? Let's figure it out together.

The promise
It's easy to see the attraction here. VirnetX is an offshoot of defense contractor SAIC (NYSE: SAI). Like Athena of Greek myth, it sprang into existence fully armed with a cache of security-related patents acquired from SAIC.

The company burst onto the computing scene with a big patent infringement win against Microsoft (Nasdaq: MSFT). Mr. Softy lost a jury trial and was ordered to pay $106 million in damages, but decided to fight on, and eventually settled the case for $200 million.

Of the settlement, 35% belongs to SAIC, according to the original license agreement. It was a very expensive campaign to wage, costing more than $40 million in legal fees. But everything worked out: the legal battle paid off, and VirnetX investors enjoyed a $23.6 million special dividend the next quarter. That's $0.50 per share for a meaty 8.4% payout.

If VirnetX can pull off that trick again, or else start licensing 4G security patents to every player in the mobile industry, investors will be very, very happy. And VirnetX is trying hard. There's one lawsuit involving six patents against networking giant Cisco Systems (Nasdaq: CSCO) and iStuff maker Apple (Nasdaq: AAPL). VirnetX has also declared several of its patents "essential" to various 4G mobile standards.

Where's the payoff?
So many irons in the fire and so much promise, and yet 24% of the float is sold short. Shares are trading some 36% below 52-week highs. Why aren't we all backing up the truck to these seemingly outsized returns?

Because there ain't no such thing as a free lunch, my friend. In order to cash in those chips, VirnetX must prove that its technology is indeed both innovative and important. And that might not be the slam-dunk that the Microsoft case implied.

For one, those "essential technology" claims are VirnetX's own. The relevant standards bodies have not, to the best of my knowledge, acknowledged or approved of any of them. That's like your Uncle Dennis declaring that his air guitar skills are essential to saving the economy. Might be true, but you just don't know. The technologies could be perfectly valid solutions to 4G security challenges, but network and phone builders might be able to design around them with totally different technologies.

As for the Microsoft case, it seems obvious that Redmond could have settled for much less -- or perhaps won outright with the right strategy. For one, paying nearly double the original damage award doesn't speak highly of Mr. Softy's performance in this case. For another, one of the patents (the '759 patent) from that proceeding has since been invalidated by a challenge from Cisco. Microsoft paid for that now-worthless patent.

Cisco has since lost another challenge of VirnetX patents. Maybe VirnetX has a legal leg to stand on after all. But it's hardly the bulked-up hunk of meat it looked like in 2010.

So in 2012, I not-so-bravely predict that VirnetX will win some battles and lose a few others -- the fickle fates of justice swing both ways. But there will also be an explosion of competing solutions developed by hardware and software giants with plenty of skin in the mobile game.

In the end, VirnetX goes home with a much smaller paycheck than earlier victories have indicated, and investors are left holding an empty bag. The current $1.35 billion market cap is just not appropriate for a development-stage company with minuscule proven sales and a highly uncertain future.

If you want to invest in technology licensing firms, why not go with a proven performer instead? OLED veteran Universal Display (Nasdaq: PANL) has several real-world contracts and is already profitable. Jack of all technologies InterDigital (Nasdaq: IDCC) is also profitable and cash-rich, with cheap shares to boot. And that's not all -- check out three more winning ideas in the smartphone wars of 2012.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.