The online travel industry is extremely competitive. Companies like Priceline.com (BKNG -0.45%), Expedia (EXPE 0.55%), and Orbitz Worldwide (NYSE: OWW) compete to sell hotel and airline inventory through their websites and apps. This is far more convenient for consumers than booking directly, and often results in lower prices as well.

These three stocks have soared over the past five years as the online travel industry has grown rapidly. Priceline alone has advanced by more than  1,600%. Revenues are growing at double-digit rates, and the margins achieved, by Priceline specifically, are impressive. But can the rapid growth and high margins continue? Or are investors in for a shock?

The three big players
Priceline is the largest online travel company, and its success has been tied to its aggressive expansion into fragmented international markets. International bookings make up around 85% of total gross bookings, with markets in Asia and Europe driving growth. The company has made acquisitions, like Booking.com, TravelJigsaw, and Kayak, in order to drive growth. The most recent quarter saw revenue grow by 26% year over year.

Priceline is by far the most profitable company of the three mentioned. The operating margin in 2012 was 35%, with the net income margin coming in at an astounding 27%. Net income has increased by nearly a factor of 10 since 2007, largely the result of Priceline expanding into new markets well before the competition.

Expedia is the second largest company, with around $4.4 billion in annual revenue compared to Priceline's $5.9 billion. Expedia is far more concentrated on the United States market, with international revenue making up only 43% of its total. The U.S. market is fairly mature, and competition is far more fierce than in many international markets. This has caused Expedia's margins to contract over the past few years. In 2012, the operating margin was 10.7%, compared to 19.9% in 2007. The net income margin fell to 7% from 11% during that same time.

Expedia is still growing quickly, with revenue jumping by 16% year over year last quarter. But Priceline has outflanked the company by pushing into new markets, and Expedia is playing catch-up. Deutsche Bank recently downgraded Expedia, causing the stock to plummet by as much as 8% in a single day.

Orbitz is far smaller than Priceline and Expedia, with revenue of around $800 million annually. Over the past decade, the company has not had a single year where net income was positive. Scale is likely the problem, as online travel is an industry where size matters. The company grew revenue by 12.4% year over year last quarter, slower than both of its competitors, and it's inability to consistently turn a profit is a big concern.

Orbitz is pushing into international markets, though. The company has partnered with Taiwan-based Eva Air to provide hotel booking for websites operated by the airline. This is similar to deals struck between Orbitz and other airlines, including Air Canada and Hawaiian Airlines. These types of partnerships are probably the best bet for the company, as competing directly with Priceline and Expedia is unlikely to bear fruit.

Priceline's big problem
Unfortunately for Priceline, there's a big difference between having high margins and maintaining high margins. Priceline had a first-mover advantage in many markets in Asia and Europe, allowing the company to achieve incredible margins due to weak competition. But as Expedia expands into these markets, there is no reason to believe that these high margins will last.

Priceline doesn't really have any competitive advantages. Management has been extremely successful in taking advantage of fragmented international markets, but there's nothing special about Priceline compared to Expedia. The U.S. market is extremely competitive, resulting in declining margins, and these international markets will eventually look exactly the same.

With no competitive advantage, Priceline's margins cannot remain elevated. Competition will eat away at its margins, and the company's epic earnings growth will slow. Priceline trades at 37 times last year's earnings and 25 times estimates for next year's earnings. If revenue growth continues to be around 20%, then margins need to at least hold in order to justify this valuation. What happens if they don't? Shares of Priceline will fall off a cliff.

The average analyst estimate for five-year earnings growth is 20.3%, very close to the expected revenue growth rate. This is far too optimistic, and it seems that analysts are falling prey to the assumption that trends from the recent past will continue indefinitely. Anyone who believes that Priceline can maintain its margins for the foreseeable future is in for a rude awakening.

Expedia trades at 15.5 times expected earnings, but with margins falling, the stock seems too pricey to consider. It's well off of its highs, but I'd want a significant margin of safety before buying shares.

Shares of Orbitz have skyrocketed this year, moving from $2 per share to $9 per share. This meteoric rise makes me believe that the stock has become overpriced, especially since profitability is a big question mark. A good argument could be made at $2-$3 per share, but not now.

The bottom line
Priceline operates in an extremely competitive industry, and eventually the markets in which it currently enjoys little competition will cease to offer such favorable conditions. Margins can go nowhere except down in the long term, and while 20% revenue growth can continue to drive earnings growth, analyst expectations are overly optimistic. Expedia has room for improvement, but the stock is still too expensive to consider. And Orbitz needs to show that it can actually be profitable before it can be considered for an investment.