When you think about it, the "Great Shakeout of 2000" moved its way up the Internet food chain in a fairly efficient manner. It was a kind of fiscal bubonic plague that started as a local problem and pushed its way outward until it had taken down everything in its path.

The scourge began at the most public face of the Internet: the commercial content and e-commerce companies. First the sickly fell, with such late, lamented, never-viable businesses such as ValueAmerica and iTurf collapsing. Next were some no-clothes-wearin' emperors like MicroStrategy (Nasdaq: MSTR).

The malaise quickly spread up to the top-level e-commerce and portal companies, such as Yahoo! (Nasdaq: YHOO), Amazon.com (Nasdaq: AMZN), and eBay (Nasdaq: EBAY). The companies that were already weak have shriveled into nothing, while those with more market strength lived to fight another day -- albeit with generally lessened expectations for future profits than before the Shakeout.

Next in line came the telecom carriers, again with the same results: The weakest have all but disappeared, while the stronger have survived but no longer possess the same swagger. Revenue projections have evaporated as things such as "dark fiber" commoditization, burdensome legacy networks, and oversized capital spending weighed heavily on these companies' perceived values.

ICG and Northpoint filed for bankruptcy. AT&T (NYSE: T) and British Telecom (NYSE: BTY) have seemingly evaporated, squashed under enormous levels of debt. Only the companies with guarantees of future revenue streams -- i.e., the Baby Bells -- have slipped by. Capital spending, while continuing, does not promise the lucre it once did for the carriers. Accumulating spiraling debt for commodity services does not an ideal business model make.

With the declining prospects for return on invested capital among the telecoms, it made sense that their buildout plans would be scaled back. This was the last stop in the Shakeout: the equipment manufacturers. As I've opined before, this is no fiber glut. In fact the demand for data transport continues to increase exponentially. It is an economic crisis, where the financial model does not exist for the technology needed to support itself -- at least, not to the point that carriers can continue to deploy facilities at breakneck speed.

And so the last chip fell, with variously situated networking companies -- such as Juniper (Nasdaq: JNPR), Redback (Nasdaq: RBAK), Broadcom (Nasdaq: BRCM), Lucent (NYSE: LU), and Nortel (NYSE: NT) -- crashing to earth one by one as their revenue pictures clouded up in a hurry.

Demand is growing while economics decline
Thus completed the rapid collapse of what were the darling sectors of 1999, as the economic justification for networkers' rapid growth and deployment fizzled. There is one remaining piece to this puzzle, though. Fact is, neither the carrier nor the consumer demand for fatter bandwidth have cooled off at all, and at some levels -- in particular the Metropolitan Area Networks (MANs) -- demand threatens to bust the existing infrastructure open at the seams. This makes for a quandary: Carriers find themselves unable to recoup their sunk costs fast enough, yet they cannot keep up with demand.

There is a simple, fundamental problem here: The quality of service and speed to market for the Internet has been so awful that, though customers are willing to pay for it, they are unwilling to do so in a way that allows carriers to pass on their up-front costs. Although hundreds of thousands of miles of fiber optics have been laid to augment MANs, even in the most-wired areas the Internet and data networks are considered of lower reliability than the switched network. Outages that would be considered unacceptable in voice communications are commonplace in data.

This has much less to do with the "idiot with a backhoe" phenomenon -- which essentially means carriers require robust routing to protect from man-made disaster -- than the fact that the network couldn't pass megabits of data per second through pipes that were only designed to handle kilobits. It grows worse when one looks at the edge of the network in the less-dense parts of the MAN, such as residential areas with their wild usage swings and their alphabet soup of xDSL, cable protocols, and fixed wireless.

These on- and off-ramps are overlaying facilities that were not designed to handle bandwidth-scaleable applications. A voice call requires a maximum of 56 kilobits per second, and since each person only has one mouth, the carriers could easily predict how much they needed to allocate to each channel. Not so for data usage, in which something like Napster can bring local nodes on a network to its knees.

From many to one
Network architecture must change to be optimized for Internet Protocol (IP), and it must do so in a way that is economical for the carriers. What's likely is the opposite of today's last-mile facilities, which have data applications laid over the top of voice networks. One such suggested path is migration from the multi-layered and infinitely complex MAN, with IP, ATM, SONET, and Dense Wave Division Multiplexing -- and their expensive interfacing requirements and provisioning difficulties -- to MAN-scaled Optical Ethernet. (There will not be a test on those terms; the issue is migration from a complicated, layered topology with competing protocols onto a simpler, singular one.)

While the core of the network is increasing rapidly toward the next generation, the MAN (or the "edge")  is not. That's not for lack of focus or addressable market, as there are billions of dollars to be spent -- Goldman Sachs put the number at $17 billion by the end of 2000 -- and billions to be made. But the existing infrastructure is so large, the market so vast, and the bandwidth demands growing so quickly that this area will continue to be a bottleneck in the BEST of cases for the next decade or more, for reasons of economics as well as sheer volume.

It makes for an interesting problem, one that may be solved by sheer supply/demand economics. The local carrier's point-to-point service is terrible? I'll go to Cable & Wireless (NYSE: CWP), Level 3 (Nasdaq: LVLT), or 360Networks (Nasdaq: TSIX), regardless of how much more it costs.

But there is another potential solution, one that is being addressed by a number of companies, such as Micromuse (Nasdaq: MUSE). Whereas Internet service until now may have been all about hardware, the economics of the next generation will be driven by software. The company that can utilize software to manage its network better will be at an advantage to the one that operates a faster core. There is an economic benefit to having a network that can run more intelligently, offer instant or remote provisioning, and can be customized for -- or even by -- the customer.

The end formation of the next-generation network is anyone's guess. In fact, anyone who tells you that they KNOW what the cloud will look like five years from now -- and what companies will emerge, gilded, from it -- is probably trying to sell you something you don't need. Investors considering this area should hitch their wagons to the leader in an area that will prove indispensable to carriers, customers, or networks.

As for the carriers themselves, the ones that provide more intelligent networks and faster provisioning are the ones that will be able to yield cash flow from their investments quicker and charge a premium for service -- a powerful combination for an investor to get behind in a hypergrowth market.

Your Turn:
The future of data networks is among the most highly debated topics in investing, as the sector lends itself to such high levels of standardization that there will certainly be big winners and an enormous level of shakeout. Got an opinion? Come speak your mind on the Fool on the Hill discussion board.

Fool on!

Bill Mann, TMFOtter on the Fool Discussion Boards

Bill Mann's dog once knocked over a tree while playing in the yard. At time of publication, Bill had beneficial ownership in Cable & Wireless and eBay. His other holdings can be viewed in his profile. The Motley Fool is investors writing for investors.