NEW YORK (AP) -- Technology created an energy revolution over the past decade -- just not the one we expected.
By now, cars were supposed to be running on fuel made from plant waste or algae -- or powered by hydrogen or cheap batteries that burned nothing at all. Electricity would be generated with solar panels and wind turbines. When the sun didn't shine or the wind didn't blow, power would flow out of batteries the size of tractor-trailers.
Fossil fuels? They were going to be expensive and scarce, relics of an earlier, dirtier age.
But in the race to conquer energy technology, Old Energy is winning.
Oil companies big and small have used technology to find a bounty of oil and natural gas so large that worries about running out have melted away. New imaging technologies let drillers find oil and gas trapped miles underground and undersea. Oil rigs "walk" from one drill site to the next. And engineers in Houston use remote-controlled equipment to drill for gas in Pennsylvania.
The result is an abundance that has put the United States on track to become the world's largest producer of oil and gas in a few years. Just Thursday, the U.S. reported that oil imports have fallen to a 17-year low.
The gushers aren't limited to Texas, North Dakota, and the deep waters of the Gulf of Mexico. Overseas, enormous reserves have been found in East and West Africa, Australia, South America, and the Mediterranean.
"Suddenly, out of nowhere, the world seems to be awash in hydrocarbons," says Michael Greenstone, an environmental economics professor at the Massachusetts Institute of Technology.
The consequences are enormous. A looming energy crisis has turned into a boom. These additional fossil fuels are intensifying the threat to the earth's climate. And for renewable energy sources, the sunny forecast of last decade has turned overcast.
This is the story of how technological advances drove a revolution no one in the energy industry expected. One that is just beginning.
Exploring energy frontiers
The new century brought deep concerns the world's oil reserves were increasingly concentrated in the Middle East -- and beginning to run out. Energy prices rose to record highs. Climate scientists showed that reliance on fossil fuels was causing troubling changes to the environment.
"The general belief was that the end of the oil era was at hand," says Daniel Yergin, an energy historian and author of "The Quest: Energy, Security and the Remaking of the Modern World."
As a result, Wall Street, Silicon Valley and governments were pouring money into new companies developing alternative forms of energy that promised to supply the world's needs without polluting.
Even oil and gas companies got in the game. BP (BP +1.13%) had adopted the slogan "beyond petroleum" in 2000 and threw millions into its solar division. Shell (NYSE: RDS-A) partnered with another company to fire up a plant to convert agricultural waste into ethanol.
So strong was the lure of alternative energy that veterans of the oil patch began fleeing for start-ups.
In 23 years at Shell, David Aldous helped develop projections that showed a booming world population and rising energy demand. He also saw how hard it was for big oil companies to find enough oil every year to replace all they sold. He left Shell to join Range Fuels, a company that promised to turn wood chips into ethanol, in 2008.
"I felt we needed faster innovation," he says.
The race for new technology
But while the national focus was on alternatives, the oil and gas industry was innovating too. New technology allowed drillers to do two crucial things: find more places where oil and gas is hidden and bring it to the surface economically.
Large oil companies such as Exxon (XOM +0.83%), Chevron (CVX +1.10%), Shell, and BP turned up huge discoveries offshore in ultra-deep water with the help of better sensors and faster computers that allowed them to see once-hidden oil deposits.
Onshore, small drillers learned how to pull oil and gas out of previously inaccessible underground rock formations.
For most of the oil age, drillers have looked for large underground pools of oil and gas that were easy to tap. These pools had grown over millions of years as oil and gas oozed out of what is known as source rock. Source rock is a wide, thin layer of sedimentary rock -- like frosting in the middle of a layer cake -- that is interspersed with oil and gas.
An engineer named George Mitchell and his company, Mitchell Energy, spent years searching for a way to free natural gas from this source rock. He finally succeeded when he figured how to drill horizontally, into and then along a layer of source rock. That allowed him to access the gas throughout a layer of source rock with a single well. Then he used a process known as hydraulic fracturing, or "fracking," to create tiny cracks in the rock that would allow natural gas to flow into and up the well.
The United States, which was facing a gas shortage five years ago, now has such enormous supplies it is looking to export the fuel in large volumes for the first time.
The common wisdom in the industry was that the process Mitchell had invented for natural gas wouldn't work for oil. Oil molecules are bigger and stickier than gas molecules, so petroleum engineers believed it would be impossible to get them to flow from source rock, even if the rock were cracked by fracking. But Mark Papa, the CEO of a small oil and gas company called EOG Resources (EOG +1.89%), didn't accept that.
"The numbers were too intriguing, the prize was so big," he remembered.
He thought there could be as much as a billion barrels of oil within reach in Texas, North Dakota, and elsewhere -- if only he could squeeze it out.
In 2003, he had a "eureka!" moment while poring over pictures of rock. Sections of a 40-foot-long column of source rock had been run through a CT scanner, the same type used to peer into the human body.
He saw something in the source rock sections the rest of the industry didn't know was there: a network of passageways big enough for oil molecules to pass through. Papa believed the passageways could act like rural roads for the oil to travel through. Fracking could then create superhighways for the oil to gather and feed into a pipe and up to the surface.
EOG began drilling test wells, and in 2005, Papa got some results from one in North Dakota that made him realize oil could flow fast enough to pay off.
"It was kind of like holy cow," he says. "My first thought was we need to replicate this, make sure it's not a freak result."
It wasn't. EOG snapped up land in a similar formation in South Texas known as the Eagle Ford Shale for $400 an acre when his competition thought it would never produce much oil. That land now goes for $30,000 per acre.
Papa thought the Eagle Ford might hold 500,000 barrels of oil. The Department of Energy now predicts it holds 3.4 billion. Some even expect 10 billion, which would make it the biggest oil field in U.S. history.
Smart drills, rigs that can walk
But even after drillers figured out how to find oil and gas deep offshore and in onshore source rock, they still needed to develop technology that would make it economical.
At the tip of every oil or gas drill is a rotating mouth of sharp teeth that chews through rock. In the past, these drill bits could only dig straight down. Now they are agile enough to find and follow narrow horizontal seams of rock.
The drilling-services company Baker Hughes (BHI +0.00%) has designed a bit that can change directions underground, without having to be drawn back up to the surface, reducing drilling time by as much as 40 percent.
Behind the drill bit, attached to a long line of steel known as the "drill string," is an array of sensors. The sensors bombard rock with subatomic particles and measure the gamma radiation that bounces back. They assess how easily electricity flows through the rock and underground fluids. They analyze the magnetism of the rock and how it vibrates -- both up and down and side to side -- while drilling.
"To the layman, it looks like dumb iron, but you'd be shocked about what's inside," says Art Soucy, president of global products and services at Baker Hughes.
All this information is sent to engineers via fiber-optic cables. They run the information through supercomputers as powerful as 30,000 laptops to create a picture of the earth thousands of feet below the surface.
The people analyzing this data -- and even directing the drill bits -- are often sitting hundreds of miles away. Shell's Pennsylvania drilling operations are directed from a center in Houston, where experienced drillers monitor the progress at several sites across the country from a single room.
And when the drilling is done, the rig itself can "walk" a hundred feet or so to another location and start drilling again. In the past, rigs had to be taken down and reassembled, which could take days. New rigs are built on sliding "shoes" that allow hydraulic lifts to shuffle the rig forward in short steps.
"It has made possible things that were unthinkable 10 years ago," says Claudi Santiago, managing director at First Reserve, a private-equity firm that invests in energy companies.
Now, drillers are finding oil faster than the world is using it. At the end of 2001, the industry had enough "proved oil reserves" to satisfy world demand for 45 years, according to BP's annual statistical review, a closely watched study. By the end of 2011 that had grown to 51 years -- even though a decade's worth of oil had been used and daily demand had grown 14 percent. And "proved reserves" refers to oil that can be economically tapped using today's technology. Tomorrow's methods could yield even more.
This is good news for a global economy that remains dependent on fossil fuels, but it's terrifying to climate scientists.
"If we're willing to go down this road of squeezing whatever petroleum we can out of the earth, we can easily get carbon dioxide levels up to unfathomable levels and put in motion what would be dramatic or catastrophic changes in our climate system," says Michael E. Mann, a geophysicist and director of the Earth System Science Center at Penn State University.
Renewables progress, but not fast enough
Renewable technologies have had their successes. The average cost of a solar power system has fallen by 31 percent in the last two years. Solar now generates six times more electricity in the U.S. than it did a decade ago, and wind produces 14 times more. Most major automakers offer some type of electric vehicle.
And this success has come despite the fact that renewable energy's major benefit -- that it doesn't pollute -- is given little or no value in the marketplace because most governments haven't adopted taxes or penalties for fossil fuel pollution.
But the outlook for wind, batteries and biofuels is as dim as it's been in a decade. Global greenhouse gas agreements have fizzled. Dazzling discoveries have been made in laboratories, and some of these may yet develop into transformative products, but alternative energy technologies haven't become cheaper or more useful than fossil fuels.
Solar, wind, and geothermal sources together accounted for 4.8 percent of U.S. power generation last year. Ten percent of U.S. gasoline demand was satisfied with corn ethanol, but ethanol and other fuels made from non-food sources have yet to hit the market.
"In many cases, renewables aren't ready for primetime yet," says George Biltz, vice president for energy and climate change at Dow Chemical (DOW +0.00%), which continues to work on a host of renewable technologies.
Likewise, electric cars have not enjoyed the success many expected. The battery alone in an electric car costs as much as a new gasoline-powered car, and electric vehicles are not selling nearly as fast as once projected. General Motors (GM 0.29%) expected to sell 60,000 Chevy Volts globally last year, but sold just half that many. Sales of Nissan's (NSANY +0.00%) all-electric Leaf grew 22 percent around the world last year to 26,000, short of Nissan's projected 50 percent growth.
The cost of wind and solar power has declined, but the price of electricity made with newly cheap fossil fuels has fallen too, making it harder for wind and solar to compete.
"Renewables are now under scrutiny. They haven't made the kinds of quantum leaps we have seen in the oil and gas industry," says First Reserve's Santiago, who now shuns investments in alternatives.
David Aldous, the former Shell executive, learned that lesson while trying to turn wood chips into ethanol at Range Fuels. The system that fed the chips into a gasification chamber didn't work well, and the project failed.
"Things don't always scale from the petri dish to the demo plant and then to the commercial plant. It's just part of building up a new industry," Aldous says.
Range went out of business in 2011, and it was hardly alone. Dozens of biofuel, battery and solar companies failed even though federal and state governments supported alternatives with loans and grants, and mandated their use. Others are limping along.
Pacific Ethanol, which traded near $300 per share in 2006, now trades for 27 cents. Amyris (AMRS +0.00%), an advanced biofuels company, traded near $34 a share as recently as last year, but now trades at $2.65. The battery maker A123 was forced to file for bankruptcy protection last year, three years after going public. An index of clean energy companies that was first traded in March 2005 is down 70 percent since then. A similar index of traditional energy companies is up 73 percent over the same period.
The next 10 years
This dark period for alternative energy could last for years. With government debt soaring and no more worries about running out of oil, many renewable subsidies are being scaled back.
"The world is completely different now," MIT's Greenstone says.
But there are still hundreds of companies, including fossil fuel giants, working on new renewable-energy projects. ExxonMobil is investing in Synthetic Genomics, a company started by the geneticist J. Craig Venter to try to create strains of algae that will produce fuels. BP and Shell continue to work on ways to turn plant waste into fuels.
California, meanwhile, set the nation's most ambitious renewable energy goals and is on track to meet them. One-fifth of the power delivered by the state's three biggest utilities now comes from renewables, not including large hydroelectric dams. By 2020, that portion will rise to one-third.
President Barack Obama in March proposed using $2 billion in federal oil and gas royalties to invest in clean energy technology research. Obama is also expected to promote renewables through pollution regulations, if not with new laws.
And for all the world's newfound oil, prices are still high because developing nations are consuming more.
"It's not time to write the epitaph yet," Aldous says. Eventually the global economy will fully recover, he says, and demand for energy of all kinds will increase.
"New sources of energy are going to be in vogue again," he says.
Experts didn't see the oil and gas boom coming five years ago. It's certainly possible the world will change direction again in the next five years.
But EOG's Papa says oil and gas companies will just invest in even more sophisticated technology. He estimates that current techniques pull only 6 percent of the oil trapped in source rock to the surface. Learning to double that would yield yet another enormous trove of hydrocarbons.
"Now we go into the next phase of technology," he says. "How are we going to get the rest of it out of the ground?"
