In the oil and gas industry, the downstream segment – also known as refining and marketing – is focused on the final stage of the integrated oil and gas process. In a nutshell, the upstream companies go out and get the oil out of the ground, the midstream companies store and transport it, and the downstream companies refine it.

Specifically, refining refers to the process of converting crude oil into usable petroleum products – such as gasoline and diesel – while marketing refers to selling the product to customers. While most of the integrated major oil and gas companies have branded retail gasoline stations, downstream operations have often been the least profitable part of their business.

This is why, historically, independent refiners – those involved exclusively in the downstream segment – have been viewed as the boring lot out of the bunch, receiving sparse attention from investors.

Investors' avoidance of refining companies has largely been predicated upon the highly cyclical nature of the industry and its historically tight margins, which have averaged about $9 per barrel over the past 20 years or so. Refiners' vulnerability to commodity price volatility has been another concern, with average benchmark crack spreads – a proxy for refining margins – having varied wildly over the years.

But recently, geographically advantaged American refiners have staged a dramatic turnaround, prompting several commentators to claim that a "golden age" for the industry has arrived.

A new "golden age" for refiners?
Judging by recent performance, their case appears solid. The past year was nothing short of remarkable for U.S.-based refiners, with many registering gains that easily exceeded 50%. For instance, shares of HollyFrontier and Tesoro rose nearly 85% over the course of 2012, while Valero (NYSE:VLO) gained more than 60% and Marathon Petroleum (NYSE:MPC) surged nearly 90%. These performances handily trounced the S&P 500 Energy Index.

So what explains elevated profitability among U.S. refiners this time around? Evidence suggests that the phenomenon is unique to North America, as opposed to being global in nature. While the previous "golden age," which occurred from 2003 to 2007, was driven more by cyclical factors, this one appears to be driven by deep structural changes in the North American energy landscape. Let's take a closer look.

Widening Brent-WTI spread
Total U.S. oil production has been on the rise over the past four years. Last year, domestic crude oil production increased more than ever before in the history of the U.S. oil and gas industry, and it shows no signs of slowing down any time soon.

However, despite massive capital spending by midstream operators, such as Denver-based DCP Midstream Partners (NYSE:DCP), which plans to invest a whopping $5 billion-$7 billion in new growth projects between 2011-2015, the U.S. simply does not have the necessary pipeline infrastructure to transport all that oil to distant parts of the country. As a result, a lot of it ends up as inventory in the nation's main oil storage hub at Cushing, Okla.

The resultant glut at Cushing has helped drive down the price of domestic oil, often benchmarked to West Texas Intermediate (WTI), in comparison to the global price, which is linked to Brent. As the difference between these two price benchmarks – known as the Brent-WTI spread – has grown, something of a geographically determined rift has developed between refineries.

Location, location, location
For instance, refineries on the East Coast, which lack access to cheap inland oil due to insufficient pipeline infrastructure, have languished. Several were even forced to close down last year. In fact, Hess (NYSE:HES) recently announced that it will be shutting down a refinery in Port Reading, N.J., by the end of this month.

In sharp contrast, refiners with access to cheaper grades of oil – many of them located in the Midwest – have benefited tremendously because they can sell the refined product at higher globally determined prices.

For instance, Valero, whose fourth-quarter results blew consensus estimates out of the water, chalked up its impressive profit growth to cheaper input costs. At its Gulf Coast and Memphis refineries, the company replaced all of its imported crude with significantly cheaper domestic oil.

Marathon Petroleum, which also had a stellar fourth quarter, acknowledged that similar factors drove its solid performance. The company's access to lower-priced crude inputs boosted margins substantially, with its refining and marketing gross margin averaging $10.45 per barrel for 2012, up significantly from $7.75 per barrel in 2011.

These refiners' improved margins, due to a wide Brent-WTI spread, mark an important development. Historically, WTI actually traded at a slight premium to Brent. But since 2010, Brent prices have been on the rise and, at times, reached a $25 premium to WTI. And certain grades of inland oil have been even cheaper than WTI, at times trading below Brent by as much as $40.  

What's next?
While the Brent-WTI spread has been falling since December, it recently rose sharply following an announcement that Seaway, a major crude oil pipeline, is encountering some setbacks as it aims to expand capacity from 150,000 barrels per day to 400,000 barrels per day. But some analysts are suggesting that this is a short-term issue and expect the spread to narrow meaningfully by year's end as new pipeline capacity is brought online.

However, we saw this exact same argument last year, when analysts expected the reversal of the Seaway pipeline in May to dramatically lower the Brent-WTI spread by the end of the year. Seaway, which is operated as a 50/50 joint venture between Enterprise Products Partners and Enbridge, was expected to substantially alleviate the glut of oil in the Midwest once the direction of its flow was reversed, hence causing the spread to narrow.

But that theory didn't pan out at all. Instead, the spread steadily increased from July all the way to December, suggesting that many people either underestimated the true size of the crude oversupply or overestimated the speed with which Seaway would alleviate it. With the U.S. Energy Information Administration projecting another record year of oil production, there is a good chance that the spread may not contract as quickly as some are expecting.

Fool contributor Arjun Sreekumar has no position in any stocks mentioned. The Motley Fool has no position in any of the stocks mentioned. Try any of our Foolish newsletter services free for 30 days. We Fools may not all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors. The Motley Fool has a disclosure policy.