"Believe me, I'd be fine if these companies fail, but the whole economy is on the line," President George W. Bush warned a congressman in Sep. 2008, according to his just-released memoir Decision Points. "The son of a bitch is going down if we don't step in."

He's not the only one who felt that way.                                                 

"All of corporate America's dominoes were lined up, ready to topple at lightning speed," Warren Buffett wrote in an op-ed for the New York Times this week. "Nor was it just business that was in peril: 300 million Americans were in the domino line as well."

Two famous men from nearly opposite political spectrums coming to the same conclusion: Government intervention was absolutely necessary to prevent the financial crisis from becoming a calamity.

"Only one counterforce was available, and that was you, Uncle Sam," wrote Buffett. "Yes, you are often clumsy, even inept. But when businesses and people worldwide race to get liquid, you are the only party with the resources to take the other side of the transaction. And when our citizens are losing trust by the hour in institutions they once revered, only you can restore calm."

Here's Bush again: "If we're really looking at another Great Depression, you can be damn sure I'm going to be Roosevelt, not Hoover." Speaking to the nation in a primetime Sept. 2008 address, he continued: "I understand the frustration of responsible Americans who pay their mortgages on time, file their tax returns every April 15, and are reluctant to pay the cost of excesses on Wall Street. But given the situation we are facing, not passing a bill now would cost these Americas much more later."

Enough ink has been spilled arguing whether bailing out the financial system was fair. The answer is clear: Of course it wasn't. It was ugly. It was costly. But Buffett and Bush tackle an important yet often overlooked topic: What would the cost have been had we not bailed them out?

No one knows. But here's a safe bet: Inaction would have cost far more than intervention.

Unemployment jumped from 5% in 2008 to 10% in 2009. Fewer people working meant that federal tax receipts plunged by $419 billion. Had there been no bailout, and unemployment instead jumped to, say, 12% or 15%, lost tax revenue would have easily tacked on an extra $100 billion or more per year to the deficit, compared with TARP's estimated final one-time cost of $50 billion. That even seems conservative. Bush's economic team reckoned that letting Ford (NYSE: F), GM (NYSE: GM) and Chrysler go under, let alone the banks, would have lowered tax receipts by $150 billion per year. And don't forget the cost of unemployment benefits for all those lost jobs.

Then there's an issue my colleague Richard Gibbons once brought up. During the fall of 2008, the FDIC insurance fund only had enough money to back 1.01% of nationwide deposits. Had there been a widespread banking panic like those during the Great Depression, this fund would have been wiped out almost immediately. Then what? The FDIC would have sucked money straight from taxpayers' pockets, thanks to its $500 billion line of credit at the Treasury. There's another enormous cost TARP warded off.

How many of those criticizing the bailout's cost take these alternatives into account? Few. The real outrage of actual money spent overwhelms the intangible benefit of potential costs averted.

But forget costs. There's the larger issue of moral hazard. Bailing out bad behavior spits in the face of capitalism. That's why so many were opposed.

Bush gives his thoughts on this: "I decided that the only way to preserve the free market in the long run was to intervene in the short run." 

Buffett also weighs in: "In this extraordinary emergency, you [the government] came through -- and the world would look far different now if you had not."

Charlie Munger, Buffett's partner at Berkshire Hathaway (NYSE: BKR-B), takes this a step further, repeatedly noting that Germany allowing its economy to collapse in the 1920s was a driving force behind the rise of Adolf Hitler. "So I think when you have troubles like that you shouldn't be [complaining] about a little bailout," he said earlier this year. Other historical accounts back this up. His point isn't literally that another Hitler would take over if we didn't bail out the banks. It's that financial collapse tends to lead to social collapse. 

On a less morbid note, the moral-hazard-fueled criticism that banks got off scot-free is slightly exaggerated. Citigroup (NYSE: C) shares are down 93% since 2007. AIG (NYSE: AIG) is down 97%. Bank of America (NYSE: BAC), down 78%. Fannie and Freddie are both down 99%. Capitalism wasn't thrown out amid the bailouts -- just temporarily weakened. Companies that made bad decisions paid dear prices -- but not quite as dear as they should have. Letting that marginal moral hazard slip through was likely less costly and ultimately fairer to everyone else than the alternative. That's the point Buffett and Bush are in harmony on.

None of that is to say the bailouts were enjoyable. They were terrible. Hundreds should go to jail, said Motley Fool CEO Tom Gardner. And everything humanly possible should be done to ensure we never have to go down that road again.

But here's my guess: Had the government sat idly by during the financial crisis, the same people condemning its decision to intervene would instead be criticizing it for being asleep at the switch.

What do you think?

Check back every Tuesday and Friday for Morgan Housel's columns on finance and economics.