In his best-selling book The Big Short, Michael Lewis describes a trader named Danny. A Wall Street investment bank came to Danny offering to set up a fancy trade that looked too perfect, too easy. "I appreciate this," Danny told the bank, "but I just want to know one thing: How are you going to f--- me?"

"And the salesman explained how he was going to f--- him," Lewis writes. "And Danny did the trade."

That passage might be the best rebuttal to the letter outgoing Goldman Sachs (NYSE: GS) employee Greg Smith wrote for the New York Times yesterday.

After 12 years with the bank, Smith quit this week, claiming "the interests of the client continue to be sidelined in the way [Goldman] operates and thinks about making money." This is disturbing and, no doubt, true. Smith showed with bravery and honor that he had had enough.

His most damning warning is that if Goldman's culture doesn't change, the firm risks losing clients -- mostly managers of hedge funds, private equities, and mutual funds -- and ultimately going out of business. "I truly believe that this decline in the firm's moral fiber represents the single most serious threat to its long-run survival," Smith wrote.

CNBC host David Faber disagrees, and he makes a strong point stemming from his own experience with dozens of Goldman clients while working on a documentary:

None of them would go on camera. But when you sit down and have dinner with them, they all know this. This is not news to them, that the client doesn't come first. Goldman may say it differently, but it's not news to their clients. The one thing I would take issue with in [Smith's letter] is the idea Goldman will lose clients. Because at this point, they haven't, even though most of the clients I spoke to are fully aware of how Goldman plays them.

They are, in other words, just like Danny -- aware that Goldman is taking advantage of them, but seemingly OK with it.

Asking why is important.

In one sense, they may not have much choice. With Bear Stearns and Lehman Brothers out of business, competition on the street is low. If you want something fancy done, odds are Goldman will be involved, whether you like it or not.

But there's another reason that I think has been lost in this debate. The institutional managers that banks like Goldman call clients can and will make a fortune even after Wall Street's shellacking has been administered. Consider:

  • After conducting a massive a research project, IBM concluded that global money managers overcharge investors by $300 billion a year for failing to deliver returns above the benchmark. Another $250 billion in fees goes to wealth advisory services that fail to beat stated benchmarks, and $51 billion a year is charged by hedge funds that fail to meet targeted returns.
  • According to David Norman, former CEO of Credit Suisse Asset Management, UK investors pay 8 billion pounds a year in hidden fees to financial advisors, capturing upwards of one-third of all investor returns.
  • Fees charged by equity mutual fund managers captured one-third of all investor profits generated by those funds from 2000 to 2010, according to Yoseph West of Vuru. After management fees, two-thirds of actively managed stock funds have underperformed the S&P 500 over the last three years, according to Standard & Poor's. About the same portion have failed over 10-, 15-, and 20-year periods. Most charge fees of 1% to 2% a year for this service.
  • Last year, SAC Capital Advisors' main fund returned 8% -- barely on par with the Dow Jones. Its boss, Steve Cohen, was paid $600 million, or more than the GDP of six nations.
  • The average hedge fund charges a hefty 2% management fee (plus a portion of profits), yet average returns have been astonishingly bad. According to author Simon Lack, if all the money ever invested in hedge funds had been invested instead in Treasury bonds, the results would have been twice as good.

You can go on and on all day with stats like these. They tell the same story: The clients that Goldman and the rest of Wall Street rip off are skilled at ripping off their own clients, thank you very much. Each is part of the same game of inflating expectations and overcharging fees -- a system summarized best by the title of Fred Schwed's classic book, Where Are the Customers' Yachts?

And frankly, some of the fees charged by mutual funds and hedge funds are more egregious than anything Goldman can dream of. Which is more obscene: Goldman charging a few percentage points to sell a security, or a money manager taking one-third of the profits that security product generates for his investors? Or worse, still paying himself eight figures when the security blows up and his investors lose their shirts?

That's what bothers me about all of this. What's sad about Goldman abusing its clients is not that its clients lose. It's that Goldman's behavior is merely representative of a system where the clients' clients lose. That's you, the 401(k) investor, the pension fund beneficiary, and the retired schoolteacher. They're the ultimate losers of the Wall Street game in which Goldman is just one example -- and probably not even the worst -- of putting personal interests before clients.