There are big names in the investment industry, and then there are big names -- like Vanguard, Fidelity, and T. Rowe Price (NASDAQ:TROW). These big dogs didn't get where they are by playing it safe and ignoring investor demand. They've built their businesses around providing what investors want. And now one of those same big players is breaking new ground and boldly going in a new direction -- far away from its more humble beginnings.

Hedging its bets
Roughly two years after first testing the waters for feasibility, Vanguard appears to have taken another step toward offering its first hedge fund. The fund shop recently filed an exemptive relief application with the SEC looking for the go-ahead to launch an absolute return fund.

If the fund goes forward -- and Vanguard says that's still a big if at this point -- then it would offer "alternative" strategies and will have leeway to invest in the usual stocks and bonds as well as commodities, futures, and currencies. It would also have the ability to short securities and employ leverage, as most hedge funds are wont to do.

According to Vanguard, the absolute return fund would initially be available only as a part of the firm's Managed Payout funds, which attempt to convert investor assets into a long-term income stream throughout retirement. These funds would be able to invest up to 20% of assets in the absolute return fund, in an attempt to juice long-term returns. Vanguard feels that this off-shore hedge fund would be able to invest in a wide range of alternative strategies at a lower cost and with lower risk than the Managed Payout funds could on their own.

You've come a long way, baby
Vanguard has certainly come a long way from its day of being primarily a low-cost index fund provider. Although the shop became famous for its indexed offerings, Vanguard soon expanded into actively managed funds, typically outsourcing management to another top-tier manager. The shop has had great success in this arena, producing several highfliers. For example, Vanguard PRIMECAP (VPMCX), sub-advised by a team at PRIMECAP Management, invests in large growth names like Google (NASDAQ:GOOG), Oracle (NASDAQ:ORCL), and Microsoft (NASDAQ:MSFT). Now closed to new investors, the fund has outperformed 97% of all large-cap growth funds in the past 15 years.

As exchange-traded funds burst onto the investment scene, Vanguard again responded to investor demand, offering a wide range of rock-bottom cost ETFs. These offerings range from narrowly focused, sector-specific funds to the broadly diversified Vanguard Total Stock Market ETF (NYSE:VTI), which invests in blue-chip names like Exxon Mobil (NYSE:XOM) as well as more mid-sized names like Southwest Airlines (NYSE:LUV).

Some reservations
It would be silly to expect the fund giant not to explore new ways to make money. But I'm a bit hesitant about a Vanguard hedge fund.

For one, I'm not sold on the concept of "absolute return" funds to begin with. Oh, the idea is great -- a fund that seeks to provide positive returns in all markets. That's actually a very solid investment thesis.

The problem comes in actually implementing such a strategy. Such a fund is still investing in the same securities that the rest of the market is, so to keep its head above water during difficult times, riskier strategies are oftentimes employed. That means a greater chance of something going wrong. There's also no guarantee that an absolute return fund will live up to its hype of generating positive returns in all environments.

In fact, of the handful of mutual funds that run absolute return strategies, most have ended up trailing the market in recent years while still managing to charge more than the average actively managed fund -- much more in some cases. An absolute return focus is a good thought, but execution frequently leaves a lot to be desired.

Wait and see
It's still too soon to tell exactly how a potential Vanguard hedge fund would shake out. It's still in the initial stages, and actually getting the fund on its feet will take some time.

Nevertheless, if Vanguard commits to going forward, I'm willing to bet that the company will do a careful and thorough job building this fund. The shop's well-honed and time-tested due diligence process will no doubt be employed when looking for a team or multiple teams to run the new portfolio. Vanguard has a terrific track record of identifying and keeping tabs on managers, so I'm willing to give the benefit of the doubt here.

At least at first, a Vanguard hedge fund wouldn't have a big impact on investors, since you'd have to buy one of the shop's Managed Payout funds to gain access. But if the absolute return fund does well, it may be made available to more Vanguard investors. I'll reserve a final judgment on Vanguard's latest move for now, but I'll be watching closely to see how this next stage in the fund shop's evolution turns out.

For more insider investing and personal financial planning tips, check out the Fool's Rule Your Retirement service, which provides top-notch retirement and mutual fund advice. You can start your free 30-day trial today.

Amanda Kish is the Fool's resident fund advisor for the Rule Your Retirement investment newsletter. At the time of publication, she did not own any of the funds or companies mentioned herein. Google is a Motley Fool Rule Breakers recommendation. Motley Fool Options has recommended a diagonal call position on Microsoft, which is a Motley Fool Inside Value recommendation. The Fool owns shares of Oracle and has a disclosure policy.