For many years, Merck (NYSE:MRK) has been a leader in developing drugs designed to fight so-called "bad" cholesterol, thus reducing the risk of clogged arteries and heart disease.

But with Mevacor and Zocor having lost patent protection, and with the therapeutic value of Vytorin and Zetia being questioned, Merck is making one of the industry's biggest R&D bets on drugs to raise levels of "good" cholesterol to improve heart health.

For investors, Merck's R&D requires a leap of faith -- or at least patience. There are just a handful of good-cholesterol-raising drugs on the market, and the field has been filled with disappointments. Merck has compounds in late-stage clinical testing for two classes of these drugs, but it will still be several years before those studies yield results.

Bad versus good
Today's most popular therapies are aimed at reducing bad cholesterol, or low-density lipoproteins (LDL). When too much LDL is in the bloodstream, it can cause a buildup of plaque and restrict blood flow. When a blood clot forms, it can cause a heart attack or stroke. Bad news.

The biggest anti-LDL weapon is the drug class known as statins, which reduce the liver's production of cholesterol. Major brands include Lipitor from Pfizer (NYSE:PFE) and Crestor from AstraZeneca (NYSE:AZN). On the generic side lurk versions of Merck's Mevacor and Zocor, as well as Pravachol from Bristol-Myers Squibb (NYSE:BMY).

Merck also has Zetia, which also reduces LDL, even though it isn't a statin; instead, it cuts absorption of cholesterol in the intestine. Zetia and Vytorin (a combination of Zetia and Zocor) are products of Merck's joint venture with, and later acquisition of, Schering-Plough.

Sales of Zetia and Vytorin have been fading. Year to date (YTD), Vytorin's sales of $1.5 billion have declined 17% versus the same period last year. Zetia's sales have fallen 10% to $1.51 billion.

And that happened before the presentation of clinical-trial results at an American Heart Association meeting earlier this month. Abbott Laboratories (NYSE:ABT) released results showing that Zetia didn't do as well at keeping arteries clear as did its own Niaspan, a drug that raises good cholesterol. Even though the test's methods provoked some controversy, and Merck challenged it, Abbott said the test confirmed its own strategy of trying to reduce the clogging of arteries.

That brings us to the other side of the cholesterol coin, and where Merck is headed next.

Trying to raise HDL
There's also high-density lipoprotein (HDL), or good cholesterol. The American Heart Association says high HDL levels appear to reduce the risk of heart attack, while low levels increase the risk. Some researchers believe HDL can even reduce the rate of plaque buildup in arteries.

Right now, Abbott is the king of HDL. Each of its three HDL-boosting prescription drugs contains an extended-release version of niacin (which is actually simply a B vitamin).

The biggest seller is the aforementioned Niaspan, at $601 million in sales YTD, up 6.4%. Abbott also sells Advicor, which is Niaspan plus generic Mevacor, as well as Simcor, which is Niaspan plus generic Zocor.

But there's one key drawback that explains why Abbott's HDL drugs' sales are modest versus the multibillion-dollar sales of LDL drugs such as Lipitor. In addition to boosting HDL, niacin causes flushing of the skin, leading to redness, itching, and sensations of warmth.

Merck wants to break Abbott's grip on the HDL market by developing a niacin drug with minimal flushing. Its candidate, Tredaptive, combines niacin with an antiflushing agent; it's won regulatory approvals in 39 countries, including Mexico, the U.K. and Germany. But in the U.S., where it was called Cordaptive, the Food and Drug Administration rejected the drug in April 2008.

Merck later explained that the FDA wanted more safety and efficacy data. The agency wanted Merck to wait for the results of a 20,000-patient clinical trial, which isn't due to be completed until early 2013. Merck could use some data from it to answer the FDA's objections early, but even so, it won't file a response until sometime next year.

Another good-cholesterol effort
A second class of drugs seeking to raise HDL are called cholesteryl ester transfer protein inhibitors, or CETP inhibitors. None have come to market yet from this class -- in part because one of these drugs nearly ate Pfizer a few years ago.

Pfizer spent nearly $1 billion developing torcetrapib, only to halt a late-stage clinical trial in December 2006, after an independent safety review panel found that patients taking torcetrapib plus Lipitor had a higher death rate than those taking Lipitor alone. Ouch!

After that news broke, scientists and analysts wondered whether torcetrapib's problems indicated the whole CETP class was damaged or whether it was specific to that compound. Merck believes the latter, and it's forging ahead with its own CETP inhibitor drug.

Last year, Merck started a phase 3 trial with anacetrapib. Just last month, Merck said that a phase 2 trial with anacetrapib (alone or with Lipitor) showed an HDL-raising effect in the higher-dose regimens, even eight weeks after patients stopped taking the drug. Very encouraging.

What it means for investors
If Merck can successfully develop an HDL-raising drug, sales could be as big as Lipitor's in its heyday. Unfortunately, that possibility's still several years away, at best.

In the meantime, Merck has to compete against decent pipelines from rivals like Novartis (NYSE:NVS) and GlaxoSmithKline (NYSE:GSK). The acquisition of Schering-Plough helped the company, though it will take time to fully integrate.

With future sales of Vytorin and Zetia uncertain, and with its big blood-pressure drug Cozaar/Hyzaar coming off patent in 2012, Merck will continue to need more help from more products to prevent atherosclerosis of its shares.

More Foolish analysis of drug company mergers:

Fool contributor Robert Steyer doesn't own shares of any companies cited in this story. Pfizer is a Motley Fool Inside Value selection. Novartis is a Global Gains selection. The Fool has a disclosure policy.