Fitbit (FIT) recently claimed that one of its corporate wellness programs, which placed fitness trackers on employees, reduced the healthcare costs of Fitbit users by 22.5% (almost $1,300 more than the control group) after two years. While that figure looks great for future bulk sales of Fitbit devices to companies, critics quickly pointed out that the study had some glaring flaws.

Image source: Fitbit.

What's wrong with Fitbit's study?

In the study, 900 employees out of the company's workforce of 20,000 volunteered to be studied and received partially subsidized Fitbits. Fitbit then formed a non-Fitbit control group, consisting of people who roughly matched those volunteers' ages, health conditions, and genders.

The Verge pointed out that while this study technically compared Fitbit users to non-Fitbit users over a two-year period, more active people were likely to volunteer for the Fitbit group, while less active ones were likely to be placed in the control group. Therefore, Fitbit should have used a "randomized" control trial -- where half the volunteers didn't receive a Fitbit -- to ensure that the difference in healthcare costs was caused by the Fitbits instead of differences between participants.

Fitbit's claim of 22.5% healthcare savings for Fitbit users also glossed over the fact that healthcare costs in the control group also fell 9.3%. Since Fitbit didn't use a randomized control trial, there was no way to tell if the difference was merely caused by the volunteers being more active, other changes in healthcare spending, or if they truly benefited from wearing Fitbits. Some people also dropped out of the program before the two years ended, but they were recorded as "less active users in the final results."

Image source: Fitbit.

The numbers were also skewed by active users using Fitbits much more than average ones. For example, 350 participants used the device for at least 274 non-consecutive days over two years. It wasn't surprising that those users saw the biggest drop in healthcare costs, but those figures boosted the average savings of the Fitbit group. These factors all make Fitbit's claim of 22.5% healthcare savings for Fitbit users seem disputable.

Why this study matters

This study matters to Fitbit because the company is trying to expand its corporate wellness programs into a meaningful stream of revenue. The company hopes that big contracts with companies like Target (TGT -0.38%), which ordered Fitbits for 335,000 employees last year, can boost bulk orders of its devices that would widen its moat against rival fitness tracker and smartwatch makers. Fitbit doesn't regularly disclose how much of its revenue comes from corporate wellness programs, but less than 10% of its sales came from that segment last year.

The validity of the corporate wellness study isn't Fitbit's only headache. A recent study in Singapore found that 90% of Fitbit users stopped using the devices within 12 months, a JAMA study found that fitness trackers can provide a false sense of security in weight loss programs, and The Verge claimed that the Charge 2 recorded inaccurate distances. Earlier this year, Fitbit faced a class action lawsuit claiming that the heart rate monitoring features in the Charge HR and Surge were inaccurate.

Should investors be worried?

It seems like many of Fitbit's problems are caused by the company sponsoring its own studies. If it let third parties conduct their own studies on Fitbit's efficacy, the results might hold up better. However, third party studies could also be less flattering and reveal that the devices don't really cut healthcare costs as much as Fitbit claims.

Fitbit stock has already fallen more than 50% this year due to slowing sales, falling margins, and rising competition. But there is some good news amid all that gloom -- its share of the global wearables market rose from 24.9% to 25.4% between the second quarters of 2015 and 2016, according to IDC.

During that same period, Xiaomi's share fell from 17.2% to 14%, while Apple's (AAPL 0.57%) share plunged from 20.3% to 7%. Those numbers indicate that Fitbit still has a big advantage in the enterprise space -- since Xiaomi doesn't have a strong presence in the U.S., and Apple Watches are pricier, have a weaker battery life, and are only compatible with iOS devices.

The road ahead...

But looking ahead, Fitbit must keep defending its niche against cheaper fitness trackers and smartwatches, which threaten to eventually converge and render "sports performance" wearables like Fitbit's obsolete. Those efforts will likely boost expenses and reduce margins. Securing more big enterprise customers like Target could prevent that from happening, but not if the company's claims of lower healthcare costs are repeatedly shot down.