It's no secret that software-as-a-service (SaaS) stocks have been huge beneficiaries of COVID-19. It's not that they offer anything to help combat the virus. It's just that worldwide lockdowns have vastly accelerated the transition to the cloud.

With years of such transitions now compressed into a few quarters, many SaaS companies are reaching sales goals far faster than ever expected. On the surface, that's what happened for Smartsheet (NYSE:SMAR) -- a company that provides data analytics tools for corporations.

It's a stock that I owned and even called out as a top small cap buy for 2020. But there was a specific detail in the latest earnings report that caused me to sell my small position entirely.

Finger pressing red sell button on keyboard

Image source: Getty Images.

But first, the impressive numbers

On the surface, everything about Smartsheet's fiscal 2021 first quarter was impressive. In the chart below, ACV stands for "annual contract value."

Metric Q1 2021 Q1 2020 Growth
Revenue $85.5 million $56.2 million 52%
Customers with ACV over $50,000 1,040 518 101%
Customers with ACV over $100,000 391 189 107%
Average ACV $3,866 $2,675 45%

Data source: SEC filings.

On top of all this, dollar-based net retention came in at 132%. That means existing Smartsheet users -- those who were around in early 2019 -- not only stayed with Smartsheet but spent 32% more with the company on average.

Everything about this is impressive. More customers are signing bigger contracts. That not only means more recurring revenue, it means Smartsheet's technology is becoming more deeply ingrained in customers' operating systems.

At the same time, however, the company is still bleeding money. It lost over $28 million in free cash flow during the quarter. And though it withdrew its free cash flow guidance for the year, management said it would likely come in below the low-end of previous guidance. In other words, it will likely be worse than an outflow of $11 million.

And yet, that wasn't the part that gave me the most pause.

The key metric

The metric I was most focused on, however, was billings. Because Smartsheet is a subscription-based company, it can collect a full year's worth of revenue from a customer in January, but it won't all show up on the income statement at once. One-fourth will be recognized in the first quarter, one-fourth in the second quarter, and so on.

The advantage of billings is that it gives you a better look into the future. Customers who sign longer, more valuable contracts will show up in this figure -- even if it takes a while to show up in sales.

Here's what that trend looks like.

Chart showing billings growth over time at Smartsheet

Chart by author. Data source: SEC filings.

The blue bars represent how much billings grew from one quarter to the next; the red bar from one year to the next. The key here is that revenue is growing faster (52%) than billings (30%).

In the SaaS stocks I invest in, the opposite is usually true -- because so many companies are flocking to the platforms, billings (or "bookings" or "remaining performance obligations" or whatever word they use) grow faster than sales.

What does this tell me? Perhaps Smartsheet isn't as mission-critical as I thought. Perhaps the company is losing out to the competition. Perhaps it's actually easier than I assumed for a customer to take all of its data and load it up on another cloud-based tool.

It might be a little bit of all those reasons. But what I feel relatively sure about is this: In a time when many SaaS companies are seeing growth accelerate as people move to the cloud, Smartsheet is headed in the opposite direction. For the second quarter, management expects billings to grow at a mid-point of 16% -- another significant reduction.

Should you follow me?

In the end, Smartsheet could still be a great investment. The company is clearly hanging onto the customers it already has. Many would be happy for just that in a post-COVID-19 world. 

And perhaps Smartsheet is simply helping out small and medium-sized businesses by offering them assistance and reduced prices now in return for much larger gains later.

But I believe there are simply too many excellent SaaS companies out there to invest in to keep my money with one where growth is decelerating so rapidly, in a time where I believe it should be moving in the opposite direction.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.