2017 will go down as year two of the "restaurant recession," an industry-induced slump caused by heavy competition from restaurant over-expansion. Many chains turned in lackluster performances thanks to falling foot traffic, but some had a downright awful year. Excluding the small fry of publicly traded companies and looking just at companies with market caps over $200 million, The Habit Restaurants (NASDAQ:HABT) did the worst in 2017 as of this writing, and  Fiesta Restaurant Group (NASDAQ:FRGI) and DineEquity (NYSE:DIN) are neck-and-neck for taking the spot as second-worst-performer.

HABT Chart

Data by YCharts.

Traditional dining losing its appeal

According to data from research group TDn2K, full-service dining has taken the biggest hit during the restaurant industry downturn. DineEquity, parent of casual-dining options Applebee's and IHOP, has been no exception. The company took a hit on impairment charges from falling value of Applebee's, and foot traffic has been in a steep slide all year.

Through the first nine months of the year, comparable sales at IHOP and Applebee's were down 2.5% and 7.3%, respectively. When adjusting for the one-time impairment charge on Applebee's, earnings per share were at $3.42 for the three-quarter period, a 26% drop from 2016's $4.63 during the same time period.

The company is working on a rebound strategy that includes revamping its menus, but things could get worse before they get better. Management forecasts foot traffic being down again to finish out its final quarter of the year. 

Applebee's ribs with a side of French fries, broccoli, and a beer.

Image source: Applebee's.

Fast casual in a pinch, too

Traditional dining isn't the only segment of the industry struggling. Fast casual -- dwelling somewhere between full-service dining and fast food -- has been rapidly expanding, but that can come at a cost. Fast casual is responsible for much of the industry over-expansion, and foot traffic trends have been hurting profit margins.  

Fiesta Restaurant Group, which operates fast-casual chains Pollo Tropical and Taco Cabana in the southeastern U.S., had revenues dip 6.2% during the first nine months of the year in spite of 14 new locations being opened. The culprit: falling foot traffic. Comparable sales for Pollo Tropical and Taco Cabana were down 8.5% and 7.2%, respectively, during the nine months ended Oct. 1. 

The company has revitalized its menu and launched new ad campaigns. Prior to Hurricanes Harvey and Irma over the summer, management said sales were beginning to rebound. Now that conditions in the chain's primary markets are getting back to normal, only time will tell if Fiesta's rebound is for real.

The better-burger market gets crowded

There are a lot of brands expanding in the fast-casual burger concept, making for one of the most hotly contested segments of the restaurant industry. California favorite The Habit has faced challenges as a result. Because of new options popping up to divert diners' attention, The Habit's foot traffic has suffered this year.

The company's total revenue increased 17.3% in the first nine months of 2017, driven by several dozen new locations that brought its total count to 195 at last report. That's enough top-line growth to keep the overall picture positive for the chain. Profits have dipped, though, with earnings per share at $0.17 for the three-quarter period, versus $0.20 during the first three quarters of 2016.

A group of three people sitting beside a window eating burgers and a salad at The Habit.

Image source: The Habit.

That's a result of slowing comparable sales, turning what was once a strong tailwind into a headwind for The Habit. In years past, growing traffic at existing stores was able to offset some of the expansion costs, but not anymore. The chain is still successfully growing its version of the better burger, but with plenty of new entrants vying for attention, expect that growth to keep bumping up against resistance for awhile.

What's in store for 2018?

2017's worst-performing restaurant stocks all have one thing in common: cutthroat competition. With the industry shrugging off concerns and growing its base of locations, 2018 looks like it will offer up more of the same.

Will 2017's restaurant losers repeat in 2018? Maybe, but the restaurant industry is highly competitive and popularity of various brands are always in ebb and flow. Don't fear investing in restaurant stocks, though. Even in the midst of difficulties, some businesses are beating the trend.

This article represents the opinion of the writer, who may disagree with the “official” recommendation position of a Motley Fool premium advisory service. We’re motley! Questioning an investing thesis -- even one of our own -- helps us all think critically about investing and make decisions that help us become smarter, happier, and richer.