Alphabet's (GOOG 0.37%) (GOOGL 0.35%) Google and its driverless car initiative began back in 2009, but the futuristic notion of driving from point A to point B without touching a steering wheel goes back long before that. It looked as if Google was getting close to making driverless cars a reality when it announced in December it would be spun off into a separate unit under the Alphabet umbrella sometime this year.

The decision to run the car unit as a distinct entity led some to believe that its self-driving cars were inching closer to going mainstream. Alas, it appears one of consumers' biggest fears as it relates to a Jetson-like driving experience could prove to be a major roadblock.

Image source: Google.

Survey says
As part of the deal with the state of California -- where many of its self-driving cars are being tested -- Google is required to submit reports detailing the results of its efforts. After logging over a million miles on the road, there have been some hiccups, which is to be expected during this initial phase.

However, Google's latest report to the state highlights more than a few disturbing incidents that could put a damper on its plans to introduce a driverless car fleet to the public. From Sept. 2014 to Nov. 2015, there were a whopping 341 "disengagements," which is Google's way of saying test drivers were either prompted, or felt they needed, to take back control of the vehicle.

Of the hundreds of disengagements reported, failure of the technology was the cause for testers to take over on 272 occasions -- an unsettling number, to be sure. But the remaining 69 problems were even more disconcerting, and they speak directly to why two-thirds of consumers responded to a recent poll saying they were not interested in self-driving cars because the thought of "losing control" was unsettling.

The 69 times drivers took the wheel of the self-driving vehicles of their own volition were due to incidents in which "safe operation of the vehicle requires control by the driver." These included situations in which others on the road were driving erratically, or the rider was simply uncomfortable and wanted to assume control of the vehicle.

According to data Google receives whenever one of its cars disengages, there would have been 13 accidents involving either another car or object if not for the voluntary disengagement.

With all of those miles under its self-driving belt, the relative number of would-be accidents may not be alarming. However, according to a study, Google's self-driving cars are currently about 15 times more likely to be in an accident than "traditional" vehicles.

Does it matter?
There's a reason most every major auto manufacturer and folks like automotive components manufacturer Delphi (DLPH) -- which showed off its own driverless cars at the recently completed computer electronics show (CES) in Las Vegas -- are diving into autonomous vehicles. According to one estimate, by 2020, there will be 10 million self-driving cars on the streets.

That's an awfully big opportunity for the Googles of the world, let alone an upstart like Delphi, with its $18 billion market capitalization. If Delphi is able to carve its own niche among of all those driverless cars expected on the road in four to five years, it could really move the revenue needle. Apparently, Delphi is doing something right, in that its self-driving cars took one of the spots as a "top pick" at CES.

Unlike Delphi and the auto manufacturers' driverless car initiatives, Google has the added benefit of boosting its digital ad revenue, too. It's not difficult to imagine cruising along in a Google driverless car -- that just happens to be running Android on its monitor -- and surfing the Internet, or checking messages with the usual collection of digital ads.

Still, investors shouldn't expect Google, let alone newcomer Delphi and its ilk, to introduce self-driving cars to the masses anytime soon. As the report to California makes abundantly clear, consumers' uneasiness with giving up the reins to their automobiles is warranted. While that may not put the kibosh on Google's driverless car plans, it could set it back years.