Machine Learning 101

Excitement around artificial intelligence and machine learning has been buzzing around in sci-fi and tech enthusiast spheres for over 50 years now, but only recently has research around it started taking off at exponential speeds.

In this Industry Focus: Tech clip, our analysts talk about the fascinating origins behind modern machine learning tech, and how the unprecedented advances we're seeing today almost never came to be, what machine learning is and why companies across the board are so excited about its potential, some of the most exciting advances we've seen in the past few years, where artificial intelligence could go from here, and more.

A full transcript follows the video.

This video was recorded on Jan. 5, 2018.

Dylan Lewis: Before we get too deep in conversation, I think it's probably good for us to formally define AI and deep learning/machine learning, because those are two terms that are going to be coming up again and again, and it's good to just get that out of the way.

Eric Bleeker: You know, I hate to do a history lesson.

Lewis: I love history lessons.

Bleeker: But you kind of have to start with it in AI. I think your average person has a lot of natural skepticism around artificial intelligence that's completely warranted. The reason for that is, the term "artificial intelligence" was coined in 1956.

Lewis: That's a long time ago.

Bleeker: Over 60 years. There's been a lot of boom and bust. In the '60s, people were saying it was around the corner in the '80s. The biggest thing was expert systems, where you're going to have this computer as your co-CEO telling companies how to run. So if you've been a longtime market observer, it's good to have some natural skepticism about this. But, we have had several incredible recent breakthroughs. And what the breakthroughs are really rooted in are that 50 or 60 years of progress, because a lot of that research was actually right, and it was finding the right ways to approach this problem. Now, the limiting factor was, in the '70s, '80s, computers were slow.

Lewis: Yeah, they just weren't capable of doing everything that they can now.

Bleeker: Yeah, artificial intelligence requires tremendous amounts of, No. 1, data. And when we used to have entire rooms to store megabytes of data, that's not cost-effective. Second, it required processing power. So I think the root of the current AI boom actually goes back to the birth of the internet, because it poured so much capital into the broader technology industry. And I think a lot of the forward-looking companies that many investors see as something not related to AI, truly were thinking about it at that time. There's a famous example of Larry Page at a cocktail party mingling --

Lewis: Alphabet (GOOGL -1.23%) (GOOG -1.10%) CEO.

Bleeker: Yeah, and co-founder. He was commingling, and at the time, he ran into Kevin Kelly, who worked for Wired. And Kevin Kelly expressed some skepticism, saying, "Hey, I just don't get it. You guys are a great search engine, but you're still a search engine that's free, and I don't quite get this." And his response was, "Oh, no, we're really an AI company."

Lewis: And that was back in the early aughts.

Bleeker: Yeah, more than 15 years ago. He saw this wave that was coming, and he saw that Google's product was essentially an AI product. And we had a lot of work around machine learning at that time. The definition of machine learning is essentially code that writes itself. You input a lot of data and you have software that basically finds very advanced patterns and doesn't need to be explicitly coded. So, what are the applications of machine learning? You've seen a lot of it in Web 2.0. It's Google, it's how Facebook filters your feed, it's how Twitter looks for trolls, it's doing a lot of that work that's really behind the scenes in every application. A company like Amazon optimizing its entire supply chain, they use machine learning for that process, it's the underlying factor that makes everything hum. But you might say, "Hey, Eric, that's interesting." Kind of interesting, maybe. [laughs]

Lewis: I think it's very interesting. I wouldn't be having you on the show if I didn't think it was very interesting.

Bleeker: "But everyone has been talking about AI changing everything, and I don't know that it sounds like it's changing everything." And that's an astute point, listener out there. I'll say, the real breakthrough started around 2011, and that is deep learning, which is a sub-discipline of machine learning. And essentially what deep learning does is, it replicates the way the human brain learns, which is building connections between synapses. Essentially, though, our brains are hugely powerful. We have 15 quintillion synapses --

Lewis: I'm not going to even venture a guess at how many zeroes are in that number.

Bleeker: A computer, you're talking billions of transistors. So there's still a huge jump between those two areas. So, what had happened was, Google had hired someone who was, I believe, an intern, of all things, and he was one of the literal dozens of people in the world who was still studying this idea that had come out in the '70s about recreating the human brain. He said, "I'm working at Google, a place where there is unprecedented data that didn't previously exist, and unprecedented power," thanks to all the power to process all their cloud architecture and products, "what if I started applying this to identifying," what else, "cats on the internet?" And what we found was, all of a sudden, a quantum leap in computers being able to identify cats.

And taking this back to the start really quickly, what is artificial intelligence? Well, it's teaching computers to think like humans. It's bridging that gap. Why was a calculator smarter than all the way back in the '60s, and yet looking at a cat and saying, "Oh, that's a cat," the most simple, human thing that you do, fractional, without any thought, beguiles computers? And once you figure that problem out, what can you do with it? So deep learning figures that out. And all of a sudden, this thing catches on like wildfire at Google. There's a couple of employees from this guy's division, they take over to their Google Translate division. I'm talking hundreds of the smartest people in the world work on this problem for a decade, and they essentially say, "We can do better than you." And that group says, "No way, don't even try. But, it's Google, we have to let you try, but you can't do it. Oh, and by the way, don't do French. You'll never beat us in French." So do you know what those two guys do?

Lewis: They go French.

Bleeker: "We're going to beat you in French." And I believe it only takes them a matter of months to topple the combined works of hundreds, as I noted, of the smartest engineers in the world working on one of the most complex problems. At this point, it was an all hands-on deck at Google, and in 2012, I believe, deep learning was in a handful of projects. I'm talking, count it on one hand. And within three years, it was in over 1,200 projects at the company. It literally invaded every single product. I mean, self-driving cars had actually plateaued. For all the hype in self-driving cars, they stopped updating their self-driving car progress at this time. And then they put deep learning in it and it was like an explosion.

So it's one of those enormous moments of serendipity in technology. If this intern hadn't started at Google, the right place with the right amount of data, the right processing power to start this discipline that a few dozen people in the world were looking at, we might be years behind technology. And I think very few people understand this kismet moment and the explosion it started. And that's where we're truly at in AI, because now this is going everywhere. Most companies are a couple of years behind Google, I'll say. And I remember meeting with NVIDIA in December of 2014, and they had relatively recently started working on the product, and deep learning in their products, and they had said, "We're using deep learning to identify 39% of objects in our self-driving car model." Like, oh, that's nice, but it'd better be a lot better than that.

Lewis: Right, yeah.

Bleeker: And then I met with them in July of 2015, and they said, "Eric, we're over 90. And the progress is increasing." And that was my holy cow moment. I went back and talked with a lot of people that owned NVIDIA at that point, it's up over 1,000%, for being a company on the cusp of it. In any case, there's my explanation of why artificial intelligence, why this deep learning is this big bang moment that progresses to a whole new level, and why we're now sitting here at the moment and going, "This is different than the '60s. This is different than the '80s." That skepticism, I believe, needs to be gone. Now, the question is, how big is this, what can it change and how can I start getting my crystal ball out to see how this changes everything in five years?

Lewis: So, to distill that down to a couple of bullet points. I think what we have is, there's been this long, maybe, ideology or philosophy of what AI could be or might become. That meets the computing power that we currently have. It also meets all these amazing data collection and Big Data practices that come into vogue once the internet really takes off. You have things like search engines and social media companies. And you have the, by nature, very rapid, exponential ascent of progress that machine learning creates, because you have machines learning and then being able to take the best-performing machines and replicate that out. I remember you had this presentation at Fool Fest, which is one of our annual get-togethers, and you showed, I think it was a clip from NVIDIA testing, what was it, computers playing hockey?

Bleeker: Yeah, robots.

Lewis: And the explanation he had provided, I forget who was narrating --

Bleeker: Jensen Huang, their CEO.

Lewis: Their CEO. He was saying they would basically test out all these different robots, attempting to score a goal in hockey, in this very simple game of hockey. And they would take whichever one had the best approach, best calculated approach, give it to all the other ones for the next iteration and so on and so on and so on. And when you have that testing environment, that just means it's not going to be a straight, linear line of progress. It's going to shoot up real quick.

Bleeker: Yeah, and I don't know how quickly I want to bring this into sci-fi land here, but what was amazing about that was, UC Berkeley had actually been teaching robots using these techniques in the real world. But it's so hard to learn at that rate. You need to go reset the puck, you need to have someone hand it to them, and it can only learn from so many examples. So, what NVIDIA had done was built a real-life simulator, and they called this reinforcement learning, and it totally revolutionizes the way that robots can be something completely new.

Now, I want to note, where does this get crazy? Google, we're always updated on how they've done 3 million miles and data is so important to deep learning. What they discovered was, they could actually start stimulating all their driving. So they have 3 million miles. They actually went and built a city out in the real desert that's like a real city, and what they do is simulate a thousand times more miles in their simulations to look for weak points where their self-driving car doesn't know how to handle the real-world situation. And only when they identify problems in their simulation do they bring their cars out to their city and train it over and over again to figure out how to do that.

Lewis: So they're taking issues that they're recognizing in digital simulations, and then forcing the computer system to actually deal with it in the physical environment of being out in the street.

Bleeker: Yeah. So this changes everything. It's truly wild.