What is the black box in artificial intelligence?
It might be really surprising to learn, but a lot of AI machine learning models don't really do work that their programmers or operators fully understand. Although they seem to be getting the right answers, they don't really explain how they work or give any hints about what's going on under the hood. To put it in more human terms, they can't tell you what or how they're thinking.
Because AI is such complicated programming, it's easy for the program to take on a metaphorical life of its own. It's still limited to the functions programmed within it, but sometimes the software comes up with accurate answers without using an accurate process. When this thinking is obscured, it's said to be working in a "black box," also known as "the black-box problem."
Why does it matter if machine learning is white-box or black-box?
An AI's ability to generate an accurate answer is the entire purpose for having it. If the answer's process is hidden from observers, and no one is really sure what it's doing, it's difficult to trust it for precision work. That's why white-box AIs like xAI are so important. With a black-box AI, you get an answer, but if you need to know for sure that the machine understands its job, you simply can't. With a white-box AI, the AI can tell you exactly what it's doing, why it's doing it, and how reliable it is.
For example, an AI in healthcare that's meant to read chest X-rays may be able to identify a case of COVID-19 pneumonia a high percentage of the time, but if there's a black box involved, you can't say how it knows that. In one case, it was observed that the AI wasn't identifying pneumonia at all but observing that the probability that a particular machine's usage would predict COVID-19 pneumonia was high. Although it came up with the right answer often enough, it did it the wrong way, making the results of that AI system sketchy at best.
If this had been done with a white-box AI like xAI, researchers would have known immediately that the AI was utilizing the data incorrectly and would have been able to retrain it right away. Since this happened in a black-box AI, it took some time to work the bugs out, and if it had been fully implemented, it would have given doctors an AI that was essentially guessing all the time and worthless.
Related investing topics
