In 2019, the total number of Deep learning research papers published in arxiv.org is more than 8000. Deep Learning has solved hard problems such as language translation and cancer detection with very good results. Jobs in deep learning are the most sought after by people who want to build a career in Data Science. Kai-Fu-Lee (who headed Google in China and also worked in Apple and Microsoft) has said ‘There’s only been one AI breakthrough’ and that is Deep Learning.
What is Deep Learning?
Deep Learning had its moment of truth in 2012 in the Imagenet Challenge. Since then it has been used to crack very difficult problems in computer vision, language and speech processing - something that humans have not been able to solve so far in the entire history of evolution.
Is it true that Deep Learning does not work?
Partly because chatbots like Microsoft’s Tay made some real gaffes when chatting with users and had to be pulled out. Or Google’s image recognition algorithm classified humans as gorillas. Examples like the above have earned a lot of bad press for Deep Learning and have created a ‘trust’ problem.
The 'Deep Learning' debate
It is good to see debates on what Deep Learning is capable of and what it is not.
This week there was a debate by Yoshua Bengio and Gary Marcus which covered some aspects of it. People who use Deep Learning to solve difficult unsolved problems know the limitations of the technology. After all, this technology is just 7 years old and there is a long way to go before it can be used as a general-purpose ‘intelligent’ tool to solve all AI problems we can think of.
Our trust in automobiles and airplanes has happened over many decades. And we still have failures. All technologies go through a maturity cycle and have to be tested with several use cases and scenarios to make them robust and trustworthy.
Deep Learning has been mistakenly identified as the technology that can achieve Artificial General Intelligence (AGI). It has been compared to the human brain, just because it uses artificial neurons to enable predictions. But making such comparisons today is absolutely wrong.
The human brain has evolved over more than a million years and has been trained with zillions of use cases and data. Nature has tried many competing alternatives to come up with the current structure of the brain.
A very simple neural network can achieve human-level competence or better that. A complex neural network can carry out the translation in more than 100 languages - something a human being is unlikely to perform.
So there are enough data points to prove that Deep Learning networks have performed very well in specific tasks - Narrow AI. The cost and effort to perform the same tasks by human beings will be enormous. The opportunity cost for not solving these problems is very high. It's true that there are some fundamental enhancements that have to be made to make this technology better.
Deep Learning networks are data guzzlers, cannot generalize well, cannot handle out of distribution (OOD) cases, have a high energy requirement for training, do not have common sense, are not emotionally intelligent, cannot perceive the world with senses, do not really develop understanding like humans and need constant updates to the model to handle newer scenarios. This list will keep going till we come to a point in time when all such problems can get addressed to make Deep Learning networks function similar to a human brain.
A key point to note is that the expectation setting has been very wrong. The chart below shows that Deep Learning is at the peak of inflated expectations.
Bad communication and hype have created a perception that Deep Learning can achieve AGI. Nothing can be far from the truth. Deep Learning will be one of the technologies to achieve AGI, only after the above problems get addressed. When Deep Learning networks have made ‘stupid’ mistakes we have ridiculed them. But the key issue is that the training itself has been biased by humans. So we play a very important role to create models that are unbiased and ethical so that we can trust them to make informed decisions.
Deep learning is a breakthrough technology to process and make sense of unstructured data. There are estimates that unstructured data is growing at a rate of more than 50% every year.
FAMGA companies have used Deep Learning to solve many challenging problems in the last few years and are trying to monetize them now. There is no viable alternative technology available to analyze mountains of unstructured data, get insights and make future predictions. Research scientists are using Deep Learning techniques to solve complex societal problems e.g. climate change. It is a data-driven world today and we have to evolve with data. We have to make more sense of data and utilize our understanding of data to progress humankind.
Deep Learning is a promising technology that we have to invest in and make it better to augment our intelligence.