History of Artificial Intelligence

Is AI just a temporary hype, like virtual reality and cryptocurrency? Or is there more behind it?

Hardly any other topic in today’s technological world is discussed as persistently as AI. For the pessimists, Europe and Germany are threatened with losing touch with the USA and China in global competition and losing their prosperity. Others, on the other hand, paint the specter of an army of millions of unemployed people on the wall in view of the performance of AI and ML, which is no longer needed thanks to intelligent, learning robots.

Contemporaries with a positive attitude, on the other hand, see huge potential in AI in particular — especially for a location like Germany, which has already successfully responded to the challenges of the Internet of Things with Industry 4.0. But what does reality really look like?

Dartmouth Conference — The Birth of AI

AI looks back on a longer history than some people might believe.

Already in the 40s and 50s, scientists from different disciplines started talking about the possibility of creating an artificial brain.

In 1950 Alan Turing published a groundbreaking paper in which he speculated on the possibility of developing machines that could think. He noted that “thinking” is difficult to define and as a result developed his famous Turing Test. This test was about the fact that if a machine could manage to have a conversation that is indistinguishable from a conversation with a human being, then it was justified to say that this machine could think.

The Turing test was the first serious proposal in the philosophy of artificial intelligence.

In 1956, the famous Dartmouth Conference was held, attended by the leading researchers and scientists of the IT industry at the time. It was at this conference that the term artificial intelligence was first used.

The 1956 Dartmouth Conference was the moment when AI got its name, its mission, and its key players, and is generally considered the birth of Ki.

The years 1956–1974 after the Dartmouth Conference are now considered the golden years of AI. It was the era of discoveries and breakthroughs in the field. During this time, programs were developed which were met with pure astonishment by the public.

Computers were from now on able to solve algebra problems, prove mathematical theorems, and even learn English. Few people would have thought at that time that machines could even achieve such intelligent behavior. There was great optimism among researchers. It was even predicted that a fully developed intelligence machine would be developed within the next 20 years.

Many early AI programs used the same basic algorithm. To achieve a goal (like winning a game or proving a theorem), they would walk towards it step by step (by making a move or logical inference), as if they were walking through a maze to find the solution, and return a few steps back whenever they reached a dead end. The main difficulty was that for many problems the number of possible paths through the “labyrinth” was simply astronomical (a situation known as “combinatorial explosion”). The researchers would reduce the search space by using heuristics or “rules of thumb” that would eliminate many paths that would probably not lead to a solution.

First AI Winter 1974–1980

1974–1980 was the time of the first AI winter. In the 1970s, the AI was exposed to criticism and financial setbacks. AI researchers had not properly assessed the difficulty of the problems they faced. Their enormous optimism had set expectations incredibly high, and when the promised results failed to materialize, the financial resources for AI were steadily reduced. The agencies funding AI research was frustrated at the lack of progress and eventually cut almost all funding for undirected AI research. At the same time, the field of connectionism (or neural networks) was almost completely shut down for 10 years.

There were many reasons for the setbacks in the field of AI. But the main reason was limited computer power: there was not enough memory or processing speed to achieve anything really useful.

The Spirit of Optimism 1980–1987

From the 1980s onwards, things finally took a turn for the better. At that time, many companies around the world were developing special AI systems called “expert systems”. An expert system is a computer system that mimics the decision-making ability of a human expert.

Expert systems are designed to help people solve more complex problems like an expert by deriving recommendations for action from a knowledge base. Via if-else-statements, human knowledge can be presented in a way that is understandable for computers.

At the same time, the Japanese government began to invest massively in AI.

Another encouraging event in the early 1980s was the revival of connectionism or artificial neural networks in the work of John Hopfield and David Rumelhart. Once again, AI was on the up and up.

The second AI Winter 1987–1993

The business world’s fascination with AI rose and fell in the 1980s in the classic pattern of an economic bubble. In the late 1980s and early 1990s, AI suffered a series of financial setbacks. More than 300 AI companies were closed, went bankrupt or were taken over at the end of 1993, ending the first commercial wave of AI.

1993–2011, Major Developments in AI

The more than half-century-old AI territory has finally reached some of its oldest goals. AI has been used successfully throughout the technology industry, albeit somewhat behind the scenes. Part of the success was due to increasing computer performance, and another part was achieved by focusing on certain isolated problems. However, the reputation of AI, at least in the business world, was not at all positive, as it could hardly be used for commercial purposes.

One of the groundbreaking successes of the AI, was IBM’s Computer Deep Blue, which in 1996 managed to defeat the then chess grandmaster Kasparov in six games of chess under tournament conditions.

In the early years of the 21st century, Big-Data-Era was born. The era of large amounts of data. Together with ever cheaper and faster computers, machine learning techniques were used to solve problems in industry and business. But it was to take until about 2011 before the great AI revolution began, which continues to this day.

2011 — Now, Deep Learning Revolution

In addition to Machine Learning, another technique of AI came into the limelight at this time. Deep Learning is based on the concept of so-called artificial neural networks. Both terms will be explained in detail in the next lesson along with some more terms related to AI. At this point, I would like to ask you to consider Machine Learning and Deep Learning as two approaches to the implementation of AI and not to question exactly what it is all about.

Both technologies had been around for many years until then. But while Machine Learning dates back to the 60s, the foundation for Deep Learning was only laid in the 80s. And since that time, Deep Learning has hardly stepped beyond the publications in scientific papers.

While machine learning had already been used for commercial purposes for years, the artificial neural networks of deep learning did not become mass-producible outside research until around 2011.

In fact, all major breakthroughs in AI in recent years can be attributed to deep learning and artificial neural networks. Without Deep Learning we would not have self-propelled cars, no computer translators that have literally replaced dictionaries. No intelligent language assistants like Alexa and Siri, no chatbots and many other technologies that we now take for granted would simply not exist.

The artificial neural networks are our best approach to real artificial intelligence. A machine that learns, even as Alan Turing imagined it back then.

In fact, there is even debate if the Turing test has already been passed. In May 2018 Google demonstrated to the general public how the app Google Assistant, which is based on neural networks, was able to reserve a hairdresser appointment by phone. The person on the other end of the line did not even notice that it was an app.

3 years earlier, in 2015, the company DeepMind built a complex neural network that was able to beat the world’s best player in the Chinese game Go.

The special thing about this game was that there are more possibilities for every move than there are atoms in the universe. For this reason, the machine could not calculate many moves in advance like in chess. Instead, the AI really had to “learn” to play this game. Just like a human. AI developed what you might almost call intuition.

Where this is going in the next few years can only be speculated upon. The fact is that AI-based on Deep Learning not only performs amazingly well in everyday life but has countless commercial applications that more and more companies all over the world are taking advantage of.

But it wasn’t until around 2011, plus or minus one, depending on who you ask, that the conditions were right for a true AI revolution that made the amazing technologies I mentioned earlier possible.

This AI revolution, or what some already call hype, is still going on today and the end is far from being reached — on the contrary, as we will see later on.

There are many reasons why it took so long for the almost decades-old Deep Learning technology to become marketable and to almost take over the AI segment.

In my opinion, there are exactly two reasons:

  • Data
  • Computing power.

Data and Computing Power

The amount of data we generate in this world is growing exponentially. Since the 1980s, the world’s technological capacity to store information has doubled about every 40 months. From 2012, 2.5 exabytes (2.5 × 1018) of data will be generated daily. Based on some forecasts, the global data volume will grow exponentially from 4.4 Zettabyte to 44 Zettabyte between 2013 and 2020.

Machine learning and deep learning algorithms require data to learn to perform specific tasks. However, artificial neural networks benefit much more from large amounts of data than machine learning algorithms. The accuracy of the deep learning algorithms becomes better and better as the amount of data increases. On the other hand, the algorithms of machine learning reach a threshold at some point. Beyond this threshold, their accuracy cannot be improved with a growing amount of data.
This means that the large amounts of data in today’s numbers are like a never-ending fuel for the artificial neural networks of deep learning.

As a rule, deep learning models are much more complex than machine learning models. Artificial neural networks have many parameters that have to be optimized. Accordingly, there are many more mathematical calculations. Before the time around 2010/11 the computing power was simply not sufficient to train very complex deep learning models. On the one hand, the training of these models took very long and on the other hand, the necessary computing power was very expensive.

But in the time of greatest need, the gaming industry came to our aid. The graphics cards, which were originally designed to render computer games, are perfectly suited to perform the necessary calculations.

The graphics cards are able to parallelize many simple operations. Other than a CPU, which performs complex calculations, but only very few in parallel.
As you will see later in this course, the mathematical operations in a neural network are not that difficult. There are just a lot of them. These can now be executed in parallel on a graphics card. This reduces the time needed to train a deep learning model enormously compared to conventional CPUs.

Over the last few years, GPUs for deep learning have been optimized even further. Meanwhile, even GPUs especially designed for deep learning are released. And even more, exotic hardware is being developed for this area of AI. For example tensor processing units that beat GPUs even further.

A Bright Future

So as we see we are living at a very convenient time for AI. Especially when it’s based on deep learning. The computing power we need for neural networks is increasing from year to year and is getting cheaper and cheaper. On the other hand, the amount of data that we can use to make deep learning models even more powerful is also growing.

In the commercial field, Deep Learning is becoming one of the most advanced and useful technologies. Companies use Deep Learning to gain valuable insights from their large amount of data. This is done to deliver innovative products and better customer experience, thereby increasing revenue opportunities for their market.

The increasing use of deep learning technology in various industries such as automotive, finance, advertising, and telecommunications is driving the growth of the deep learning market. The increasing adoption of cloud-based technology, the frequent use of deep learning in the analysis of big data, and the increasing applicability in healthcare and autonomous vehicles are accelerating growth. The growing demand for deep learning for database systems, fraud detection, and cybersecurity is also driving growth.
According to forecasts by Kenneth Research and Zion Market Research, the global deep learning market is expected to experience annual growth of 46% and 49% respectively in the period 2019 to 2024.
The research institute Tractica comes to similar conclusions. According to a report by the institute, sales of AI-based products and services will grow from 4 billion to 37 billion US dollars between 2019 and 2025.

Deep learning today is the Internet in the mid-90s. Those who position themselves correctly today and begin to exploit the potential behind it will win in the long run. Especially the companies with huge amounts of data.

This spirit of optimism is slowly beginning to arrive in Germany.

According to IDG’s study on Machine Learning and Deep Learning 2019, around 30 percent of German companies intend to focus intensively on Machine Learning and Deep Learning in the coming year. CEOs and IT departments, in particular, are pushing this technology. According to the study, Machine Learning and Deep Learning also rank third in the list of the most important topics in the IT sector, behind Cloud Computing and Cyber Security. Furthermore, almost 60 percent of German companies already use at least one AI application — and the trend is rising. Around 40 percent of AI users want to use it to make internal processes faster and more efficient