Introduction

The basics of modern Artificial Intelligence is machine learning. The most pronounced part is statistical machine learning, with other parts such as pure computing approaches having been left behind by recent advancements. While the wider area of statistical machine learning has been very successful, the recent advancements demonstrate that the specific branch emanating from Artificial Neural Networks is very successful in achieving human equivalent performance in a variety of applications.

Deep learning and statistical machine learning at scale requires usually significant amount of computing resources. The field of deep learning benefited from the proliferation of GPUs originally developed for high end graphical processing and gaming. Another application having fuelled the proliferation of GPUs has been the development of the Bitcoin network where they have been extensively used in the early years for distributed verification of financial transactions. This triad of applications have enabled an ever increasing availability of higher end computing capabilities at a lower cost, allowing research to produce capable systems to demonstrate the power of Deep Learning at scale. To provide an account of the power available by these units we must consider that a typical GPU nowdays boasts thousands of times more computing power of what would be considered a High Performance Computing Cluster for solar scale and weather systems simulation, a couple of decades ago.

This proliferation of GPU enabled technologies and assorted software development frameworks such as Pytorch, and Tensorflow, has made deep learning the most modular, versatile, and scalable solution to for Artificial Intelligence, giving rise to the wide range of applications from computer vision and image generation to language, and advanced information retrieval.

In this context we are going to focus on deep learning and cover all the necessary standard machine learning algorithmic approached required for a profound understanding of Deep Learning.