Education

[Home]

Table of Content

Educational resources for beginners
Resources for intermediate to advanced practitioners
- Advanced books tbd
- Organized collections of important papers tbd
List of important conferences and workshops tbd
List of important journals tbd
Tools & frameworks tbd

For each entry provide:

a short summary
your name
date of entry

Educational resources for beginners

Books

Machine learning (general)

Information Theory, Inference,and Learning Algorithms
[webpage] [pdf]
David MacKay
Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo codes, and digital fountain codes - the twenty-first-century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, the book is ideal for self-learning, and for undergraduate or graduate courses. It also provides an unparalleled entry point for professionals in areas as diverse as computational biology, financial engineering and machine learning.
[Added by Mehdi Cherti on August 21, 2020]
Mathematics for Machine Learning
[webpage] [pdf]
Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. To be published by Cambridge University Press. 2019
The book is not intended to cover advanced machine learning techniques. Instead, it aims to provide the necessary mathematical skills to read those other books. It consist of two parts: 1) Mathematical foundations, 2) Example machine learning algorithms that use the mathematical foundations.
[Added by Susanne Wenzel on October 10, 2019]
An Elementary Introduction to Statistical Learning Theory
Sanjeev Kulkarni and Gilbert Harman. Wiley, 2011
[webpage]
For those interested in developing a strong foundation before jumping into the deep end, this is (arguably) the best place to start. It is a beautifully written, easy to understand book that focuses on the theoretical foundations of learning.
Note: This book is available in electronic form via the FZJ central library.
[Added by Fahad Khalid on December 14, 2018]
Pattern Recognition and Machine Learning
Chrisopher M. Bishop, Springer Information Science and Statistics, 2006
[webpage] [pdf]
A comprehensive introduction to the fields of pattern recognition and machine learning. It is aimed at advanced undergraduates or first-year PhD students, as well as researchers and practitioners. No previous knowledge of pattern recognition or machine learning concepts is assumed. It include a comprehensive coverage of topics such as probabilistic graphical models and deterministic inference methods, and emphasizes a modern Bayesian perspective.
[Added by Susanne Wenzel, December 2018]
An Introduction to Statistical Learning: With Applications in R
R. Gareth James, Daniela Witten, Trevor Hastie & Robert Tibshirani. Springer Texts in Statistics. Second Edition, 2009
[pdf]
This book provides an overview of statistical learning methods. It is accessible to a very wide readership outside mathematical and computer sciences. It is a good starting point for beginners to acquire the basics.
[Added by Sarah Genon, December 2018]
The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
Trevor Hastie, Robert Tibshirani & Jerome Friedman. Springer Texts in Statistics, 2013.
[pdf]
This book covers the content of the previous one (An Introduction to Statistical Learning) but goes deeper into the mathematical foundations and cover a broader spectrum of methods. It is accessible to a wide readership but requires a basic academic background in statistics.
[Added by Sarah Genon, December 2018]

Deep learning and Neural Networks

Neural Networks and Deep Learning
Michael Nielsen. Determination press, 2015
[webpage]
Freely available online in HTML format, this is a beautifully written book that introduces neural networks in a unique way. It constitutes both theoretical and applied aspects of learning with shallow and deep artificial neural networks. The contents go all the way from implementation of a neural network from scratch in Python, to the derivation of the fundamental equations of backpropagation, with further insights into the inner workings of deep networks. This book is a resource for developing a thorough foundation in neural networks. In terms of the various types of deep network architectures, it only covers introductory material on Convolutional Neural Networks (CNN). Other network types and advanced material are not covered.
[Added by Fahad Khalid on December 14, 2018]
Deep Learning
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. MIT Press, 2016
[webpage]
Perhaps considered as “the book” on deep learning, it is an excellent resource. The authors are forerunners in the field, and have made many important contributions. For each topic, not only is there a general introduction and explanation, there are references to important papers throughout the book. In certain cases, reading the book helps develop a basic understanding of the concept, which then serves as good preparation for delving into the papers. A wide variety of topics is covered (at least the most important aspects), each of which discusses the mathematical foundations as well as insights from research. Please note that this is an advanced text and might appear a bit intimidating without a good foundation.
Note: Individual chapters are available free of charge in PDF format via the book's website.
[Added by Fahad Khalid on December 14, 2018]
Fundamentals of Deep Learning: Designing Next-Generation Machine Intelligence Algorithms
Nikhil Buduma. O'Reilly Media, 2017
[webpage]
This book starts by providing a brief introduction to neural networks and certain essential fundamentals. From then on, it covers a wide variety of topics in deep learning such as convolutional networks, autoencoders, sequence analysis with recurrent networks, memory augmented networks such as Neural Turing Machines and Differential Neural Computers, and finally deep reinforcement learning. Each topic is accompanied by code examples in TensorFlow (without Keras though), which lowers the barrier when moving from concept to realization. Moreover, most chapters include references to important relevant papers. However, the text can be a bit difficult to following in certain places. Nevertheless, overall, it is a very good resource, especially for those working with TensorFlow.
[Added by Fahad Khalid on December 17, 2018]

Tutorials and Videolectures

Die Lernplattform für Künstliche Intelligenz
https://ki-campus.org/
Collection of different material about AI and ML
webpage is German, but it also provides and links to suited english material
[Added by Susanne Wenzel, August 20, 2020]
Spinning Up in Deep RL
webpage
A comprehensive educational resource by OpenAI about deep reinforcement learning (deep RL)
list of key papers
[Added by Susanne Wenzel on October 23, 2020]
DeepLearning Bootcamp
A repeated event covering different aspects of full stack deep leaning and thought by great practitioners from academia and industry (openAI, Facebook, Tesla, Uber, Kaggle, Stanford, UToronto) that now offer latest material as online course, for free.
https://fullstackdeeplearning.com/
From the webpage:
- There are many great courses to learn how to train deep neural networks. However, training the model is just one part of shipping a deep learning project. This course teaches full-stack production deep learning:
  - Formulating the problem and estimating project cost
  - Finding, cleaning, labeling, and augmenting data
  - Picking the right framework and compute infrastructure
  - Troubleshooting training and ensuring reproducibility
  - Deploying the model at scale
- The course is aimed at people who already know the basics of deep learning and want to understand the rest of the process of creating production deep learning systems. You will get the most out of this course if you have:
  - At least one-year experience programming in Python.
  - At least one deep learning course (at a university or online).
  - Experience with code versioning, Unix environments, and software engineering.
- We will not review the fundamentals of deep learning (gradient descent, backpropagation, convolutional neural networks, recurrent neural networks, etc), so you should review those materials first if you are rusty.
  [Added by Susanne Wenzel, August 20, 2020]
YouTube channel by 3Blue1Brown
Neural Networks
Great intro into multilayer perceptrons with very nice visualizations as starting point for beginners in deep learning. It requires no background and is intended to convey the intuition of deep learning. Following videos deal with the training of NNs by backpropagation. Below the video you will find further readings.
[Added by Susanne Wenzel, March 29, 2019]

Survey papers

and other easy to understand papers
...

Resources for intermediate to advanced practitioners

Advanced books

Organized collections of important papers

List of important conferences and workshops

probably sorted by domains

List of important journals

probably sorted by domains

Tools & frameworks

Tutorials and samples
...
Information about how to reach out to colleagues experienced in the tool of interest
...

[Home]