Getting Started with Machine Learning

A community-driven place to get started with machine learning and AI. This list is not definite, nor sequential, but we hope it’s a good starting place for anyone looking to get into the field. All resources mentioned in this guide are free, and include a little description of why they are useful. Each section has a set of starting points (usually courses, books, blog posts, etc.), relevant papers and project ideas. The most helpful starting points in each section are marked with a ⭐️.

Remember: The two best things you can do are building stuff and running your own experiments.

Contents

1. Basics
2. Mathematics
   • Calculus
   • Linear Algebra
   • Statistics
3. Neural Networks
4. Computer Vision
5. Natural Language Processing (NLP)
6. Reinforcement Learning (RL)
7. Generative Adversarial Networks (GANs)
8. Ethics
9. Community
10. Contributing

Basics

• ⭐️ Stanford CS229: Stanford’s introductory machine learning course. Even though the code is quite outdated, this course explains many fundamental ML algorithms. It also has a section on the mathematical foundation required. Additional links: Python exercises/solutions - Notes - Coursera.
• Practical Deep Learning for Coders (fast.ai) (part 1 & 2): The best course to start with if you are looking to quickly train your own models. This course teaches many clever tricks to improve accuracy and overall performance.
• Deep Learning Book: This book covers everything you need to know to start reading papers. Be sure to write your own code while reading.
• Machine Learning Crash Course: Google’s fast-paced, practical introduction to machine learning (TensorFlow).
• Dive into Deep Learning: An interactive deep learning book with code, math, and discussions. Provides NumPy/MXNet, PyTorch, and TensorFlow implementations.

Resources

• Google Colab: Free GPUs, shareable notebooks.
• r/datasets: Feed of datasets to fuel your own projects.
• Dataset Search: A Google for datasets.

Mathematics

• ⭐️ The Matrix Calculus You Need For Deep Learning: Digging deeper into the math behind neural networks. Understanding this text will be of tremendous help.
• Learning Math for Machine Learning: Essay on why math is/isn’t important in engineering, ML.
• A Programmer’s Introduction to Mathematics: Be sure to read this if you have never read a college level math book before. It not only teaches fundamental math, but also introduces the reader to succesfully reading mathematical literature.

Calculus

• Calculus Learning Guide: realistic learning plan for Calculus.
• Essence of calculus by 3Blue1Brown: Explore Calculus, the study of change, in an intuitive manner.

Linear Algebra

• ⭐️ MIT OCW on Linear Algebra: An in-depth course on everything you need to know about linear algebra and more. It includes exams with solutions.
• Essence of linear algebra by 3Blue1Brown: Build a visual intuition for linear algebra. You probably won’t need to know linear transformations, but this course presents some key insights in how everything fits together.
• The Matrix Cookbook: Reference sheet with many key formulas.
• Computational Linear Algebra for Coders: Linear Algebra taught with a focus on computing.

Statistics

• ⭐️ MIT OCW on Statistics: Interactive, very comprehensive course on statistics. While the video lectures are not available, you can find all readings here. Each reading has corresponding questions. The exams with solutions are available. This course ends with linear regression: the approximate start of machine learning.
• probabilitycourse.com: Free, online verion of ‘Introduction to Probability, Statistics, and Random Processes’ by Hossein Pishro-Nik.

Neural Networks

• MIT 6.S191 Introduction to Deep Learning: Nice overview of what neural networks can do. Includes Python demos.
• DeepMind x UCL | Deep Learning Lecture Series 2020: covers many aspects of deep learning.

Papers

• Gradient-based learning applied to document recognition: one of the classic ML papers.
• Theoretical issues in deep networks

Project ideas

• Build an CIFAR10 classifier.
• Write a neural network from scratch in Python and NumPy.
• Write a basic NN library with layers, activations functions and a training loop.

Computer Vision

• ⭐️ Stanford CS231n - Convolutional Neural Networks for Visual Recognition: Industry standard course on computer vision and CNNs.
• A guide to convolution arithmetic for deep learning: Nice guide on the mathematical principles behind convolutions. Covers padding, stride, pooling, transposition, etc.

Papers

• ImageNet Classification with Deep Convolutional Neural Networks: One of the first papers on large convolutional networks. Even though it’s already outdated at this point, it remains a must-read for anyone looking to understand ConvNets.
• Deep Residual Learning for Image Recognition
• Densely Connected Convolutional Networks

Project ideas

• Write an Imagenette, Imagewoof, Imagewang classifier.
• Build a model that generates new images of cars.
• Implement ResNet in NumPy, JAX.

Natural Language Processing

• ⭐️ Stanford CS224n: Natural Language Processing with Deep Learning: Stanford’s undergrad course on NLP. Includes a lot of additional materials.
• A Code-First Introduction to Natural Language Processing: fastai’s course on NLP.

Papers

Coming soon - Open a PR!

Project ideas

Coming soon - Open a PR!

Reinforcement Learning

• ⭐️ Spinning Up in Deep RL by OpenAI: A large collection of RL tutorials by OpenAI, one of the leading AI research institutes.
• Reinforcement Learning: An Introduction: Very nice overview of many intuitions, mathematical foundations and algorithms you will encounter in reinforcement learning.
• RL Course by David Silver (Google DeepMind): very well organized RL course by one of the creators of AlphaZero.

Papers

Coming soon - Open a PR!

Project ideas

• Write a TicTacToe bot.
• Write a chess bot.
• Build a stock trading bot.

Generative adversarial networks

• DeepMind: Generative Adversarial Networks
• Lectures 5 and 6 of Berkeley’s Deep Unsupervised Learning.

Papers

• Generative Adversarial Nets
• Conditional Generative Adversarial Nets
• Wasserstein GAN
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• cGANs with Projection Discriminator
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• BigGAN
• Progressive Growing of GANs for Improved Quality, Stability, and Variation, A Style-Based Generator Architecture for Generative Adversarial Networks and Analyzing and Improving the Image Quality of StyleGAN

Ethics

• ⭐️ Practical Data Ethics: A practical course of fairness, biases, and more.
• CS 294: Fairness in Machine Learning: List of good readings on fairness.
• 21 fairness definitions and their politics: Talk from the FAT conference.
• NIPS 2017 Tutorial on Fairness in Machine Learning: Presents a ‘toolkit’ researchers and engineers can use to improve fairness.
• Montreal AI Ethics Institute Publications
• Montreal AI Ethics Institute Living Dictionary
• Montreal AI Ethics Institute State of AI Ethics June 2020 Report

Papers

• Ethics of Artificial Intelligence and Robotics
• Concrete Problems in AI Safety
• A Survey on Bias and Fairness in Machine Learning

Project ideas

• Write a blog post detailing the importance of ethics in AI.
• Explore and report the ethics of real world datasets.

Community

You can’t learn hard things alone. Fortunately, there’s a great community ready to help out.

Resources

• gettingstarted.ml ;)
• paperswithcode.com: Machine learning research with corresponding code. A great place to learn to implement literature.

Blogs / publications

• distill.pub: interactive articles about new machine learning research, or new viewpoints on existing work.
• OpenAI Blog
• DeepMind Blog
• Google AI Blog
• Colah’s Blog: Very good explanations of intermediate ML topics.
• Montreal AI Ethics Institute Blog

Newsletters

• ImportAI: Weekly newsletter by an OpenAI director.
• DataScienceWeekly: News, articles and jobs related to Data Science.
• Data Elixir: Machine learning, Data visualization, analytics, and strategy.
• Deep Learning Weekly: Features the latest in industry, mobile ML, academia and more.
• The Algorithm: Weekly AI newsletter by MIT Technology Review.
• Open Data Science: Open Data Science’s weekly newsletter.
• Montreal AI Ethics Institute AI Ethics newsletter

Discussion

• r/MachineLearning: The latest in ML research (advanced).
• r/LearnMachineLearning: A subreddit dedicated to learning machine learning.
• arxiv-sanity.com: An organized directory of the latest ML and stats research papers.
• Official r/LearnMachineLearning Discord server: Probably the best place to get your short questions answered quickly.
• Deep Learning Course Forums: For questions related to any of fastai’s courses.
• stats.stackexchange.com: Statistics Stack Exchange
• math.stackexchange.com: Mathematics Stack Exchange
• Stack Overflow: questions related to programming / software engineering.
• Montreal AI Ethics Institute Learning Community
• Montreal AI Ethics Institute AI Ethics workshops

Papers

• Ten simple rules for getting started on Twitter as a scientist

Contributing to gettingstarted.ml

Thanks for even considering contributing! This guide is community driven, and every pull request is appreciated. Whether you are fixing a typo, adding or updating resources - everything will help make this guide better for everyone. Please be careful when submitting your own content, this is not a place for self promotion but we do value good learning materials. On behalf of every beginner, thank you!

---

Edit on GitHub · Created by Rick Wierenga