As part of our time during the Google AI Residency, we wanted to find a better way to study and understand important machine learning papers and ideas. We found that many papers often assumed a set of requisite knowledge, which prevented us from deeply appreciating the contribution or novelty of the work.
To this end, we designed Depth First Learning, a pedagogy for diving deep into machine learning by carefully tailoring a curriculum around a particular paper or concept in small, focused discussion groups. So far, we've created guides for InfoGAN, TRPO, AlphaGoZero, and DeepStack.
1. A small self-contained group convenes online to understand a particular paper in depth with one or two preselected group guides.
2. The guides plan ahead, writing a document that maps the dependencies from the target paper back to sound source material and problems sets.
3. The group discusses the dependencies over 5-6 weekly sessions. They write up solutions to problems and add comments on the document itself, creating a living representation of their understanding.
The document is made public afterwards and all credit goes to the guides and the group.
Since our launch, we’ve received very positive feedback from students and researchers around the world. Now, we want to run new, online classes around the world.
"This is so wonderful — a one stop shop for understanding fundamental topics building up to important papers, at depth."
Ashish Vaswani (Senior Research Scientist, Google Brain, first author of “Attention is all you need”)
“You’re building the book on modern machine learning.”
Nal Kalchbrenner (Staff Research Scientist, Google Brain, ex-DeepMind, author of PixelRNN, WaveNet, WaveRNN)
“Awesome initiative — we can use a lot more resources like this in our community."
Peter (Xi) Chen (CEO of Covariant.AI, ex-OpenAI, first author of InfoGAN)
We are looking for guides to lead new DFL study groups, each based around a high-quality curricula targeting a particular paper in machine learning.
Towards this end, we are launching the Depth First Learning Fellowship. Chosen fellows will each be given a $4000 grant to help design the curricula for and lead a DFL group. The group will run for 5-6 weeks remotely via video conferencing.
We will support each group by finding 6-8 motivated students with the relevant background experience, and connecting fellows with experts in the field to act as advisors.
Here are the characteristics that we are looking for in candidates:
• Exhibit mathematical maturity: Fellows target meaningful papers which depend on fundamental concepts in mathematics, statistics, and information theory. They should understand the fundamentals in order to plan a curriculum.
• Effectively communicate scientific ideas: A primary responsibility of the fellow is effectively communicating ideas central to research papers and moderating discussions among group participants. Previous experience with teaching or organizing classes is a bonus.
• Commit strongly to make the DFL sessions a success: This primarily entails designing curricula with sufficient rigour and exercises for the classes. From previous iterations, we estimate this to take around 12 hours a week. We will help you in this process with regular feedback.
• Value group learning: Encourage positive interactions in study sessions.
Thanks for all of the applications! We received interest from an astounding 113 people, and we are now going over the list. If you applied, you should have received an email from us.
Applications are now closed.
