Is "Learning by Doing" a Myth?

by Avilay Parekh

I am a slow learner. I have to learn the fundamentals of a new skill before I can apply it in some real world project. On the other hand, if I don't apply my newfound skill fast enough, I tend to forget it. But never ever have I been able to learn something while simultaneusly applying it. Which begs the title question.

Exhibit A: Learning Python

In early 2013, I spent a week or so learning Python. And by learning I mean getting familiar with the basic syntax, runtime concepts, and standard libraries. After that I started using it in my own startup to write Machine Learning code using scikit-learn. More than that, I used it for any new side-project that I did, even if it was not ML related. For example I used it for building listy - a TODO web app. And gradually I became good at Python. Soon I was able to confidently use it for other real world client projects when I was contracting after my product startup failed.

Exhibit B: Deep Learning skills

In 2015, a friend and I set about to solve the newly launched Diabetic Retinopathy Kaggle competition with the hope of simultaneously learning about computer vision and Convolutional Neural Networks. At that time I had no Deep Learning skills to speak of. When I tried to use Tensorflow in all of its 1.x ugliness, I quickly became frustrated, because I could not even follow the tutorials.  It took me a good part of the next two years to grok Deep Learning before I could think of doing something like that again.

Exhibit C: Asymmetric Adversarial Self-Play

These days I am trying to become an expert in Reinforcement Learning. In my day job at Unity Technologies, I am doing a very cool project that applies concepts of asymmetric adversarial self-play to a gaming scenario. And I am failing miserably at it. I know my way around single agent "traditional" Reinforcement Learning, but I know very little about adversarial self-play. And learning that with a deadline hanging over my head is not working. At all.

"Learing" what?

The first thing that these experiences taught me was that there are grades of learning. When I was learning Python, I was building upon my existing expertise in programming. At that time I was transitioning from using Ruby for my day-to-day work to using Python. Therefore the incremental effort required to learn Python was not that great. Which is why I was productive within a week. When I set about to learn Deep Learning/CNNs I had very little exposure to Neural Nets in general. Most of my ML related work was using Support Vector Machines, Collaborative Filtering, and Linear Regressions. Jumping in feet first into an application like classifying Diabetic Retinopathy only served to frustrate me. It matters a lot what I am attempting to learn. If it is some brand new skill where I have very little background, jumping in feet first is a bad idea. I need to first learn the fundamental concepts and try my hand at a bunch of toy problems before I take on a real problem.

"Doing" what?

The second thing that these experiences taught me was that there are grades of "doing". With Python, where my learning curve was pretty gentle, I was able to start doing real world projects within a week. But with adversarial self-play, where my learning curve is pretty steep, I am not able to execute on a real world project. For these kind of skills I need play projects to practice my skills on. Let me unpack this phrase a bit. I use this term instead of toy projects because these are real world projects which are hard to do. By contrast, toy projects are easy and usually have existing working solutions, e.g., classifying MNIST dataset is a toy project. But classifying Diabetic Retinopathy is a play project. Another defining characteristic of play projects is that they are not mission-critical, i.e., the cost of failure in these projects is not high. This gives me the time and space to experiment with my newfound knowledge and tinker with the problem space. This soldifies my learning like nothing else does.

"Learning" vs. "Applying"

When do real world but mission-critical projects come in? Like almost every competent engineer, I tend to get very creative at problem-solving when faced with a deadline or any other type of constraint. But in order to effectively problem-solve I need to draw upon my existing skillsets. Which is why, I need to have done a couple of play projects with my newfound skill. This way it will have solidified enough so that I can draw upon it in my time of need. And that is when I graduate from learning the skill to applying it.

Learning Playbook

In short my learning playbook looks like this:

  1. Learn the fundamental concepts and the theoretical underpinnings of the new skill. Use toy projects during this phase of learning.
  2. If it took me over a month to learn the fundamentals, then do a couple of play projects to solidify the learnings. This will possibly take a significant amount of time as well.
  3. Apply my new skill in mission-critical projects.

Photo by Jordan Sanchez on Unsplash