Stuff I'm Bad At: The Art of Abstraction

"Learning is any change in a system that produces a more or less permanent change in its capacity for adapting to its environment."

-- Herbert Simon 


Usually, I write about things that I'm comfortable with, and maybe even reasonably good at doing. Because a blog called "Stuff I'm Bad At" would be full of parallel parking, deciding how much food to order, and writing thank you notes on time. Who wants to read that? 

But today, I'm going to talk about an immense learning challenge I have, and it's going to be way more interesting than anything else I could write about. My problem lies in the task of abstraction. Specifically, I have trouble abstracting away from the internals of the "black box" of what a program is doing without mixing it in with the interface (what the user or, in our case, programmer, sees). This applies both to understanding and writing code, and it is the biggest hurdle I face on my path towards becoming a software engineer.

"Wonderful, but not incomprehensible."


What I love about programming is that it can explain and generate wonder, all at the same time. As Herbert Simon writes in The Sciences of the Artificial:

For when we have explained the wonderful, unmasked the hidden pattern, a new wonder arises at how complexity was woven out of simplicity.

The idea of unmasking simplicity is very much connected with the concept of abstraction, which I refer to in the context of programming as the practice of concealing unnecessary or irrelevant details of a program, and surfacing only its essential features. A programmer who is able to abstract her way around a seemingly complex system and grab ahold of only the relevant details of that system to apply to her particular task has potentially endless power. For me, learning how to do this is akin to discovering a new natural law. 

Programmers rely on abstraction to build clean and intuitive interfaces that allow others to interact with their programs. An interface, broadly speaking, is the meeting point between an inner and outer environment.

For example, imagine you have a mug (hopefully you don't have to imagine too hard because mugs are pretty awesome). Your mug is made of a solid material, has a capacity of 16 fluid ounces, and has a handle -- these qualities serve as affordances that guide you to the mug's proper use: to hold liquid and physically maneuver that liquid with one hand. The mug is your interface to drinking coffee, tea or other beverage. In order to understand the mug's intended task, you don't need to know how the mug was constructed, or whether it's ceramic or porcelain or glass. You just need to know that it's a mug. 

Applying this model to a code example, consider Underscore's function signature for the array method _.indexOf:

_.indexOf(array, value, [isSorted]) Returns the index at which value can be found in the array, or -1 if value is not present in the array.

  _.indexOf([1, 2, 3], 2);
  => 1

Just like how anyone can pick up a mug and know what it's for, a programmer should be able to read the _.indexOf function signature (the interface) and quickly grasp the essential features of the function it represents, without needing to "go to the mug factory" by diving into the code that comprises the function line by line.

And just like how a mug is designed to present its most obvious use to the external world (the outer environment), so too must a programmer take care to design an interface with clarity and precision; by naming functions and inputs appropriately, for example. This helps account for weirdos like you and me who otherwise might try to use their program in some way it was never designed for.

So, what happens when we breach the mug factory and go "under the hood" of _.indexOf?

From the Underscore documentation:

_.indexOf = createIndexFinder(1, _.findIndex, _.sortedIndex);

That doesn't look so bad, right? As it turns out, _.indexOf is simply an invocation of a generator function, createIndexFinder, which takes in 3 parameters. Let's find that helper function...

function createIndexFinder(dir, predicateFind, sortedIndex) {
  return function(array, item, idx) {
    var i = 0, length = getLength(array);
    if (typeof idx == 'number') {
      if (dir > 0) {
          i = idx >= 0 ? idx : Math.max(idx + length, i);
      } else {
          length = idx >= 0 ? Math.min(idx + 1, length) : idx + length + 1;
    } else if (sortedIndex && idx && length) {
      idx = sortedIndex(array, item);
      return array[idx] === item ? idx : -1;
    if (item !== item) {
      idx = predicateFind(, i, length), _.isNaN);
      return idx >= 0 ? idx + i : -1;
    for (idx = dir > 0 ? i : length - 1; idx >= 0 && idx < length; idx += dir) {
      if (array[idx] === item) return idx;
    return -1;

Welcome to the mug factory! At this point, you've likely forgotten why you're even here. I have, and I'm writing this thing.

My challenge is to avoid taking unnecessary tours of mug factories, while also building my own super awesome mug factories. I'm making good progress on the first part of the task: it takes some discipline, but I've definitely become more sensitive to internal and external flags telling me when I don't need to know the internals of some piece of code, and it's OK to move on. 

What's much, much harder for me is building the mugs and the mug factories, or programming both the interface and the "black box" code that you shouldn't worry about.

It's a super weird sensation to know that you're writing code that a future version of yourself will deliberately ignore, and at the same time having to remain cognizant of how Future You and other programmers will interact with the interface to that code. Coding becomes something of an out-of-body experience.

For me, the real problem is that I've never been particularly good at thinking in abstract terms. My mind jumps into details and keeps digging until every possible nugget is unearthed. This impacts how I reason about a program because I can easily lose sight of what the hell it is I'm supposed to be returning.  

In other news, I've learned that I have far less mathematical intuition than I previously assumed. Apparently being able to do basic math in your head really fast doesn't actually make you a mathematical thinker.

"The best thing that can happen to a human being is to find a problem, fall in love with that problem, and live trying to solve that problem, unless another problem even more lovable appears."

-- Karl Popper 


I am in love with my problem. I'm positively obsessed with it. Who could wish for a better, more lovable problem? We all should be so lucky to be made aware of any deficiencies impeding us from doing our best work. Otherwise, we would never know to improve!

After sharing my situation with a number of people who I respect and admire, I've put together a few strategies to help address this challenge:  

  1. Draw pictures. I've found that mapping out a problem on paper helps me tremendously, which makes sense since the very act of drawing something to represent something else is a method of abstraction. 
  2. Write your test suite first. As the maybe 3 readers of this blog know, I'm a big fan of test suites. By keeping your eye on the prize, you're less likely to go astray. At the very least, always know what a function or a program is expected to return to the person who calls it. Write it down in a comment at the top of your code if you need to.  
  3. Stay vigilant. I spend a good portion of my day wrestling with myself to stop thinking so much like myself, and imagining what it's like to think in this totally different way. It's a little exhausting. Eventually, it will become less so (I hope). 
  4. Find examples. When I witness a classmate making a cognitive leap, I'll ask how they got from Point A to Point B. I find it super interesting to hear how others visualize and reason about problems, and I hope that I'll eventually get to a similar level of intuitive cognition. 
  5. Have empathy. This particular strategy is fairly easy for me to enforce. It feels very natural for me to imagine how someone else would feel in one situation or another, and this will help me build better interfaces that account for a variety of potential users. 

If you have other strategies to share, please do so in the comments!