Six months ago I didn't understand the concept of abstraction. Now it comes up almost daily. It’s foundational to my thinking on everything from software to entrepreneurship. I can’t believe how simple it seems. When I finally grokked abstraction, it felt like my first taste of basic economics. Given a new framework, something that had always been there, intuited but blurry, came into focus.
The implications extended well beyond software. GPS and cell phones were now abstractions. Move up a few layers there's Uber. Every food item at the grocery store became an abstraction waiting to be combined with other abstractions in the best tasting way.
To the software veteran this may seem pedestrian, but for me it was a powerful new way of seeing. Marc Andreesen's quote, "Software is eating the world," took on new meaning. The light bulb came on. "Software is eating the world", I thought. Wasn’t it obvious? Software had been improving our lives by encapsulating the complexity of the world for decades. Thanks, in part, to abstraction.
So, what had changed? Why did I now understand? I joined a software company whose business is abstraction. Our system's value hinges on the combinatorial nature of its components. A hundred components that don't communicate build nothing, while twenty that do build an amazing number of applications. What lets them communicate? The right abstractions.
How do we recognize the right abstractions? Like so many things, we frequently find them based on what they are not. We tend to know intuitively the way things should work. If they don’t, there’s often been an abstraction violation.
Recently a D3 scatterplot was working well for me, until I gave it a date for the X-axis value. It should have worked. Why didn’t it? It was hard coded to support numbers. The changes to clear up the abstraction violation were simple, but this illustrates my point. With the right abstractions in place, we come to expect things to just work. When they don’t, we notice. When we notice, we fix them. When we fix them, our world works better. Repeat cycle.
Steeped in examples of abstraction, I couldn’t help but understand.
The right abstractions are how a set of components from the Xap Store …
get used in this data-flow by a developer . . .
and become this data application for a data scientist . . .
and this data application for a researcher . . .
You may notice, these two “Xaps” (web-based data applications built with the Exaptive Studio) aren’t the same. One models State of the Union Addresses, the other Congressional hearing documents. They’re arranged differently. They’re styled differently. But they share the same topic-modeling algorithm and visualization and UI components, code modules from different languages and frameworks written without knowing anything about each other that are nevertheless combinatorial.
Exaptive CEO, Dave King, recently spoke on the difference between modular and combinatorial code. The distinction hinges on the right abstractions. I won't steal Dave's thunder, but, to summarize, modularity is an ability to connect with something else, even just one other thing. Combinatorial is modularity plus. It is the loose coupling of inputs and outputs that enable unexpected combinations and connections, reusability plus repurposing. (Check out Dave's talk here. It's worth watching!)
This ethos of combinatorial over modular helped one of our data scientists build a web-based data application and enabled me to develop a financial dashboard prototype in 45 minutes (despite being a novice programmer). And all of it boils down to the simple but powerful concept of abstraction. In the right hands, there’s no chance it won’t fundamentally change the world. Again and again. Like it has, time after time. Iteratively making the world a better and better place. That’s something I should have learned in school.