I’m just sitting here after another virtual therapy session. Thinking. I know this feels a bit personal, as I usually focus on data visualization, but that’s actually what this letter is about. About my personal journey with data visualization. About getting better at the creative process. About how seeing something we try to imagine makes us feel.
First though, I want to talk about decisions and how we think about them. I also want to talk about what I don’t hear as often: how we feel about them. More precisely I want to talk about how we think and feel during decisions. When we are making a data visualization we make tons of big and small decisions by the time we publish it. How often do you think about how you feel while you are making those decisions?
https://observablehq.com/@enjalot/emotion-tree
How many times have your groaned in your head “ugh, this is going to take forever” when you realize you need to reshape the data? That’s feeling overwhelmed, maybe bored and a little scared. What about those times you say “woah, I didn’t expect to see that!” That’s joy, curiosity and inspiration. There’s always the crunch-time “why did I ever agree to do this project!?!” which is of course feeling stressed and frightened.
Then there are the countless moments that pass without intense feelings but are nonetheless felt. Choosing a higher contrast color scheme, feeling the difference it makes to your eyes. Choosing the formatting and font of the axis tick marks and feeling the slight comfort of additional context. Choosing the sort order, the statistical metric, to filter or not to filter the outliers and missing data, feeling the uncertainty as each choice is made. These are the choices of our profession, which are often discussed in terms of perception and statistics, correctness and effectiveness. I’d like to focus on how it feels to make these choices.
In September of 2011 I made my first “live coding” video about D3, a 40 minute quest to make a bar chart coded in the terminal. I don’t recommend watching it 10 years later, for one thing we are 5 versions further along with D3. I did skim through it though and realized that it does highlight the cost of all of those decisions. As I narrate my thought process I need to type quite a bit of code to enact a decision, if I get it at all wrong I need to stop and think about the implementation. Then I plod along trying to implement a series of decisions heavily based off prior experience and conventional wisdom, rather than the dataset in front of me. All of that feels frustrating.
Today in 2021 I can make an incredible amount of decisions about a bar chart almost at the speed of thought. The implementation of D3 has been abstracted into heuristics and defaults in Plot. Here is the same bar chart remade in less than two minutes:
https://observablehq.com/@enjalot/simple-bar-chart-plot
This feels good, it makes me eager and energetic, I am often awed by what pops up rather than anxious about what is causing my visualization not to show up at all. I get to be surprised and curious because of how quickly I can try out a different decision, I can be confident in my trials instead of nervous about my implementation. If you’re interested in the technical differences that enable these different feelings, I recommend Toph’s excellent Plot for D3 users. If you’re curious about how these improvements were arrived at I recommend starting with Mike Bostock’s 10 years of open source data visualization.
The thing is, Plot isn’t done and neither is Observable. Ive been on my own quest to pursue these feelings while I create for a long time, and it continues. For me it started with Tributary, when EJ Fox and I saw the Bret Victor Inventing on Principle talk in 2011 it opened our eyes to a new way of thinking. We realized we could build tools to improve the way it felt to code, to learn and to design.
This was fun and educational time, we kept adding features and fun things that let us try new ways of making. After a year or so though, the codebase itself got too complicated for me to think about and I would feel dread anytime I wanted to fix or change something. I stopped being able to add to it. At that time I started working for Lever as a “full stack” engineer and learned a lot about product development and software engineering. These skills enabled me to think more clearly about building software and they came in quite handy when I decided to start Blockbuilder in 2015:
At this time bl.ocks.org had grown as an incredibly powerful and popular resource for learning D3. Creating and consuming examples was (and is) one of the best ways to learn a technology with such combinatorial possibilities. Creating examples at that time required quite a bit of comfort with the command line, an environment and way of thinking that many people coming to D3 were not familiar with. Using the same technical underpinnings as Tributary I was able to make an in-browser editor compatible with bl.ocks.org.
This felt great, people who wanted to make examples, or try messing with existing ones didn’t have to think about the process of saving, editing and deploying them. They could just do. As with Tributary though, with time, the weight of software development built up and I was no longer able to move the project forward.
This past weekend after 5 years of serving the D3 community, and a couple of years of barely keeping it running I finally decided to put it to rest. Of course it’s open source, and we’ve archived over 40,000 blocks that were indexed by our search engine, but it is no longer running as a service. This makes me feel sad, I’ll miss what this phase of my journey. I’m grateful that I don’t have to miss the people that it connected me with or the ideas it gave me. I’m already feeling a bit lighter as I move forward in my current phase where I get to work with a phenomenal team on helping people think with data. I feel excited about that.
I’d like to hear how you feel. Let me know <3
It would be impossible to overstate the impact that you (and to a lesser extent block builder) had on the community—so I won’t try 🤪. Thanks so much for these reflections and all you do