Human driven decisions

I enjoy the sleep tracking function in Apple Watch. It’s effortless, delightfully designed, and it gives me detailed data on how I have slept every night. But it doesn’t give me any significant knowledge I didn’t already have about how I sleep. It doesn’t help me sleep better. If I have a hard time falling asleep, or wake up in the middle of the night, or if my kid repeatedly head-butts me, I’ll know.

If I wanted to sleep better, I wouldn’t need sleep tracking data to help me. I know I sleep better when I exercise regularly, don’t go to bed too early and don’t sleep next to a 3-year old with nightmares.

But enough about me, let’s talk about software development.

In software development, we love measuring things. Throughput! Lead times! Errors! Performance! We measure anything that can be defined by a number, and we obsess over the numbers.

We collect as much data as possible and dig through it looking for interesting questions to ask, instead of collecting data when it’s needed to answer questions.

Because data is sometimes great, we think it’s always great. Because data is often the best — if not the only — way to answer a question, we use it even when it obscures instead of clarifies. Numbers don’t show nuance, uncertainty or complexity. They give us an illusion of exactness. Data can tell us what is happening, but not why. Data can describe the problem, but not the solution.

And once we start, we tend to use data for everything. It becomes a habit that is hard to kick. Gotta follow up on that velocity metric every week. “Hey team, we completed two fewer points this sprint compared to last. What’s happening?”

Instead we could ask ourselves a simple question: “What could we do to improve how we work?” We are professionals, we know when we are not productive, we know when we can’t focus on the right things, when things are getting in our way and we’re not doing our best work. Make it a habit to ask, listen, learn and improve. Use your judgement, work on your intuition. Get quality feedback.

The products we build are controlled by data too. A/B test every copy change and every button color. Don’t launch anything without measuring its impact.

Instead we could let our decisions be guided by quality feedback from users together with our own vision, empathy, skill, taste and intuition. If we want to make meaningful changes, if we want to reach beyond the local maximum and create something really great, we have to make sure we can optimise for those signals. Makers that understand the problem they’re solving at its core, that has deep empathy for the users of the product, will make better decisions in every turn compared to an A/B testing machine.

We see “data” as a source of objective truth, even when the answers we are looking for are subjective in their essence. We think data will give us the most reliable information, overlooking how misleading numbers can be, and discarding the knowledge that stems from human experience, emotion and intelligence.

Build relationships, engage in conversations and listen to feedback. Make human driven decisions.