Marc's Blog

About Me

My name is Marc Brooker. I've been writing code, reading code, and living vicariously through computers for as long as I can remember. I like to build things that work. I also dabble in machining, welding, cooking and skiing.

I'm currently an engineer at Amazon Web Services (AWS) in Seattle, where I work on databases, serverless, and serverless databases. Before that, I worked on EC2 and EBS.
All opinions are my own.

Links

My Publications and Videos
@marcbrooker on Mastodon @MarcJBrooker on Twitter

My heuristics are wrong. What now?

More words. More meaning?

Some people who ask me for advice at get a lot of words in reply. Sometimes, those responses aren’t specific to my particular workplace, and so I share them here. In the past, I’ve written about echo chambers, writing, writing for an audience, time management, and getting big things done.

Do you remember Cool Runnings? In the movie, John Candy is a retired bobsled champion, who uses his experience, connections, and lovable curmudgeon character to turn a rag-tag group of sprinters into an olympic bobsled team. A lot of principal engineer types think of themselves this way: they used to bobsled, they don’t bobsled, but they still know the skills and the people and the equipment.

And that worked well enough, while we were still bobsledding.

But we’re not bobsledding anymore.

Many of the heuristics that we’ve developed over our careers as software engineers are no longer correct. Not all of them. But many. What it means for a system to be maintainable. How much it costs to write code versus integrate libraries versus take service dependencies. What it means for an API to be well designed, or ergonomic, or usable. What it means to understand code. Where service boundaries should be. Where security and data integrity should be enforced. What’s easy. What’s hard.

We’ve seen this play out in small ways before. Over the last decade, I’ve frequently been frustrated by experienced folks who didn’t update their system design heuristics to match the cloud, to match SSDs, to match 100Gb/s networks, and so on. But this is the biggest change I’ve seen in my career by far. An extinction-level event for rules of thumb.

But you’re a tech leader, and you need to lead, and leading is heavily based on using your experience to help people and teams be more effective. What now?

The victorious man in the day of crisis is the man who has the serenity to accept what he cannot help and the courage to change what must be altered.1

Let me assume that you want to continue to be a valuable tech leader. You want your teams and organizations to succeed. That you’re willing to sound less smart and less sure, in interests of being right and helpful.

In that case, and I hope that is the case, your job has changed. Your job, for the foreseeable future, is to have the humility to accept that many of your heuristics are wrong, the courage to believe some are still right, and the curiosity to actively learn the difference.

You can’t throw out everything you know. Your taste, your high standards, your understanding of your business and customers and the deep technical trade-offs in your area are more valuable than ever before. This is like that fantasy that people have of going back to middle school knowing all the things they know now2. You’re ahead of the pack in many ways.

But you also need to really deeply question the things you know, and the things you assume. Before you share one of your rules of thumb, you need to deeply examine whether it’s still right.

And the way you’re going to know that, right now, is by getting back on the ice. Build. Own. Get your hands dirty and use the tools. Build something real. Build a prototype. Build a thousand little experiments in an afternoon. Challenge yourself to try to do something you previously would have assumed is impossible, or infeasible, or unaffordable. Find one of the ways that you’re worried that the new tools are going to lead to trouble, and actively fix it. Then examine the things you’re learning. Update your constants.

Over the next couple of years, the most valuable people to have on a software team are going to be experienced folks who’re actively working to keep their heuristics fresh. Who can combine curiosity with experience. Among the least valuable people to have on a software team are experienced folks who aren’t willing to change their thinking. Beyond that, it’s hard to see.

This is going to be hard for some folks. It’s hard to admit where you’re wrong. It’s hard to go back to being a beginner. It’s easy to stick your fingers in your ears and say “No, it’s the children who are wrong”. My advice is to not be that guy.

The good news? It’s as fun as hell. Get building, get learning, make something exist that you couldn’t imagine before.

Footnotes

  1. Winnifred Crane Wygal paraphrasing Reinhold Niebuhr
  2. A fantasy I have never understood. Being 13 once was enough for a lifetime, thank you very much.