Getting Big Things Done

In one particular context.

A while back, a colleague wanted to make a major change in the design of a system, the sort of change that was going to take a year or more, and many tens of person-years of effort. They asked me how to justify the project. This post is part of the email reply I sent. The advice is in context of technical leadership work at a big company, but perhaps it may apply elsewhere.

Is it the right solution?

I like to pay attention to ways I can easily fool myself. One of those ways is an availability heuristic applied to big problems. I see a big problem that needs a big solution, and am strongly biased to believe that the first big solution that presents itself is the right one. It takes intentional effort to figure out whether the big solution is, indeed, a solution to the big problem. Bold action, after all, isn’t a solution itself.

Sometimes, in one of his more exuberant or desperate moods, Pa would go out in the veld and sprinkle brandy on the daisies to make them drunk so that they wouldn’t feel the pain of shriveling up and dying. (André Brink)

Because I am so easily fooled in this way, I like to write my reasoning down. Two pages of prose normally does it, building an argument as to why this is the right solution to the problem. Almost every time, this exposes flaws in my reasoning, opportunities to find more data, or other solutions to explore. Thinking in my head doesn’t have this effect for me, but writing does. Or, rather, the exercise of writing and reading does.

The first step is to write a succinct description of the problem, and what it means for the problem to be solved. Sometimes those are quantitative goals. Speeds and feeds. Sometimes, they are concrete goals. A product launch, or a document. Sometimes, it’s something more qualitative and harder to explain. Thinking about the problem bears a great deal of fruit.

Then, the solution. The usual questions apply here, including cost, viability, scope and complexity. Most important is engaging with the problem statement. It’s easy to make the exercise useless if you disconnect the problem statement from the solution.

It is important you feel comfortable with the outcome of this exercise, because losing faith in your own work is a sure way to have it fail. Confidence is one valuable outcome. Another one is a simpler solution.

Is it the right problem?

An elegant solution to the wrong problem is worse than no solution at all, at least in that it might fool people into thinking that the true problem has been solved, and to stop trying. You need to deeply understand the problem you are solving. Rarely, this will be an exercise only in technology or engineering. More commonly, large problems will span business, finance, engineering, management and more. You probably don’t understand all of these things. Be sure to seek the help of people who do.

“Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat.

Once I think I understand multiple perspectives on a problem, I like to write them down and run them by the people who explained the problem to me. They’ll be able to point out where you’re still wrong. Perhaps you’re confusing your net and operational margins, or your hiring targets make no sense, or your customers see a different problem from you. This requires that the people you consult trust you, and you trust them. Fortunately, non-engineers in engineering organizations are always looking out for allies and friends. Most are, like engineers, only too excited to explain their work.

Engage with the doubters, but don’t let them get you down

Be prepared! And be careful not to do Your good deeds when there’s no one watching you (Tom Lehrer)

You will never convince everybody of your point of view. By now, you have two powerful tools to help convince people: A clear statement of the problem that considers multiple points-of-view, and a clear statement of the solution. Some people will read those and be convinced. Others would never be convinced, because their objections lie beyond the scope of your thinking. A third group will have real, honest, feedback about the problem and proposed solution. That feedback is gold that should be mined. Unfortunately, separating the golden nuggets of feedback from the pyrite nuggets of doubt isn’t easy.

The doubters will get you down. Perhaps they think the problem doesn’t exist, or that the solution is impractical. Perhaps they think you aren’t the person to do it. Perhaps they think the same resources should be spent on a different problem, or a different solution. You’ll repeat, repeat, and repeat. Get used to it. I’m still not used to it, but you should be.

Again, writing is a tool I reach for. “Today, I’m doubting my solution because…” Sometimes that doubt will be something more about you than the project. That’s OK. Sometime it’ll be about the project, and will identify a real problem. Often, it’ll just point to one of those unknown unknowns that all projects have.

Meet the stakeholders where they are

Most likely, you’re going to need to convince somebody to let you do the work. That’s good, because doing big things requires time, people and money. You don’t want to be working somewhere that’s willing to waste time, people or money. If they’re willing to waste time on your ill-conceived schemes, they’ll be willing to waste your time on somebody else’s ill-conceived schemes.

May your wisdom grace us until the stars rain down from the heavens. (C.S. Lewis)

The best advice I’ve received about convincing stakeholders is to write for them, not you. Try to predict what questions they are going to ask, what concerns they will have, and what objections they will bring up and have answers for those in the text. That doesn’t mean you should be defensive. Don’t aim only to flatter. Instead, tailor your approach. It can help to have the advice of people who’ve been through this journey before.

The previous paragraph may seem to you like politics, you may have a distaste for politics, or believe you can escape it by moving to a different business. It is. You may. You can’t.

Leadership willing to engage with your ideas and challenge you on them is a blessing.

Build a team

You need to build two teams. Your local team is the team of engineers who are going to help you write, review, test, deploy, operate, and so on. These people are critical to your success, not because they are the fingers to your brain, but because the details are at least as important as the big picture. Get into some details yourself. Don’t get into every detail. You can’t.

Your extended team is a group of experts, managers, customer-facing folks, product managers, lawyers, designers and so on. Some of these people won’t be engaged day-to-day. You need to find them, get them involved, and draw on them when you need help. You’re not an expert in everything, but expertise in everything will be needed. Getting these people excited about your project, and bought into its success, is important.

Finally, find yourself a small group of people you trust, and ask them to keep you honest. Check in with them, to make sure your ideas still make sense. Share the documents you wrote with them.

Be willing to adapt

You will learn, at some point into doing your big project, that your solution is bullshit. You completely misunderstood the problem. You may feel like this leaves you back at the beginning, where you started. It doesn’t. Instead, you’ve stepped up your level of expertise. Most likely, you can adapt that carefully-considered solution to the new problem, but you might need to throw it out entirely. Again, write it down. Be specific. What have you learned, and what did it teach you? Look for things you can recover, and don’t throw things out prematurely.

Other Posts

• 17 Dec 2024 » Snapshot Isolation vs Serializability
• 06 Dec 2024 » DSQL Vignette: Wait! Isn't That Impossible?
• 05 Dec 2024 » DSQL Vignette: Transactions and Durability