Focus on the Good Parts

Skepticism and cynicism can get in your way.

Back in May, I wrote Reading Research: A Guide for Software Engineers, answering common questions I get about why and how to read research papers. In that post, I wrote about three modes of reading: solution finding, discovery, and curiosity. In subsequent conversations, I’ve realized there’s another common issue that gets in engineers’ ways when they read research, especially in the discovery and curiosity modes: too much skepticism.

The chief deficiency I see in the skeptical movement is its polarization: Us vs. Them — the sense that we have a monopoly on the truth; that those other people who believe in all these stupid doctrines are morons; that if you’re sensible, you’ll listen to us; and if not, to hell with you. (from Carl Sagan’s The Demon Haunted World)

I could blame it on comment thread culture, racing to make that top comment pointing out errors in the paper. I could blame it on the low signal-to-noise ratio of content in general. I could blame it on poor research, poor writing, or incorrect data. But whatever is to blame, many readers approach technical content with their first goal being to find errors and mistakes, gaps in logic, or incomplete justifications of statements. When a mistake is found, the reader is justified in throwing out the whole piece of writing (unreliable!), the authors (sloppy!), their institutions (clueless!), or even the whole field (substandard!). It’s also a perfect opportunity to write that comment or tweet pointing out the problems. After all, if you found the author’s mistake, doesn’t that make you smarter and better than the author?

This approach gets in the way of your ability to learn from reading. I’d encourage you to take a different one: read with the goal of finding the good stuff. Dig for the ideas, the insights, the analyses and the data points that provide value. Look for what you can learn.

I’m not suggesting that you don’t carefully approach what you read. You absolutely should make sure what you believe is well-supported. Don’t waste your life reading crap. Your time is too valuable for that.

The flip side of this is relying too much on social proof. If you open the comment thread first, you’ll find that the piece you’re about to read is great or it’s crap or it’s another piece of junk published by those people (you know, them, the incompetent ones). Then, when you finally read the paper, you’ll be less smart. You’ll be biased towards confirming the opinions of others, rather than reading and understanding the material. I’m not against comment threads, but I never read them first.

Again, you can go too far in this direction. A lot of academic publishing is an exercise in social proof. Almost all the filtering we use to reduce the firehose of content down to a manageable stream depends on social proof. We use these tools because they’re powerful, and scalable. But remember than popularity with Hacker News commenters, and even publication in a prestigious conference or journal, is only weak evidence of quality. Unpopularity, and rejection, are weak evidence of a lack of quality.

An Example

Fox and Brewer’s classic paper Harvest, Yield, and Scalable Tolerant Systems contains many great ideas. The framing of Harvest and Yield is very useful, and I’ve found it’s had a big influence on the way that I have approached system design over the years. The first time I read it, though, I put it down. The parts describing CAP (Section 2 and 3) are confusing at best and wrong at worst (as I’ve blogged about before). I couldn’t get past them.

It was only after being encouraged by a colleague that I read the whole thing. Taken as a whole, it’s full of great ideas. If I had kept tripping over my skepticism, and getting stuck on the bad parts, I never would have been able to learn from it.

Other Posts

• 06 Dec 2024 » DSQL Vignette: Wait! Isn't That Impossible?
• 05 Dec 2024 » DSQL Vignette: Transactions and Durability
• 04 Dec 2024 » DSQL Vignette: Reads and Compute