Blog Posts

2025

• 20 Jun 2025    Career advice, or something like it
• 02 Jun 2025    Systems Fun at HotOS
• 20 May 2025    Good Performance for Bad Days
• 17 Apr 2025    Decomposing Aurora DSQL
• 25 Mar 2025    One or Two? How Many Queues?
• 05 Feb 2025    What Fekete's Anomaly Can Teach Us About Isolation
• 04 Feb 2025    Versioning versus Coordination

2024

• 17 Dec 2024    Snapshot Isolation vs Serializability
• 06 Dec 2024    DSQL Vignette: Wait! Isn't That Impossible?
• 05 Dec 2024    DSQL Vignette: Transactions and Durability
• 04 Dec 2024    DSQL Vignette: Reads and Compute
• 03 Dec 2024    DSQL Vignette: Aurora DSQL, and A Personal Story
• 14 Nov 2024    Ten Years of AWS Lambda
• 14 Aug 2024    Garbage Collection and Metastability
• 29 Jul 2024    Resource Management in Aurora Serverless
• 25 Jul 2024    Let's Consign CAP to the Cabinet of Curiosities
• 04 Jun 2024    Not Just Scale
• 09 May 2024    It's always TCP_NODELAY. Every damn time.
• 25 Apr 2024    MemoryDB: Speed, Durability, and Composition.
• 17 Apr 2024    Formal Methods: Just Good Engineering Practice?
• 25 Mar 2024    Finding Needles in a Haystack with Best-of-K
• 04 Mar 2024    The Builder's Guide to Better Mousetraps
• 12 Feb 2024    Better Benchmarks Through Graphs
• 06 Feb 2024    How Do You Spend Your Time?
• 23 Jan 2024    Pat's Big Deal, and Transaction Coordination
• 18 Jan 2024    What is Scalability Anyway?

2023

• 15 Dec 2023    Why Aren't We SIEVE-ing?
• 27 Nov 2023    It's About Time!
• 18 Oct 2023    Optimism vs Pessimism in Distributed Systems
• 21 Sep 2023    Writing For Somebody
• 08 Sep 2023    Exponential Value at Linear Cost
• 25 Aug 2023    On The Acoustics of Cocktail Parties
• 28 Jul 2023    Invariants: A Better Debugger?
• 13 Jul 2023    My Favorite Bits of OSDI/ATC'23
• 23 Jun 2023    Bélády's Anomaly Doesn't Happen Often
• 19 Jun 2023    What is a container?
• 23 May 2023    Container Loading in AWS Lambda
• 10 May 2023    Open and Closed, Omission and Collapse
• 20 Apr 2023    The Four Hobbies, and Apparent Expertise
• 23 Mar 2023    Surprising Scalability of Multitenancy
• 07 Mar 2023    False Sharing versus Perfect Placement
• 07 Feb 2023    Hot Keys, Scalability, and the Zipf Distribution
• 30 Jan 2023    NoSQL: The Baby and the Bathwater
• 06 Jan 2023    Erasure Coding versus Tail Latency

2022

• 15 Dec 2022    Under My Thumb: Insight Behind the Rules
• 29 Nov 2022    Lambda Snapstart, and snapshots as a tool for system builders
• 22 Nov 2022    Amazon's Distributed Computing Manifesto
• 08 Nov 2022    Writing Is Magic
• 21 Oct 2022    Give Your Tail a Nudge
• 04 Oct 2022    Atomic Commitment: The Unscalability Protocol
• 02 Sep 2022    Histogram vs eCDF
• 11 Aug 2022    What is Backoff For?
• 29 Jul 2022    Getting into formal specification, and getting my team into it too
• 12 Jul 2022    The DynamoDB paper
• 02 Jun 2022    Formal Methods Only Solve Half My Problems
• 03 May 2022    What is a simple system?
• 11 Apr 2022    Simple Simulations for System Builders
• 28 Feb 2022    Fixing retries with token buckets and circuit breakers
• 16 Feb 2022    Will circuit breakers solve my problems?
• 31 Jan 2022    Software Deployment, Speed, and Safety
• 19 Jan 2022    DynamoDB's Best Feature: Predictability

2021

• 16 Nov 2021    The Bug in Paxos Made Simple
• 20 Oct 2021    Serial, Parallel, and Quorum Latencies
• 27 Aug 2021    Caches, Modes, and Unstable Systems
• 11 Aug 2021    My Proposal for Arecibo: Drones
• 05 Aug 2021    Latency Sneaks Up On You
• 24 May 2021    Metastability and Distributed Systems
• 19 Apr 2021    Tail Latency Might Matter More Than You Think
• 14 Apr 2021    Redundant against what?
• 25 Mar 2021    What You Can Learn From Old Hard Drive Adverts
• 22 Feb 2021    Incident Response Isn't Enough
• 22 Jan 2021    The Fundamental Mechanism of Scaling
• 06 Jan 2021    Quorum Availability

2020

• 19 Oct 2020    Getting Big Things Done
• 05 Oct 2020    Consensus is Harder Than It Looks
• 02 Sep 2020    Focus on the Good Parts
• 06 Aug 2020    Surprising Economics of Load-Balanced Systems
• 28 Jul 2020    A Story About a Fish
• 23 Jun 2020    Code Only Says What it Does
• 08 Jun 2020    Some Virtualization Papers Worth Reading
• 25 May 2020    Reading Research: A Guide for Software Engineers
• 22 Mar 2020    Two Years With Rust
• 19 Feb 2020    Firecracker: Lightweight Virtualization for Serverless Applications
• 17 Feb 2020    Physalia: Millions of Tiny Databases
• 02 Jan 2020    Why do we need distributed systems?

2019

• 12 Aug 2019    Kindness, Wickedness and Safety
• 20 Jun 2019    When Redundancy Actually Helps
• 17 Jun 2019    Is Anatoly Dyatlov to blame?
• 01 May 2019    Some risks of coordinating only sometimes
• 03 Apr 2019    Learning to build distributed systems
• 17 Mar 2019    Control Planes vs Data Planes

2018

• 20 Jun 2018    Telling Stories About Little's Law
• 25 Feb 2018    Availability and availability
• 01 Jan 2018    Balls Into Bins In Distributed Systems

2017

• 28 Dec 2017    Is the Mean Really Useless?

2016

• 03 Jan 2016    Why Must Systems Be Operated?

2015

• 05 Nov 2015    Heuristic Traps for Systems Operators
• 26 Sep 2015    Is there a CAP theorem for Durability?
• 20 Jun 2015    CALISDO: Threat Modeling for Distributed Designs
• 24 May 2015    Sodium Carbonate, and Ramenized Pasta
• 11 Apr 2015    The Zero, One, Infinity Disease
• 29 Mar 2015    How Amazon Web Services Uses Formal Methods
• 21 Mar 2015    Jitter: Making Things Better With Randomness
• 03 Mar 2015    Electoral Trouble in Sybilania
• 28 Feb 2015    Does Bitcoin Solve Byzantine Consensus?
• 25 Jan 2015    A Quiet Defense of Patterns

2014

• 06 Dec 2014    Make Your Program Slower With Threads
• 30 Nov 2014    Two Farmers and Common Knowledge
• 15 Nov 2014    Exactly-Once Delivery May Not Be What You Want
• 25 Oct 2014    Ice Cream and Distributed Systems
• 12 Oct 2014    Harvest and Yield: Not A Natural Cure for Tradeoff Confusion
• 21 Sep 2014    The Essential Barbara Liskov
• 10 Aug 2014    The Space Between Theory and Practice in Distributed Systems
• 09 Aug 2014    Use of Formal Methods at Amazon Web Services
• 16 Jul 2014    CAP and PACELC: Thinking More Clearly About Consistency
• 04 Jul 2014    Two traps in iostat: %util and svctm
• 29 Jun 2014    The Operations Gradient: Improving Safety in Complex Systems
• 19 May 2014    Viewstamped Replication: The Less-Famous Consensus Protocol
• 10 May 2014    The Essential Nancy Lynch
• 14 Apr 2014    Failure Detectors, and Non-Blocking Atomic Commit
• 30 Mar 2014    The Essential Leslie Lamport
• 08 Mar 2014    Snark, Chord, and Trust in Algorithms
• 12 Jan 2014    Distributed Consensus: Beating Impossibility with Probability One

2013

• 16 Dec 2013    Restricted Transactional Memory on Haswell
• 14 Dec 2013    Hardware Lock Elision on Haswell
• 14 Jul 2013    Beyond iostat: Storage performance analysis with blktrace
• 25 May 2013    Some Patterns of Engineering Design Meetings
• 20 Jan 2013    Exploring TLA+ with two-phase commit
• 06 Jan 2013    C++11's atomic and volatile, under the hood on x86

2012

• 13 Nov 2012    Java's Atomic and volatile, under the hood on x86
• 10 Sep 2012    Are volatile reads really free?
• 10 Sep 2012    Highly contended and fair locking in Java
• 02 Sep 2012    Expect Less, Get More?
• 11 Feb 2012    Latency lags bandwidth
• 22 Jan 2012    The properties of crash-only software
• 17 Jan 2012    The power of two random choices
• 10 Jan 2012    The benefits of having data