Premature Optimization: Universally Misunderstood

Published on
Last Updated on

“Premature Optimization”

You might have come across the famous software optimisation quote popularised by Donald Knuth:

Premature optimization is the root of all evil.

– Sir Tony Hoare

It has been commonly interpreted as “don’t think about performance in the beginning, you can fix any performance problem later”. This interpretation is completely and categorically wrong.

Original Quote

It’s very common for statements to lose their original meaning when context has been stripped and that’s exactly what happened here.

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

– Sir Tony Hoare

The short version is missing a crucial part: “small efficiencies”. At the time the quote was made, “small efficiencies” referred to techniques like minimising the number of instructions used.

With the additional context, the quote takes on a significantly different meaning: it’s making a statement only about micro-optimisations ("small efficiencies"), not about performance in general.

Hoare was simply advising against micro-optimisations without finding the hotspots first: the “premature” part refers to lacking measurements.

Hotspot Optimisation: “Make it Fast Later” Fallacy

It’s quite tempting to adopt a “ship now, make it fast later” approach. While optimisations will improve performance, it won’t change the fundamental performance envelope. As tech stack fundamentals, access patterns, data dependencies and data structures are baked into a design, it’s not possible to hotspot your way out of a slow architecture.

Daniel Lemire perfectly explains this in Hotspot performance engineering fails:

Developers often believe that software performance follows a Pareto distribution: 80% of the running time is spent in 20% of the code. Using this model, you can write most of your code without any care for performance and focus on the narrow pieces of code that are performance sensitive.

…

Sadly, it does not work.

Charles Cook encounters this with a configuration client which performs a lot DCOM calls:

Again, optimizing this code would require a lot of re-working: optimization after a design has been implemented nearly always involves much more work than incorporating it into the original design.

In Performance Excuses Debunked, Casey Muratori astutely notes that if optimising hotspots was the solution to serious performance problems, software wouldn’t have to get rewritten to make it faster:

If Facebook’s performance problems were concentrated into “hotspots”, why did they have to completely rewrite entire codebases? Why would they have to do a “ground up” rewrite of something if only a few hotspots were causing the problem? Why didn’t they just rewrite the hotspots? Why did they have to rewrite an entire compiler in a new language, instead of just rewriting the hotspots in that language? Why did they have to make their own compiler to speed up PHP and Hack, instead of just identifying the hotspots in those codebases and rewriting them in C for performance?

Designing for Performance

If you want to have fast software, you must think about performance from day one. This includes thinking about the tech stack, the architecture, the data access patterns, the data dependencies, the networking and how it all fits together.

Charles Cook summarises it quite well:

Its usually not worth spending a lot of time micro-optimizing code before its obvious where the performance bottlenecks are. But, conversely, when designing software at a system level, performance issues should always be considered from the beginning.

A good software developer will do this automatically, having developed a feel for where performance issues will cause problems. An inexperienced developer will not bother, misguidedly believing that a bit of fine tuning at a later stage will fix any problems.

A Revised Quote

When having conversations about performance in the future, I’ll be using a revised version:

Premature micro-optimization is the root of all evil.

And that one, I completely agree with.

Quote Origin (Update)

Alexander Keller reached out about the origin of the quote. It first appeared in Donald Knuth’s Structured Programming with go to Statements and Knuth later attributed the quote to Hoare:

Just FYI … after giving Knuth credit for this for years, I recently saw a place where Knuth attributes it to C.A.R. Hoare. From “The Errors of TeX” on page 276 of “Literate Programming” to Hoare:

“But I also knew, and forgot, Hoare’s dictum that premature optimization is the root of all evil in programming.”

Does anybody know where this statement first appeared?

Now, the interesting part is that Hoare did not actually claim to be the author of the quote. Both Bruce Eckel and Tony Hoare credit Edsger Dijkstra.

Dear Hans,

I’m sorry I have no recollection how this quotation came about.? I might have attributed it to Edsger Dijkstra.

I think it would be fair for you assume it is common culture or folklore.

Tony.

References

← Back to Writings