1. First and foremost: measure early, measure often. It's been said so often and it still needs repeating. In fact, the more you know about performance the easier it can be to fall into the trap of not measuring enough. Measuring will show exactly where you need to focus your efforts. It will also tell you without question whether your work has actually lead to an improvement, and to what degree.

2. The easiest way to make things go faster is to do less work. Use a more efficient algorithm, refactor code to eliminate unnecessary operations, move repeated work outside of loops. There are many flavours, but very often the biggest performance boosts are gained by simply solving the same problem through fewer instructions.

3. Understand the performance characteristics of your system. Is your application CPU bound, GPU compute bound, memory bound? If you don't know this you could make the code ten times as fast without gaining a single ms because the system is still stuck waiting for a memory transfer. On the flip side, if you know your system is busy waiting for memory, perhaps you can move computations to this spot to leverage this free work? This is particularly important in shader optimizations (latency hiding).

4. Solve a different problem! You can very often optimize your program by redefining your problem. Perhaps you are using the optimal algorithm for the problem as defined. But what does the end user really need? Often there are very similar but much easier problems which are equivalent for all practical purposes. Sometimes because the complexity lies in special cases which can be avoided or because there's a cheap approximation which gives sufficient accuracy. This happens especially often in graphics programming where the end goal is often to give an impression that you've calculated something.

I definitely agree with this one, especially on the level of "don't make network calls in your hot loop".

But it should be noted that less efficient algorithms that access memory in a more efficient way (that is to say, get a higher percentage of cache hits when iterating the data) can beat more efficient algorithms that get more cache misses.

That's all to say, iterating over an array is very vast, and iterating through a map is not. Is the map faster if you only need to access a couple things? Well, depends on the dataset size, as well as the constant factor in your map access, and how much of an improvement your other algorithm is.

You should definitely measure that once you've made the change though.

Simplicity is often the best because necessary complexity will arrive on its own.

That is, if you have a rarely used slow part of the system, that might make it seem unimportant, but not if running it uses all the disk space or drains your phone battery.

Even if that doesn't happen, there are things you can optimize in the unimportant parts - you can optimize them getting out of the way of the rest of the system, like by having smaller code size and not stomping all over caches.

That is very true! I simply think of it first because it is often the biggest problem at my work. One of the extreme exceptions was optimising Wavetale for the Nintendo Switch, where we had to decrease memory usage from over 20GiB to below 3GiB.

> is actually a bit unusual as it doesn't represent an exhaustible resource.

Not in the context of game development, however! There you typically don't care about wall time. Instead, you work towards a set frame rate meaning you have a set slice of time (usually 16.7 or 33.3 ms) to go through the entire game and render loop each time.

I was about to bring a similar example: Some of our really old daily data processing at work might need some attention and optimization in the future, because we're running out of hours in the day. And we haven't found a place we can buy more hours in the day from yet.

In part, it's a somewhat rewarding topic. Thinking about queries, joining a bit differently, adding another index based on new data patterns can cut hours of runtime without incurring further resource cost.

But on the other hand, it's yet another project someone dumped on the floor and we were forced to adopt it "because of the customer". And the second or third project of PD trying to "do it right" is teetering on failure once again. Cron running shell scripts held together with chicken wire and duct tape is too strong of a stack I guess.

If jobs must be executed one after another, then you absolutely create an infinite backlog.

This is a great rule of thumb. I've seen junior engineers (and even senior in some cases) try to parallelize existing solutions before first optimizing the single threaded case.

Things that eat CPU: iterations, string operations. Things that waste CPU: lock contentions in multi-threaded environments, wait states.

You can usually build a lot of the understanding from first principles starting there. Back in the day we had to do this because there wasn't much by way of readily available literature on the subject. Actual techniques will depend or evolve based on your choice of platform or version.

E.g. 20 years ago, we used to create object pools in C++ at load time to avoid Unix heap locks at runtime. This may no longer be necessary. 15(ish?) years ago, JNI was used when the JVM wasn't fast enough for certain stuff. This is no longer necessary. 10 years ago, immutable JS objects were thought to be faster because the JS runtimes at the time were slower to mutate existing objects than to create new ones. This too, may no longer be true (I haven't checked recently). Until very recently, re-rendering with virtual DOM diffing was considered more performant than direct, incremental DOM manipulation. This too, may no longer be true.

Actually wasn't strictly true even when React came out, but it was true enough with the code that most JS developers actually wrote to lead to a change in dominant JS framework.

DOM manipulation even in 2013 used a dirty-bit system. Calling element.appendChild would be a few pointer swaps and take a couple ns. However, if you then called any of a number of methods that forced a layout, it would re-render the whole page at a cost of ~20ms on mobile devices of the day. These included such common methods as getComputedStyle(), .offsetWidth, .offsetHeight, and many others - there was a list of about 2 dozen. Most JS apps of the day might have dozens to hundreds of these re-layouts triggered per frame, but the frame budget is only 16.667ms, so that's why you had slow animations & responsiveness for mobile web apps of 2013.

React didn't need a full virtual DOM layer. It just needed to ensure that all modifications to the DOM happened at once, and no user code ran in-between DOM manipulations within a certain frame. And sure enough, there are frameworks that actually do this with a much lighter virtual DOM abstraction (see: Preact) and get equal or better performance than React.

The lesson for performance tuning is to understand what's going on, don't just take benchmarks at face value. If a call is expensive, sometimes it's conditionally expensive based on other stuff you're doing, and there's a happy path that's much faster. Learn to leverage the happy paths and minimize the need to do expensive work over and over again, even if the expensive work happens in a layer you don't have access to.

“Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” - Donald E. Knuth, Structured Programming With Go To Statements

https://www.brendangregg.com/systems-performance-2nd-edition...

1. Latency vs throughput. Oftentimes they are the same, i.e. reduce the time it takes to do something. However, when you passed a certain threshold, techniques that can optimize throughput will hurt latency, so it is important to know what you are looking for. There are also low level details if you have rather extreme latency requirement, e.g. pinning the cores, kernel settings etc.

2. Knowledge about the overall system and your input distribution. While this seems trivial, often times you can get large performance improvement by avoiding redundant work, either by caching or lazy evaluation. Some computation may only exist because they may be needed later, and these can be avoided by lazy evaluation.

3. Better algorithms. Again, this seems trivial but oftentimes people are using algorithms that are far from optimal. And even if the algorithm can be asymptotically, there may be faster algorithms for special cases or faster in practice. Optimizing special cases may be rewarding if they occur frequently. Do you really need optimal solutions? Can you allow randomization? Can you do optimization on the queries to make it faster overall without optimizing individual operations?

4. Parallelization. Can you do parallelization? Are your problem instances large enough, or individual stages slow enough to benefit from parallelization? Do you have computation that are trivially parallelizable and can benefit from offloading to the GPU? If your code is waiting on some events, can you make them async? Can you avoid locks or atomic operations in your parallel code?

5. Data structure optimization. Can you reduce the number of allocation needed? Can you make the data structure more linear and predictable so the CPU can have better cache utilization? Can you compress certain data if they are sparse?

6. Low level CPU/GPU optimizations. There are a lot of great resources out there, but only do it when you are very sure it will be worth it, i.e. they are bottleneck in your system.

OP is right that you can often improve both at the same time, it's just worded poorly.

Later though… once your system is somewhat optimized, you will tend to make latency vs throughput decisions. For most people though, slight changes to latency are the cost to large increases in throughput, but that may just be my experience.

I heard good things about his course https://www.computerenhance.com/

Agner Fog manuals are good too https://www.agner.org/optimize/#manuals

A site to look up instruction timings is https://uops.info/table.html

The full quote is as follows, "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

He specifically does not recommend writing code that is obviously inefficient. He is clearly referring to engineers optimizing routines by introducing new, additional complexity. He is not recommending that anyone write obviously, ruinously slow, bloated code.

Writing software is an art and a science.

Optimization is no different. One frequently missing part of our process is to keep a watchful eye on features that are obviously toxic to performance during development (i.e., multiply-nested for loops, many large external dependencies, introducing and frequently iterating over huge, bloated structs, etc.).

As everyone says, measure early and often, but also please don't just shout "LEEEEEEEROY JENKINS!" as you throw fireballs of slow, bloated code into the world.

His books (linked on the blog post) are also incredible, concrete resources to learn about optimizing systems.

[1] https://www.brendangregg.com/blog/2021-06-04/an-unbelievable...

I've worked with performance optimizations for years, but never touched a network connection. Because for me it's all in the context of optimizing single player video games, which primarily leads to a focus on graphics programming and GPU performance.

Well yeah, my first reaction to the question was: Optimize for what?

The question probably would have benefited from a bit more details about his job. Plattform, domain, etc.

I am also in the same boat as you, where I have 16 ms to do everything. So some of the general things we optimize for, also apply elsewhere, but many others not so much.

My main generic advice would be: things that happen only sometimes, can usually be slow, but things you need to do often ("hot spots") they need attention.

But of course this does not apply to a programm that checks for example whether the airbag of the car needs to fire, because some very rare condition was met. This code only runs very rarely - but if it does, there should be no garbage collector kicking in at that moment, no slow DB lookup, no waiting for a UI process to finish or alike (which should not have a connection anyway to the critical parts).

1. His free course: https://products.easyperf.net/perf-ninja

2. His free book: https://book.easyperf.net/perf_book (the 2nd edition is being worked on right now and there's a draft on github: https://github.com/dendibakh/perf-book)

Of course, it doesn’t cover every possible performance trick. For that, I’d also recommend the Intel Optimization Manual and Johnny’s Software Lab.

Funny, I've just discovered them via a totally different route and read a few articles. Definitely good resource to learn more about low-level CPU code optimization.

Though OP doesn't say what are they interested in. GPU code optimization, for example, is totally different bonkers Universe :D

Think of a system as a chain of bottlenecks, visualized as a set of pipes. If you can measure the metric you care about (tput, latency etc) at a component level, and put together the system’s control flow, you can spot where the bottleneck is. Optimise that component, and you will reveal the next bottleneck, now optimize that… and it goes on. To limit the fun of this exercise, it helps to do a back of the envelope calculation of what is a realistic estimate of the thing you measure in the system. Example - I want this service to do 100 emails/ sec. Now, piece by piece remove bottlenecks to achieve close that value.

To do this you must find ways to “cheat”. This can be of various forms. Better algorithms, better data structures, precomputation, caching. At some point you will exhaust low hanging fruit and need to dig into lower level aspects of the code or its compilation.

Anyway, best way to learn is to do it, go depth first and always check your work thoroughly. (It is easy to optimize yourself into a solution that is not working properly).

This is a good book. It covers most common concepts and techniques in a fairly accessible way. At they end it also shows builds up a highly optimized version of some algorithms and data structures and does explains every optimization.

Full disclosure: I'm the editor in chief for the series.

If you want one bit of advice on optimization, I can try one: follow your app architecture closely. This is where data structures that hold all of the important data live and this is what limits what is possible to achieve on performance. A lot of learning is narrowly focused to specific micro optimization techniques leaving big picture as an exercise.

1 Fedor Pikus, The Art of Writing Efficient Programs

I've learned a lot from Cary Millsap over the last 2 decades and he recently published a general performance optimization book "How to Make Things Faster" that I can recommend [1]. It's less about tools, more about the method and systematic approach for performance optimization:

[1] https://method-r.com/books/faster/

> One of the best, if possibly exaggerated, examples of the reality distortion field comes from Jobs's biographer Isaacson. During development of the Macintosh computer in 1984, Jobs asked Larry Kenyon, an engineer, to reduce the Mac boot time by 10 seconds. When Kenyon replied that it was not possible to reduce the time, Jobs asked him, "If it would save a person's life, could you find a way to shave 10 seconds off the boot time?" Kenyon said that he could. Jobs went to a white board and pointed out that if 5 million people wasted an additional 10 seconds booting the computer, the sum time of all users would be equivalent to 100 human lifetimes every year. A few weeks later Kenyon returned with rewritten code that booted 28 seconds faster than before.

https://en.m.wikipedia.org/wiki/Reality_distortion_field

A) kind of sounds like a toxic work environment

B) this is a bad way to reason about performance without knowing more. Is this actually a bottleneck? Would the heroic effort be better spent saving minutes elsewhere where its easy to save time instead of saving seconds during boot where its hard to save time and users encounter relatively rarely? Optimizing boot might be the right call but it also might not be.

Anything can be optimized. The real trick is to optimize your optimization so you optimize the right thing to get the most improvement possible as you are almost always limited by the amount of time you can spend optimizing so you can't do it all. Picking a component at random is a terrible way of doing optimization.

Also, while premature optimization is obviously a problem, knowing more about how to actually write optimized code can help you make more informed decisions earlier on in the development process.

True, although not knowing what needs to be optimized guarantees that you don't know how to optimize it :).

> knowing more about how to actually write optimized code can help you make more informed decisions earlier on in the development process.

Totally agree though, despite poking a bit of fun.

This is something that I've been decently good at for many years. If I had to give someone new to it advice, it'd be:

- learn how to repeatably measure your system without introducing too much overhead. If you get this wrong, you're going to end up tricking yourself into believing you've made an improvement but instead got lucky/unlucky.

- once you've found the repeatably-measureable hotspots, the optimization approach is going to depend dramatically on the problem domain. Optimizing for database disk throughput is different than optimizing for http server response-latency is different than squeezing more polygons into a frame in a game.

The one very important thing to keep in mind though is that you're going to need to peel back the abstractions and understand what's happening under the hood. This applies to both parts. Maybe the way to measure what's happening in your network service is to use tcpdump to capture the raw packets and see that you're sending a bunch of small writes into a socket instead of a single big write (why is that a problem? :D). Or something like NVidia NSight can provide a ton of insight into what's happening on your CPUs and GPU frame-to-frame.

I agree with this although I think it can sometimes be a red herring for those new to optimization. It's possible to spend a lot of time digging deeper when that's not really what's needed. For example, you might find a particular database query that's really slow and start looking at what configuration tweaks you can make to your database server. But maybe if you structure the application a bit differently, that query isn't necessary at all.

It's a great skill to be able to peel back layers of abstraction and continuing to optimize all the way down. But it's an equally important skill to be able to know what later really needs optimizing.

Can you tell me more about what your new job is, without releasing anything sensitive?

If you are running applications on Linux containers in the cloud, then I would recommend Brendan Gregg's blog and books (https://www.brendangregg.com/overview.html). He does a lot of knowledge sharing from his experiences at Netflix.

https://www.oreilly.com/library/view/understanding-software-...

Then optimize it - measure front end render performance/compilation times/code perf, and then do the same on the backend. Write a blog post about it.

No substitute for experience

My playbook for optimizing in the real world is something like this: 1. Understand what you're actually trying to compute end-to-end. The bigger the chunk you're trying to optimize, the greater the potential for performance.

2. Sketch out what an optimal process would look like. What data do you need to fetch, what computation do you need to do on this, how often does this need to happen. Don't try to be clever and micro-optimize or cache computations. Just focus on only doing the things you need to do in a simple way. Use arrays a lot.

3. Understand what the current code is actually doing. How close to the sketch above are you? Are you doing a lot of I/O in the middle of the computation? Do you keep coming back to the same data?

If you want to understand the limits of how fast computers are, and what optimal performance looks like I'd recommend two talks that come with a very different perspective from what you usually hear:

1. Mike Acton's talk at cppcon 2014 https://www.youtube.com/watch?v=rX0ItVEVjHc

2. Casey Muratori's talk about optimizing a grass planting algorithm https://www.youtube.com/watch?v=Ge3aKEmZcqY

Even if your system is not C++, I've always enjoyed this talk and the subsequent discussion which tackles some of the problems associated with some programming practices and the impact on performance.

CppCon 2014: Mike Acton 'Data-Oriented Design and C++' https://youtu.be/rX0ItVEVjHc

Go multi-core because async is an important optimization skillset, but other than that just build some things.

I live and breathe optimizations (it feels almost as satisfying to me as driving fast) and as an example recently I created an 11-board (one for each channel) wifi-presence-detection system in a busy wifi area and there was literally no way it was going to work without optimization. From communication protocol to having to be strict about every byte of memory, it’s working with the first principles that built the entire industry.

I watched the lectures from 2018 on YouTube: https://m.youtube.com/playlist?list=PLUl4u3cNGP63VIBQVWguXxZ...

A big piece missing from there is algorithms and data structure, likely covered in one (or multiple) other course(s).

https://www.amazon.in/Understanding-Software-Addison-Wesley-...

Here is something I wrote about extreme performance in JavaScript that is discarded by most programmers because most people that program JavaScript professionally cannot really program.

https://github.com/prettydiff/wisdom/blob/master/performance...

The best people I can hire can mostly use a computer. They can’t build something like this, or even build upon it. They need recipes to reuse to accomplish tasks, and the more something is custom, the harder it is to teach.

And product only cares about performance when a customer mentions it, which is almost never.

I long to spend time tuning race cars, but mostly I assemble sedans.

Most people capable of writing original software can easily apply the things I suggest, but most people writing JavaScript are not capable of writing original software. That doesn’t mean guidance for superior performance is out of alignment. It means there are fundamental problems with hiring and training.

Using your example I once saw a Puerto Rican racing team make a lot of money with their custom car. They took an old 80s Mazda small sedan and dropped in a large Ferrari engine and customized the suspension. This is something innovative they did to make money by winning competitions, because that pays better than just changing tires. Shops that only change tires or change oil have that does to a science to maximize human productivity, akin to copy/paste.

For example you create your own bundler, when modern bundlers are very mature and good.

For example you dismiss SSR as unnecessary and then basically roll your own. You dismiss modern frameworks out of hand then list performance improvements they can make for you (e.g. keeping state in html).

Your last two performance improvements are about not taking drugs???

I'm really pro people doing things themselves for fun and all that but this article and you comment comes across as so arrogant and condescending whilst also seeming to show ignorance (or at least willful dismissal) of exactly where modern JavaScript development is at, and present it as state of the art performance improvements.

I rolled my own bundler only because it’s tiny and without dependencies. I am not rewriting ESLint even though I absolutely dread its large number of dependencies.

FTFY

https://youtu.be/kjufVhyuV_A

The idea to classify cycles into front-end bound, backend bound, bad speculation or memory bound is brilliant. Once you know which one your program suffers from, it's easy to know what can be done to improve things.

https://m.youtube.com/watch?v=Ge3aKEmZcqY

https://m.youtube.com/watch?v=ffDXc6oup3Q

https://m.youtube.com/watch?v=pgoetgxecw8

A good talk, doesn't go deep and instead goes a bit wide. https://www.youtube.com/watch?v=6RlloT_6WxA

This one explains the black magic the CPU makes have been doing. If you going to be optimizing code you should know your hardware. https://www.lighterra.com/papers/modernmicroprocessors/

Aditional note: I've noticed C++ conventions have a habit of having performance related talks, YT is your friend.

https://news.ycombinator.com/item?id=20883860

As for specific optimizations, it requires context of what software are you trying to optimize and under what circumstances. A lot of the times you are going to see that the answer to asking if certain optimizations are worth the effort is going to be 'it depends'

It's a beginner-friendly introduction to Low Latency Programming, which involves a lot of performance optimization. Could be a good way to start your learning on the subject.

You can read one of the chapters on my blog: https://tech.davidgorski.ca/introduction-to-low-latency-prog...

Optimizing games sends you deep down a fun rabbit hole, but that will be very different than trying to optimize a Go backend server.

1) Try to understand well the architecture of your company (who is who, who decides what changes are made to computers, how they decide that, what metrics do they use, what tests and benchmarks are passed before changes, etc)

2) Try to understand your place in such architecture. Am I responsible from the overall performance? or only the performance of certain components? am I responsible about the latency of the network or the latency of the database, or both etc. Make a clear scope. This will help you to focus on which metrics do you need to follow.

3) Try to understand the company procedures. Can I refuse a change that comes from the product or marketing team? can I refuse a change that comes from developers? can I refuse a change that comes from the platform team? how much time do I have to analyze such changes before they reach production, how can I request the rollback of a unsupervised change, where can I check the performance impact of each change made on production in the past? etc

4) Try to understand what the CEO and CTO, your team and the rest of teams should expect from you. Are there any SLA o SLO for your position related to the overall performance?

5) Make clear how you are informed of ongoing changes and roadmaps? Should I spend all the week in the performance of a component that is going to be deprecated in the next sprint? etc

6) Ask doubts and questions to your team mates, or department head. They may teach you about the company workflows, past issues, past solutions, corner cases, blockers, resources, plans and guidelines.

In short... look for reading/watching material, but don't forget to look at your company too, you will find things to learn there too, and maybe things that need to change if they are important enough; or need to be clarified that they are not important enough to change, to defend your work on future performance issues, related to those things that weren't changed.

Or get a job you can handle idk.

https://news.ycombinator.com/item?id=13407192

A few people had replied, agreeing with my opinion, and giving some more details. One of them called the book "gold".

I have also posted about the book a few other times on HN, over the years.

Those comments can be found by searching hn.algia.com for comments (not stories) matching the pattern "writing efficient programs vram22".

Sorry, only noticed the typo now:

Should be hn.algolia.com

I swear, nobody actually read OP's post. The top five comments are "here's some personal advice about general rules of thumb, without any links to actual resources!"

Kudos to levodelellis, whose post is at #6 root comment and contain some links and drops some names.

I guess there are some common parts like profiling and statistics, but thats about it.

Just like the best advice on learning to write code is to write code, the best way to learn how to optimize performance is to optimize performance.

It’s really not specific to Julia, though the language does let you drill down into the details nicely.

To be honest I've not approached the material in a few years. Can you pinpoint which areas are wrong?

For example, on page 36 he asserts: "Accessing a variable or object through a pointer or reference may be just as fast as accessing it directly." While this might be true for a large/compound object (neglecting non-cached memory access vs cached memory, such as the stack), this is certainly not true in general case for simple POD types, such as ints or floats, if a compiler can't prove that between accesses the variable hasn't been modified (which happens actually very frequently). I've seen x10 speedups of computations in tight loops when I explicitly cached a value used in the loop into a variable from under a pointer/reference.

On p65 he asserts: "Assume that a function opens a file in exclusive mode, and an error condition terminates the program before the file is closed. The file will remain locked after the program is terminated and the user will be unable to access the file until the computer is rebooted." This is just hilarious not at the last modified date 2023-07-01, but even two decades ago. This could be true in times of DOS and maybe Windows 3.11. I might not remember exactly, but I think even Windows 95 have already dealt with it by tracking resources acquired by a process, and releasing everything after process termination. All WindowsNT family definitely didn't/doesn't have this issue (unless your program is a kernel-mode driver, but I'm not even sure in that, since my knowledge of kernel mode is circa ~2007 at most).

And I could keep counting, this isn't all issues, unfortunately... So there's some historical interest in the document, but...one shouldn't trust it blindly for today's things.

This is indeed an important gotcha! Let's say you have a filter and want to process an array of floats. If the coefficient is a struct member and has the same type as the audio samples, you must cache it in a local variable, otherwise the compiler might reload it from memory on every loop iteration. ('restrict' can somewhat help with these kind of aliasing issues, but you need to be careful.)

2. Read the blog posts people wrote about the 1BR challenge where they tried to figure out the fastest way to process 1 billion rows of data

3. Brendan Gregg‘s blog

4. Google/YouTube how to profile code in your desired language

https://ocw.mit.edu/courses/6-172-performance-engineering-of...

[Lemire] https://lemire.me/en/#publications

I learn a lot by reading my compiler’s and profiler’s documentation.

For Rust, the Rust Performance Book by Nicholas Nethercote et al. [Nethercote] seems like a nice place to start after reading the Cargo and rustc books.

[Nethercote] https://nnethercote.github.io/perf-book/

Algorithms for Modern Hardware by Sergey Slotin [Slotin] is a dense and approachable overview.

[Slotin] https://en.algorithmica.org/hpc/

Quantitative understanding of the underlying implementations and computer architecture has been invaluable for me. Computer architecture: a quantitative approach by John L. Hennessy and David A. Patterson [H&P] and Computer organization and design: the hardware/software interface by Patterson and Hennessy [P&H ARM, P&H RISC] are two introductory books I like the best. There are three editions of the second book: the ARM, MIPS and RISC-V editions.

[H&P] https://www.google.com/books/edition/_/cM8mDwAAQBAJ

[P&H ARM] https://www.google.com/books/edition/_/jxHajgEACAAJ

[P&H RISC] https://www.google.com/books/edition/_/e8DvDwAAQBAJ

Compiler Explorer by Matt Godbolt [Godbolt] can help better understand what code a compiler generates under different circumstances.

[Godbolt] https://godbolt.org

The official CPU architecture manuals from CPU vendors are surprisingly readable and information-rich. I only read the fragments that I need or that I am interested in and move on. Here is the Intel’s one [Intel]. I use the Combined Volume Set, which is a huge PDF comprising all the ten volumes. It is easier to search in when it’s all in one file. I can open several copies on different pages to make navigation easier.

Intel also has a whole optimization reference manual [Intel] (scroll down, it’s all on the same page). The manual helps understand what exactly the CPU is doing.

[Intel] https://www.intel.com/content/www/us/en/developer/articles/t...

Personally, I believe in automated benchmarks that measure end-to-end what is actually important and notify you when a change impacts performance for the worse.

You have three areas to study:

1. Measurement - makes you define the performance you're looking for and measure it. Until you do this it's mostly a bullshit "make people stop complaining about performance" errand that's too wishy washy to do with more than a few stabs in the dark. With containers and decent capture of samples of your load, a benchmark is pretty straightforward to set up.

2. Modeling - these models are usually little more than measured rates and latencies applied to Little's Law. Pocket-calculator math is often good enough. At worst, an M/M/1 queue.

3. Instrumentation - Figuring out how to attribute your computer's resources (memory, CPU time, iops, etc) to different parts of your code. Tracing libraries, Linux perf, and ebpf can be useful here.

There are a decent number of computers performance books. I like the ones by Jain (great, but AFAICT out of print) and Harchol-Baltar. For work, you shouldn't read them straight through but iterate through parts as you better understand the problem you're trying to solve and start choosing strategies. For the tactical side. Brendon Gregg (sp?) has some decent measurement tool books. Figure out what you want to improve and how to measure that. Then start attributing the existing performance to implementation choices that you can control. Then control those choices (e.g. change algorithm, load balance better, make design trade-offs) to improve performance.

Second there is an important engineering lesson to learn. Often there are many performance issues, with only a few acting as serious bottlenecks. Additionally sometimes the solutions to performance issues add complexity, but as an engineer you want to avoid complexity. Engineering effort is usually limited, so there is always a question of whether a performance issue needs to be fixed now or left till later.

Here is a quick example to illustrate my point. pgAdmin is a webui program to interact with PostgreSQL databases, allowing you to remotely run queries. Part of its operations fetches information about columns in a result set, in one version this code ran one query per column sequentially. So c columns, each a synchronous query to the server - almost instant on a local database with a small number of columns. However with 400 columns, and a 40ms internet link, it ended up taking at least 400*40=16 seconds to complete. In 99% of cases this code works just fine, but in a few less obvious scenarios its runtime balloons.

Another example; what happens if all the daily scheduled jobs run at the same time? https://github.com/go-acme/lego/issues/1656

The number one most important thing you can do is dive in and start profiling real-world code. Find a part of your software that is too slow or uses too many resources, and use whatever the standard profiler is for your development environment to figure out why. Performance optimization is a very empirical discipline. Yes there are general principles, but if you don't measure your baseline or your changes you won't know how good your optimization was. In my experience, the first attempt at a fix is often flat-out wrong! Doing this first will also help motivate your reading.

Once you know how to measure the performance of your software, I recommend learning the basics of modern computer architecture. At a minimum, learn about CPU caches, how they work, and how to design your code to use them effectively. I find Algorithms for Modern Hardware to be a good resource for this [1], but there are many others. Relatedly, you should have a rough idea of how long it takes for your computer to do various basic things (fetch something from memory, fetch something from cache, etc.). There's a table at [2] that gives a good idea. Don't worry too much about the absolute values--the order of magnitude is what's important.

You should also study fundamental data structures, but understand that for low-level programming 95% of the time the correct answer will be to shove everything into a simple flat array (e.g. std::vector in C++), maybe with some sort of index on top. Fancy data structures are more important in higher-level languages that are structurally unable to make effective use of modern hardware.

[1] https://en.algorithmica.org/hpc/

[2] https://gist.github.com/jboner/2841832

The generally useful rule is “measure before acting”.

There are some rules of thumb at every layer:

If you’re getting bad scrolling in a web application on a mobile phone, something is probably getting called over and over

If you’ve got an x86_64 server maxing out but the cores aren’t printing work? Zen4’s northbridge has some edge cases.

If you’re trying to melt aluminum so that exquisite optics can do extreme ultra-violet litho: weak hyper charge is very well determined empirically but there are some weird readings on muon spin.

I’m sort of kidding because this is an Endless Internet Feud, but really it’s measure and whack the hot spots.

I’ve done a bunch of this shit: if you’re not sure where to start feel free to email.

Lots of great resources in this thread, but as the saying goes; knowing is half the battle.

That's it.

1. The problem, 2. The algorithms, 3. Micro-optimisation.

The potential gains shrink rapidly as you descend this list. A lot of people start thinking at level 3 straight away, but this is pointless if you've left performance on the table at the higher levels. For example, no amount of clever bit twiddling will compensate for the wrong algorithm, and even the best algorithm is pointless if you're solving the wrong problem.