Over many years, “DevOps” practitioners applied Theory Of Constraints to our problems, ruthlessly optimizing our delivery pipelines and practices. Manual release management? Hell no, automate that. Deployment? Automate that too. Image management? 🔨 No thanks. Rolling back after we trebuchet a flaming dumpster into production? Automated. Whatever low value activity we could find in the process of getting code from product backlog to customer hands was a bottleneck to be removed or optimized.
The end result of this was that the slowest part of software delivery is testing. Since testing is why continuous delivery exists, that should have been good enough. Yes, we can make our tests faster, more automated, parallelized, etc. But when the highest value activity of a given practice is the bottleneck, you’re optimal. You have achieved “the best possible problem”.
Those habits and behaviors of optimization didn’t stop there. We kept on chopping. 🪓 We squished our integration and end to end tests down to unit tests to parallelize. At the personnel level, we pushed out anybody whom we believed could not code, indiscriminately, at the function level. We decided that testing might not be the bottleneck…the QA team was. The industry began to treat the people in these roles worse and worse. Expectations for them went up, salaries went down(while everyone else’s seemed to be going up!), we contracted the role, we offshored it, pretty much anything we could do to try and stop employing QA Engineers.
This created a self-reinforcing spiral, in which anyone “good enough at coding” or fed up with being treated poorly would leave QA. Similarly, others would assume anyone in QA wasn’t “good enough” to exit the discipline. No one recommends the field to new grads. Eventually, the whole thing seemed like it wasn’t worth it any more. Our divorce with QA was a cold one — companies just said “we’re no longer going to have that function, figure it out.” It was incredibly disrespectful and demoralizing to folks who had spent their career in QA. It also caused a lot of problems, because producing low quality software is actually a huge headache.
You can probably see where this is going by now: developers did not figure this out. Most orgs have no idea who should be doing what in terms of software quality. Those who have kept the function are struggling to find a place for it, because of the damage already done to the discipline.
It turns out, the “One Weird Trick” to faster software delivery was not “fire your testers”.
Wrecking this discipline was one of the worst kind of management mistakes — a choice to destroy something that took decades to develop, and one where the impact might not be felt for years. By the time your org has felt it, you’re likely years away from a meaningful fix.
The parts of the broken QA role we were handed are all still broken, and on the metaphorical workbench. The division of labor simply did not happen. Unsurprisingly, developers did not readily assume the duties of the role without any additional compensation, recognition, or reward. Those of us who can still remember working with a high functioning QA team can impersonate some of those behaviors, but newer engineers and managers have no idea what any of that was about, and aren’t able to tell what’s missing. The guiding principle in the following advice is that quality assurance is work. Just like any other work, assuming “somebody” will do it, letting it be invisible, is a recipe for failure. Denial is not a strategy.
It’s 2023 and it feels silly to have to write this down, but my experience suggests that I absolutely must.