Two identical pieces of code can have different specifications.

# x is an age of a person; the code checks if the person is past the retirement age in the US def is_of_retirement_age(x): return x >= 65;

# x is an ASCII character which is already known to be alphanumeric; this code checks if it’s a letter. def is_ascii_letter(x): return x >= 65;

[1] https://www.pathsensitive.com/2018/01/the-design-of-software... [2] https://www.mirdin.com

Several years ago, I did some contract work for a company that needed importers for airspace data and various other kinds of data relevant to flying.

In the US, the Federal Aviation Administration (FAA) publishes datasets for several kinds of airspace data. Two of them are called "Class Airspace" and "Special Use Airspace".

The guy who wrote the original importers for these treated them as completely separate and unrelated data. He used an internal generic tool to convert the FAA data for each kind of airspace into a format used within the company, and then wrote separate C++ code, thousands of lines of code each.

Thing is, the data for these two kinds of airspace is mostly identical. You could process it all with one common codebase, with separate code for only the 10% of the data that is different between the two formats.

When I asked him about this, he said, "I have this philosophy that says if you only have two similar things, it's best to write separate code for each. Once you get to a third, then you can think about refactoring and making some common code."

That is a good philosophy! I have often followed it myself.

But in this case, it was obvious that the two data formats were mostly the same, and there was never going to be a third kind of almost-identical airspace, only the two. So we had twice the code we needed.

I would kind of appreciate having two formats, since what are the odds they would change together? While there may never be a 3rd format, a DRY importer would imply that the source generating the data is also DRY.

One of the reasons they brought me in on this project is that besides knowing how to wrangle data, I'm also an experienced pilot. So I had a good intuitive sense of the meaning and purpose of the data.

The part of the data that was identical is the description of the airspace boundaries. Pilots will recognize this as the famous "upside down wedding cake". But it's not just simple circles like a wedding cake. There are all kinds of cutouts and special cases.

Stuff like "From point A, draw an arc to point B with its center at point C. Then track the centerline of the San Seriffe River using the following list of points. Finally, from point D draw a straight line back to point A."

The FAA would be very reluctant to change this, for at least two reasons:

1. Who will provide us the budget to make these changes?

2. Who will take the heat when we break every client of this data?

That is a great way to describe it!

Of course it is all just rows in a CSV file, but yes, it is a set of instructions for how to generate a map.

In fact the pilot's maps were being drawn long before the computer era. Apparently the first FAA sectional chart was published in 1930! So the data format was derived from what must have been human-readable descriptions of what to plot on the map using a compass and straightedge.

I just remembered a quirk of the Australian airspace data. Sometimes they want you to draw a direct line from point F to point G, but there were two different kinds of straight lines. They may ask for a great circle, a straight path on the surface of the Earth. Or a rhumb line, which looks straight on a Mercator projection but is a curved path on the Earth.

You would often have some of each in the very same boundary description!

For anyone curious about this stuff, I recommend a visit to your local municipal airport and stop by the pilot shop to buy a sectional chart of your area.

Tho if done haphazardly by someone inexperienced, you might end up with subtle divergences that might look like they're meant to be copies, and debugging them in the future by another developer (without the history or knowledge) can get hard.

Then someone would wonder why there are these two very similar pieces of code, and mistakenly try to DRY it in the hopes of improving it, causing subtle mistakes to get introduced...

All ‘is-a so it gets this code’ models can be trivially modeled as ‘has-a so it gets this code’ patterns, which don’t have that single-use constraint… so the corollary to this rule tends towards ‘never use inheritance’.

Has-a doesn't make sense for code like the literal text reuse. It makes sense for composition and encapsulation.

Edit: I'm now realizing that Python has one of the only sane multiple inheritance implementations. It's no wonder the rest of y'all hate it.

But what basically ended up happening was we had 2 codebases: 1 for that non-DRY version, and then 1 for everything else. The non-DRY version limped along and no one ever wanted to work on it. The ways it did things were never updated. It was rarely improved. It was kinda left to rot.

It sounds to me that you're trying to pin the blame of failing to maintain software on not following DRY, which makes no sense to me.

Advocating against mindlessly following DRY is not the same as advocating for not maintaining your software. Also, DRY does not magically earn you extra maintenance credits. In fact, it sounds to me that the bit of the code you called DRY ended up being easier to maintain because it wasn't forced to pile on abstractions needed to support the non-DRY code. If it was easy, you'd already have done it and you wouldn't be complaining about the special-purpose code you kept separated.

Writing it DRY in the first place would also have costs, including the alternative costs. Would it have been better to take those there and then?

3 things matter most in real estate: Location, location, location!

3 things matter most in programming: Context, context, context!

DRY – like almost every other programming tool/paradigm/principle – are very often misused by a lack of the programmer's ability to discern correctly whether the tool/paradigm/principle fits in the specific context.

It's not just a science. It's an art, too.

If you do what this guy did and are wrong, it looks like yes obviously it was the same thing and you should have DRY right from the start.

If you DRY from the start and are wrong, someone writes a blog post about not DRYing from the start.

It's a bet on which mistake is worse, because you will be wrong sometimes. DRYing from the start IMO is a worse mistake than duplication.

I’ve had a lot of success with this in embedded settings where data is piped into storage or OTA, and I want to format and pack it/send it up consistently but I might want to treat the data itself slightly differently.

[0a] More generally, I like a concept of "total" functions -- those which have sensible outputs for all their inputs. It's a bit of a tomayto/tomahto situation defining "all their inputs" (e.g., I'm personally okay using a function name like `unsafe_foo` and expecting a person to read the docs, and on the other extreme some people want sensible answers to anything the type system allows you to input), but the desired end-state is that when the project's requirements change you don't muck around with the ABI and implementation of `count_or_maybe_sort_for_these_three_special_customers_or_else_hit_the_db(...)`, or whatever much more generic and very wrong name the method actually has; the individual components are already correct, so you make the changes at the few methods which are actually wrong given the new requirements.

[0b] Another way of thinking about it is whether the two things should always change in tandem. For two largely overlapping beaucratic data formats? Maybe; there's a comment somewhere in this chain suggesting that they'll never go out of sync, but I'm a bit paranoid of that sort of thing. For the particular data structures that are currently shared by those formats? Absolutely not; if one diverges then you can build the new structure and link it in. The old structure is still valid in its own right.

Was that necessarily a bad thing and something that must be corrected for that code base?

I usually follow the same rule of thumb until I find myself repeatedly updating both at the same time. If I can't update one without updating the other then they must be the same thing and its time to DRY.

Don't Repeat Yourself when updating code.

Yes, there was some ongoing maintenance of this code where both versions had to be updated. The original author was not a pilot and was unfamiliar with some of the nuances of FAA airspace. One of the reasons they brought me in was that I am a pilot and knew how the FAA's data should be interpreted.

In the end, not a huge deal, but it was annoying when I had to make the same changes in two places.

While they were wrong in this case, I would say it was a reasonable move to not DRY based on the code pattern itself at the time. And that's the big difference imo - DRYing based strictly on the structure of code vs business processes.

That's not what people want to hear - they want a silver bullet; a set of criteria for which DRY'ing could be determined from the onset!

In reality: Had he DRYed it up from the beginning, you would probably have complained about a codebase that needs to corner cases into account deep in the code - the story had been turned around.

Yes, it is annoying to get into an existing code base.

So if a new customer comes and need an integration to a system we already support, even if we think they'll start out being identical, we just copy the code.

Thing is, these things evolve. Suddenly we have to patch over some process-related issues in the other system for customer A, while customer B does not have that issue. Now we can fix A's integration without worrying at all about affecting B.

Of course we write library and helper functions, and use those actively throughout, so we only repeat the "top level" stuff.

As I recall, it was after I saw the two implementations. I got curious and looked at the original FAA data and specifications and saw how much the two kinds of airspace have in common.

DRY is not just about code duplication, it’s about information/knowledge duplication, and code happens to be one representation of information.

Hyper focusing on code duplication quickly gets into premature optimization territory, and can result in DRYing things that don’t make sense. Focusing on information duplication leaves some leeway for the code and helps identify which parts of the code actually need DRY.

The difference is important, and later editions of the Pragmatic Programmer call this out specifically. But the concept of DRY often gets a bit twisted in my experience.

It is not a problem to /have/ the same code 2, 3 or even 4 times in a code base. In fact, sometimes just straight up copy-paste driven development can be a valid development technique. Initially that statement horrified me, but by now I understand that just straight up copy-pasting some existing code can be one of these techniques that require some discipline to not overdo, but it's legit.

And in quite a few cases, these same pieces of code just start developing in different directions and then they aren't the same code anymore.

However, if you have to /change/ the same code in the same way in multiple places, then you have a problem. If you have to fix the same bug in multiple places in similar or same ways, or have to introduce a feature in multiple places in similar way - then you have a problem.

Once that happens, you should try to extract the common thing into a central thing and fix that central thing once.

It feels weird to work like that at first, but I've found that often it results in simpler code and pretty effective abstractions, because it reacts to the actual change a code base experiences.

I once worked in a year-old startup of ~5 developers that found it had written the same line of code (not even copy-pasted, it was only one line of code so the devs had just written it out) 110 times. A bug was then discovered in that line of code, and it had to be fixed in 110 places, with no guarantee that we'd even found all of them. This was a very non-obvious instance of DRY, too, because it was only one line of code and the devs believed it was so simple that it couldn't possibly be wrong. But that's why you sometimes need to be aware of what you're writing even on the token level.

That's why we have principles like "3 strikes and then you refactor". 3 times fixing a bug isn't too onerous; even 4-6 is pretty manageable. Once you get to 20+, there starts to be a strong disincentive to fixing the bug, and even if you want to, you aren't sure you got every instance.

You absolutely do have to be careful. But even so, it's arguable that having to update something in 100 different places is better than updating in one place and having it affect 100 different paths where you only want 99 of them (this is some hyperbole, of course).

Yes, it will be, because copy-pasted code is never the same verbatim. First and foremost, name changes make it almost impossible to identify different copies. Then, there are different tweaks for each copy to make it suitable for the context. I always DRY early, because it's always free to copy-paste later.

Making changes quickly get onerous when the query (or slight variation on it) is pasted into multiple places. Nowadays my org has started to use Power BI so there is also multiple dashboards that all need to be updated.

At a practical level, these situations sucked. Someone had to search for the common expression, look at each instance, decide to change it to the central place or not. They spent 2-3 days on that. And then you realize that some people were smart and employed DRY - if they needed that one expression 2-3 times, they'd extracted one sub-expression into a variable and suddenly there was no pattern to find those anymore. Those were 2-4 fun weeks for the whole team.

But at the same time, I think people learned an important concept there: To see if you are writing the same code, or if you're referring to the same concept and need the same source of truth, like the GP comment says. I'm pretty happy with that development. Which is also why my described way is just one tool in the toolbox.

Like, one of our code bases is an orchestration system and it defines the name of oidc-clients used in the infrastructure. These need to be the same across the endpoints for the authentication provider, as well as the endpoints consumed by the clients of the oidc provider - the oauth flows won't work otherwise.

And suddenly it clicked for a bunch of the dudes on the team why we should put the pedestrian act of jamming some strings together to get that client-id into some function. That way, we can refer to the concept or naming pattern and ensure the client will be identical across all necessary endpoints, over hoping that a million different string joins all over the place result in the same string.

In such a case, early or eager DRY is the correct choice, because this needs to be defined once and exactly once.

We've all been there though, at some point in our careers. Possibly multiples of times (try changing thousands of "echo" statements to call a logger because it was initially meant to be a simple script that just kept growing).

It sucks but I've also been on the other side, where it was DRY but 20% of the calls to the function now needed different behavior. Finding all of those usages was just as hard.

Been there - now unless it is a very simple / throwaway code, I always start with logging setup from the start. It also helps with print based debugging because you can tune the output.

DRY is almost always a good thing to do. Coupling superficially similar code is definitely not a good thing to do.

I've taken to calling that activity (removing syntactic redundancy that is only coincidental) "Huffman coding".

> Let’s get something out of the way up-front. In the first edition of this book we did a poor job of explaining just what we meant by Don’t Repeat Yourself. Many people took it to refer to code only: they thought that DRY means “don’t copy-and-paste lines of source.” That is part of DRY, but it’s a tiny and fairly trivial part.

> DRY is about the duplication of knowledge, of intent. It’s about expressing the same thing in two different places, possibly in two totally different ways.

> Here’s the acid test: when some single facet of the code has to change, do you find yourself making that change in multiple places, and in multiple different formats? Do you have to change code and documentation, or a database schema and a structure that holds it, or...? If so, your code isn’t DRY.

The goal of DRY as a refactoring, is first-and-foremost to obviate such developer errors.

And therefore — if you want to be conservative about applying this "best practice" — then you could do that by just never thinking "DRY" until a developer does in fact trip over some particular duplication in your codebase and causes a problem.

This. Ironically the example on TFA is vulnerable to this issue. Each of the deadline setting methods has a copy of the validation ensuring that the date is in the future. If it's discovered that we need to ensure deadlines are set no later than the project deadline (since that wouldn't generally make sense), it's awfully easy to only update one and miss the others, especially after code has been added and these implementations are no longer visually near each other. I'm not saying that this means the code must be DRY'ed, but it is a risk from the beginning of the project, so one that needs to be weighed during initial implementation.

In fact, the name probably already gives some hint as to what sorts of human orgs this sort of principle would be applicable in.

And, well, the vast majority of orgs require some sort of coordination.

I.e. what you said. Couple logic that needs to be coupled. Decouple logic that shouldn’t be coupled.

Optimization is specialization (which is the opposite of DRY): to enable DRY you likely need to generalize the problem (i.e. abstract) such that you remove duplication.

- Performance - Code structure / abstraction - Data structure - Team organization / org structure

I'd argue that DRY (and a focus on abstractions more generally) are optimizations of the codebase. Not all optimizations are optimizing the same thing.

One thing that really goes against the usual programming grain is DBMSes. We're taught to always decouple/abstract things, but I'm convinced that it's impossible to abstract away your DBMS in most applications. It's just too big of an interface, and performance considerations leak right through it. It's always one of the selling points of an ORM, "you can switch databases later," and then nobody actually switches.

And in response, people will complain that they're being dismissed with "you're doing it wrong!" Because that happens with everything in programmer-land.

In the book, the authors explicitly call out that many people took the wrong idea from the original writing. They clarify that DRY is not about code, it's about what they call "knowledge", and that code is just one expression of it.

People can still disagree, but the original intent behind DRY is very well articulated.

Luckily, our compilers and interpreters have gotten so good and advanced that, in 95%+ of cases, we need not make premature “optimizations” (or introduce hierarchies of “design patterns”) that sacrifice readability for speed or code size.

They actually have a Fortran example of "optimized" code that's quite difficult to follow, but allegedly faster according to the comments. But they rewrote it to be more readable and ... turns out that's actually faster!

So this already applied even on 197something hardware. Also reminds me about this quote about early development of Unix and C:

"Dennis Ritchie encouraged modularity by telling all and sundry that function calls were really, really cheap in C. Everybody started writing small functions and modularizing. Years later we found out that function calls were still expensive on the PDP-11, and VAX code was often spending 50% of its time in the CALLS instruction. Dennis had lied to us! But it was too late; we were all hooked..."

And Knuth's "premature optimisation is the root of all evil" quote is also decades old by now.

Kind of interesting we've been fighting this battle for over 50 years now :-/

(It should go without saying there are exceptions, and cases where you do need to optimize the shit out of things, after having proven that performance may be an issue. Also at scale "5% faster" can mean "need 5% less servers", which can translate to millions/dollars saved per year – "programmers are more expensive than computers" is another maxim that doesn't always hold true).

The old salty professor who taught numerical physics at my uni insisted that function calls were slow and that it was better to write everything in main. He gave all his examples in Fortran 77. This was in the 2010s...

Was he, though? I mean, yeah having to push and pop a call stack does indeed require more work than not having to do that. However, compilers can and do inline and optimize out function calls.

And what's the real performance impact of calling functions a constant number of times outside of the hot path? Is an untestable spaghetti salad of things better than a few hypothetical push and pops?

There's wisdom behind Knuth's remarks on premature optimization.

I wanted to see what compile flags were used by guix when compiling emacs. `guix edit emacs-next` brings up a file with nested definitions on top of the base package. I had to trust my working memory to unnest the definitions and track which compile flags are being added or removed. https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages...

It'd be more error prone to have each package using redundant base information, but I would have decoded what I was after a lot faster.

Separately, there was a bug in some software aggregating cifti file values into tab separated values. But because any cifti->tsv conversion was generalized, it was too opaque for me to identify and patch myself as a drive-by contributor. https://github.com/PennLINC/xcp_d/issues/1170 to https://github.com/PennLINC/xcp_d/pull/1175/files#diff-76920...

This gives us reuse without obscuring the real definition.

I automated this in my emacs to be able to "macroexpand" the current buid file in a new buffer. It saves me a lot of time.

At a high level, at first glance, the code might look good and it "makes sense". But once you want to understand what's happening and why, you're jumping through five different classes, two dozen methods and you still don't know for sure until you run a test request against the API and see what shows up where in the debugger. And you realize your initial glimpse of understanding was just window dressing and actually nothing makes sense unless you understand every level of the abstractions being used.

It's suddenly a puzzle to understand another software developer instead of software engineering.

Now I work in C#, we have a lot of classes with a few functions, a lot of helper functions. Doesn't matter since it's so easy to use the tooling to build a mental picture - let alone refactor it in an instant if that variable name feels a bit off, or we think a function is not used (such things were always a risky exercise in Perl).

We refactored one insurance based project to use generic base classes extensively since all insurance shares some attributes and features - this really helped cut down complexity of changes and overall just reduced code on the screen to sift through. I had a lot of fun doing this, I'm a weirdo who almost likes deleting code more than writing it. Once you hit the lowest level it is a little less intuitive due to being generic but at the higher levels we mostly work at, it's simpler, and rolling out a new product we get a lot of stuff for free. They got a long way copy-pasting the product logic (4 or 5 product lines) but at this point it made sense to revisit, and I sneak a bit more in each time I have a change to do.

"Usually" being the keyword and what the article is all about IMHO. I work in a codebase so DRY that it takes digging through dozens of files to figure out what one constant string will be composed as. It would have been simpler to simply write it out, ain't nobody going to figure out OCM_CON_PACK + OCM_WK_MAN means at a glance.

When code is serial, with comment blocks to point out different sections, it is much easier to read, follow, and debug.

This is also a little bit of a tooling problem

I don't know the codebase, but to my mind that level of abstraction means it's a system-critical string that justifies the work it takes to find.

interface MyProps { thing: boolean; bar: string; baz: string; stuff: string; }

Kind of wondering what I'm missing now.

If "I don't want to maintain three copies of this" is your reaction unifying likely makes sense.

But that assumes the maintenance would be similar which is obviously a big assumption.

Having 0% DRY is probably bad, having 100% DRY is probably unhinged

Your task is to deliver a good product, not necessarily good code.

the problem is even that in concrete terms can be controversial. everyone wants to minimize maintenance work; not everyone agrees on what kind of code will achieve that.

Imo the very best approach is a codebase that's small enough that you can just do chunky refactors every so often rather than building in extensibility as a "thing". Not applicable to all problem spaces (I'd hate to do this for UI code), but for a lot of stuff it works really nicely.

For me this often looks like an external DSL/API that stays relatively constant (but improving), with guts that are always changing.

Yes. Especially at the beginning when it's critical to ensure that the logic is correct.

You can then go back and DRY it up while making sure your unit tests (you did write those, right?) still pass.

PS: same applies to "fancy" snippets that save you a few lines; write it the "long way" first and then make it fancy once you're sure it runs the way it's supposed to

I worked in Perl. Yes it has a reputation for being hard to read, but that was not the problem. Our scripting was pretty basic and easy to read. It's the loose typing, the runtime evals, the lack of strict function parameters, no real IDE, “Only perl can parse Perl” - the fact you can load a module from a network share at runtime, import it, and call a function, based on a certain run flag - and so on. Refactoring was always a mine field and there was a lot I wanted to do in my old job but could not justify it due to the risk.

So while - as you say - best practices can be at odds with each other - dev teams might be following both over time, just prioritizing one in some phase while completely disregarding it during another.

[1] E.g. the UI of the actual product might pivot multiple times at phase 1 because the product has yet to find its niche or core offering. While at a later stage the focus might be on massive scaling, either in numbers of devs or rolling out the product in new jurisdictions. Other phases might be a maintenance one, when an "offshore" team is given ownership or a sundown of an application.

You got it entirely backwards. Readability becomes far more important with the size of your project.

When you get a bug report of a feature request, you need to dive into the code and update the relevant bits. With big projects, odds are you will need to change bits of the code you never knew they existed. The only way that's possible is if the code is clear and it's easy to sift through, understand, and follow.

> You need a system of code management (documentation, diagram, IDE, tests, etc), to explain in a human-friendly way what the hell is going on.

That system of code management is the code itself. Any IDE supports searching for references, jump to definitions, see inheritance chains, etc. Readable code is code that is easy to navigate and whose changes are obvious.

As someone who has read 10,000+ lines in order to track down surprising behavior in other people's code, I can say without a doubt that readability still matters at that scale.

Code management systems can sometimes be helpful, but they are no substitute.

Ravioli code is a real problem though. Saying small chunks are readable is not enough. The blast radius of a five byte change can be fifteen code paths and five million requests per hour.

A number-crunching app or a data processing pipeline packed with spaghetti business logic is far harder ti read.

>A project with a state machine, commands, strategy patterns, etc requires an awful lot of repetitive boilerplate.

Yeah, good thing we're not a java shop...

I think this is where AI could be helpful in explaining and inspecting large codebases, as an assist to a developer.

That's a great point. Everyone lauds the benefits of chatgpt/copilot in generating new code, but I'm starting to learn that the places they might shine is onboarding onto projects and preliminary code reviews. What LlMs excel at is context, and they should excel in activities where context-awareness is key.

Project manager: Sam is working on a deadline validator. You made a deadline validator last sprint right? Could Sam use yours?

Me: No, unfortunately not. My deadline validator enforces that deadlines are in the future and are aligned with midnight UTC, to ensure correct date calculations in the database. The deadline validator Sam is working on does not enforce those restrictions. Sam's deadline validator will be applied to user input for an entirely different field, where deadlines don't have to be at midnight and are just as often in the past as in the future. In fact, Sam's validator only checks that a deadline string has the expected format and is within twenty years of the present day. My validator operates on timestamps sent as integers from another service, not string values uploaded by users.

Project manager: So your deadline validator is not reusable at all? That's unfortunate. Is there something we could have done differently to avoid this redundant work?

You two have been talking about this longer than it took Sam to implement by now ;)

My experience has been that the worst code was also the most poorly tested, if at all. In many cases, you can't really test the code without refactoring it, but you can't refactor it without risking regressions due to lack of tests.

Breaking this cycle requires going back to requirements, whether explicit or by painstakingly inferring every valid use case supported by the intentions of the original code, even if its defects meant it couldn't actually serve those use cases anyway and so cannot even act as a reference implementation for those use cases.

Once you've understood the intended behavior of the old code enough, you now have a test suite to use for any future code. This is usually the hard part [1], and it's going to seem that way because once you have it, finding the simplest code that passes all tests is just programming. Importantly, even if a future maintainer disagrees with you on the best solution, at least they can rewrite it without worrying about regressions against any tested case.

Aside: Performance regressions are more difficult to detect but preparing standard test workloads is a necessary part of that too.

After you ship this rewritten solution you're going to get user issue reports that you broke some edge case nobody has ever explicitly considered before but someone had somehow come to rely on. Now you're only adding a test and logic for that one edge case, you know that no other case was broken by this change, and that this case will never be broken again.

Now you have leverage over the complexity of the project instead of giving it leverage over you. Now you're free to refactor any which way you prefer, and can accurately judge the resulting code entirely on its own merits. You know you're comparing two correct solutions to the same problem, neither one is subtly hiding bugs or missing edge cases. Your code reviewers will also find it much easier to review the code because they can trust that it works and focus the review on its other merits.

[1] You know if your problem domain is an exception better than I do, like if you work on LLVM.

grug begin feel repeat/copy paste code with small variation is better than many callback/closures passed arguments or elaborate object model: too hard complex for too little benefit at times

hard balance here, repeat code always still make grug stare and say "mmm" often, but experience show repeat code sometimes often better than complex DRY solution

something i have learned the hard way is that DRYing out too fast paints you into architectural corners you don't even know are there yet.

Now I have the experience that most of my projects I only understand after months (or years!) of development, observing the users and testing. Only after I have experience in the project I can actually know what should be the focus of my engineering.

In WET code (write everything twice) everything looks primitive, as if it was written by complete newbie, and every change needs to be added at multiple places, but each change is trivial and time to finish is predictable. I would go as far as calling the code boring. The most difficult thing is to resist the temptation to remove the duplicity.

Like many things in software knowing when to do something and when not to do something can be hard. Premature DRY as the article mentions can lead to difficulty when eventually the use case diverges. Re-implementing everything every time everywhere is also silly. As mentioned in other places, I like the rule of 3, if you have 3 examples the point strongly a certain direction that's probably a good one to follow.

I imagine it's difficult to keep all of them in your head

This only scales so far. After some point, it's very easy to run into cases where you meant to change something everywhere but forgot/didn't know about others. Not to say everything should be so compartmentalized as to restrict change, but there is a balance to be had.

Those may be good principles to think about when you are starting out with programming, but the key is to gain experience trying to solve problems in different ways.

Over time, you get better at making the right decision about whether duplicative code should be refactored or left alone.

To be fair, we can’t always make the “right” decision, but we can at least try to make the best decision we can based on the knowledge we have.

Until your ‘if’ soup reaches all the locations in your codebase, and now you have 10 different places with too many if statements instead of one. Likely all touching slightly different things, so you can’t simply copy from one file to the other any more.

    def set_deadline(deadline):
      if deadline <= datetime.now():
        raise ValueError("Date must be in the future")
    
    set_deadline(datetime(2024, 3, 12))
    set_deadline(datetime(2024, 3, 18))

Now, as far as teaching best practices goes, I also dislike this post because it doesn't explicitly explain the pros and cons of refactoring vs not refactoring in any detail. There is no guidance whatsoever (ie: Martin Fowler's Rule of Three). This is Google we're talking about, and newer developers could easily be led astray by nonsense like this. Addressing the two extremes, and getting into how solving this problem requires some nuance and practical experience is much more productive.

> There is no guidance whatsoever (ie: Martin Fowler's Rule of Three).

That is completely unfair imo. Although not properly motivated, the advice is all there. "When designing abstractions, do not prematurely couple behaviors that may evolve separately in the longer term." "When in doubt, keep behaviors separate until enough common patterns emerge over time that justify the coupling."

Simplified maxims like "Rule of Three" do more harm than good. Don't couple unrelated concerns is a much higher programming virtue than DRY.

As someone that's made a best selling technical course, I strongly disagree.

It's 100% laziness and/or disregard for the reader.

The reason examples are as bad as they are is that people rush to get something published rather than put themselves in the audience's position and make sure it's concise and makes sense.

It's not like webpage space is expensive. There's plenty of room to walk through a good example, it just requires a little effort.

Right at the top of the page:

> A version of this post originally appeared in Google bathrooms worldwide as a Google Testing on the Toilet episode. You can download a printer-friendly version to display in your office.

So no, there isn't room for a longer example.

It is not the webpage space. It is people's limited attention spans and ability to focus. A complex example is needed to properly motivate certain concepts, but too complex an example also contains too many other details that the reader gets bogged down/distracted from the main concept being discussed.

At least that is my hypothesis for why almost all programming books and tutorials have terrible examples. I am happy to be proven wrong.

Coming back to the article, I looked at some of the previous articles from the same series, and to me it feels like a very conscious decision to only include 3-4 line code examples.

On the other hand, what beginner starts with Neo4j and Cypher? Is there really enough of them to justify a whole course? Apparently there are, it just feels weird to me.

It's really catered for beginners, people that have next to no knowledge of graph databases or Neo4j and want to get up to speed in just a few hours.

I imagine some people may not even be super technical, but may want to learn just the basics of querying a DB at work to get some basic info out of it.

Apart from lessons there are also exercises for people to practice what they just learnt, and I do my best to point out gotchas and keep it mildly entertaining with a gentle progression in difficulty.

    def set_deadline(deadline):
      if deadline <= datetime.now():
        raise ValueError("Date must be in the future")
    
    def set_task_deadline(task_deadline):
      set_deadline(task_deadline)

    def set_payment_deadline(payment_deadline):
      set_deadline(payment_deadline)

    set_task_deadline(datetime(2024, 3, 12))
    set_payment_deadline(datetime(2024, 3, 18))

So the argument from the article...

> Applying DRY principles too rigidly leads to premature abstractions that make future changes more complex than necessary.

...does not apply to this example.

There clearly are kinds of DRY code that are less easy to reverse. Maybe we should strive for DRY code that can be easily transformed into WET (Write Everything Twice) code.

(Although I haven't worked with LISPs, macros seem to provide a means of abstraction that can be easily undone without risk: just macro-expand them)

In my experience, it can be much harder to transform WET code into DRY code because you need to resolve all those little inconsistencies between once-perfect copies.

My personal goal is to get things done in as few lines of code as possible, without cramming a bunch on one line. Instead of coming up with fancy names for things, I try to call it by the simplest name to describe what it's currently doing, which can be difficult and is subjective.

If we wanted to define a function which crashes like the example, I would probably write this:

    def throw_past_datetime(dt):
        if dt <= datetime.now():
            raise ValueError("Date must be in the future")

    dt = datetime(2024, 5, 29)
    if dt < datetine.now():
        # Handle past date gracefully?

Problem with unintentional coupling isn't that you can't undo it. It is that someday someone from some other team is going to change the method to add behaviour they need for their own use case that is different from your own and you won't even notice until there is a regression.

My experience is mostly in Ruby though, so I'm not sure how well it generalizes here :-)

> Yeah, that was written by and he couldn't abstract his way out of a paper bag

I guess the real lessons were the crappy decisions that someone else made along the way.

:)

Right. It's not an optimization problem!

Remember in school when you learned to turn a truth table into a Karnaugh map and then use it to find the smallest equivalent logic expression? Well, your code is not a Karnaugh map, is it?

Premature dry'ing is the same as premature engineering. And lest someone go 'oh so YAGNI is all you need'... no, sometimes you are going to need it and it's better to at least make your code easily moldable to 'it' now instead of later. Future potential needs can absolutely drive design decisions

My whole point is that dogma is dumb. If we had steadfast easy rules that applied in literally every situation, we could just hand off our work to some mechanical turks and the role of software engineer would be redundant. Today, that's not the case, and it's literally our job to balance our wisdom and experience against the current situation. And yes, we will absolutely get it wrong from time to time, just hopefully a lower percentage of occasions as we gain experience.

The only dogma I live by for code is 'boring is usually better', and the only reason I stick by that is because it implicitly calls out that it's not a real dogma in that it doesn't apply in all cases.

(Okay, I definitely follow more principals than that, but don't want to distract from the topic athand)

You wrote that code 4 years ago with tons of abstractions designed for some day someone not having to repeat themselves... but it's been years and they've never been useful. However I've had to dig through a dozen files to make the change I needed to make which by all rights should have been entirely contained in a few lines.

My most common reaction to a new codebase is "where the hell does anything actually get done" because of silly over-abstraction which aspires to, one day, save a developer five minutes or three lines of copied code.

IMHO - some of the clean code books have ruined the industry as much as the virtues of microservice preachers have.

The second number goes to the Javascript tooling.

And here's the kicker: If you were aware the individual was taking inspiration from a book easily in such a way, what made it difficult for them to take inspiration from you through a dialogue? Hopefully your arguments were stronger than your HN comment.

So DRY isn’t an obvious default for me.

There is no question in my mind that this should always have been built as either a single pipeline with some parameters to cover the two use-cases, or perhaps as a toolbox of reusable components that are then used for the overlapping area. But I expect the mentality at the time the second one was being stood up was that it would be less disruptive to just build the new stuff as a parallel implementation and figure out later how to avoid the duplication.

More importantly, cleverer people than me said "premature optimization is the root of all evil"

If you want a specific example look at something that needs to be performant, i.e. in those 3%, let's say OpenSSL's AES implementation for x86, or some optimized LLM code, you'll see the critical performance sections include things that could be reused, but they're not.

Also the point Knuth is making is don't waste time on things that don't matter. Overuse of DRY falls squarely into that camp as well. It takes more work and doesn't really help. I like Go's proverb there of "A little copying is better than a little dependency."

Source: Donald Knuth on the Lex Fridman podcast, when Lex asks him about that phrase

In raw performance this could manifest as issues with data duplication bloating structures and resulting in cache misses, generic structures expressed in JSON being slower then a purpose-built struct, chasing pointers because of functions buried in polymorphic hierarchies. But I doubt that any of this would really matter in 99% of applications.

In some cases it makes sense, like writing SIMD/etc. specific assembly for compression/decompression or video/audio codecs, but more often than not the readable version is just as good -- especially when compilers can do the optimizations for you.

A lot of times I've found performance increases have come from not duplicating work -- e.g. not fetching the same data each time within a loop if it is fixed.

If you measure the code's performance and see the ++i/i++ is consuming a lot of the CPU time then by all means change it, but 99% of the time don't worry about it. Even better, create a benchmark to test the code performance and choose the best variant.

But it's your life to live. Who cares what someone else thinks?

Also, your digression about modern compilers is irrelevant to the context of the quote, since Knuth talked about premature optimization at a time when compilers were much simpler than today.

If you're talking about consolidating some code from one in-process place to another in the same language, you're mostly right: there's only going to be an optimization/performance concern when you have a very specific hotspot -- at which point you can selectively break the rule, following the guidance you quoted. This need for rule-breaking can turn out to be common in high-performance projects like audio, graphics, etc but is probably not what the GP had in mind.

In many environments, though, DRY'ing can mean moving some implementation to some out-of-language/runtime, out-of-process. or even out-of-instance service.

For many workloads, the overhead of making a bridged, IPC, or network call swamps your algorithm choice and this is often apparent immediately during design/development time. It's not premature optimization to say "we'll do a lot better to process these records locally using this contextually tuned approach than we will calling that service way out over there, even if the service can handle large/different loads more efficiently". It's just common sense. This happens a lot in some teams/organizations/projects.

A really common example is overhead of polymorphism, although that overhead can vary a lot between stacks. Another is just the effect caused by the common complaint about premature abstraction: proliferation of options/special cases, which add overhead to every case even when they don’t apply.

There is the "single source of truth" problem where you need to compute something exactly the same way at all points in the software that need to compute that thing. In these cases you really want a single library implementation so that the meaning of that computation does not accidentally diverge over time since there is a single implementation to maintain.

There is the "reuse behaviors in unrelated contexts" problem where you want to create implementations of common useful behaviors that can be abstracted over many use cases, often in the context of data structures and algorithms. In these cases you really want generics and metaprogramming to codegen a context-specific implementation rather than sharing a single implementation with a spaghetti mess of conditionals.

DRY works best when it fits neatly and exclusively into one of these two categories. Cases that fit in neither category, such as the practice of decomposing every non-trivial function into a bunch of micro-functions that call each other, are virtually always a maintenance nightmare for no obvious benefit. Cases that fit into both categories, such as expansive metaprogramming libraries, become difficult to maintain by virtue of the combinatorial explosion of possible implementations that might be generated across the allowable parameter space -- the cognitive overhead grows exponentially for what is often a linear increase in value.

That has approximately nothing to do with DRY. At best it might technically be a violation of DRY if (and only if) some of those micro-functions are identical, but the correct way to fix that is to recompose them into the non-trivial (and non-repeated) functions they're more usefully expressed as. And more often it's just a totally independent refucktoring that makes the codbase worse entirely orthogonally to 'DRY-ness'.

Whenever I have to have a "mental model" of the code, I know I screwed up.

This way, the reader can just do a global grep to find all the places to change, and you don't have to list them in each place and keep them in sync.

I think the example in the article does a bit of that as well. The example sets a deadline on a thing (a task or payment) by validating the deadline against the current time, and then presumably doing something else that isn't shown. The article argues that in the future a task might have different validation requirements than a payment, and they're only coincidentally the same today; so it would be foolish to abstract the deadline setting logic today. BUT, the reality is that the real coincidence is that payments and tasks have the same set of validations, not that the logic to validate a deadline is coincidentally the same. In my opinion "good" code would be fine to have separate set_task_deadline and set_payment_deadline methods, but only one validate_deadline_is_in_future (or whatever) method, alongside other validation methods which can be called as appropriate by each set_x_deadline implementation.

Disclaimer: the code is so short and trivial that it doesn't matter, I think we can all assume that this concept is extrapolated onto a bigger problem.

Routines that are conceptually identical should share an abstraction. A concept might benefit from an abstraction even if it is only used once. It is never too early to add more intuitive abstractions to your code.

On the other hand, code that is only coincidentally similar in execution should never be forced to share an abstraction.

The duplication is typically bad because it leads to inconsistency which leads to bugs. If your code is highly cohesive and loosely coupled, this is less likely [across independent components].

And on this:

> When designing abstractions, do not prematurely couple behaviors

Don't ever couple behaviors, unless it's within the same component. Keep your code highly cohesive and loosely coupled. Once it's complete, wall it off from the other components with a loosely-coupled interface. Even if that means repeating yourself. But don't let anyone make the mistake of thinking they both work the same because they have similar-looking interfaces or behaviors, or you will be stuck again in the morass of low cohesion. This is probably one of the 3 biggest problems in software design.

Libraries are a great help here, but libraries must be both backwards compatible, and not tightly coupled. Lack of backwards compatibility is probably the 4th biggest problem...

There are still good reasons to DRY early on. Actions that need to be performed synchronously, rather than acquiring the same lock in several places, consolidate your code so you're so there's at most a few places you acquire and release that lock. Cache invalidation, having a single class for reading and writing some piece of data makes it much easier to keep the cache consistent.

Or rather, don't assume the same thing in two different places, especially not implicitly.

Avoiding code duplication mostly follows from that.

One way that helps with that is creating abstractions. I don't mean clever grandiose abstractions, those are extremely hard to get right. I mean precisely those abstractions that factor shared code.

But even before that, it's important to get the control flow right to minimize doing the same thing multiple times in the codebase. Because even when the implementation is in a central place, calling it multiple times from different locations is what you say, duplicating assumptions.

However, very often, unstated assumptions sneak in.

For example, assuming to understand how to interpret a certain string, which usually causes all kind of escaping issues.

The problem with DRY occurs when it contravenes this principal - when deduplication is too expensive and/or unlikely to decrease the cost of future mutations enough to be worth it.

The proposed problem isn't a binary - that you should or shouldn't make the assumption yet - but rather that the assumption has a cost based on what you believe is likely to occur in the future and the value produced by making the assumption now needs to outweigh the cost.

I see a lot of attacks on DRY these days, and it boggles my mind. Maybe it is being conflated with over-engineering/paramterization/architecting. I don't know.

But I do know that having to fix the same bug twice in the same code base is not a good look.

a) a terminal running a text mode app extracting data from a mainframe

b) Excel

c) Outlook

Their sole job was to copy data from the terminal, paste it into Excel on a daily basis, save it as a CSV file, and send it as an attachment to the address monitored by a script that was responsible to running data processing pipeline. Because of the manual nature of the job and the way data was presented in the terminal there were errors, which were really unpredictable. I was not allowed to talk to the person doing this job or visit them in their office so it took me three months to find out that what was shown on the terminal screen was essentially a 80x25 version of a punchcard and that position of the fields mattered. Sometimes the user would not copy the whole screen, sometimes there would be an extra character added while pasting data into Excel (not always a stray "v"), sometimes a gremlin character would be added between the mainframe and the terminal (there must have been a serial connection somewhere). Forget proper encoding, JSON or XML, it was raw data, really raw. When I started working on the pipeline it would break on every incorrectly formatted record, when I finished the job and left it was only barfing on 5% bad records and those that it could not process would be neatly put aside an emailed to the support people responsible for dealing with this client. There was a lot of repetition initially, but I then discovered patterns (random offsets, encoding errors, extra characters) and that allowed me to build a set of generic classes to quickly implement problem-specific handlers.

It's right to be concerned that a bunch of teams might be a wasting time implementing the same thing slightly differently, or that you'll end up fixing the same bug across all these "copies".

Often that kind of duplication is good for the business though, and the platform team doesn't have the insight to see all the divergent requirements on the horizon. Letting the teams innovate separately without having to coordinate changes in all these DRY-ied up systems can be the best way to support them.

Or even more abstractly, zero codebase chaos is not the optimal amount.

For every purported best practice, there are teams/orgs that painted themselves into a corner by getting carried away and others that really would have benefited from applying it more than they did.

In the case of DRY, it's an especially accessible best practice for inexperienced developers and the project leads many of them become. Many many teams do get carried away, mistaking "these two blocks of code have the same characters in the same sequence" with "these two delicate blocks of code are doing the same thing and will likely continue to do so"

Having advice articles floating around on both sides of practices like this helps developers and teams find the guidance that will get them from where they are to where they need to be.

Context, nuance, etc, etc

Best practices can be like that. "Here's something to avoid!" "OK, I'll back far away from that." Yeah, but there's another cliff behind you, of the opposite error that is also waiting to wreck your code base.

Listen to best practices. Don't apply them dogmatically, or without good judgment.

Huh. When you put it that way, that's actually a good point. In my experience, competent programming will try to consolidate repeated code, and then cite "because DRY" if asked why, but I can't think of any case where I or anyone else competent started with "needs more DRY" as the original motivation (as opposed to "this is a incomprehensibly verbose mess" or the like).

Conversely, starting with "don't repeat yourself [and don't let anything else repeat itself]" as a design goal does seem to correlate well with cases where someone temporarily (newbie) or permanently (moron/ideologue) incompentent followed that design principle off a cliff.

I absolutely agree with this, and the only thing I would add is that is difference is even more pronounced in codebases using a dynamic language.

Sure it's not easy to navigate a bowl of duplicated spaghetti, but navigating opaque DRY service classes without explicit types is a nightmare.

Luckily as an industry we've realized the benefits of static typing, but your point still holds true there.

My experience is the opposite. The less code, the better. I just spent a week on refactoring UI automation test code where they had copied the same 30 lines of code into almost 100 places. Every time with an ID changed and some slightly different formatting. It took me a few days to figure out that these sections do the same thing so I decided to introduce a function with ID as parameter. It was a lot of work to identify all sections and then to make sure they are really equivalent.

Saved us 3000 lines of code and now we can be sure that timeouts and other stuff is handled correctly everywhere. An we can respond to changes quickly.

that's DRY to me. Don't copy/paste code. Introduce functions. Ideally in the simplest way. When you have functions, you declare the same behavior everywhere.

To get internet points easily, write something of that:

“Clean code is overrated”

“SOLID is holding you back”

“Tests are less important than profits”

“KISS is the only important principle”

“Declarative programming is only suitable for pet projects”

“Borrow checker is the plague of Rust”

and so on.

Good way to go at it is to isolate the functionality that is used many times and to pull it aside in its own function (or similar). That's just good code practice and also makes it easy to refactor and modify as needed.

Shareable components are more effort to maintain, so just creating them because they consolidate code is not always a good idea. You really want to have positive ROI here and you only get that if you actually reduce maintenance burden. For raw code duplication that doesn’t have a maintenance issue on it‘s own, the bar is a lot higher than most people think.

It would have been OK if they'd used the right methods on Buffer but they didn't.

These encoding/decoding methods are a very good example of code that should be centralized, not least so you can write tests for them. (It is a favorable case for testing because the inputs and outputs are well defined and there are no questions of whether execution is done like you might encounter testing a React component) It is so easy to screw this kind of thing up in a gross way or an a subtle way (I'm pretty sure btoa's weirdness doesn't affect my application because codepoints > 255 never show up... I think)

There's the meme that you should wait until something used 3 times before you copy it but here is a case where two repetitions were too many and it had a clear impact on customers.

As with anything dogmatic, it truly depends. There are times when the abstraction cost isn't worth it for a few semi-duplicate implementations you want to combine into a single every-edge-case function/method.

One of my biggest gripes (related to the post) is the data structure

   create table student (
      ...
      applied_date                         datetime,
      transcript_received                  datetime,
      recommendation_letter1_received      datetime,
      recommendation_letter2_received      datetime,
      rejected_date                        datetime,
      accepted_date                        datetime,
      started_classes_date                 datetime,
      suspended_date                       datetime,
      leave_of_absence_start_date          datetime,
      leave_of_absence_end_date            datetime,
      ...
      graduated_date                       datetime,
      ...
      gave_money_date                      datetime,
      died_date                            datetime
   )

If you think before you code you might settle on this design

    create table history (
       student_id                         integer primary key,
       status                             integer not null,
       begin_date                         datetime not null,
       end_date                           datetime
    )

If "No code" is to really be revolutionary it's going to have to have built-in ontologies so that programmers get correct data structures for situations like the above that show up everyday in everyday bizaps where there is a clear right answer but it is usually ignored.

1. The first table structure is a flat non-normalized table structure that trades normalization for easy to query and select computed properties

2. Second structure is a normalized table structure that trades the normalization for joins.

It is easy to write a query for the first that gets a list of students names and the dates they applied. That query is harder for the second one. On the other hand figuring out what state a user was in at time t could be a very hard problem with the first table.

My experience with the first is that you find corrupted data records, one cause of that will be that people will cut and paste the SQL queries so maybe 10% of the time they wind up updating the wrong date. Systems like that also seem to have problems with data entry mistakes.

The biggest advantage of #2 is ontological and not operational, which is that in a business process an item is usually in exactly one state out of a certain set of possible states. Turns out that this invariant influences the set of reasonable requirements that people could write, the subconscious expectations of what users expect, needs to be implicitly followed by an application, etc.

Granted some of the dates I listed up there don't quite correspond to a state change, for instance the system needs to keep track of when a student started an application and when the last document (transcripts, letters, etc.) has been received. With 5 documents you would have 32 possible states of received or not and that's unreasonable, particularly considering that a student with just one letter and a very strong application in every other way might get accepted despite that. It's fair to say the student can have an "open application" and a "complete application". Similarly you could say the construction of an airplane or a nuclear power plant can be defined by several major phases but that these systems have many parts installed so if the left engine is installed but the right engine is not installed these are properties of the left and right engine as opposed to the plane.