Once you learn to see it though, it's hard to ignore. Now I can tell instantly if code is over-engineered. Unfortunately It seems like maybe 99% of code is over-engineered. The developer's incentive to maximize their own lock-in factor and billable hours are powerful forces. Even developers who appear to be totally robotic and ego-less are often guilty of over-engineering. It works on the subconscious mind. Few are ever able to escape the mindset because they are not fully conscious. They are not thinking about every single line of code that they write. They decide on some goal and then churn out whatever code first pops into their heads to get incrementally closer to that goal... Not realising that, at each step, there were many alternative paths that which were superior.

In my experience these are rarely, if ever, the reasons for over engineering.

"Bad" design can be fine if it's kept simple.

I was thinking of good code as something I had to invent, but now I feel like it's more like something I have to discover.

But, for contemporary software developers in general, I'd guess more often it's either resume-driven development (e.g., add the big complex new framework keyword to your resume) or not yet having enough experience to know what complexity is worthwhile for a situation.

I had a team once implement the MediatR pattern for a

When I asked them to remove it, it became a political fight that went to the VP because they were convinced that sort of flexibility was a good idea. Fast forward a year and they have a new technical leader and he thought I wanted that complexity until we had a conversation and when I mentioned I didn't like it he confessed to me he wanted to rip it all out.

People value the wrong thing too often.

Believing flexibility is needed still suggests a lack of understanding.

Values are those things that help guide you in the face of imperfect information. Not having a crystal ball that can predict the future perfectly means you have imperfect information.

This leads to a certain kind of decoupling, although with a higher tolerance for coupling that can kept in check by static analysis.

It took me a month to realise that the staff developers very much revelled in and protected their bad code and bizarre domain choices.

It was so far gone that there was no way to get rid of them and the product was just slowly dying and burning the remaining cash.

Then the mergers happened and they all got let go, only retaining the name for brand power and the entire stack was quietly moved over to another similar product which was rebranded.

Separately, there were absolutely very large consultancies that had a programming style/rules based on making their implementations difficult to read/modify, needing to call their COE back in to fix their code or add features - with it being very hard to modify. Talking entire codebase structured with ridiculous levels of abstraction and annoying code style. Bad integrations requiring their tooling to work and make sense of etc.

They target traditional orgs where the management just wants to get a project through and then bleed them over years.

I think personality can account for this without any reference to incentives, which come in to explain how this personality problem can be so common among successful engineers.

So much of the grug-brained argumentation is same-side violence, is developers roasting developers. Assigning bad motives & declaring immoral bad people among us probably takes already same-side disdain & amplifies it, forments real & powerful hatred. Yes, various petty or criminal over engineering happens some, yes. But I usually think there are simpler Hanlon's Razor explanations that aren't even the individual's fault, are just the story of an org: an organizational body where so many constituent members of the body-whole have so little idea what others are doing or have done, and access to so few who can offer informed situational-appropriate wisdom for whatever dark end a squad finds itself tasked with.

In some ways, this has been the actual story of open source. We have not been end user facing. We have grow communities of practitioners with individual knowledge & experience that ports & can be compared & discussed with others, from outside our company. We get peership that companies can rarely afford or find after they grow mid-sized.

The Influx folks just talked about their 3.0, & replacing their custom built engine with Flight, DataFusion, Arrow, and Parquet. Maybe they might miss one box or two on neat nitty gritty optimizations (maybe this guy elsewhere in this thread, insisting on diy'ing binary encodings because protobuf doesn't have 16-bit ints can find some deficiencies, https://news.ycombinator.com/item?id=38078133), but the nice to haves of ecosystem integration seems like an intangible, but one that lets little good things out there plug in & help out, and that seems invaluable. https://news.ycombinator.com/item?id=38013714 https://www.influxdata.com/blog/flight-datafusion-arrow-parq...

The top comment to me epitomizes the worst of grug brainedness. It's smug & certain of itself, a strong & divisive Hanlon's violation of high degree. That kind of blue-on-blue attitude to me is far worse than 99% of the people who end up over engineering. Few organizations have real defense, lack coherency, to help themselves away from complexity, or they have such tight constraints that they overprune & disaffect those who do care and have good ideas. These are organizational issues, and having an org that can somehow see itself & what's afoot is the huge challenge. Stop hating on devs, please.

I don't think this makes sense. For example currently I have a big inclination to think bad about developers working for Vercel/recommending NextJs because they are invariably the same ppl and do it because of monetary benefits. The intentional over engineering they are adding to make a profit is insane and evil. There is no way I can make this point without "hating" the people doing it.

1. I like interesting puzzles. A lot of code - especially commercial code - is pretty boring if you do it right. I find myself subconsciously pushing for features that will be fun to implement. And by "fun", I mean, features that will overcomplicate everything.

2. While I'm in the middle of programming something, all the choices that I make seem straightforward and necessary. Its only later when I step away from my code and then try to understand it with fresh eyes do I notice what a mess I've made of everything.

I also think that a lot of the time the most obvious solution to most problems is quite complex. It takes wisdom, skill and domain knowledge to know where to look for simple solutions to any given problem. Simple, clean solutions are rarely obvious.

"Oh, it looks like we're slowly implementing a halfbaked message queue. Lets use a standard message queue instead." "Oh, we're slowly building up a custom, buggy, binary framing protocol. Lets just use protobuf/msgpack". "How about instead of writing a custom RPC protocol to fetch data, we just use REST over HTTP. And then we can put nginx in the middle to cache our backend responses, and we can throw out our custom cache."

Developers grow. It starts with simple code that doesn't work. The next step is, complicated code that solves the problem, in messy unmaintainable ways. The next step is writing super clean, almost boring code, that's highly readable and "dumb" and does exactly what it's supposed to do.

The other thing to realize, a lot of great code doesn't just spring from the developer's hands in its final form -- it's extensively edited and rewritten into its final, good, form.

Not entirely facetiously, I think that, for engineers, there's real skill in creating simple solutions to simple problems—not, for example, finding the general instance of the problem and solving that, when the problem is unlikely to recur and crafting the perfect general solution delays delivery on what's actually in front of you.

(I know Perl's not fashionable any more, but I've always liked its design philosophy of "make easy things easy, and hard things possible." It seems like a slogan that can be adaptable to how to solve problems, though I'm not sure of the absolutely perfect analogue. Hmm, maybe I'm trying to solve the general instance of a problem ….)

1. linked list is a generic data structure with a relatively simple interface

2. an application with 1k configuration values is generic in that it can handle everything, but is in no way simple.

This is true, but I think also illustrates the phenomenon. It's tempting not to solve the specific problem in front of you, because a more general problem might be easier and more elegant to solve—and this mindset can easily lead one into, at worst, never solving the real problem; or, at less worst, solving a problem that's so far generalized that no one else looking at your code can tell why it's doing what it's doing.

(It can also happen that the more general problem doesn't have a simpler solution. If I want to print a string that has a few hard-coded values in it, formatted a particular way, I could develop a formatting spec and write a formatting library to process it, which is surely the right solution to the general problem—but, if the specific problem is likely only going to arise once, then it may be both easier to understand and a better use of time just to put in the hard-coded values.)

That's what factorio is for!

https://projecteuler.net/

By using protobuf/msgpack you lose the ability to precisely control the layout and encoding of your data on the wire. Most applications don't care, but this results in your wire representation being defined by "whatever protobuf says".

Say I want to transmit an unsigned 16 bit integer with protobuf. How do I do that? The documentation doesn't include 16 bit integers as a datatype, so I'd probably have to wrap it in 32 bits and/or use some varint stuff. It would be simpler to just write a big endian 16 bit int though.

I wish there was a simpler alternative to protobuf that gives more control to end users and doesn't try to be smart. Until then, making your own binary protocol is not over-engineering.

If I really wanted to squeeze out bloat, I’d try to use https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_PER_%... (which has a paid standard, but isn’t well known) before resorting to a completely ad hoc protocol.

Something like "use fixed length 8/16/32/64 bit signed/unsigned integers in big endian, length prefix can be 8/16/32 bits, bool is 1 byte (00 = false, 01 = true)" etc, without extra stuff like varints or bit packing, which a lot of current formats are doing.

In short, just use the most straightforward way of encoding while also using the least amount of data. Big endian for ints is very common, simple and relatively compact if you only use the bit width that you need.

Developing good instincts for this stuff takes a lifetime.

That is a very good point. If your code doesn't look boring, then you probably doing something wrong.

EDIT: I ironically wrote way too long here. Grug say better:

> note, this good engineering advice but bad career advice: "yes" is magic word for more shiney rock and put in charge of large tribe of developer

People can write perfect, simple, DRY code if they have the time to and are incentivized to. In most cases you're rewarded for launching the thing and showing one's "technical prowess" with the amount of work / intellect / design skillz™ required to launch the thing. The natural conclusion of this is that everything becomes a bloated, over-engineered mess of kludge solutions that gets rewritten every 3-7 years.

I haven't seen any data on this but I'd guess the "rewrite half life" is correlated with turnover / average tenure, so even if people tried harder to write not-over-engineered code, it'd probably get rewritten anyway. As a perfectionist, this truly bothers me, but I find sometimes thinking harder / spending more time on the best _simple_ solution is rewarded less than building the complicated overengineered thing. I'm sure better organizations exist but I have yet to find one in 9 years as a SWE...

Open Source projects are actually the best counter-example to this that I can think of, but even then the best libraries sometimes get rewrites or new versions when they've changed hands from one maintainer to another, and I'll note that financial incentives are very different between open source and the types of enterprise-y cruft most people working full time as SWEs on HN probably see. It's like comparing a well crafted academic paper to a lazily written work email.

In my experience, people trying to make code DRY also wind up writing over-complicated patterns and abstractions to make it so.

I think a large amount of over-engineering is likely due to people applying patterns where they don't need to, or building unnecessary abstractions, or otherwise doing what they think is "good code".

Most folks aren't incentivized or simply don't bother to think through things and try harder. The "this way looks more smart so I made it all complicated" thing definitely happens too, but in eng orgs like I'm part of where we go through design reviews to try and cull that sort of thinking, the other less-intentional version is still common enough to be a problem at scale.

Regardless of the learnings you had having merit, you don't see a problem with the mental model you've developed, if it's output is giving the same answer 99% of the time?

In itself it doesn't mean it's necessarily wrong, but since you're assessing the quality of other people's work I'd assume you have a heavy bias in there.

The smartest are those who can make complex problems look simple, with simple solutions.

The problem isn't usually the code, it's the people in charge not giving the right instructions, shifting the goal post, allowing feature creep, or the deadly sins of rewrites, large refactors, unrealistic deadlines.

Over-engineering is just fine in the real world. The problem is when that causes cost and deadline overruns. Those can be controlled for, even if the complexity can't. But finding a manager who knows how to manage software teams effectively and simultaneously keep his bosses from digging their own graves is even rarer than an engineer who writes simple code.

No one can, and no one is. All lines of code are over-engineered, just some more so. No amount of thought will get you to the perfect solution; it's an unapproachable asymptote. We're all creatures of time and there's a limit to how many cycles we can afford to spend iterating in design space to try and hit the right note. (Not to say we can't get better with practice, though.)

Hah, I just experienced this when I asked my brother for a simple comparison of about 7 criteria in a 3 column table, to include in a report.

He gave me 12 criteria not entirely from the data source I was looking for, and a cost calculator to boot.

On the other hand, over the course of a few weekends he coded up a beautiful baroque bastard of an excel macro that saved us thousands of man-hours over 5 years.

In my experience (including my own development), overly complex designs accrete, as opposed to start off complex.

They usually seem to begin with "This is simple, let me just do this...", then, when we run into Roadblock A, we design in a mitigation, and so on...

Eventually, we have a ghastly chimera.

Other times, it comes from trying to coerce software written for one purpose, into another, and the glue code is kinda messy.

Also, there was an article mentioned here, about "Don't design a general-purpose framework."

I can concur with that. The app I'm releasing now, has a server component that is, in my opinion, way too complex. I designed it, and implemented it, so I get to say that.

The deal was that I originally developed it as a general-purpose framework. It has a layered architecture, and I did heavy-duty unit testing of each layer, as I was writing it.

It works very well, is fast, and secure.

But way too complex, as this app is its only implementation. It handles a lot of stuff this app never touches, like trees of user permissions. I have a very simple, rather "flat" permission structure, so a whole shitton of code never gets used. It was tested heavily, works well, but will never be used. I don't like having unused code paths, but I don't have the luxury of time to remove it (I have removed some, but there's plenty more, where that came from).

If I were to rewrite it (I won't -see "works very well", above), it would be much simpler.

I suspect this phenomenon explains a lot about how this industry has developed over the last few decades. Any significant software nowadays requires a team of baby sitters just to keep operating. Nothing is ever done and everything keeps changing for no good reason.

The biggest factor I've found in overengineering is the lack of a long term roadmap. If you need to build a feature X, and you engineer the bare minimum and need to put in the same amount of hours to do X+1, your management is going to be upset that you're taking too long to ship. You already had it 80% (in terms of feature complete) of the way there, why is that extra 20% as hard as the first 80%? So the engineers build up scar tissue. If you have to handle a ton of cases that you don't understand, why not build a CaseHandlerFactory so you can add the next feature faster?

A clear roadmap of "this is what we want in 1 year" will help solve over engineering. Otherwise engineers are incentivized to make their code as configurable, modifiable, and extendable, as possible, regardless of cost or business need. Not to mention all the additional time trying to figure out "canonical" data models that will "future-proof" the applications interfaces. If you have to iterate quickly (which is not as common as agile folks wish), you need to build up a raport with leadership to help them understand that speed comes with trade offs: faster might mean 'more work to change later', while slower to market today might mean faster iteration down the line. These discussions ARE valuable for leadership, as sometimes they need a quick win because of a Q3 earnings hole or contract, and sometimes they are willing to make longer term investments.

You move too quickly, they call your solutions hacky, you move too slowly, they call it overengineered. Everyone needs to be on the same page of what the change is trying to do: win short term, win long term, or somewhere in the middle.

I've never understood this, maybe I'm just a bad engineer. I mean, it sucks having to pivot, but all those configurable and extendable pieces take hours trying to get the design right. And you only end up using ~1% of them, and then something you didn't foresee happens as well. I always end up spending months of my life saving days doing the change-overs.

And that's before we get into the problems with onboarding a new engineer (or being the one onboarded) into the kind of hellscape that the overly configurable application turns into.

Can subscribe to this, 16 years career. Even a horizon of 3 months seems unattainable.

I call it a "management problem", but maybe that's because I'm not in management..

Also, I think most developers just don't want to do the same thing twice. And most developers really are writing the same software over and over again through their careers, with minor changes. So they need to change something to keep it interesting, and the only things they can easily change are technology and methodology.

Honestly for me over-engineering is usually the result of the opposite. Thinking too much when writing and having preconceived ideas about what a codebase ought to look like.

It was Casey from Handmade Hero IIRC who called his style of programming "compression based", effectively just writing code and factoring out what belongs together incrementally. Abstracting things out as they repeat, not consciously by design. I've taken this up more and more as a way to program.

Some fellow devs seem to live creating big beastly complex abstract PatternFactoryClassBuilderGenerators for simple one off use cases which should be quite simple.

Having devs and PMs on board with adding estimations and spending the time actually doing that refactor on the second or third time you're following a pattern is the tricky bit. It pays dividends long term though as you maintain velocity.

I think the general inclination here towards static typing is due to this, rather than any evidence that statically typed languages lead to higher quality software. Engineers just love puzzles. I'm also looking at you, Rustaceans...

runs for his life

The best answer I can come up with is that people just seem to have differently wired brains. For me, static typing- even fairly sophisticated static typing, is simple. It makes the code simpler, easier to reason about, easier to refactor, and with a sufficiently expressive type system it lets you build things in a much more intuitive way than you could otherwise. It’s not about solving puzzles for the sake of them- types remove a big part of the puzzle by letting me explicitly write things down- and letting the compiler keep track of the details.

Certainly plenty of people don’t see it that way, and I’ve heard a lot of people make similar arguments about dynamic typing being simpler and more expressive. I don’t think they are lying but I see a big pile of inscrutable pain when I work in large dynamically types codebases.

I know I’m right about my experience, and I trust other people are right about theirs, so there must be some significant divide in how we conceptualize code that makes one persons elegant simplicity another’s intolerable complexity.

Every tool has the potential to be misused in accidental complexity.

But the JS function we would have written would have required someone using it to read and understand it, and the TS function (without using 'any') needs to fully express what its inputs and outputs can look like.

Because of this TS actually tends to guide me towards writing more "Grug brained" code, because I refuse to use 'any' (and throw away TS benefits) and using generics usually requires a trip downstairs for a fresh cup of tea.

I have a side-gig as an external examiner for CS students, and well, a lot of the stuff we grade our students on are things that I’ve had to “unlearn” myself throughout my career, because usually complexity and abstraction aren’t going to work out over a long period of time for an IT system. This obviously isn’t something that’s universally true. I do tend to work in non-tech enterprise organisations (or startups transitioning into enterprise) and in this space a lot of what is considered good computer science just doesn’t work. It’s everything from AGILE, where you project processes will trip you over as you try to juggle developers who both need to build and maintain things at the same time. To how we try to handle complexity with abstractions, and how those abstractions sometimes lead to “generic” functions where you need to tell a function 9001 different things before it knows how to perform “because it just evolved”. It’s in everything really, like, we teach students to decouple their architecture and it’s absolutely a good thing to teach CS students, but the result is that a lot of them “overengineer” their architecture so that you can easily swap which database your system is using (and similar) in a world where I’ve never actually seen anyone do that. Anecdotal, sure, but I did work in the public sector where we bought more than 300 different systems from basically every professional supplier in our country, and you’re frankly far more likely to simply replace the entire system than just parts of it.

But how are you going to know that when all you’ve been taught is the “academic” approach to computer science by teachers who might have never experienced the real world? Being self-taught isn’t really going to help you either. I can’t imagine how you would even begin to learn good practices in the ocean of “how to” tutorials and books which are essentially little more than the official documentation on a language and maybe a couple of frameworks.

> The developer's incentive to maximize their own lock-in factor and billable hours are powerful forces

This part, however, I disagree with. Again this is very likely coloured by the fact that I’ve mainly worked in the space where developers both build and maintain multiple things, often at the same time. But I’ve never met developers who wanted to do this. In fact I only meet developers who genuinely want their code to be as easily maintainable by others as possible because we all absolutely hate breaking off from actual development to fix a problem. That being said, I do think there is a natural risk of ending there accidentally if you haven’t “unlearned” a lot of the academic CS practices you’ve been taught. Especially because there is a very good chance you didn’t really “learn them right”.

At least STEM subjects are tested against repeatable natural phenomena or mathematic validation.

A CPU is indifferent to you complexifying yourself, ad infinitum. It didn't have to be like that.

Microservices

grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too

seem very confusing to grug

>complexity very, very bad

>given choice between complexity or one on one against t-rex, grug take t-rex: at least grug see t-rex

> grug very like type systems make programming easier. for grug, type systems most value when grug hit dot on keyboard and list of things grug can do pop up magic. this 90% of value of type system or more to grug

Juniors at my job routinely ship code that breaks due to null access in production, Sentry tells me. During intensive development periods that's about 1 detected null-access bug per day per junior developer.

Using a proper type system with static checks would probably help immensely by pointing out "Hey, this can be null. You sure?" in their IDEs...

Also, you can have completion even without static typing.

> big brain type system shaman often say type correctness main point type system, but grug note some big brain type system shaman not often ship code. grug suppose code never shipped is correct, in some sense, but not really what grug mean when say correct

That's just rude and uncalled for.

I have shipped code mainly in in C, PHP, Python, Haskell and typed Python. The incidence of bugs that make it into production is much lower with typed languages. That's one reason to like it.

It also makes refactoring much, much easier. I can check whole code base for broken callers when I change something widely used and get reliable results in seconds. That helps immensely with iterating on a growing code base.

But is it wrong? Based on your stated experience, not even you seem to buy into type systems that force formal proofs – instead accepting lesser type systems that make tradeoffs between catching some problems (along with, most importantly, providing popup magic!) and not bogging you down in every little detail needed to prove total correctness.

Yes, because people don't "not often ship code in statically typed languages". It's obviously wrong.

Clearly type systems live on a spectrum with varying degrees of sacrifices made, with each sacrifice compromising some ability to check for correctness, but giving back some other advantage, improved developer productivity being one possibility.

It is not a question of static typing or not – it is how complicated of a type system do you really need? Even Grug agrees that dynamic typing is not sufficiently complicated for most circumstances, but maybe a primitive static type system somewhere in the middle of the spectrum that helps catch the most grievous of errors (and provides popup magic!) with a sprinkling of automated testing to fill in the gaps where the type system falls short is good enough?

It's just a joke. I'm almost certainly one of the people he's poking fun at and I don't think we should get bent out of shape. He obviously thinks that C or Java levels of static typing are fine/good, and thinks that fancier stuff like found in Rust and Scala is a waste of time. I think he's dead wrong, but that's life.

I think you're not the "shaman" he's referring to.

By "big brain type system shaman" I understood someone who doesn't necessarily write code, but instead, e.g. sells courses around the topic, getting people hyped about it, etc.

He uses the word "shaman" previously when talking about Agile, and there are many who fit that picture of working on selling courses, and not on developing software.

10 years later, and I've shifted more towards being a "Grug brain" developer. Now, I focus on the simplest solution that could possibly work, knowing that it's probably not perfect. But that's okay, because it gets me closer to what is correct, allowing for iteration.

The best thing you can do as a developer is to delete code! Right now, we have a requirement that we've been living with for two years that suddenly isn't a requirement anymore. I can't tell you how excited I am to go through and rip out a whole bunch of code, because it makes everything simpler.

It's like I'm FEELING the space being freed.

This is an infinite cycle with JS though. Someone tired of JS complexity writes a simple JS lib (SJSLib), SJSLib attracts people for simplicity, SJSLib grows complex because it has to support all the web things, someone tired of SJSLib complexity writes a simple JS lib...

I say this as someone who started with raw JS and fought IE5/6 for years, jQuery then saved us all, skipped over AngularJS because it was meh, React beta finally came around and loved it for FP ideals, now wade thru piles of transpiling / hot-reload / Typescript and React fat-libs. Have also written a few projects with both Intercooler (and now HTMX).

HTMX is overall solid, but this ain't the first wagon to come through town.

So the cycle spoken of would be broken at this sentence. “SJSLib grows complex because it has to support all the web things.”

I laughed at this, then I checked.

    _="on load wait 5s then transition opacity to 0 then remove me"

    _="on htmx:error(errorInfo) fetch /errors {method:'POST', body:{errorInfo:errorInfo} as JSON} "

If someone thinks it's ruined because they found out ahead of time that there are vampires in it... yeah, that's their problem.

I'd love to see some real critiques of htmx.

Personally I think the web's big ongoing challenge has been figuring out how to update the page well, and we keep trying all kinds of attempts. That causes fatigue, and some of the ideas are wild or grow crufty over time. Htmx seems like a back-reaction, to insist on grug brained less/YAGNI.

And even though the future is uncertain, I still see work on things like signals as saintly. There's a lot of different pokers in the fire, refining & trying different angles. We're still pioneering this front, other fronts.

The quest feels similar to so many others. MobX and Svelte and others have made great journeys already. That the journey is still irresolute, that we do keep tangling with the complexity beast: rather than that being a sign of weakness, as the conservative/grug minded might take it for, I think and I hope it's appropriate & due effort to reach a simple but good. Rather than a just simply simple approach.

- Writing Vanilla JS ES6+ is not that hard, actually it is pretty fun.

- Toolchain for JS/TS/CSS was bad but is pretty good now. esbuild is simple and great.

- Dodging learning the above two things does not serve the dev.

Others have to do with foundational aspects of the project:

- HTMX is in denial of need to support native clients. IIRC, the official recommendation is that native clients should parse HTML for values.

- HTMX is still just a javascript library dependency with syntax that must be learned by whoever picks up the project.

IMO HTMX's buzz can largely be attributed to anti-frontend or anti modern frontend.

This thread describes this idea: https://twitter.com/DanaWoodman/status/1682075711266496512

Native clients are deliberately out of scope, but scraping HTML isn't the suggested solution. Rather, it is having a JSON API next to the hypermedia API: https://htmx.org/essays/splitting-your-apis/. This does have the downside of duplicate work, but does bring stability to the API without introducing something like GraphQL.

And yes, you will need to learn HTMX to use HTMX. But the surface of HTMX is generally quite small, and composes well with knowledge about HTML/CSS.

i like vanilla es6 too. i've enjoyed using it with htmx, rather than using intercooler or alpine.

Say you’re writing a true web app. You’re making a mobile view, and the user has scrolled down and loaded some items (and partially inserted them as the scroll bar has moved down.) They add a card representing a few items to a cart, which 1. changes the look of the card and 2. the number that floats above the cart icon in the header.

Do you

1. Return the updated card and insert it as normal, then pass the HTML to figure out the number to change the cart icon with and call JS to change that icon?

2. Trigger a full page reload, meaning that both items update at the cost of losing the scroll position?

3. Break convention with HTMX and call a JSON API that will let you return the updates primitive values, and maintain the display logic changes in JS?

4. Have a weird hybrid JSON + HTML API that return multiple responses for each part of the DOM tree that needs changing and rely on some custom JS to do both updates?

If you need to modify the card and something else, probably using hx-swap-oob is the best approach to update the cart icon.

On some occasions it can be worth the lesson to let them try but it's a good thing to remember that the people who came before us weren't all complete idiots and there is generally a reason why they did what they did.

Sometimes it is gross old code that needs to be replaced, but even then it often still contains a hard fought record of all of the corners and edge cases you need to understand and handle to build anything in that domain.

The worst sort of tests are integration tests which secretly depend on another integration test having run first, which will be true 99% of the time, until a change you make changes the order.

Integration tests can give unclear signals when they are flaky, but when they are engineered well they will give a much clearer signal that things work when they pass and that something is broken when they fail.

It's harder to engineer a good integration test - this includes making them isolated and independent e.g. of test ordering or indeed, anything else.

That's an example of bad integration tests. Well engineered integrations tests don't do that.

If the system is crappy/stateful/implicit, and you somehow manage to write nice/clean/stateless integration tests against it, I'd argue that the tests won't be close enough to the expected running of the system to tell you anything useful about it.

If I set up an integration test that sets up a database from scratch and tears it down and tests only the behavior of the app in that rigidly defined context, then yes, it will be useful. It will tell you how the code behaves in the scenario you've created.

Bad integration tests will share state with each other - e.g. by using the staging DB.

> Bad integration tests will share state with each other

The real-world system shares state.

> If I set up an integration test that sets up a database from scratch and tears it down and tests only the behavior of the app in that rigidly defined context

... constructing a particular set of circumstances which will never occur in the real-world system.

What do you find unlikely about a scenario where a test uses an app in a realistic way (e.g. with a browser) set up in a realistic context (e.g. with some fixed sample data) to reproduce a realistic scenario (e.g. a bug that already happened)?

I wouldn't say that isolation and realism are completely orthogonal but I find that well engineered integration tests are usually able to reproduce 90% of bugs sourced from production while unit tests can often manage only 10 or 15%. Bug in the SQL? Browser is involved? No can do.

Neither would I. I'm arguing that when you write a test method, you deliberately make the choice to include some kind of 'before-all' method, or not.

The reasons you would choose to include a 'before-all' method will vary from case to case. Let's say you're testing an addUser method. If you choose to isolate its state to avoid 'test flakiness', it is you making the call that addUser is flaky when run against shared state.

What is it about your application code that would make you think that addUser is flaky enough to need a clean slate to run against? Why not change the application code instead?

Not really. I would always purge anything that would cause tests to share state. I wouldn't do it on a case by case basis.

>What is it about your application code that would make you think that addUser is flaky enough to need a clean slate to run against?

The user already existing in the database? The behavior of the app would change in that case. Something has to wipe the db clean to test that scenario.

Thats why tests shouldn't share databases.

If a team has simple code, then tests can help a lot. However, if a team does not have simple code (and usually they don't) then it's better to spend time simplifying the code than writing tests.

> best weapon against complexity spirit demon is magic word: "no"

> sad but true: learn "yes" then learn blame other grugs when fail, ideal career advice

Complex wisdom from grug.

The Grug Brained Developer - https://news.ycombinator.com/item?id=31840331 - June 2022 (374 comments)

I have not used Go, but what I have heard makes it seem like it is designed in a way that would encourage the second approach.

Enterprise projects build in single digit seconds, test suites fly by, project builds to a single binary (with embedded resources), most third party dependencies follow the established interfaces (which means plug and play) and so on.

It's the one language that I feel confident in, despite having worked in several other languages for many more years than Go.

https://youtu.be/k9Zbuuo51go

1. It’s not actually overcomplicated, but the people saying it is haven’t thought about it hard enough to realize this.

2. It is overcomplicated, but it’s such a tangle of complexity that fixing it would require the people pointing out the problems to basically do it over from scratch.

#2 is usually the result of a very experienced developer being overwhelmed by the amount of complexity coming out is the vastly larger amount of inexperienced developers around them. It’s much easier to add complexity than it is to fight it.

And this matches my own dev experience cathedrals of complexity pale into comparison that code that's easy to throw away and rewrite to meet changing requirements or scope.

iterative development with time to rewrite > cathedral development > "iterative" development without time to rewrite

I think that most teams actually do "iterative development without time to rewrite" so cathedrals of complexity would actually be an improvement.

https://www.backendbanter.fm/episodes/024-behind-htmx-carson...

Ah so this is the purpose of UML!

I believe there are some antipatterns like singletons and globals that on the surface look grug-brained but are actually complexity multipliers.

Rich Hickey gave his own practical definition of complexity and followed it in his design, and sometimes the results are very unintuitive. For example, transducers in Clojure are actually simple (by his definition) because they de-couple transformation from the context. Also, by his definition HTMX approach (aka PJAX, aka HTML-over-the-wire) would be more complex then JSON + client side rendering, because it couples together multiple things: network, routing and rendering.

now say others no listen and do opposite of what grug say for month after month. code complex. code very complex

what grug do now

> I feel the exact same about grug. I don't think people actually agree on what's simple, so it's pretentious to pretend your "simple" is the obvious one that a caveman would agree with.

Simple/simplicity is often one of the most complex things to discover. I see a lot of people resolute & certain that everything around them (except what they do) is complex & needs to be bent into simplicity. It feels dangerously weak & clutching after authority.

The grug brain philosophy is simplicity at all costs, unless absolutely unavoidable.

The big brain philosophy as grug sees it is reusability at all costs, unless absolutely unavoidable.

The issue with this philosophy is that it tends to lock in first generation design choices and makes iteration more difficult.

It’s true that simplicity is difficult to find, but iteration is the key to finding simplicity.

I agree.

A common approach to simple is “just start with the first thing that pops into your head and see how far you get.” I guess we could describe that as “simple to think of.”

A far less common approach to simple is “think through the whole problem, then remove everything that you don’t need.” This is what I’d call “a simple solution”, but note that it takes a lot more work to find it.

But I see plenty of anti-intectual anti-whatever attitudes founded in other forms of disdain. And I think grug reflects a broad/broader spectrum of negative biases.

One example, I've had an incredibly hard time getting folks to switch from dirty cobbled-together-with-StackOverflow shell scripts to Ansible, which is a just more sight-readable consistent experience. Or to zx or anything not just badly written shell scripts no one collaborates on.

People swear worse is better, are YAGNI (you ain't gonna need it) up the wazoo against 90% of everything. I think grug picks a lot of things to bash on, and a lot of devs do to.

I adore simplicity, but I find it challenging & long & arduous to hammer out of things. It's not fast to produce, or low thought. It's not made by resolutely sticking to lo-fi paths & mentalities.

We do need to be aware of the hazards at the other side, of ridiculous complexity & absurd/unnecessary systems engineering: yes! And I think grug offers some good almost-koans to reflect on, illuminates real hazards well. But I also think it's actively harmful & pernicious to put grug-brainedness on a pedestal, to go about actively disbelieving against any possibilities that might possibly be done simpler.

We seem to agree that simplicity takes iteration. Grug presents it as "oh I'm just dumb grug brain, I dunno" but in practice there is a lot of infighting in tech, and those who insist too loudly on aiming too high are not even IMHO as dangerous as those too loudly insisting on too low. We have to keep engaging possibility from all sides, & keep finding out what balances do work.

Ansible can be used quite elegantly/simply, but the ecosystem as a whole is totally infested with "big brains" who insist on complicating things to make them more Reusable™.

There are really guys out there who will argue that your 20 line shell script actually needs to be two Ansible Roles each consisting of 10 files spread out among 2 layers of directories, and can be published to the Galaxy.

(In reality you can turn that 20 line shell script into a 30 line idempotent, single-file Ansible playbook, which is SOMETIMES worth it)

Grug no make small problem big.

Grug humble.

I think it does a pretty good job of identifying if you're fighting your tech/language or doing things the "intended" idiomatic way. The difference is usually at least an order of magnitude in LOC.

https://youtu.be/v79fYnuVzdI?si=2iEdgEgx3Q-7RyI_

(Zathras Wrong Tool)

Anything below your chosen level of simplicity has no features.

Anything above is too complex.

You are at the true simplicity optimum. Your manager is the one who doesn't get it. Terrible guy. Understands nothing. Unlike you, true artist, pure simplicity.

grug speak only true

``` Intwoduction

this cowwection of thoughts on softwawe devewopment gathewed by gwug bwain devewopew

gwug bwain devewopew not so smawt, but gwug bwain devewopew pwogwam many wong yeaw and weawn some things awthough mostwy stiww confused

gwug bwain devewopew twy cowwect weawns into smaww, easiwy digestibwe and funny page, not onwy fow you, the young gwug, but awso fow him because as gwug bwain devewopew get owdew he fowget impowtant things, wike what had fow bweakfast or if put pants on

big bwained devewopews awe many, and some not expected to wike this, make souw face

THINK they awe big bwained devewopews many, many mowe, and mowe even definitewy pwobabwy maybe not wike this, many souw face (such is intewnet) ```