Today that type of concern would seem absurd, and we've gotten used to the idea that flying a plane and building it are separate skillsets and jobs. After all, we are comfortable with aviation engineers designing the aircraft without knowing how to mine the bauxite, smelt it to aluminium, and machine it into aircraft parts.

As comforting as it is to have an overview of how to trace a complete path from logic gate to web request, it isn't necessary. The decrease in the number of engineers with that overview isn't a cause for alarm, or a problem to be solved. It is just a sign that "tech" has matured, and it is stratifying into a set of interlocking disciplines. The same happened with every other subject in human history, and we've been just fine...

Take, for instance, the "Yes, let's all go back to coding in assembly!" line -- The thing is: For a really really long time after high-level languages had become mainstream, you really did still have to know assembly to be a programmer, even if you did most of your work in, say, C, or Pascal. That's because compilers for high-level languages and their debugging tools were initially a "leaky" abstraction. When your programme failed, you had to know assembly to figure out what went wrong and work your way back from that to what you could do in your high-level language to fix the problem. Nowadays, compilers and debugging tools have become so good, that those days are mostly gone, and you really don't need to know assembly any more (for most practical intents and purposes).

But the problem we have today: We pile on layer upon layer upon layer of leaky abstraction without ever giving it the time it needs to mature. We're designing for shortening the amount of time a developer spends on getting something done, under the completely misguided assumption that the developer will never leave the "happy path" where everything works as designed. This is neglecting the fact that a developer spends most of their time debugging the situations that don't work as designed. Usually, if you make the "happy path" time more productive with a side-effect of making the "unhappy path" time less productive, that amounts to a net-negative, and that's the big problem.

I really hate the state of modern software. We have so many layers of utterly unknowable abstractions that it isn't even possible to understand what your code actually ends up doing.

And that's how we ended up with Electron, which I think is the pinnacle of shitty software driven by the unsustainable paradigm of libraries on abstractions on libraries.

I wouldn't bet that most pilots know more about how the planes they fly work than software devs know about their computers. For one, most planes today rely heavily on computers. Do they teach electronics in "aviation"?

I just did not find it fair to say "pilots know how planes really work, but software engineers don't know how computers really work". Both are waaaay too complex for one person to actually understand fully.

I recommend reading “Flying Blind” for more detailed accounts of the precursor lion air flight that almost crashed.

Or are you saying that they knew, but somehow did not do it?

Not at all saying that they were incompetent. On the contrary, passenger planes today are IMO much more complex than one desktop computer loading a web page: passenger planes are a group of many computers doing safety-critical stuff in order to maintain a giant machine up in the air.

I don't see how one can say that software engineers don't really understand computers, but pilots do really understand planes.

As a JavaScript/Web/Fullstack developer I don’t live in that world. I live in a world of giant stupid frameworks. The only purpose of these frameworks is to supply an architecture in a box and put text on screen in a web browser. If a task cannot be performed using only the API provided by that framework then it must not be worth doing as it’s clearly far beyond the capabilities of the developer. There is far more to this software platform than merely putting text on screen in a web browser, for example: accessibility, security, performance, test automation, A/B testing, network messaging, architecture, content management, and so on.

God forbid you take the giant stupid frameworks away. It’s like castrating a person in public and then laughing at their great embarrassment. Many developers, some of whom shouldn’t be in this line of work to begin with, have built their entire careers around some framework API and absolutely cannot write code without it. The emotional insecurity is very real, as well as the completely inability to write original applications.

Building a gas turbine engine without training? Forget it.

The only electrical engineering a pilot needs to know is the difference between volts and amps, and what it means when a breaker pops. The EEs who design the avionics are not so fortunate.

You have probably 80% of people using React that have absolutely no idea how the three main functions in it's API work. That's the equivalent of a 747 captain having no idea what the TOGA button does and only knowing that that's the one you press to do a take off or a go around.

Yes, a cab driver does not need to understand automotive engineering because a cab driver is, in the non-pejorative, technical sense of the word, unskilled labor. Is that really the analogy you want to make though?

Software devs have zero control over the tooling, languages, networking and hardware.

It looks like they have a choice because the map of available options constantly shifts... but they're metaphysically locked to near-identical options in the universe of potential ways.

As long as computing is largely US based, it will always be this way. It's treason to go off-piste, in a large way.

I think that's a highly dismissive and ignorant view of what software development, as a value-creation endeavor, actually is.

The responsibility of a software engineer is not mapping high-level constructs to low-level details. The responsibility of a software development engineer is to implement systems that meets the business requirements, and operate on those systems at the abstraction level that makes sense to the problem domain.

It is entirely irrelevant what machine code is running, or even what machine is running the code, just like being able to model fluid flow over the control surfaces of an airplane is entirely irrelevant to steer the plane. A pilot needs to know how to control the plane using the plane's interfaces. Being able to whip out a computational fluid dynamics model is entirely irrelevant for a pilot if all they want to do is turn left/right.

High-level languages and abstraction layers are the key to simplify and speed up delivering value. No one should care about what pages of virtual memory their application is writing to if their goal is to serve a webpage in multiple continents.

This take is outright wrong. One of the most basic business requirements is turnaround time for features, bugfixes, and overall maintenance, which ultimately means minimize operational costs.

All production-ready application frameworks are designed to provide standardized application structures out-of-the-box that hide the implementation details that don't change and make it trivial to customize the parts that change more often. Backend frameworks are designed around allowing developers to implement request handlers, and front-end frameworks are designed around allowing developers to build custom UI elements from primitive components, provide views to present data, and fill in handlers to respond to user interactions. Developers adopt these frameworks because they don't have to waste time reinventing the wheel poorly and instead can focus on the parts of the project that add value.

What you describe is a training failure, but your thoughts on the matter are an economic failure. The goal of software is eventual cost reduction via automation. I say eventual because software is always a cost center and its value is not immediately realized.

What you describe is employment, which is not the same thing. The least friction path to employment to turn candidates into commodities to ease selection and risk of rejection post-selection. Once employed the candidates perceived value is often measured in things you describe, which rarely translates into any kind of value add to the business. Churn is burn, which increases employee engagement but almost always increases operational costs. The way to decrease operational costs is with automation, which includes things like CI/CD, static analysis, test automation, and so forth. These automation efforts are not measured in churn.

That contrast is why many software developers are API monkeys, because its what they are hired for and what they are rewarded for. That is why software developer return on investment is not defined by business requirements. Many employers need people to perform low effort work and do not wish to invest in formal training. This is all measurable.

Say more? What do you see as the trivial non-framework path (choose any example problem you like)?

Otherwise a valid example is this one file that creates a complete OS-like GUI in the browser awaiting content typically populated from WebSocket messaging: https://github.com/prettydiff/share-file-systems/blob/master...

I don't think you have a very good grasp on the issue.

All production-ready application frameworks are designed to provide standardized application structures out-of-the-box that hide the implementation details that don't change and make it trivial to customize the parts that change more often.

That's what they are used for: to ensure developers do not have to reinvent the wheel poorly, and to provide very flexible ways to change the things that are expected to change the most often.

Front-end frameworks are used to help develop user interfaces. Describing user interfaces as "put text on screen" already shows you have a very poor grasp on the subject and are oblivious to fundamental requirements.

Unwittingly, you're demonstrating one of the key aspects where frameworks create value: gather requirements and implement key features that meet them, so that people like you and me who are oblivious to them don't need to rearchitect their ad-hoc frameworks to support them as an afterthought.

It should be noted that those who try to make the same accusations you've made regarding complexity aren't really complaining about complexity. Instead, they are just manifesting that they are oblivious to key requirements and as they are oblivious to them then they believe they can just leave huge gaps in basic features without any consequence.

https://en.m.wikipedia.org/wiki/Virtue_signalling

These frameworks provide value to the employer, not the developers, because it eases candidate selection and turns otherwise unqualified developers into less capable commodities. In that regard the value is entirely regressive because it requires more of the less capable people to perform equivalent work that does not achieve disproportionate scale, which is the economic goal of automation. If the given developers only return value directly proportional to their manual efforts they are merely overpaid configuration experts on top of data entry.

Its not clear the author is demanding 'from first principle'. Second, not everybody in university needs the knowledge of flight or the ability to do it themselves - and there are alternatives to flight.

Finally, there is a upfront and physical limits to what can be done to the process of flying. Computing and the framework to think about them are endlessly malleable (and applicable).

If I take some of the BS being sold today and transplant them back into the aviation metaphor i can only describe it as a donkey ride being sold as flight. So many customer don't have the knowledge to tell the difference.

What resonates with me here (and why i don't like this comparison) is what i believe are the ingredients critical to progress. That is: Only when we've figured out how to make something simple can we build on those foundations and take huge leaps forward. There is a limit to how far or high we go when things get too complex. Similar to the idea of technical debt but at the scale of humanity.

Tragically we're very bad at recognizing those eureka moments in history when things became an order more simple because naturally we look a back and assume they're obvious.

Mentour Pilot video on the issue: https://invidious.protokolla.fi/watch?v=e5AGHEUxLME&t=1

fixing one leak and assuming there isn’t any other is not the right strategy, is my point.

It's good to have people that do know how computers operate and how we use them, but specialization is also good thing.

People assume that because their phone/laptop is locked down and unable to spark curiosity that it must be the same for kids today, but I think they just become a little more un-curious themselves.

Yeah it's built on some abstractions, but I do think there's some valleys among the tall abstraction mountains that the curious few venture into, just as it's always been.

Yes, c64 BASIC lets you POKE at any memory address you want, but I was getting a pitch about a complex 3D collect-a-thon with FPS elements, from a 10 year old. That's also kind of cool.

Yes some people are experts and some are just adequate. Nothing new here. Not everyone can be a messiah like the author, who spends half of this short essay telling us how great he is.

This article has almost nothing to do with abstraction. The author loses interest in his own thesis after the first two paragraphs.

I know the current mainstream state of the art is inefficient for sure because I've built an SDK which I use for my own projects and I'm at least 10x more productive with it than with mainstream frameworks I used during my day job and the code is way easier to read and maintain. I could show the code to a junior dev who doesn't know any frameworks and they will be productive with it. I wrote a no-code BaaS (Back end as a Service) platform in 2 months part time using my SDK. I highly doubt I or anyone else could have done this in 12 months full time using mainstream tools and frameworks.

Many experts in drywall installation before drywall screws were popularized swear that screws are slower and worse. However, an expert nailer and an expert screwer both complete jobs just as quickly (if not faster for the screw-adherent), and just as well.

I look at the current situation of UIs in Rust, every one of which is unappealing, and think I could build my own. The basic structure and paradigm is so simple! Then, I think... Unicode support. Right-to-left text display. Affordance for text readers.

All very important, but things I could ignore for myself, but not in a publicly shared library. And... ugh. Either I would have to spend years figuring that stuff out, or pull in a number of other libraries which adds layers of abstraction and complication. So I fire up Civ 6 instead.

Screws are superior as an end product. Old houses will tell you that with the amount of creaking and slop that builds up with nails.

But there are videos floating around about the expertise of some old nail drywall experts. Holy smokes it could be fast. They are probably right about the speed.

Companies are made of people who, hopefully, have people in decision making roles who have context and the knowledge to make the right decisions. There are reasons why we use "these frameworks" (be it frontend or backend). You may not like the reasons but corporate programming isn't just about language purity but what makes the company money, of which there are a lot of factors like hire-ability, maintainability, continuity, etc.

What happens when your company uses your bespoke SDK and you leave? You may think its easy to teach someone else how to use it but there's a lot more that goes into how companies make technology decisions.

To give you a rough sense:

- My SDK has some front end components and back end components connected via WebSockets using a client/server framework I wrote years ago and have been maintaining.

- It's all declarative so for example, for the back end, I don't write much code, I just declare the models, what fields they have, what views of the data are exposed and specific access controls. The back end is mostly a large JSON object. I don't want to go into too much details but the way it's set up, it's very flexible and you can model almost any kind of data and relationships with little to no code. It guides you towards a good architecture which makes good use of database indexing (so it performs and scales well). To connect the back end models and front end, it's a twist on the old CRUD concept so that it is conflict-free. I went down a completely different path to GraphQL and I think the result is simpler, more efficient, provides better access control, simpler caching and code (or should I say markup/JSON) is much easier to read and maintain. As you build your system using it, it guides you into making optimal architectural decisions at every step, keeping complexity as low as possible (as opposed to GraphQL which, by virtue of its extreme flexibility allows complexity to grow out of control).

- The front end components provide ways to render lists and objects in complex ways (e.g. grouped, filtered based on relationships between different models; all declarative). Components are hooked into the model backend in a particular way so they update in real time by default. Real time updates are delivered to the front end efficiently. Only relevant components/views update themselves and they do so automatically. Pagination is automatic and specified declaratively as part of the HTML and in accordance to limits specified in the back end JSON. Access control is enforced automatically based on the rules specified on the back end in the JSON object.

I'm almost at a point where I can build entire complex apps using only HTML on the front end and JSON on the back end with essentially no code.

The ideas and approaches you talk about evoked some of the concepts from that paper for me. It talks a lot about separating accidental complexity and infrastructure so you can focus only on what is essential to define your solutions.

I think you are missing the point of no-code. Your solution isn't even low-code, you just created a framework with well-thought components to start a project. If you need to integrate a third-party component that uses code, you'll need to integrate that code in your HTML/JSON or create a new abstraction in JS to use in the HTML.

disclaimer: I'm creating a "similar" platform and got curious by what would set your SDK apart from mainstream frameworks but I don't think what you described is anything different. For the view part, is it using something like htmx, react or pure JS?

HTML is markup, not code. JSON is object notation, not code. I used very little code to build my service. I think this type of highly flexible low-code is the right path to no-code.

I know from experience that people with zero tech knowledge can be taught HTML and CSS in a few weeks.

They most definitely are code. They are not programming languages, and writing them is not 'programming', but they are still code.

This resonates with me strongly. Everyone seems to know how to do things by rote memorization ("to do X, add this line to your config file") but nobody knows how to go off-script if you need to do something slightly different (not-quite-X or a variation on X). Worse, people will waste your time trying to steer the conversation away from not-quite-X back to X. (Insert the parable of the man searching for his lost wallet under a streetlight because that's where the lighting is best.)

The internet of 1995-2005 was built on mediocre solutions. It was a blast.

Deep-dive experts are not a dying breed, they're not a limited resource with secret knowledge from a time before abstractions. We have more of them now than ever before. They're not defined by having worked all the way up and down the stack, but by having the curiosity to look into layers that are not their own, because not only are we adding more abstraction layers on top, we're also changing some of the layers down the stack: NVMe, WebGPU, WebAssembly, QUIC, AVX512 !

But deep-dive experts are a luxury that most teams don't need, and one of the most important skills of a technical manager is, I would argue, to know when they absolutely must hire one.

However, a discussion over the misalignment of incentives in modern society can definitely be had. I would trust the pilot generally much more than a business software developer, researcher, or banker because the pilot will be the first to arrive in a crash.

There is a lot of asinine software in the world and that’s fine. Lots of “real problems” are pretty asinine and don’t require heroic engineering feats.

Just slapping some bullshit together is good enough in a frightening number of cases.

Hate it too myself, but I have come to accept I can either prove them wrong by building my own business and outcompeting them through my supposed superior engineering or just swallow that I’m yelling at the clouds.

A sizeable percentage of which is made without source control, unittests, and just slapped together like the rest you described

Hospitals don't check software quality when making purchasing decisions (much beyond "this button doesn't work")

This! The worst piece of low quality garbage I've seen from the inside is hospital software. Nobody really cares or understands software on the hospital side, so anything goes. The product is inherently complex, so there are only a small number of large companies who don't care either since they can rely on customer lock-in, no matter how crappy their service is.

"Fix IT: See and solve the problems of digital healthcare"

https://www.waterstones.com/book/fix-it/harold-thimbleby/978...

I guess something like certified engineering may alleviate some of these issues.

If we build bridges like this.. oh.

> Some students today apparently don't even know what files and folders are

…to a slashdot thread that links to a PC Gamer article that rephrases and links to a Verge article that links to Stack Exchange questions.

[1] https://medium.com/the-engineering-manager-guide/cargo-cult-...

And 2 of the people did not understand his questions and he never even tried to clarify or consider that the problem was with his question.

“I asked a “React.js expert” to compare different SPA approaches such as direct DOM manipulation, MVC driven client side templating, component-based DOM manipulation and compile-to-vanilla-JS“

I honestly don’t understand what he is asking either. What is compile to vanilla js? Is he talking about compiling typescript to js? And what does that have to do with the dom? Also what is “MVC driven component based dom manipulation”? Like I feel like I am dealing with someone who read some design patterns book and dings anyone who doesn’t use the exact same terms he does. Not someone who has a superior understanding of development.

In a nutshell, I think people who truly enjoy the satisfaction of doing a thing well are spending more time, generally, trying to truly understand things than those who just want to do their job and be done with it.

But what does that curiosity get us? I can't help but think of this fortune(6), and wonder how many non-curious people would even get it:

    A novice was trying to fix a broken Lisp machine by turning the power
    off and on.  Knight, seeing what the student was doing spoke sternly:
    "You can not fix a machine by just power-cycling it with no
    understanding of what is going wrong."  Knight turned the machine off
    and on.  The machine worked.

Is it? this forum is full of it in the form of blog posts and replies. And it is one of the things that makes it great. By the other hand, social media is also full of people boasting about their accomplishments but more often than not is just marketing.

Some people choose to interpret any challenge as a personal attack. ¯\_(ツ)_/¯

Imagine if Kubernetes was closed source & binary distribution only. Understanding abstractions is, I think, why being open source has become table stakes for infrastructure software.

* https://www.gnu.org/philosophy/free-sw.en.html

* https://www.gnu.org/philosophy/shouldbefree.en.html

Go is different of course. Your insight is very good. Not being able to dig into source code when needed is unimaginable.

That's not really true, obfuscating JavaScript can be quite effective at making it difficult to study the program's workings. It can't be perfect at this, of course, but neither is conventional ahead-of-time compilation to machine code.

My entire career so far i spent trying to get onto teams where others were doing the same as me but had more experience. In the years i spent on those teams i learned a lot. But as time has gone on a lot if those people have left into management which just isn’t the same as directly working with them on problems. I actually have also done the same recently.

When i look at almost anyone today they have no desire to understand what is going on with the tools they are using. Further more most of upper management pushes me to enable this by building more abstraction on top of abstraction they do not understand so when it breaks I or my team can fix the actual issue. Ensuring the developers have an easy out for anything that is outside knowing their framework and some basic syntax.

I always wonder if people felt the same way about me if they came from a background where they had to know assembly or other lower layers of the stack. At any rate even in the current environment I haven’t been worried about finding new work if needed. Their seems to be a very short supply of people anymore who know how things work under the hood or even have a desire to figure it out when it breaks.

Google -> you don't need a longterm memory, just Google it.

So with good LLMs (and the latest iteration of chatgpt is really good at a lot of things you don't want bore yourself with), you don't need to process logic and abstraction as it will do it for you.

This is not yet there for everyone but I think it will work the same. Lazy the mind and have less and less rigor where the abstractions get more abstract and also more shallow.

[1]: https://xkcd.com/1227

And I don’t think we will become mindless code monkeys: I think we will end up telling the computer what we need in english but us having 0 clue or memory on how it achieves that. That’s the ultimate ‘too abstract’ issue.

Rust uses algebraic datatypes with ML-style type condtructors and Hindley-Milner type inference to achieve, for example, types like Option> or Option or Option, so that you're forced to check for the case of a null pointer (None), and if it's not in an Option, then it's guaranteed not to be null. This was originally conceived of mathematically in the 1980s and then implemented in ML.

The Hindley Milner type system was devised in 1969 from the typed lambda calculus.

Golang was influenced a lot by Communicating Sequential Processes, a formal system for describing concurrent behavior in systems, first described in 1978 by Tony Hoare.

Graph theory is a field of math that comes up a lot in computer science, in marketing, networking, compilers (register allocation and functional programming language interpretation).

The only innovation I can think of that came purely from tinkering is Rust's borrow checking system, and they're still trying to formalize that while improving it.

Derivations from first principle are always important.

I am a pretty firm believer that antilock brakes are a bad abstraction that might cause fewer accidents, but often more dangerous accidents than they prevent.

They avoid a class of accident caused by the brake’s locking limiting your ability to steer. They cause a whole class of accidents where you hit things at a higher speed than you otherwise would have because your ability to actually slow down is greatly diminished. It’s a trade of braking distance for control. Basically we’re prioritizing rapidly swerving around an obstacle over less controlled but far more rapid deceleration.

I really don’t think this trade off makes any sort of sense in anything but the most sparse rural environment. In urban and suburban areas, swerving blindly around an obstacle will just mean hitting something else. Yes, you missed the car that pulled out in front of you but now you’re either throwing your vehicle into pedestrians on the sidewalk or into oncoming traffic. Both cases likely a far worse outcome than the accident you are taking evasive actions to prevent. The sanest option becomes just to hit the obstacle you would have been able to stop for were it not for antilock brakes.

In my eyes, the most fundamentally frustrating part, and what makes them a bad abstraction, is that the problems antilock brakes solve are entirely preventable by human intervention. Namely, pumping the brakes. The class of accidents antilock brakes cause are largely unavoidable. You can stop lessen their affect by not fully depressing the brake but it is still a much longer deceleration than no antilock brakes at all.

On gravel/snow, ABS perfoms worse. But 99% of the time you likely are not in such a context.

I was curious about your statement, so I looked it up:

> ABS increases stopping distances on surfaces covered with gravel, snow, or other loose materials. In such situations, a locked tire digs into the snow or gravel, pushing it forward and forming a wedge in front of the tire, which brings the vehicle to a stop

Source: https://itstillruns.com/do-brakes-work-ice-snow-6162289.html

[1] https://youtu.be/ERE9EtOWZMU?si=s35N-IzuyPLuEoKD

With anti-lock brakes, you're protected if you've never learned anything and you just try to put the brake pedal to the floor.

If you know how brakes work, you're worse off than if you don't have anti-lock brakes, since you can no longer properly control your brakes. In other words, we're punishing people who learn and know how to do things in order to ostensibly protect people who can't be bothered to learn.

Consider how many people drive with their headlights on, but no other lights. It's because automakers are selling "features", so it feels like we're actively encouraging people to think and pay attention less. Unfortunately, these "automatic" lights aren't truly automatic, and the value of having simple off and on states is lost because of these "features". It's actively unsafe.

That sounds nice on paper, but in reality you need training in order to overcome the instinct to smash the pedal through the floor... and you need regular practice to avoid reverting to instinct. Needless to say very few people can _actually_ take advantage of manually controlling the brakes.

I had to explain the problem with a single wire signalling bits in a series, recipient having to de-serialize them into some data structure. Then I had to explain that TCP emulates such a single wire using small packets.

I think that they have understood, but it was a funny feeling explaining this to someone who routinely deserializes form and JSON data, then serializes them into SQL queries, then deserializes query results in order to serialize them into templates or JSON.

The layers and layers of indirection obscured what HTTP is for me, and it took me too long to understand it.

Have a look at https://learnbchs.org/easy.html for a real "oh that's what HTTP is?" moment.

I've worked in software QA also and abstraction is the bane of debugging, especially when you can't even see the code in the libraries you're using. Proprietary binary blobs in embedded systems are the worst.

Priesthood. See: Foundation by Asimov.

Not all technology feeds into the cycle of tool bloat, we just need to get better at choosing our tools wisely instead of letting somebody's marketing department influence our decisions.

Talking about abstractions, during my past month, I was reading nand2tetris, and it's a compelling experience if you understand the exercise you are doing, which is not about building a computer from first principles; it's much more than that.

It makes you really understand what's going on behind the layers of abstracting that have been raised. Sometimes, understanding every layer of complexity is impossible, but depending on the area you are in, we need at least to try to understand the roots of it.

However, this is not for everyone; ask a musician if they are really in the weeds of why the instrument is producing music (the physics behind it!). They are probably aware that it's vibrating air, but, in general, they won't know the theory behind it.

Everything is built on abstractions, and that's OK, with time we will add even more on top of what we have; now, the issue is when those abstractions lock you in with a mindset that prevents you from creating something new without relying on those same abstractions that help you build stuff.

Many discoveries and inventions were made because they knew the layer of abstractions on top of it, and they just started again from scratch. Even Figma, the software, is built on a new core of concepts based on how the web was working back in the time [1]

[1] https://madebyevan.com/figma/introducing-vector-networks/

A big difference is that, unlike computing, their instrument probably won't stop working because of some subtle change to physics introduced by a seemingly-unrelated change made to the universe by some other party in the musical instrument / air / molecules / atoms / quarks "stack". Theirs is a world with some assurance of stability.

Ours is a field built upon shifting sands. Knowing what the foundations are that the edifice you've constructed sits upon allows you to affect repairs when it crumbles unexpectedly.

Finally get to use this bit of knowledge: 'effect' (the verb) was the word you wanted there.

Computers are a relatively new invention. There are still people around from the times when knowing the entire stack from top to bottom was not just valuable but necessary. They worked during periods when abstractions were far leakier than they are today and far less reliable but also far simpler.

Some of those people have persisted with the attitude that knowing the stack top to bottom is still necessary as the complexity of those stacks have grown in complexity geometrically. This does not signify wisdom, only age.

- a musician knowing how to fix their instrument vs a programmer knowing how to fix their keyboard/computer

Or

- a musician fixing some harmony, or progression vs a programmer fixing a bug in code

Otherwise it does not feel like a fair comparison.

- Knowing how to change strings is comparable to knowing how to use the browser console

- Knowing music theory is comparable to understanding programming language theory

- Being competent on the fretboard is comparable to being competent in one programming language. Strumming/fingerpicking could be considered another language.

---

And then for the controversial one...

- Only knowing how to use music-making software is like only knowing how to use low-code applications for development.

dodges tomatoes

Not a great metaphor. First, most instruments were invented by people who had no knowledge of the theory, not even of sound involving vibrating air. Second, many musicians do understand some of how their instrument works. It's not hard to understand how a guitar works, how to tune it, and how to fix a broken string. Most can't fix a broken body or neck, but they do understand why it stopped being playable. Hell, I knew a flutist who simply started to build her own recorders. Zero knowledge of physics, but after a few years producing great sounding period instruments.

Knowing the nitty-gritty of the acoustics theory and the related math may not help you much as a musician... (that being said, if you're building a home studio or doing any mixing/mastering, you'll find it helpful to learn about physics of standing waves etc).

However, the details of how frequencies come to form the well-tempered scale with all the tradeoffs and imperfections, and the low-level details of music theory, would be something many musician nerds would actually know.

Sometimes I'm learning a framework, and I'm in the part of "...you write this easy syntax and it outputs pure HTML!", which is great. But then the next line you read "also, when you use [obscure symbol], it does [obscure thing] in order to respect [obscure concept]". You have no idea what to do other than start googling those 3 new things, none of which are explained properly or at all in the current "super simple framework" documentation. I think even naming should make sense.

How is Clojure a layer of abstraction that had to be removed and replaced with Java? What do you think that Clojure was abstracting exactly?

Same question for Typescript. In fact, Typescript is a layer on top of Javascript so I could say that you actually added one more here.

But operationally yeah I agree, the stricter separation could make it easier to use a VM to get stuff done.

> Replaced docker with VMs

This doesn't make sense to me, as Docker and VMs are not the same thing. Replace LXC with VMs, sure.

It won't be better than docker, for sure, but if you're going to use VMs and stay sane, you'll need to reinvent it.

This is what happens when you turn Engineers into Devs. Software Engineering used to be viewed more as a profession but orgs have been chasing the holy grail of commoditizing software development. It’s not assembly line work and never will be.

What are examples of these places where people are not given the time to do their job right?

Every generation has some people more interested in the depth of their field than others. So why do we see fewer of the actual experts?

Well, we don't, we just see more of the surface level developers being able to contribute real economic value to society. They were just kept out by the more demanding requirements before.

If anything, it's a good sign we've come to have this luxury problem.

the new layers of abstraction have not made programmers dumber, they have added more programmers to the stack.

we still have and will have people designing kernels and CPUs for fun.

I am not personally familiar with any of those OSes (other than reading about the...bizarre TempleOS), but many of them have last release dates within the last 5 years.

Not to say that OP is incorrect. But intelligent people were opposed to abstraction already in ancient times. Socrates said about the invention of writing [1]

> For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them.

From today's perspective it seems absurd to oppose written information. The net benefit on society is clearly positive.

I am not saying that every abstraction is a net positive. But the example demonstrates that it is easy to oppose abstractions, even if they turn out to be positive in hindsight.

[1] https://www.historyofinformation.com/detail.php?id=3439

- how does it really work?

- what are its dependencies?

- what are its failure modes?

- what are its side effects?

- why are we doing it this way?

- how long will it keep working?

- are there any alternatives?

- if so, why aren't they in use?

- and what can we do about it?

This is particularly crucial with regard to materials and energy, as well as social norms, societal institutions, and cultural expectations.

Sound advice imho:

"If you understand the nature of the beast, you know what it's capable of."

Rarely -if ever- necessary to know all the gritty details. But it sure helps to understand the structure of what's underneath, or how it uses whatever is underneath that.

For example: I'm no mechanic who knows the outboard ICE on my boat like the inside of his pockets. But I do know it's a 2-cylinder, 4 stroke engine, how those cylinders sit in the engine block, how to unscrew the sparkplugs & check those, what the various other parts & hoses are for, and (roughly) how much fuel it should consume for trip x, y or z.

Imho that's about the level of understanding programmers should have about the tools of their trade.

We haven't used _too many_ abstractions. We've merely leveraged abstractions in a way that makes a different tradeoff. A small number of abstractions trades off deep expertise for narrow perspective. A larger number of abstractions allows us to trade off a wider perspective for a shallower understanding.

Neither is wrong. Both are useful. Different situations will call for each.

I sometimes feel like there are multiple specialties (for which different roles, and often different people are required) where someone only has to work for way less than full-time (maybe even as low as 5% of the time). When we hire specialized people, they do the needed 5% and then have to find additional work to do to fill in the time. So we get more complex tools that incremental improvements, we get more abstractions, we get more overall complexity.

Semi-related: I'd love to see a single video game that modernized the game experience as you played: Text mode only -> text mode + static VGA graphics -> 2D sidescroller -> Wolfenstein-quality -> DOOM-quality -> Quake-quality -> Half Life 2 quality -> modern-AAA.

But the problem today is that very few people, in tech not just the general public, understand the low level stuff, especially among new people entering the field.

Furthermore software abstractions tend to be far more leaky than abstractions in the physical world you mention. And when they leak you need someone who understands what's going on under the hood.

Actually I don't think this knowledge is most important for people forming their own businesses at all - they have many other problems to sort out. But I think it absolutely is valuable working for someone else. These people are the "goto guys" and, ultimately, a "technical insurance policy" for the companies they work for.

A company producing tech products without at least a few people on staff who can dig into and fix almost anything is in a difficult position ultimately.

Then those external dependencies and services are updated independently and the corresponding abstractions need updating.

Then external services are replaced and new abstractions are introduced.

Most technologies are never "finished" and instead they are updated cause a domino effect of downstream updates.

most people only chase the next thing you can build on top of what we have today. they only look back inside the layers when the current tech is not enough to achieve your goals out of the box.

a recent example i see is with the quantization and tinyml developments in machine learning. while it is easier than ever to create a model and to run it, the underlying architecture, previously only up to the people designing the frameworks, is now finally being looked at. only because the LLMs cannot fit inside the memory of elusive enterprise GPUs as easily as you'd like.

in no other instance would most people care about how numbers are stored in memory in the past 15-20 years in writing software. i think necessity is mother of invention, and that would probably still apply to dealing with abstractions going forward.

Kubernetes is great, but working with a tool that installs an in-cluster REST API that calls another in-cluster REST API that renders objects to be consumed by another in-cluster REST API that will also render objects to be consumed by multiple in-cluster REST APIs that will, eventually, produce containers that are externally accessible through a glorified NGINX config (with L4 filtering done by iptables, if you're lucky) can get extremely tiring.

(To be clear, this is still better than scripts that would call scripts that would write values to files/databases all over the place that are mutated by other scripts since Kubernetes does a really good job of enforcing interfaces between boundaries)

Programming is more like building with legos than building or using a car. So much of programming is composing units of work, even if you are starting at a low level of abstraction. And those units are very generic, very reusable, with little need for each unit to know about the goals of the end result until you get very high in the stack. In other words, we specifically avoid using specialized parts unless it becomes necessary.

I would wager that >90% of the useful, beautiful, functional software that I enjoy using every day was written by people who don’t know how to write assembly. Whether you measure it by lines of code, work hours, whatever. And that’s okay. They had time to add features and work on their business model rather than having another go at correctly loading their data into the CPU registers.

Having worked with a fair amount of devs who are very clever, but instead of having a sense of wonder and curiosity about the tech their systems rely on, they show a disdain for it. The kinds of fixes I’ve seen almost go into prod are scary, e.g. “the packaging system will not pack this 3rd party exe in with our code, so I’ll rename it to txt” - anything to avoid understanding the lower level mechanism and devise a sane solution. And management is thrilled because they deliver on schedule; engineers who protest the “working” approach are, at best, placated with a tech debt story in the backlog.

Your comment and many others here start their criticism by conflating builders with end users - does every taxi driver need to be a mechanic? This comes off as a bit of a strawman since the author is referring to building software, not simply consuming it. If the person who designed my car is just selecting prefab components on the strength of blog posts and industry hype, with weak knowledge of how they are built, I’d be worried.

it’s very likely that those skills are immaterial to their job. It’s also very likely that they will never need those skills.

All of computing is layering abstractions and no one - no one - understands all of them. The author cherry-picks their own favorite layers as being “what kids these days don’t understand” while ignoring their own ignorance of other layers.

One does not need to understand alternating current in order to plug in a vacuum.

The technology that underlies vacuums is a lot more mature and stable than those that in modern software development. So even if modern-day vacuum engineer doesn’t understand AC at the same depth as their predecessors, it’s less likely to be the Achilles heel that the author is referring to for modern devs having little understanding of computer technology.

You can be a professional in the field, but until you understand all the layers of abstraction, you aren't an expert. You can't diagnose those deep problems and fix them.

Professionals are paid to work in an area, and are thought to know enough not to be horribly dangerous.

Experts, on the other hand, are supposed to understand and have some competence in all the layers of complexity/abstraction present.

It can take decades to reach expert status in a given area.

A few weeks ago I had occasion to talk with a working computer security professional, and asked him about data diodes[1], and often they are used... he'd never heard of them. Often here on HN, I make comments about capability based security[2], and everyone mistakes it for the permissions flags on smartphones. This tells me there aren't many experts in the field of computer security.

The same is true in other fields, you can be a CNC machinist, but until you know about the Whitworth 3 plate method[3], you're not an expert.

[1] https://en.wikipedia.org/wiki/Unidirectional_network

[2] https://en.wikipedia.org/wiki/Capability-based_security

[3] https://www.wadeodesign.com/flatness-3-plate-method.html

Let's not inflate titles here. I've noticed what the author is writing about and will offer another example. Remember when the bad-guy hackers used to make hacking tools and exploits available for free? Before that was big business? Then a bunch of kids would leverage their hard work to cause trouble. Do you recall what those guys were called? Script kiddies. A lot of so-called professionals these day are little more than script kiddies. That's not to say they aren't effective or useful (the old hackers caused plenty of trouble) but they really don't know the internals of the tools they use. They can keep things going until something weird happens.

I'm not sure what I think of this state of affairs. Not everyone can go deep, but I do feel the bar has been lowered too far in many cases. It's like millions of small components... NPM: because nobody can be bothered to figure out some problem and write 50 lines of code themself.

Power steering, for example, simplifies steering thanks to a small mountain of abstractions built up over the years. Doubly so if you have adjustable power steering (i.e. you can select "sport" steering vs "comfort" steering). https://en.wikipedia.org/wiki/Power_steering

Abstraction layering is annoying, but I think abstracting hard things infinitum ad nauseam is, overall, a good thing!

Forces you to at least admit to yourself that you probably lack a lot of knowledge. Makes you humble and hopefully qurious.

And as soon as this is clear, the attitude follows. Should you know how every aspect of your code is compiled or interpreted? Not really. Should you realize that every automation takes resources, creates accidental knowledge, and introduces its own probability of failure? Yes.

It's true that a compiler is merely translating a high-level language into the actual machine code, but it's simultaneously true that you can (generally) talk sensibly about the high-level language, and maybe even prove theorems about it, without making any reference to how it will be realized on the machine. That's a good abstraction.

The better your abstractions, the more your automations make sense, and the more you can reason about them. Automation without abstraction gives you biology, which is a gnarly mess of accidental complexity that's very difficult to reason about and control.

That said, I'm going to start thinking of my code more in terms of creating efficient and low-risk automation, and less about creating nice abstractions.

You see, there is no such thing as a "good abstraction" or a "bad abstraction". In math, algebraic systems are either isomorphic or not. There is no qualitative quality to a fact. It is only in software, abstraction acquires this immeasurable quality of goodness, or leakyness, or even fashionability.

"Automation in disguise" is not a good term either. But I find it slightly less misleading and such as it brings less false promises in the connotations.

And I'm being nitpicky because the word "abstraction" brings in false connotations. It implies that you can reason about things on different levels as if they were equivalent. But in software, they never are. And I agree with the author in that this pretence of equivalency just doesn't hold and adding more and more abstractions on top of each other is not the way to go if you want to understand how things work and how to make them work predictably and efficiently.

I don't see how my claim diminishes the importance of naming.

> Computer science commonly presents levels (or, less commonly, layers) of abstraction, wherein each level represents a different model of the same information and processes, but with varying amounts of detail.

See: https://en.m.wikipedia.org/wiki/Abstraction_(computer_scienc...

If you remember that there is very specific very concrete automation behind every abstraction, you don't stack them up until the pile becomes impossible to either control or understand.

Oh, I couldn't agree more! And it is not just tools ans systems, it is everything from power generation to infrastructure to society and economics.

When start to work like that, the DevSecOps example used in the article but the same happens basically everywhere else too, it spills over into everything else. And yes, this is dangerous in deed.

The security issues expand beyond what is described in the article: you assume the security is solved by a stack of layers where is not. A single issue impacts everything doesn't matter where in the stack it is.

Someone else pointed out surface level knowledge of power tool users... but thats always been the IT industry.

IMHO. You either have "this" ability or you don't. There will always be a few people who can think and design complex systems and then everyone else who thinks its easy but only understands it surface level.

People increasingly just know about using third-party technology instead of building their own.

One approach to fix this is to ban usage of third-party technology entirely.

Personally I think cloud is a pretty big abstraction layer, I prefer to work on non-cloud and hate proprietary terms and disconnected tooling for common problems.

To understand a solution admins need to understand how it works via code and config access.

Having many abstractions on top of each other leading to inefficiencies is a problem, but that is not a problem of the number of abstractions, but rather the poor composition.

Honestly, I liked it. The font defaulted to Consolas on my Windows machine and it was both readable and not too distracting, oddly enjoyable.

Nice to know I'm not imagining this. Platform Engineers seem to be going extinct, with the Software Engineer taking over the role, and doing it badly.

It makes me wonder if we are a hitting a pyramid depth in tech that exceeds a practical limit for the human mind and life-span.

One solution is to get a better mind. Interesting coincidence that there are few in the works...

(A technician discovers how to do simple arithmetical calculations in a world where no remembers arithmetic since its all automated away.)

More information regarding the short story on its Wikipedia page: https://en.wikipedia.org/wiki/The_Feeling_of_Power

The short story itself can be found online.

This is sometimes a misalignment with organizations, though other times a company is playing games (e.g., priority is to appear externally as having growth or progress).

The abstraction becomes real.

This example sounds like a bad choice of framework, or insufficiently skilled devs.

I don't think the problem is that we have too many abstractions. Abstractions are useful, they allow people to focus on where they are different, avoid wasteful duplication of effort (when done right), and compensate for the non omniscence of everyone. They come with a cost, you need to know it.

There's a clear tendency to just buy into new trends, anf that's always been the case. Maybe the smaller scale of the industry back then made trends smaller and the choice more limited.

I think there's clearly an issue with the lack of interest to understand what's under the abstractions. Is it training, habit, culture, a change of who is a developer today? Don't know.

It was also much harder to be a dev 15 or 20y ago without having to know at least some C and some system. Stuff seemed more brittle as well so you had to fiddle. Deployment was mostly manual and artisanal so you had to copy files over manually, run commands, shit like that. Honestly the piece of mind and safety that frequent deliveries and "devops" brought is so good that I'd find it comical if anyone would suggest to me to go back.

So I don't think there's really too many abstractions. I think that sometimes they're the wrong choices, and sometimes people don't care enough about what's under their immediate interface

https://sites.google.com/site/steveyegge2/practical-magic

Meh, zoomers didn’t invent incompetence and slow, ugly software. Not everyone is cut out to write aerospace grade software and, thankfully for those people, not everyone needs it.

https://youtu.be/ZSRHeXYDLko?si=zR7Lq2jSxexWUCra

Awesome particle.

the individuals deemed to be "experts" rely on closed source software to the point that the software is the real expert

The anecdote about these kind of security person is interesting, and it's not hard to sympatize with him, but he is missing the point: The industry seems to need someone to run these kind of pre-made security tools. These jobs are not pointless (perhaps they would be if the people who actually "know" other abstraction layers didn't create software that suck), they are solving some problem, people are working full time and getting paid for them. And the fact that they exist does not mean these jobs make sense or that the tasks they focus on are the right or the wrong abstraction.

> What good does an abstraction do when it breaks and nobody any longer understands how the technology under the hood works?

Not many programmers know of a kernel works internally (processes are just another abstraction). Not many know how compilers work and translate high level code to machine instructions either (there is probably no person in the world who understand all the parts of LLVM/GCC). The amount of programmers who know how CPU instructions translate to transistors is very, very rare.

Yet all these abstractions sort of work. People argued back in the day against programming in high level languages, nobody cares about these people, because the kind of problems that can be solved with high-level programming languages can't really be solved with assembly. Abstractions don't appear because companies are stupid, people are trying to solve problems with software and they need to get some concrete task done. Doing the quick hack does not mean that they are doing something wrong, it means that they are focusing into doing something right at another level. And if you can't understand that, it's _your_ fault.

Of course, plenty of times companies are doing stupid things, but that's the nature of the problem, companies try to do different things, some of them succeed, some don't, some succeed despite being horrible and some fail despite being brilliant on paper. So abstractions are created all the time, and there is a continuous dialectic between that abstraction and its usefulness, which is not measured by the opinion of other programmers, but by the success of the companies adopting and following these trends. For some people who knows a lot about systems programming and administration, it may feel stupid that these days we have people with cloud certificates who are in charge of "orchestrating" scalable and fault-tolerant platforms in the cloud, but know very little about how Linux systems work underneath. But it turns out that these people can get things working, even if they don't do it as well as you would do, and that's something that matters - it means that the abstraction sort of works, even if it leaks some times.

I guess it's not easy to spend decades learning things only to wake up one day and realise that large parts of your knowledge has been abstracted out and automated (ie. made less relevant, and thus less valuable in the job markets). But that's how things are in this field...