Those bolts being loose (and they are BIG bolts) would mean multiple people in the installation process didn’t do their jobs, and signed their life on the line saying they did.

When I did maintenance, there was someone (QA) there to witness every torqued bolt, inspect every safety wire and installed part.

There is something rotten in Boeing.

The same management that drove it to ground?

The fish rots from the head. And these constant problems sure do sound like a new company culture of cutting corners instead engineering first.

Rushing to judgement merely obscures the true responsibilities.

I've heard this since they killed hundreds of people in the two crashes. Why are they being protected? They have names.

The NTSB is staffed with people familiar with the industry - it has to be. Thus, a significant portion of these people are former Boeing employees. And historically, many NTSB staff have gone to work for Boeing (and other industry giants) after their stint at the NTSB as well. It's not that large of an industry after all. This is known as the Revolving Door.

Regulatory capture

It's more a question of culture (oversimplifying, sales vs engineering) and this is harder to change most of the time. Apparently, even the Max debacle was not enough.

> But the nearest Boeing commercial-airplane assembly facility would be 1,700 miles away. The isolation was deliberate. “When the headquarters is located in proximity to a principal business—as ours was in Seattle—the corporate center is inevitably drawn into day-to-day business operations,” Condit explained at the time.

Oh man, wouldn't want that to happen.

> With ethics now front and center, Condit was forced out and replaced with Stonecipher, who promptly affirmed: “When people say I changed the culture of Boeing, that was the intent, so that it’s run like a business rather than a great engineering firm.”

Indeed.

https://www.imdb.com/title/tt11893274/

That was a funny one. They replaced him with his chairman... hence, more of the same.

  > that was initially a regional airplane but is now being used for transatlantic flights.

Alternative is wrong bolts, or sabotage.

But more possible - one lazy QA ghosting.

In my experience, no one puts their job on the line over silly reasons like this, unless there is intense pressure (unrealistic deadlines, heavy workload and poor working conditions) that makes mistakes like this inevitable. I wouldn't be surprised if an honest independent review of either company found ridiculous cost-cutting measures and/or emotional overload.

Or did anyone decided to cut corners by using an old batch/skipping degreasing/not putting glue because they were late on delivery?

Though it should be!

This thing is obviously not just an interior part, look at the meat in those castings, and it’s obviously safety critical, look at the cotter pins on other bolts. Sounds like it was going to be installed behind interior paneling and not inspected every day. For something like that, every important bolt should be secured by secondary methods, torqued and witnessed installed correctly. This looks like a failure in engineering (not having wire on this bolts), then a profound failure in assembly with multiple people not doing their jobs (not torquing, not witnessing, faking logs), risking the lives of passengers.

If this happened at cruising altitude and speed, people would have died. I can’t find the flight number but I believe 9 people died when a jet lost cabin pressure and a piece of the plane while decompressing during cruising altitude over water.

With a seperate follow through by another party to check the work.

That's SOP for US | AU | UK | EU military air mechanical crews.

Unfortunately, Boeing did not know they had other issues with the plug door bolts.

Especially since most shops have pretty much tossed professional career QA out the window.

Software QA provides nothing of value to software development; having it as a dedicated function works against the overtly stated goals of the function and counterintuitively acts to degrade quality within software by mandating strict top down process and brittle end-to-end testing.

Although Software QA is intended to be an independent verification body that provides engineering organizations with tools and resources, in practice they function as a moral crumple zone [1] within the complex socio-technical defense industrial system, being one of the groups that the finger will be pointed to when something goes wrong and absorb shock to the business in the event of a failure. As a result they have a strong incentive to highly systematize their work with specific process steps, to shield them from liability, which can be applied generically to all projects.

Good software teams build quality into projects by introducing continuous integration, unit testing, creating feedback, and tightening these feedback loops. This acts to find problems quickly and resolve them quickly. Software QAs need for high level, top down, generic systemization requires them to work against these principles in practice. Bespoke project specific checks, such as unit testing, is not viewed as contributing to the final product and is discouraged by leadership who see it as waste.

To give an example of how these dynamics destroy quality in software. I once found a bug in software on a piece of test equipment where a logarithmic search function was not operating on a strictly sorted list. When I pointed this out to my leadership I was told that if we changed any part of code, it would require a new FQT, which would be too expensive to conduct and was not in the budget. Although the bug would have been trivial to solve, and was clearly wrong and would not provide any benefits by remaining in the test equipment software, the process required for changes prevented solving the issue.

[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757236

I don’t work in this industry, but this seems fairly ridiculous on its face: software is not at all like manufacturing.

In manufacturing, there’s a design and a manufacturing process, and a critical function of QA is ensuring that the manufactured produce is manufactured to spec.

With software, the software is written, compiled, and then repeatedly copied. And something should verify that it’s copied correctly, but this is straightforward and boring.

So software QA ought to be much more like the kind of validation that happens when designing hardware, not like the kind of testing and validation that happens as products are manufactured.

You need some level of specification so you know what you’re building, but you have to keep in mind that the final code defines what the behavior truly is. Sometimes, that behavior unintentionally becomes part of the specification because users begin to rely on it.

I do like the fact that you both used hyperbole to succinctly illustrate the dangers of veering too far in either direction though :)

It's asymptotic. By the time you reach a human who is as dumb as an actual computer, the specification _is_ the code.

I think that's why people always tell each other to not take things at face value.

Of course there is a big difference between sw and hw QA, in the thing that they test, and how they test them.

But they are also very similar. Any QA department has to think about ways that things can go wrong, and what things to test for, how to test, which testing methods, which standards to handle, keeping certifications, etc. During testing you also need to keep reevaluating if you actually are catching each problem/bug and how to implement changes in your company that decreases the amount of problems or increase the amount that you catch.

I think in that way there's a lot of overlap in thinking about business processes and how to identify problems with them.

Of course once a specific binary gets tested and approved by QA it shouldn't matter if it gets copied or whatever as long as you make sure its the same binary (by a checksum for example).

But still making sure that errors don't reach the customer, is vital in any QA. If errors does happen, QA is the department that can make sure that it doesn't happen again. And ofc be able to proof in court that you did your due diligence if something does happen.

"Nah, I don't need to slouch anymore in this moral crumple zone!"

"We need this new feature in our program!"

"If we implement this it means the management fell in a moral crumple zone"

"What seat would you have to have for your flight?"

"Anywhere, but not in the moral crumple zone, please"

Them: “Please review this design.”

Me: “Ok, sure, when do you plan to start coding?”

Them: “Oh, it’s already in beta.”

Me: “So you can’t do anything with my feedback, but you’ll say I reviewed it?”

Them: “Well…”

Me: “You’re putting me in a moral crumple zone here!”

Suppliers (to include software) were expected to manage the quality of the product they provided; the purchaser would focus on how they managed the process, not in the compliance of every part.

This had a chance until software process was tossed in the name of "agile".

Anyhow, the fix for this was created and written. But we never got to put it into production. The reason: the company didn't have a lab test facility that could put a sufficient load on the software to prove it. Even though we were getting field failures because of this issue that were getting a bad rep, we couldn't fix it because even though the old code was known to be buggy, we couldn't prove the new code. So the process said we couldn't ship it.

"Hey Bob I know you're a competent engineer, but don't worry about specifying a certain type of bolt or loctite, the untrained assembly personnel will figure it out. I'm sure they won't let 200 people die in a plane crash."

Agreed, for good software teams.

I would content that most software teams at most companies are not good.

Which is to ask, with an average to bad software team is it better to have integrated or separate QA?

Ideally we’d always have good software teams, but in the real world sometimes you have to build software with bad teams.

Leaders have options, they can do things like reduce scope, increase budget, increase schedule, or full on abandon or cancel the project. These are all options available to leaders, but they require tradeoffs and decisions to be made on a project by project basis.

It is scalable to have a strict process that everyone has to follow, then impose a watchdog to enforce it on a wide scale. It may not be better to have separate QA, but it is easier for those in charge.

If I'm trying to build something with undertrained, demoralized, underpaid engineers... it's not optimal to use methods intended for self-motivated, high-performance teams.

And nothing says there must be company-wide mandates. Maybe this area gets a formal, independent QA team, but this other area doesn't.

My experience just doesn't bear out that collapsing the QA function into development always leads to better outcomes.

I've seen the opposite happen too often, and QA be the sole bulwark between idiocy and customers.

Here's a real, perhaps unexpected counterpoint. Say you have a good software team. How do they build good software with bad management?

Consider the classic statistic "most drivers think they are above average".

I posit that the same is true of software teams, almost every team will self-assess as above average, i.e. good. Those teams will then imagine that, being good, they build quality into the process and very little verification QA is done.

I have worked as a software consultant for 15 years now. I've worked with at least 40 separate software teams in that time. Every single team manager would pep talk with "this is the best team I've ever seen". Some of this is obviously blowing smoke to get people to work harder and feel good. But over the years I've had candid conversations with managers and realized that most of the time the genuinely think their team is really good, truly top 10-20%.

Here's the rub. Being a consultant, I'm almost always brought in by higher level management because something is going horribly wrong. The team can't deliver quickly. The software they deliver is bug ridden. They routinely deliver the wrong software (i.e. incorrect interpretation of requirements.)

Often times these problems are not only the fault of the development team, management has issues too. But in every single case, the development team is in dire straits. They have continuous integration sure, and unit tests, and nightly builds, and lots of green check marks. But the unit tests test that the test works. The stress tests have no reality based basis for expected load. The continuous integration system builds software but it can't be deployed in that form for x, y & z reasons, so production has a special build system, etc...

In 15 years I have never once encountered a team that would not benefit from a QA team doing boring, old school, black box manual testing. And the teams that most adamantly refuse to accept that reality are precisely those that think they are really top tier because they have 90+% unit test coverage, use agile and do nightly builds.

So, my question is, do you (I don't mean the specific "you" here, rather everyone should ask themselves this, all the time) think that most bad software teams know they are bad? Including the one you are part of? Would it really hurt to have some ye olde QA, just in case, you know, you are actually just average? :)

Also, did you run across any orgs where they basically refused to use a process like Agile, and instead just did ad-hoc coding, insisting that this was the best way since it worked just fine for them back when they were a 5-person startup?

You usually had a few personality archetypes:

- The most technical dev on the team, always with a chip on their shoulder and serious personality issues, who had decided to settle for this job for (reasons)

- The vastly undertrained dev who was trying to keep up with the rest of the team, but would eventually be found out and tossed, usually to blame for a major issue

- The earnest and surprisingly competent meek dev, who presumably didn't have enough confidence to apply to a better job, but easily could have made it on merit, work ethic, and skill

- The over-confident dev who read a bit of SDLC practice, and could see every tree while missing the forest

The key is that, aside from the incompetent person, they had all always been working there for awhile. Consequently, there wasn't good or bad health and quality: there was just "the system" (at that company) and dealing with it.

And none of these folks ever worked at 5-person startups. ;) I think it was definitely more an issue of SDLC "unknown unknowns" they should be doing, than willful decisions not to.

Yes, generally I join teams and work as an engineer or sometimes as a team lead, so I'm talking to all the team members.

Most start up teams are composed of junior developers, often pretty smart people. Usually 5 or fewer years of experience. Many times these are people who have already accomplished stuff they didn't think they could do. So that generally means that yes they think pretty highly of themselves. To a degree it is quite justifiable, they tend to be very accomplished but in a narrow domain. Unfortunately they don't realize that their technical accomplishments in a specific field does not mean that they are experts everywhere. Their managers understand that these are smart people and assume again that this is therefore a good team.

Non start ups that I join are usually just plain dysfunctional.

> Also, did you run across any orgs where they basically refused to use a process like Agile, and instead just did ad-hoc coding, insisting that this was the best way since it worked just fine for them back when they were a 5-person startup?

Usually more the opposite. In my experience I come across teams that are sure they must not need any help because they follow all the rules in Scrum and have great code coverage metrics.

It is really common to see this kind of thing. I call it "the proxy endpoint fallacy". It can crop up anywhere that there is something that can be measured. In that example, it would be confusing adherence to Scrum with having a working SDLC or perhaps confusing code coverage metrics with the objective of having bug-free releases.

This isn't a software only fallacy. In politics, GDP is often confused with societal well-being. Always be wary of your metrics and change them as required to keep you tracking your actual goals.

> To give an example of how these dynamics destroy quality in software. I once found a bug in software on a piece of test equipment where a logarithmic search function was not operating on a strictly sorted list. When I pointed this out to my leadership I was told that if we changed any part of code, it would require a new FQT, which would be too expensive to conduct and was not in the budget. Although the bug would have been trivial to solve, and was clearly wrong and would not provide any benefits by remaining in the test equipment software, the process required for changes prevented solving the issue.

I've seen this happen where it was a bad thing, but also where it was a good thing.

It's all about risk.

What risk does the software defect pose to the mission? What risk is inherent in making any change to the software? Noting that even trivial changes can be fat-fingered and thus are a source of risk. I've seen it go wrong this way: a seemingly trivial change was made, but the developer accidentally checked an extra file into source control, causing a further defect.

And then: what is the cost of mitigating these risks? Maybe the software defect is as trivial as its fix. Maybe an acceptable fix would be to write up a workaround in the documentation.

I don't think it's always wrong to say no to fixing issues. I also don't think it's always right that a separate QA department contributes nothing to the organization, even if they act as a handbrake on the software developers (sometimes, precisely because they do that). Human factors are real.

You think that QA is a liability shield, but that is only a side effect of the work that they actually do.

The task of QA is exactly that: an entity that tries to assure that the quality is up to some standard. Even in favourable conditions mistakes happen, so how do you make sure as a company that not 1 in every 100 product are faulty and tarnishes the good reputation that your company has spent so much time and money on to build? You hire a QA to make sure problems get caught before delivery.

But if all humans make mistakes, and QA is human, how do you make sure that the QA doesn't make a mistake? A never ending chain of QAs expecting each other?

No of course not. One thing that helps with reducing errors is to have a rigid protocol that is followed to the letter everytime. Pilots, for example, have a preflight checklist that they have to run every time they operate the plane.

The rigid protocol of QA teams is therefore an essential part of their jobs.

Although from your standpoint as a developer it might seem strange that QA is 'preventing' you from fixing a bug, it is actually very reasonable.

Especially since you work in the defence industry, I hope you understand that it is very important that the software that operates radars, planes, missiles, bombs, etc is working exactly as expected. Understandably there is a great deal of effort made to assure that when those things are needed they work exactly to spec.

So in your example it is probably very reasonable that any change you make needs to go through some rigorous process. The fact that it 'only' was about test equipment, doesn't matter because test equipment is just as, if not more important as the stuff it tests.

The reason why QA has the side-effect of being a 'liability shield' is that it gives companies the ability to argue (and proof) after the fact that the company did their due diligence in making sure that the product was to spec.

Especially certification is basically to get an external organisation to approve your QA. In that case if you get sued you can rightfully claim that you did everything that was legally asked of you, and if there is blame, then it is the certifying company using insufficient standards.

No. Good software teams are led by competent, technical management. Managers who aren't afraid to get down into the dirty details. Managers who aren't afraid to roll up their sleeves and write code if they need to.

The process doesn't matter. The management of what is or is not important does. Agile is just one process out of many.

Imagine an accounting team led by someone who never did accounting in their life: "Just make the numbers work out! I don't care how you do it! My bonus is at stake!"

For some reason, developers seem remarkably blind to the skills other roles and disciplines require. Only a developer can do that, everyone else is basically useless fluff. Maybe it's a form of arrogance or just deep unself-awareness.

Although I don't deny it can help to have the background, it is not necessary to be a good manager of something. Also seen plenty of good techies promoted to management and failing badly.

I stand by what I said, although my experience is in the technical domain, not finance or law. Maybe those departments are different, I don't know.

I guess if you have a manager who is making technical decisions, they are really a hybrid manager/contributor role. Maybe that works better in smaller organisations.

He has no experience to lead the team in high pressure situations. Like production being down.

He can't truly have a first person understanding of the work of the people who he manages. He has to rely upon others to tell him who's good and who's bad. That sets up a pecking order.

He can't help or mentor engineers with design decisions, or provide a historical context.

He doesn't understand the technology so there's an immediate communication and knowledge barrier that has to be overcome between him and his directs.

He doesn't feel the pain of a bad decision, because he's not coding it, and he can't emphasize with them since he doesn't code.

He tends to push feature development without fixing technical debt. Again that's pain he personally doesn't feel.

Simply not true however that a good manager can't lead the team in a high pressure situation. I'd say that exactly what a good manager could do well. Obviously they won't be making overtly technical decisions, that's what you are for. They can make business decisions, provide cover, get resources, communicate to other stakeholders... All the bits that need doing but would be a huge hassle for the techies who are trying to fix the issue.

The head of accounting should be an accountant, not a surgeon.

And even at the executive level of a hospital, you would want people who have spent their careers in healthcare, rather than, say, architecture.

...or perhaps with no managers at all. I'm less and less convinced of the importance of management in engineering except to give investors an illusion of control.

Let's say you have a bunch of school children and architects create a skyscraper. I've given both groups the process to design a skyscraper.

So in both cases, I should end up with a safe building?

This is driven by Pournelle's iron law of bureaucracy [1], which says that people who promote the bureaucracy rather than the mission of the bureaucracy will get promoted within the organization and come to dominate its decision making.

For example, in schools, administrators make more money than teachers. This is despite both groups having similar levels of education and intelligence. The reason for this is that administrators know the laws and regulations of the environment they’re working in and ensure the continuity of the organization. Despite not directly contributing to the organization’s stated mission of education, they are in charge of the organization and take more benefits from it.

Software QA has similar dynamics. A QA department may start out making good faith contributions to the organization. Eventually there are product failures, eventually leadership needs a scapegoat to show they’re doing something, and eventually QA takes the blame. People get moved, demoted, or fired. QA realizes its risk, and takes steps to mitigate it. They create a highly systematized workflow and process, adopt or introduce standards. Then assert that following process equates to good outcomes. When bad outcomes occur, they point to their strict adherence to following process as evidence of innocence.

If the process does not support the work or mission, that is a cost they are happy to impose on other functions to deal with. This is the final state until a system disruption happens.

[1] https://en.m.wikipedia.org/wiki/Jerry_Pournelle

The case I'm talking about does not have a separate QA department, but QA people as part of every software team. If a product fails, that team is responsible, so software devs are in the same boat as QA. They focus on learning from these failures, so no scapegoat is needed. Process does get followed, but not as a defense mechanism, but because not doing so introduces noise that is an obstacle to improvement. In case of bad outcomes, people do point out that they followed process because then it is clear that the process is involved in the failure and should be improved.

Unfortunately, companies with that kind of culture are rare.

An equivalent software QA to building planes would be to verify a known process with existing tooling.

I’m regularly critical of Boeing Defense (particularly space contracts where I’m a huge Boeing skeptic), but I think people are pretty off base if they think Boeing is just completely incompetent.

Airliner safety is insanely good. Just vast seas of competence, but when there’s a super rare failure, the incorrect impression people get is that Boeing (or Airbus) is just full of incompetency. Almost nothing that humans do is held to the same standard. Not spaceflight, not software, not healthcare, and certainly not automotive.

Flying a 737 Max with a bad door and without the fix to the angle of attack sensor is probably still better per mile than driving. In spite of going at 10 times the speed and miles above the Earth.

You can almost argue it’s held to a higher standard than it should, slowing development of cleaner aviation (and therefore killing more people in the future due to tertiary effects of climate change, etc).

It kind of annoys me when comment sections are filled with people talking about how incompetent Boeing is. It feels like out of shape slobs on their La-Z-boy chairs talking about how incompetent or slow some professional sports players are. Like, airliner safety is just a totally different league than almost anyone else plays in. On the worst day, their better than almost anyone else is on their best.

Because I dug it up for another comment, commercial carriers operating under Part 121 (roughly: scheduled passenger and cargo operation) had 4 fatal incidents in the last 10 years. [0]

Totalling 6 deaths.

In 10 years of US commercial carrier aviation.

One of those was literally 'the engine exploded and threw part of the turbine into the cabin (and also shredded some of the wing)'!!

Which resulted in 1 person dying and a successful landing.

[0] https://news.ycombinator.com/item?id=38921664

The two MAX 8s that fell from the sky were 100% Boeing's fault and could have happened in the US. If 5% of airline traffic is in the US you can renormalize those hundreds of dead and you get dozens dead.

I can't find specific numbers but estimates say about one in three has a military background. That's an awful lot.

No matter how good they are and how prescient, that doesn't help them if the aircraft computer decides it's stalling, forces a nose down and they cant fight the controls.

But, even if we assume omnipotence from these American pilot gods, and assume they can fly outside the bird and Superman-style catch it, they are still only 30% of American pilots. Just another population to normalize out.

But that's not what happened. According to every report, it is possible to take back control, it's just very much not intuitive and the situation was confusing.

According to the Seattle Times

> However on both accident flights, the angle-of-attack sensor failure set off multiple alerts causing distraction and confusion from the moment of takeoff, even before MCAS kicked in.

> On the Ethiopian Airlines flight, for example, a “stick shaker” noisily vibrated the pilot’s control column throughout the flight, warning the plane was in danger of a stall, which it wasn’t; a computerized voice repeating a loud “Don’t sink!” warned that the jet was too close to the ground; a “clacker” making a very loud clicking sound signaled the jet was going too fast; and multiple warning lights told the crew that the speed, altitude and other readings on their instruments were unreliable.

You can find some of the US reports complaining about MCAS here https://s3.documentcloud.org/documents/5766398/ASRS-Reports-... and it includes

> I manually positioned the thrust levers ASAP. This resolved the threat

Then there's

> B737 MAX First Officer reported that the aircraft pitched nose down after engaging autopilot on departure. Autopilot was disconnected and flight continued to destination

> I called "descending" just prior to the GPWS sounding "don't sink, don't sink." The Captain immediately disconnected the autopilot and pitched into a climb

Another

> Takeoff and climb in light to moderate turbulence. After flaps 1 to "up" and above clean "MASI up speed" with LNAV engaged I looked at and engaged A Autopilot. As I was returning to my PFD (Primary Flight Display) PM (Pilot Monitoring) called "DESCENDING" followed by almost an immediate: "DONT SINK DONT SINK!" I immediately disconnected AP (Autopilot) (it WAS engaged as we got full horn etc.) and resumed climb

Not surprising given that pilot training is really really expensive. Airlines love former military pilots because they are a significantly lower financial risk for them. Put them into type rating and off they go, it's rare that one ends up as a dud.

Okay, awesome. But how much of that was luck with the 737 Max that they didn't crash on US soil by US airlines?

People do this with everything though, and air travel induces a large amount of fear in the populace. Not only are we not generally comfortable flying in the air for obvious reasons, but when it happens almost everyone has to concede control to a few people in the cockpit and on the ground. Driving, even if exponentially more dangerous, affords the illusion of control of one's outcome, given driving or having someone you know driving, and control over the vehicle maintenance, etc, as well as familiarity with the control and mechanism of the vehicle. These things don't exist with airplanes for the vast majority of people.

So, you can see why there is a need to find a human component to air travel problems, because that is something one can fix (fire the incompetent people, fine them, whatever), as opposed to all of the other things which must be accepted or rejected entirely.

It is entirely in line with human nature to do this, regardless of its accuracy or effectiveness.

And there's good reasons for that. Spaceflight actually is regulated pretty strictly (partially, because any spaceworthy rocket is effectively a missile), and space pilots and tourists both sign up for such missions fully knowing that they will have a very significant chance of dying one way or another - there simply hasn't been enough human spaceflight activity to work out and understand all the failure modes, unlike with other forms of transportation.

Humans, unlike birds, aren't naturally wired to travel by air... they need to be able to trust their lives to a significantly higher degree to someone else behaving like they should, because unlike in a car they have zero control (or the illusion of control) in an aircraft.

Additionally, the inherent security risk of an airliner is very high: what is a widebody airplane at its core? Hundreds of tons of weight, a decent portion of which is fuel, propelled at near-supersonic speed, and only two people in control of it. Anything goes bonkers and you can get thousands of people killed and injured (see 9/11).

In contrast, cars, even trucks, have way less capability to cause damage simply because they weigh so much less. The only thing that comes close is railways, and hell I don't get what the US is doing there, there's barely any regulation compared to European standards (see the videos I linked at https://news.ycombinator.com/item?id=38725988).

I suspect it's rather a case of parallel evolution between McDonnell Douglas brass and software startup culture, since cost-cutting culture goes back many decades (remember "Chainsaw" Al Dunlap[1] ?) — but I wonder if there's a more direct influence.

[1] https://en.wikipedia.org/wiki/Albert_J._Dunlap

"With impressive clarity, Downfall: The Case Against Boeing reveals corporate corruption that's enraging in its callousness and frightening in its scope."

https://en.wikipedia.org/wiki/Downfall:_The_Case_Against_Boe... https://www.netflix.com/hk-en/title/81272421

It's inherently short-sighted unless forced to do otherwise by legislation. Cutting small corners pays off A LOT until the hammer falls, so there's a massive advantage to doing it / you need to do it if competition is doing it, or you eventually shut down as they take all your business.

It's inherently a race to the bottom. Sometimes that's a net gain, sometimes it isn't.

Other systems had incentives such as, get it running by such and such date or have yourself and relatives sent to inhospitable place. So people rushed flawed designs into production.

That said, upper management at Boeing needs a shake-up. People need to get fired. They need to do what Intel is trying and that is to get more engineers in charge, or at least grant them veto power on designs.

It should be a lesson against dogmatic pursuit of absolutes: capitalism comes in a wide range of flavors, and the worst is if it’s completely unrestrained. Communism produced worse and worse results the further it got from any sort of public accountability, etc.

The two problems that I see is that the concept of nuance is somewhat at odds with having a simple concept to teach kids at school, and there’s always a group which is more motivated to game the system than the average person who really just wants to hang out with their friends, raise a family, etc. rather than play political games. Boeing didn’t start it by any means but they’ve benefited enormously from decades of reduced oversight and elevated pay driven by a sort of cartoon libertarianism where letting people get enormously rich will motivate them to build great things unfettered by “red tape”.

They have, but post-Thatcher neoliberal capitalism has taken the existing perverse incentives and made them exponentially worse. We're on a course heading straight to feudalism, just with fancy titles with legal rights replaced by economic might.

I'm not carte blanche defending capitalism - its a mixed bag but it sure outpaces the competing systems put forward to date. It does need some stronger safeguards against industry self regulation - that has a bad track record.

What anti-capitalist sympathizers, in my view, don't realize is that this is due to people being in the loop. These economic systems are merely vehicles, some better than others, but the conductors are people, be they communists or capitalists. At least with capitalisms there is a delayed regulator (negative feedback) in communism it's up to the system to decide if it needs to modify itself.

Think of railway signalling systems, control-by-wire bits of modern cars, medical equipment, etc. Where the design of the software is formally proven, and the implementation verified to ensure it fits the design.

1% of 10000 is 100.

.01% is 1.

If someone came up to me and said, "Hey I can save you 99% of expected costs with 1% of your profit.", I might go for it.

Is that what you meant by "the problem wasn't the software?" Because the pilots should have been trained to unplug the computer to stop it from crashing the plane?

In the 737 MAX, the only way to disable auto-trim also disables powered trim (the thumb buttons). As grand parent says, at a certain step in the sequence it was not physically possible for a pilot to trim the plain back to stability manually. It simply can't be done.

In the 737 ng, there was a button to do just that. That would have been useful.

And that's even ignoring the fact that all symptoms were very different from those present in a runaway trim situation as described in the manual and learned by the pilots.

The user of the parent comment is remarking about time.

The aircraft can be certified without MCAS?

By what I read, MCAS is there for to avoid entering into an aerodynamic stall when the aircraft is approaching a high angle of attack, due it's using larger motors for what classical 737 was designed for. It's balancing an unbalanced aircraft using software to repeatedly adjust the horizontal stabilizer.

It is not my field, but I'm not even sure if it should be called to trim, it sounds like a euphemism for what's going on.

They introduced MCAS in the aircraft for to balance by software a hardware issue, a big design negligent issue which can lead to stalling. It is beyond to trim an aircraft, and because of this there is a big difference in the scale of the values that the algorithm manages from a trimming.

It is not my field, but I think it is not a simple factor, and that it should not be put this over the Pilots like if it were a normal aircraft that received a simple update. Every pilot flying that plane should have been warned that it was not a classic plane with a classic update.

If this type of behaviour by aircraft manufacturers becomes the norm, costs over safety, we as passengers will suffer it, as other passengers unfortunately suffered it, while they blame the Pilots. In addition that nowadays the China's aircraft manufacturing industry wants to enter global market. Some days ago I read they want permission (homologations approvals) for to enter in the European Union.

PS: They also cut costs retiring backup sensors, delegating responsibility for a vital system due the MCAS to the buyer as if it was an unimportant feature; disaster was the order of the day. And the spending cuts were not limited to that, as we have seen in recent days.

> They introduced MCAS in the aircraft for to balance by software a hardware issue, a big design negligent issue which can lead to stalling.

> Every pilot flying that plane should have been warned that it was not a classic plane with a classic update.

I was mean,

> They introduced MCAS to use software to attempt to balance an aerodynamically unbalanced aircraft with a high stall tendency, in order to avoid designing a new aircraft.

> Any pilot flying that aircraft should have been warned that it was a plane that didn't want to fly aerodynamically, with software forcing it to fly without backed redundancy. It was not mere trimming.

Afaic, the fault apportionment was Boeing documentation > airlines >> pilots > Boeing technical design.

If you listen to podcasts, these two episodes of Causality are excellent:

https://engineered.network/causality/episode-33-737-max/

https://engineered.network/causality/episode-50-737-max-ethi...

Back when I flew regularly before covid, I was tempted to create a bunch of these and hand them out to the flight crew for the flights I flew on.

Edit: Also, how does the fault lie with the airlines? Boeing didn’t document the existence of MCAS in the flight manual or training materials.

>> Because the pilots should have been trained to unplug the computer to stop it from crashing the plane?

Yes.

The fault lies with the airlines because I don't for a second believe they didn't put pressure on Boeing to get the MAX certified without mandating retraining.

And then once that was done, didn't dig into the details too hard about what changes were made.

I have a low tolerance for 'I set up all the conditions and incentives to encourage you to break the law... but you should take all the blame when it explodes.'

At some point, the customer has to take some responsibility for what they asked for.

Retraining has its own problems. No matter how well retraining is done, pilots still make mistakes from doing the right thing for the previous plane that is the wrong thing for the one they are currently flying.

Adjusting airplanes to fly the same way is a major safety advantage.

Unreported by the media, there was another MAX incident before the first crash. The crew had no knowledge of MCAS, but did follow the emergency runaway trim procedure, and continued the flight and landed safely.

Turn off the motor, and the trim is manual. There is a crank right there in the cockpit. If it is too hard to turn, change aircraft configuration to reduce the forces required to. Pilot know how to do this. This pilot stuff, they understand the forces on the flight controls and what impacts them.

Boeing made an engineering mistake. The pilots also made an operational mistake. Unfortunately, both mistakes at the same time were fatal.

I pray that pilot training has improved. And that Boeing has made systems level changes to the aircraft that will preclude it happening in the future.

And that is how aviation becomes safer every year; at a significant cost of customers lives.

"Significant" might be inaccurate.

It looks like FAA Part 121 accidents over the last 10 years with fatalities have been... 4. [0]

For a total of 6 fatalities.

[0] https://www.ntsb.gov/Pages/AviationQueryV2.aspx; 2018 (1 passenger fatality) https://www.ntsb.gov/investigations/Pages/DCA18MA142.aspx ; 2019 (3 crew fatalities, cargo flight) https://www.ntsb.gov/investigations/Pages/DCA19MA086.aspx and (1 passenger fatality) https://www.ntsb.gov/investigations/Pages/DCA20MA002.aspx ; 2022 (1 ramp fatality) https://data.ntsb.gov/carol-repgen/api/Aviation/ReportMain/G...

> The probable cause of this accident was the inappropriate response by the first officer as the pilot flying to an inadvertent activation of the go-around mode, which led to his spatial disorientation and nose-down control inputs that placed the airplane in a steep descent from which the crew did not recover.

I'm sure there's been a study somewhere that attempts to untangle all the factors that differ between commercial carriers and GA, to see which safety is most sensitive to -- continuous highly professional maintenance, highly trained and experienced crew, rigorous airliner certification regime, etc.

Also, overspeeding the airplane makes it much harder to turn the manual trim wheel. The cockpit voice recorder on the EA flight recorded the overspeed warning horn, which the crew did nothing about (they were at full power, should have pulled the throttles back).

The LA crew restored normal trim twenty-five times before crashing. What they never did was turn it off after restoring normal trim.

What Boeing did (and is STILL doing) is expect pilots to know or remember obscure NON-PILOTAGE (and in the case of MCAS, BURIED) trivia to prevent disaster.

Now... what's the more-responsible approach? Expect pilots to pilot, or expect them to recall an ever-growing list of workarounds to incompetent system design?

Software QA when actually practiced is more advanced now than airline QA.

...eh, I think "when actually practiced" is doing a lot of carrying there.

What do you mean by "actually practiced".

Outside of the aerospace and healthcare industries, I'm not sure there are many software shops that are doing QA to a level I would like to trust anyone's life with.

also software is the least likely comparison I would have made; software quality is a shit-show on a general level, and the vast public is quite aware of this every time a subway timeboard blue-screens or gets frozen on an AMI screen, or the POS machine that they're forced to interact with at work does something equally as stupid.

Almost certainly systemic issue though, so that sucks. Sucks real bad.

They need to get a Tiger Team or whatever together to look at everything with a shipment config, and make sure those "ship kits" don't leak into the real actual airplane configuration. This is . . ok, this is really manufacturing 101 stuff, but well, things happen.

I'm in the industry, but haven't touched the MAX, so take this with a grain of salt.

[1] Basically a "shipping" or train configuration

Sure, it failed, and it isn't perfect.

But planes have had a long track record of being absurdly safe.

EDIT: I saw the pictures of bolts with pins and bolts without pins. The ones with pins cannot get loose, the others can. Let's see what happened.

It's not a control surface, but it is a "moving part." That's what's baffling to me, that they spent a lot of effort building this hinge and pin roller system, and designed the door to hinge open up to 15 degrees.

It makes me wonder if there's maintenance procedures that at some point would require the operation of that door to successfully complete. Otherwise, the mechanism itself seems so incredibly overwrought, with lots of additional bolts, castle nuts, retaining pins, and even sprung hinges at the bottom.

Does anyone know why this "plug-type non-plug door" is built this way?

It needs to be usable depending on how many passengers the interior is configured for.

So it has all of the door bits there. Maybe some parts like the emergency escape slide are not installed.

e: I should be clear that it's not usable as an emergency exit, as configured by Alaska. However the operator could choose to activate it later and install a usable exit.

If you are correct, then the implication is that the concern extends beyond door plugs for MAX-9 737s to all emergency exit doors on all models of aircraft sharing this design. This is somewhat reminiscent of the huge problem with the 688 (Los Angeles) class submarines, where the discovery of a faulty weld that had passed inspections raised doubts about all welds.

If there was a reconfiguration to a seating standard that required the extra exit, the plug would be removed and a proper door would be installed, with the associated interior pieces.

This is true.

However there's still common hardware in there to allow the plug to be installed and maintained. This is why it's a complicated set of kit vs just bolting in a permanent fixture.

Alaskan airlines chose a 178 passenger configuration for their 737-9, and so are not required to have a mid-cabin exit door.

Lion Air's chosen to go with a 221 passenger configuration, and so are required to have an operating door.

Obviously changing up the number of seats isn't done on-demand, you'd need to go for a refit/maintenance cycles.

But if Alaskan decided to change density, or sold the aircraft to someone else who decided to change density - then they could go and do this.

[1] https://youtu.be/nw4eQGAmXQ0?t=305 "The Boeing 737 Technical Channel"

It sounds counterintuitive but without grease a bolt (or machine screw) will bind early with a high torque well before it is correctly tensioned lengthways. The torque is just a proxy for the tension and it is this tension which is needed to fasten your components together as intended.

The grease means that when the torque to turn the bolt reaches the correct value then the bolt is also under the correct tension instead of being because it got stuck in the thread half way.

And it's very important to understand that in most mechanical design with fasteners, the fasteners provide tension, and friction between 2 faces actually carries the load. Too little tension then fasteners can be sheared off. Too much, then the fasteners may not have remaining strength available for loads that act to add tension to the fasteners.

Using a torque wrench to reach target tension is normally only about +/-30% accurate. Usually design margin allows for this but in very critical applications where margin is not feasible, calibration is done on a sample of the same materials, etc, or a more direct measurement done. More direct measurement can mean hollow fasteners that allow you to measure the amount of stretch, use of ultrasonic measurement to measure stretch, washers with integrated strain gauges, or cleverly designed "tension indicating" washers or fasteners. There are many types so I just those keywords for anyone interested.

I am not saying that this design required such complex methods and sizing these fasteners should not be difficult. There is probably a mistake or lack of control in the assembly process.

Is there something I am missing here?

After manufacturing tension tends to drop over time so starting off with a '+' may not be entirely bad assuming it isn't extreme and that it doesn't cause the materials to deform more than permitted. The way I understand it: you apply a certain torque to a fastener in order to get to minimum levels of tension and friction (which still have an engineering reserve) on the fastener itself to guarantee a seal and to stop the fastener from coming loose, so under-tension is far worse than over-tension as long as the over-tension does not result in damage to fastener or the materials, and the allowed tolerances for over-tension are quite large (up to +150% or so normally before any permanent deformation would occur).

Unless you are using 'stretch' bolts which tend to elongate to accommodate any over-tension to end up with something quite close to the intended value. This stretching tends to be non-elastic so you'd have to replace a stretch bolt every time you unfasten it or there is a pretty good chance that it will break and/or that the threads under the ending position of the nut will have deformed so that they end up being stripped if you refasten them because the nut will travel a bit further on every refastening.