Poor Deming never stood a chance
surfingcomplexity.blog102 points by todsacerdoti 10 hours ago
102 points by todsacerdoti 10 hours ago
> for thermostat B there are many more outliers. We’d say that [...] thermostat B is not [under statistical process control]. (In practice, you’d draw a control chart to identify whether the system is under statistical control).
I did draw the control chart, and thermostat B is definitely under statistical process control: https://xkqr.org/info/xmr.html?baseline=33,97,41,65,72,71,64...
(To be clear, this is not a diss of the article. It is hard to create fake data that look realistic at first glance but are not under statistical process control. That's just how good control charts are at separating signal from noise.)
The main point that I did not see mentioned in this piece is that Deming should only be applied to MANUFACTURING environments, because things like engineering are too chaotic to identify processes or trends in the engineering itself, and trying to control those engineering processes with SPC doesn't really improve the quality of the engineering, it just adds stress, makes things take longer, and probably lowers the quality of the thing that is being engineered.
Obviously, if a quality issue is detected in manufacturing, there may be some steps that engineering could take to improve the manufacturing process and make things stable enough to obtain meaningful statistics. This is part of the Deming feedback process, and part of the System Engineering Life Cycle.
I think you're confusing Deming with statistical process control.
It is true that SPC works best for the non-chaotic parts of product development and manufacturing alike. There are parts of product development that are non-chaotic, and SPC works just fine there, too.
In addition to SPC, Deming had strong opinions on how organisations ought to work and these are relevant also for product development. These are things like
- Understand the underlying customer need.
- The leaders shape the output of the organisation by shaping its processes.
- It is cheaper and faster to build quality and security into the product from the start instead of trying to put it in at the end.
- Close collaboration with suppliers can benefit both parties.
- Have leaders skilled in whatever their direct reports are doing. Use them as coaches normally and as spare workers in times of high demand.
- Collaborate across parts of the organisation instead of throwing things over walls.
- Don't just tell people to do better. Show them how they can do better. Give them the knowledge, tools, and authority they need to do better.
These are just as relevant for product development as for manufacturing. If anything, even more so, thanks to the chaotic nature of product development.
I think Donald G. Reinertsen did a good job in his books applying Deming to the design process.
The chief problem I have with Reinertsen (and it's not his fault, at all) is how difficult it is to get people to buy in to the idea that cost of delay exists, let alone buy in to measuring it.
The core issue with the article is that author mixes up bad management and "fog of management" with the fact that financial results have a disproportionate amount of influence in how things are organised. Every team and employee should do their part to contribute to the financial targets every quarter and within the fiscal year. Which clashes with Deming's points 11b and 12b [1].
_________
> Every team and employee should do their part to contribute to the financial targets every quarter and within the fiscal year
The inevitable result of this is however the devaluation of the future. Eg if the statement was true, it'd be the R&D workers responsibility to hand in their resignation ( or their managers layoffs) if their product won't get paying customers within the same fiscal year... And the same applies to any other long term expenditure/investment that company might be considering. E g building a new fab/production line etc pp
So no, that statement of yours is not actually true. It should not be entirely ignored, but it should not become a leading cause unless you want to run the company in the ground.
> trying to control those engineering processes with SPC doesn't really improve the quality of the engineering, it just adds stress, makes things take longer, and probably lowers the quality of the thing that is being engineered
Totally depends on the scale. For pizza-sized times with a neighbourhood pizza shop sized impact, sure. Large scale projects without controls & feedback loops in place will fall apart; see: Scaling teams: https://archive.is/FQKJH
If you'd follow some medium to large scale projects (like Go / Chromium), the value of processes & quality control, even if it may seem at the expense of velocity, becomes clear.
The great insight of Deming's methods is that you can (mostly) identify the difference between common and special causes mathematically, and that you should not attempt to fix common causes directly - it's a waste of time, because all real-life processes have random variation.
Instead, what you want to do is identify your mean and standard deviation, plot the distribution, and try to clean it up. Rather than poking around at the weird edges of the distribution, can we adjust the mean left or right to get more output closer to what we want? Can we reduce the overall standard deviation - not any one outlier - by changing something fundamental about the process?
As part of that, you might find out that your process is actually not in control at all, and most of your problems are "special" causes. This means you're overdriving your process. For example, software developers working super long hours to meet a deadline might produce bursts of high producitivity followed by indeterminate periods of collapse (or they quit and the whole thing shuts down, or whatever). Running them at a reasonable rate might give worse short-term results, but more predictable results over time, and predictable results are where quality comes from.
Distributed systems is also a way to be throughly humbled by complexity: https://fly.io/blog/corrosion/
This is a very trivial treatment of Deming and I’m surprised how it makes its way to the top of HN. The arc from Walter Shewhart to W.E. Deming is a bedrock foundation in an Industrial Engineering curriculum. These men paved the manufacturing process quality principles of modern industrialization. Drucker was about management science, truly an apples to oranges comparison.
Deming was a statistician first, yes, but he also had strong opinions in terms of management science/philosophy. These opinions came from a perspective of systems theory and understanding variation.
Management Science? Only management science I read so far (with actual measured outputs and ideas) was Peopleware. Everything else was more like philosophy. Has anyone ever measured, long term results from multiple management methods? What I saw when I looked into it was simple - the Toyota Way was the model for a lot of successful companies, including Pixar.
Peopleware is extremely old, and if you were to crack open a modern MBA text you'd find statistics and statistical process control type of thinking integrated everywhere, in all the MBA subjects. Management being soft and opinionated ended a long time ago, but then again, "the future is unevenly distributed" so who knows what conceptual envelope you find yourself.
The analogy I used with the team was that, set the goal, present the map, and figure how to make a better map. Drucker was about the goal with a given map. It is not uncommon for people receiving the OKR not resonating with it. Sometimes they actually have insight into making a better map, but if OKR is OKR, one just have to follow, people swallow their thoughts
This topic is very relevant in the age of agentic AI when every decision is a statistical next token prediction “trained” on some loss function. AGENT.md, SOUL.md etc are just smoke and mirrors of The Wizard of the Oz.
Eventually manager as a profession will be replaced by tools, just like computer as a profession, editor as a profession.
The evolution of computer science will be manager science. There is more than loss function and KPI.
It is also worth noting that US management is notoriously bad at the actual management. Toyota v. US car manufacturers did not look like a fair fight when Deming was in the ascendant, and it is hard to tell given the scales involved but it looks a lot like the US has been outmanoeuvred in all aspects of industry by the Asians.
US companies are generally a better bet though, because despite the handicap of being run by Americans, they are hosted in a country that generally believes in freedom and rule of law which means they have an unfair advantage even if they do a sub-par job of making the most of what they have.
Exceptions abound in the details.
The thing is, the Toyota methods relies on people on every level to work to improve processes. If you're an employee and know you'll be there 10 years down the line or even until you retire, you have an incentive to improve said processes.
Now check most Western companies: since the 70 / 80, everything is about reducing headcount. Lay-offs, outsourcing, offshoring, now the concept of spending your whole working life at the same company feels like a fever dream. So why would an employee try to improve things for the company when they know there is no future for them there? Better improve their own career and future prospect. So yeah, things like Kaizen are doomed to fail until things change.
This is rather like my observation about British car companies in the late 20th century:
- large factory of British workers + British management: strife, strikes, disaster, bankruptcy (British Leyland)
- small factory of British workers + British management: success, on a small scale (lots of the F1 industry, McLaren etc; also true of non-car manufacturing)
- large factory of British workers with overseas management: success (Nissan Sunderland, BMW era Mini, etc)
Where does someone like Rover fit in to your matrix? If I can respectfully recommend. If you can go have a read of "We sell our time no more" by Paul Stewart.
Tory governance and fiscal policies had all the responsibility for Leyland, Hillman and more importantly Rover.
I have to admit it's not that deeply researched. That book sounds interesting.
For Rover in particular, https://en.wikipedia.org/wiki/Phoenix_Venture_Holdings stands out. Similar to Maplin and Toys R Us, an example of the owner-management taking a large amount of cash out of a business that was declining.
What's particularly interesting here is that one of the success cases (BMW era Mini) was built on a Rover design. They picked it up and ran with it - basically gutted the electrics and power-train to match their existing systems and supply chains - but it was already in flight when BMW came on.
That factory was fascinating to work in, looking back on it I saw a lot of Deming-compatible stuff going on that I wasn't equipped to recognise at the time. There was strong German representation in factory management, lots of interaction with people coming and going from Munich all the time. But the production line staff had a large agency contingent so it didn't have the "job for life" ethos that the Toyota Way would say is essential.
Fortunately, once we impose a wealth tax on corporations we can solve this. Billionaire corporations should not exist.
I don't think the problems of US corporations are due to being over-capitalized, they're all to do with interactions with the political and media sphere plus unnecessary conflict with the staff.
If no corporation is allowed more than 1 million dollars (10 years of median income) then 10 average people can counter a single corporation. That’s a good ratio.
That's too arbitrary, and you say that as if a billion dollars is a lot of money.
It's not.
There was a time when millionaires were considered 'rich'. Now that's just a retiree, in most housing markets, who's paid off a house. Or even a townhome... and in some places, a condo!
It doesn't matter "whether it should" cost that much, that's irrelevant for my example. The point is, being a millionaire isn't a big deal. It's common. It doesn't mean wealthy.
Likewise when a company is large, and has infrastructure all over the world, and is worth much of a T, a B is nothing. Cash reserves in the billions is really not all that much, just fiscal prudence.
An alternate is that "banks should get free money, by forcing all companies to borrow money for capital projects". Because if you tax companies for "wealth", then they'll just spend all that capital on loan payments.
I feel people have such weird ideas about taxation. People see "oh no, someone has free money!" then get excited and want to tax. What? The goal of taxation isn't "take money from anyone we can", nor is it 'wealth redistribution', it's instead 'how to pay for joint projects' that all of society benefits from.
Losing track of that last bit, is when people stop asking "should we tax" and instead say "they have money, so tax"
You write: > The goal of taxation isn't "take money from anyone we can", nor is it 'wealth redistribution', it's instead 'how to pay for joint projects' that all of society benefits from.
But I think the author of the comment you were replying to had a different goal in mind. I think their goal was "prevent corporations from getting too big".
We can and should debate whether that is a goal we should be trying to achieve, but if it is then progressive taxation for companies might be a way to achieve it.
We might presume that was the goal, yet it wasn't explicitly stated. And many have a goal of generic wealth redistribution, and will inject such into any conversation about large companies.
One might note the unrestrained concern about fluid capital acquisition, in the post I replied to. It's not having billions in infrastructure that was cited, nor having a large number of employees, both metrics of size, but instead having fluid, unused capital.
If we wish to constrain upon size, there needs to be nuance, conjoined with the specific industry, and even sub-industry. Some capital equipment costs can be enormous. Should we work to prevent financing such via stored profit? Should we work to force companies to finance, then pay off, just to feed the banks, rather than store and then spend?
Should we tax so that "big ideas" may never occur?
I think far more would be gained by ensuring taxation just stays fair between smaller and larger company structures. There's a lot of book-keeping that can be done as a large company, to hide profits, that cannot be done when you're a small mom and pop.
Fundamentally stock markets won the world of business, so everything has a horizon of a financial quarter.
Hence, every action of a company needs to be measured against the upcoming quarterly results.
OKRs et al are great at that.
Who cares about quality/sustainabily. We just want the stock go wheeeeee and get our bonuses.
Not sure about your opening thesis. The vast majority of employees work for privately held businesses and from personal experience of working in such companies, “management” by OKRs and the like is common in companies who are not listed on stock markets also.
There'll inevitably be cargo culting driven by MBA curriculums and "they're making a lot of money, let's do what they did" without examining the specifics of the situation to distinguish luck from judgement.
Maybe it’s my limited intellect but I found Drucker to be a lot easier to understand.
Where Deming reads like a science paper, Drucker reads like an installation guide.
Not necessarily limited "intellect", but rather limited background knowledge.
Deming requires quite a bit of knowledge and understanding in failure/success modes. The core tenet of Deming is that every output is a result of some process and, therefore, output is controlled by controlling* the process itself. Look at your process and tackle failure modes in this priority list.
Drucker, on the other hand, puts the process under the fog of war and basically says deploy pressure on process outputs and let the process adjust itself. It requires much less understanding behind the processes to make sense.
* - Process control in Deming is mostly about variability.
So following Drucker would be the cause of a lot of "every metric becomes a target" in management?
Obviously the truth is messier than that, and it's worth noting that Drucker later recognised the toxicity of Management by Objectives and disavowed it. Quite a bit of OKR literature is devoted to avoiding it becoming its progenitor, MBO.
Worth adding that Deming (after Shewhart) recognised two kinds of variation: special cause (specific the work item in question) and common cause (an artifact of the process). That knowledge work involves a lot more of the former than does manufacturing does not excuse inattention to the latter.
> Drucker later recognised the toxicity of Management by Objectives and disavowed it.
Reminds me of a seminal treatise for Waterfall by Royce[0], where he basically says it’s fraught with issues, but can be coerced into something semi-usable. Not exactly a ringing endorsement. I think that paper is used as the template for all Waterfall work.
that kind of ties in with the article's thesis; deming's approach is more scientific in the classic sense of taking observations and using those to build up your mental models, whereas drucker proposes a one size fits all recipe for managing roadmaps.
This made me spit out my coffee…
> One of the virtues of OKRs is that they are straightforward for managers to apply.
Or you can just abolish the fiat soft money system, let the corporations go out of business, let efficient small companies take their place and then you'd have founders who actually care about long term results in charge and they could manage the company however they want. If they do a bad job, they'd go out of business. If they do a good job, they'd stay afloat or maybe even grow a little. And over time, all the companies with incompetent leaders would be wiped out, everyone would get a fair shot at being a leader and everyone would end up in their rightful place and capitalism would function as it was designed.
You could read The Long Twentieth Century by Giovanni Arrighi.
Fiat money is not the problem, the financialization of economy is actually a common by-product of aging great monetary powers. The US chose to become a monetary power in 1945, rejecting Keynes' Bancor proposal.
Then in 1971, it found it couldn't keep it working, due to the very reasons Keynes explained to them at Bretton Woods. Arrighi argues this has happened 4 times already.
So Fiat money and the financialization of life is just an outcome of something else - that being a monetary superpower is just not sustainable.
The tulip craze and many other market excess crashes occurred under non-fiat currency schemes. The money system being fiat or metals backed or crypto is incidental to "let the corporations go out of business, let efficient small companies take their place". If you want smaller corporations and more competitive marketplaces, it's anti-megascale taxes and anti-monopoly regulation that can achieve that.
[dead]