The author talks about how software creation processes at large organizations are an artifact of how large organizations operate, but in all 3 of the 3 cases, I would ask “why does the large organization need to be that way?”
Most obviously, why do executives need to be the proxy to customers? Why can’t development teams simply talk to real customers? This isn’t just an abstract idea in agile, it grew out of actual Japanese product development practices practiced at large organizations: Toyota, Canon, and others, and documented in “The New, New Product Development Game” HBR review article that was so influential to early agile.
The point that in large organizations, most of the work is coordination, again demands the question, why? It’s been understood since at least World War I by some military planners (with organizations far larger than Google) that coordinating dependencies was far more complex than reducing or minimizing them. Goldratt wrote about it when designing a project management system for Theory of Contraints (indeed, you could argue this is a fundamental learning of ToC). And one of my favorite software conference talks of all time is Mary Poppendeck’s excellent “Tyranny of the Plan,” where she notes that as computer systems have been used in planning, we seem to have become more confident but no more competent in coordinating, rather than focusing on flow, in large-scale projects.
Finally, on the importance of software created at large organizations, I agree, something that will have millions of users on day one has a greater responsibility, but that doesn’t mean that loads of bureaucracy and checking are the pathway to quality software. First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services? The often provide access to even the most extreme edge cases, but they do so by reducing the quality of service to everyone else. Anyone who’s ever filled out their own tax forms in the United States knows that it covers every base of possibly income, but 80% of people really only need to be concerned with 2-3 common forms, and 99% could simply be asked about 10 forms or so. Instead we have to answer questions for “directors of foreign corporations who also happen to be Us citizens,” instead of just requiring those people to fill out an additional form. And, of course, to (probably badly mis-)quote Deming, “you can’t check quality into a product.”
I would turn it around in the author: yes, the software practices operate this way in large orgs because large orgs are structured differently—but why do large orgs need to be structured that way? Is it inherent when absentee owners with low domain context (shareholders) pass ownership over to a manager? Is it because hierarchies insulate good but not great managers from genuine value creation as long as they play politics well? Is it because these are first order ways to understand complexity, and again, low-context absentee owners aren’t going to do the work to understand the more complex dynamics at play?
> The point that in large organizations, most of the work is coordination, again demands the question, why?
You could say something similar about large-scale distributed software systems. A lot of it is just the nature of distributing and managing work across a collection of "nodes" or, as the case may be, people.
>First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services?
This is the only part of your response that doesn't quite sit right with me. There could be many "highly scrutinized and bureaucratic functions" out there that are working very well, you just don't notice because they work so well. There could be a selection-effect here.
Quality is a big deal for me[1]. But I think you're defining "quality" too narrowly in this context. "Quality" could also mean "allows everyone, at scale, reliably, to do what they need to do." The US Tax Filing system (and its associated software) meets that goal.
Oftentimes, broadly accessible services are lower quality than more personalized ones, to the consumer.
As an example in US government bureaucracy, government software teams digitizing forms at one point weren’t allowed to utilize features like autofill or automatically filling fields based on previous answers because it would relatively disadvantage users using paper forms.
Government capabilities do need to serve everyone, and from the perspective of the whole society that is beneficial, but they are often are of low quality to the individual consuming them for this very reason.
Let’s exclude taxes, because obviously many people would hate them under any circumstances. Does the government do a good job providing the other services people interact with regularly? Do people love their visits ti the DMV? Are they satisfied with their interactions with the police? Heck, in my town, just renewing a dog license is a pain.
> The US Tax Filing system (and its associated software) meets that goal.
I disagree with the argument that the US Tax Filing system meets the goal of:
> "allows everyone, at scale, reliably, to do what they need to do."
It may do so for easy / common cases of W2 salaried employees but step a little outside of the norm (foreign sourced income, tax treaties etc.) and software gives up and shows you a PDF of relevant forms and requires you to become an expert in tax code and to keep your own multi year running calculation of carryovers and things to proceed. I'm glossing over all of the detail about how complex this really is but wouldn't expect the average, even very intelligent person to succeed in filing a correct return without a professional's help.
my anecdata is that I have always filed manually by myself, but every time had a small adjustment made by the IRS... indeed filing a correct return the 1st time seems close to impossible.
Most obviously, why do executives need to be the proxy to customers? Why can’t development teams simply talk to real customers? This isn’t just an abstract idea in agile, it grew out of actual Japanese product development practices practiced at large organizations: Toyota, Canon, and others, and documented in “The New, New Product Development Game” HBR review article that was so influential to early agile.
The point that in large organizations, most of the work is coordination, again demands the question, why? It’s been understood since at least World War I by some military planners (with organizations far larger than Google) that coordinating dependencies was far more complex than reducing or minimizing them. Goldratt wrote about it when designing a project management system for Theory of Contraints (indeed, you could argue this is a fundamental learning of ToC). And one of my favorite software conference talks of all time is Mary Poppendeck’s excellent “Tyranny of the Plan,” where she notes that as computer systems have been used in planning, we seem to have become more confident but no more competent in coordinating, rather than focusing on flow, in large-scale projects.
Finally, on the importance of software created at large organizations, I agree, something that will have millions of users on day one has a greater responsibility, but that doesn’t mean that loads of bureaucracy and checking are the pathway to quality software. First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services? The often provide access to even the most extreme edge cases, but they do so by reducing the quality of service to everyone else. Anyone who’s ever filled out their own tax forms in the United States knows that it covers every base of possibly income, but 80% of people really only need to be concerned with 2-3 common forms, and 99% could simply be asked about 10 forms or so. Instead we have to answer questions for “directors of foreign corporations who also happen to be Us citizens,” instead of just requiring those people to fill out an additional form. And, of course, to (probably badly mis-)quote Deming, “you can’t check quality into a product.”
I would turn it around in the author: yes, the software practices operate this way in large orgs because large orgs are structured differently—but why do large orgs need to be structured that way? Is it inherent when absentee owners with low domain context (shareholders) pass ownership over to a manager? Is it because hierarchies insulate good but not great managers from genuine value creation as long as they play politics well? Is it because these are first order ways to understand complexity, and again, low-context absentee owners aren’t going to do the work to understand the more complex dynamics at play?