There is a wonderful corollary in basic statistics modeling that success tends to be a long-tailed distribution.
Reason: the easiest framework that allows modeling "the resulting random variable R is zero if any of the incoming variables vₙ is zero" is the product formulation,
R = Πₙ vₙ
which by taking the logarithm of the individual variables becomes a standard central-limit-theorem prediction; as a result success in such contexts follows a log-normal distribution[1], which is long-tailed, even if the individual factors fₙ = log vₙ do not look very normal.
This then leads to a growth pattern called Gibrat’s Law[2].
I always took this to be 'there is only one success, there are many failures', there is only one zero, but many values other than zero. It seemed kind of logical.
> zero has many properties that other numbers do not
In particular, zero is the additive identity [0] in almost all the "usual" number systems in which it appears. That is roughly speaking, it is the only A such that X + A = X, for all X, where X and A are elements of the number system (e.g. field of real/complex numbers).
The integers form a group under multiplication though and a key property of a group is that every element has an inverse. So how does the definition hold if there is nothing that could be considered an inverse for the number 0? Curious about this... I never thought about it before.
That's idiomatic, not an inherent property of the numbers themselves. You're basically arguing that because we do it in one context we do it in another context, but it doesn't answer the question of why.
Well, an interesting fact is that every nonzero rational number has two equal representations in some bases, e.g. 0.999...=1. Zero is then is the only rational number with just one representation.
Well, the convention in maths is that we never add trailing zeros, since they wouldn't as any information. Unlike in other sciences, where they represent measurement precision.
I'm (trying to) view the video, player cooperation willing....
A bit longish, though if this is a single take, I'm impressed by the presentation (I've yet to see a cut in the first 15 minutes or so).
"Similarity" and "difference" are relations, and as the speaker notes, relative to viewpoint. I've done some earlier research and thinking into the meaning and distinction of value, where utility-value (related to economic value) is based on relations as well -- to the producer or creator of an item or service, as well as the user or owner's use value. And these are not generally intrinsic.
Aristotle's Categories are a good starting point for considering relations, even now.
I just started reading Thomas Sowell's newest book Discrimination and Disparities and on the first page I encounter this passage:
"When there is some endeavor with five prerequisites for success, then by definition the chances of success in that endeavor depend on the chances of having all five of those prerequisites simultaneously. Even if none of these prerequisites is rare—for example, if these prerequisites are all so common that chances are two out of three that any given person has any one of those five prerequisites—nevertheless the odds are against having all five of the prerequisites for success in that endeavor."
Sounds like... the Anna Karenina Principle. He goes on to point out that this simple and reasonable model produces extremely skewed distributions for success in any particular endeavor.
"One conclusion is that we should not expect success to be evenly or randomly distributed among individuals, groups, institutions or nations in endeavors with multiple prerequisites—which is to say, most meaningful endeavors. And if these are indeed prerequisites, then having four out of five prerequisites means nothing, as far as successful outcomes are concerned. In other words, people with most of the prerequisites for success may nevertheless be utter failures.
Whether a prerequisite that is missing is complex or simple, its absence can negate the effect of all the other prerequisites that are present. If you are illiterate, for example, all the other good qualities that you may have in abundance count for nothing in many, if not most, careers today. As late as 1950, more than 40 percent of the world’s adult population were still illiterate. That included more than half the adults in Asia and Africa."
Bear in mind that anthropologists are not historians. Anthropologists are dedicated to the study of human culture, which is related to but no the same as the study of human history. Guns, Germs, and Steel chiefly argued that geography was a much stronger determining factor in the development of human societies than culture. Unsurprising that those dedicated to studying human culture weren't a fan of this framing. Contrary to the claims made in that subreddit, Diamond's high level points have been widely influential and positively received by historians. The claims made in that subreddit are not at all representative of the attitude my university history professors had towards Diamond's work.
Furthermore the claims that the book promotes a racist outlook is absurd. Guns, Germs, and Steel is one of the most anti-racist explanations to the disparities in levels of development that there is. One of the chief points of the book is that these disparities are due to being in the right place at the right time rather than racial superiority.
I think it's easiest to just call him a geographer, that really fits his work the best, and his most interesting ideas are geographical at their core.
Nonetheless, the idea that geographic determinism is mainstream among historians is not true. I'd imagine it's assigned in the early years of college because it has broad appeal across disciplines.
And over half a dozen historians at my university spoke highly of the book when I discussed it with them. I'm going to put the opinions of history professors above anonymous posters on the internet. Many mainstream historians are shifting away from the "great man" style of history that puts emphasis on culture and actions of individuals, and towards a more geographic lens of history.
The bulk of the criticism I've encountered (besides attempts to equate it with racist environmental determinism akin to Herodotus) are that it gave insufficient attention to advancements in human knowledge and understanding that changed the landscape of advantageous and disadvantageous geography. This doesn't hold much water, from what I could see in the book. In fact, one of the central points of the book is that access to the New World (which was brought by advances in maritime navigation) drastically altered the course of Western European history. Prior to that, South and East Asia were broadly speaking more developed than European societies with greater population density and larger urban centers.
If you actually read through the responses in the search I linked, you'll notice that the discussion is a little more nuanced than "it's perfect" or "it's trash". It's certainly not perfect in terms of analysis of historical evidence—there are far too few primary sources to support his claims directly, and the trends he points out can be picked apart in particular—but it still has compelling evidence that would be considered valid in other disciplines. I suspect your own professors had a more nuanced opinion than simply "spoke highly of". That doesn't sound like an academic to me.
Additionally, you should check the subreddit out. It's very aggressively moderated and many of the posters are openly credentialed professors or are actively doing graduate research. I have found it's also a great place to keep your eyes open for preprints.
I did read through the responses. The highest upvoted comment on the post "What do you think of Guns, Germs and Steel?" comes from someone that explicitly says that they are not a historian. The bulk of the criticism comes in the form of people alleging environmental determinism and the denial of human agency, which isn't actually a criticism of the book but rather a rejection of book's core claim that geography is a more decisive factor in human development than society and culture. That, and a healthy helping of people alleging western chauvinism, despite Diamond explicitly rejecting the narratives of Western social or racial superiority and pointing out that Western Europe made little contributions to Eurasian development until the mid-late 2nd millennium AD.
Sure, my conversations with professors were more nuanced: one talked about how the geographic analysis prompted them to examine history through the lens of quantifying available resources to a society and levels of urbanization. They agreed with the core theses of the book, and said that it offered good refutations to the common layman's narrative of Western cultural superiority. I could spend the rest of my evening recounting the conversations in the comments here but for the purposes of this discussion saying that they "spoke highly of" the book conveys what is necessary.
> Anthropologists are dedicated to the study of human culture, which is related to but no the same as the study of human history
This doesn’t seem quite right to me, as the child of a professional anthropologist and historian. My general impression is that the difference between anthropology vs. history (not to mention sociology & political science) has more to do with methodology than subject. Think of ethnography vs. archival research. The subject is often heavily overlapping.
They are different subjects. Anthropology is the study of human cultures and societies. Framing this as "ethnography vs. archival research" is a roundabout way of saying the same thing:
>Ethnography: The scientific description of peoples and cultures with their customs, habits, and mutual differences.
Ethnography means you embed yourself in a group, do a bunch of interviews, etc. It is a very hands-on and 1:1 kind of method. Your definition does not really adequately describe it. https://en.wikipedia.org/wiki/Ethnography#Features_of_ethnog... (Wikipedia isn’t a perfect source here, but gives some idea.)
This is a very different approach than sitting in an archive and reading old documents or running statistical analysis on government databases or ....
But it is possible to investigate the same types of subjects using these different methods.
> these disparities are due to being in the right place at the right time rather than racial superiority
The problem is that many of the supposed adaptive phenotypes were conjecture or based on poor science. Such sloppy characterizations, when combined with the seeming authoritative weight of genetics, is fodder for racism.
It's like the stereotype in the U.S. that blacks are genetically better at singing and sports. It's a "compliment" that actually serves to justify racism--the implied corollary is that to be good at those things is to be maladaptive at intellectual endeavors. There's no real scientific evidence to back it up. What "evidence" exists is far better explained by environmental factors (i.e. racism), but superficially it all seems intuitive because the entire society is constructed upon that narrative. (Actually, traits like athleticism positively correlate with intelligence, but that's irrelevant because the stereotype was flawed from inception.)
The only way to suppress such racist tendencies in humanity (to characterize and group people en masse and to rationalize post hoc our behaviors) is to be highly skeptical about proposed substantive genetic differences, particularly those which seemingly justify existing cultural or social differences. When they do exist they almost never (if at all) operate in the way we originally believed. Even recent scientific history (as in past couple of decades) is littered with hypotheses and evidence of this sort that turned out to be faulty.
I'm not sure I follow. Are you talking about Diamond's book? Diamond not only did not attribute the course of human history to genetics or phenotypes, he has argued against such an explanation on multiple occasions. He attributes the higher development of Eurasia to geographic features: an environment more conducive to agriculture, and has a large amount of land in the latitudes conducive to building dense urban centers, among other things.
This is why the allegations of racism hold little to no water: by explaining these disparities with geography, the book is a decisive counterargument to the narrative of genetic superiority.
Would you find it racist to say that people living closer to the equator have darker skins? While there's no evidence for any complex adaptive differences between groups of people there are certainly adaptive responses to specific environmental hazards that we can see. People's levels of melonin in their skin balance skin cancer against vitamin D deficiency against folate deficiency, at least for their ancestral environment. Some groups of hunters living in the far north are harmed less by a nearly all meat diet than most people would be. Many groups living at high altitudes have higher quantities of red blood cells or similar adaptations. And since Guns, Germs, and Steel came out we've located genes influencing the immune system that trade off resistance to parasites versus resistance to contagious disease and found that Native Americans tend to have genomes that make the former trade while settled peoples in Eurasia tend to have genes pushing more in the later direction. So Jared Diamond's assertions about disease resistance have been entirely born out.
> The problem is that many of the supposed adaptive phenotypes were conjecture or based on poor science.
What phenotypes? It’s been a while since I read GGS but as far as I remember the argument of the book lies directly on the environment, not on human adaptation on that environment.
Since reading the book I’ve seen a lot of comments claiming racism on Diamonds part that to this day I can’t understand. I’ve seriously considered the possibility that there’s two different editions of the book, because I find it hard to justify any racist view using the book.
> That is, natural selection promoting genes for intelligence has probably been far more ruthless in New Guinea than in more densely populated, politically complex societies, where natural selection for body chemistry was instead more potent.
> ....
> That is, in mental ability New Guineans are probably genetically superior to Westerners....
It's hard to pinpoint anything as boldly suspect as what he wrote early in the book, but in searching what stood out to me was his casual use of words like "evolved" and "evolution". In various parts it's ambiguous whether he perceives a genetic component to the evolution of various political societies.
I thought I remembered a part where he says some seafaring peoples score higher on spatial reasoning tests, from which he infers an evolutionary genetic adaption. But I couldn't find it.
> Besides this genetic reason, there is also a second reason why New Guineans may have come to be smarter than Westerners. Modern Euro- pean and American children spend much of their time being passively entertained by television, radio, and movies. In the average American household, the TV set is on for seven hours per day. In contrast, traditional New Guinea children have virtually no such opportunities for passive entertainment and instead spend almost all of their waking hours actively doing something, such as talking or playing with other children or adults.
Almost all studies of child development emphasize the role of childhood stimulation and activity in promoting mental development, and stress the irreversible mental stunting associated with reduced childhood stimulation. This effect surely contributes a non-genetic component to the superior average mental function displayed by New Guineans.
> That is, in mental ability New Guineans are probably genetically superior to Westerners, and they surely are superior in escaping the devastating developmental disadvantages under which most children in industrialized societies now grow up. Certainly, there is no hint at all of any intellectual disadvantage of New Guineans that could serve to answer Yali's question.
This is part of the prologue where Diamond rejects this genetic explanation, as well as Classical environmental determinism:
> A GENETIC EXPLANATION isn't the only possible answer to Yali's question. Another one, popular with inhabitants of northern Europe, invokes the supposed stimulatory effects of their homeland's cold climate and the inhibitory effects of hot, humid, tropical climates on human creativity and energy. Perhaps the seasonally variable climate at high latitudes poses more diverse challenges than does a seasonally constant tropical climate. Perhaps cold climates require one to be more technologically inventive to survive, because one must build a warm home and make warm clothing, whereas one can survive in the tropics with simpler housing and no clothing. Or the argument can be reversed to reach the same conclusion: the long winters at high latitudes leave people with much time in which to sit indoors and invent.
> Although formerly popular, this type of explanation, too, fails to survive scrutiny. As we shall see, the peoples of northern Europe contributed nothing of fundamental importance to Eurasian civilization until the last thousand years; they simply had the good luck to live at a geographic location where they were likely to receive advances (such as agriculture, wheels, writing, and metallurgy) developed in warmer parts of Eurasia. In the New World the cold regions at high latitude were even more of a human backwater. The sole Native American societies to develop writing
arose in Mexico south of the Tropic of Cancer; the oldest New World pottery comes from near the equator in tropical South America; and the New World society generally considered the most advanced in art, astronomy, and other respects was the Classic Maya society of the tropical Yucatan and Guatemala in the first millennium A.D.
(some paragraphs later)
> Nevertheless, we have to wonder. We keep seeing all those glaring, persistent differences in peoples' status. We're assured that the seemingly transparent biological explanation for the world's inequalities as of A.D. 1500 is wrong, but we're not told what the correct explanation is. Until we have some convincing, detailed, agreed-upon explanation for the broad pattern of history, most people will continue to suspect that the racist biological explanation is correct after all. That seems to me the strongest argument for writing this book.
And the answer he offer to refute the racist biological explanation is a geographic one.
> Why did New Guineans wind up technologically primitive, despite what I believe to be their superior intelligence?
That's a direct quote where he says flat-out what he believes. It sets the tone for the entire book and the implications about how societies evolve. In the context of everything else, one could reasonably infer that he believes that Eurasians conquered the world because they genetically evolved to develop authoritarian, centralized societies where intelligence took a back seat to being a pawn in a hierarchical political machine.
But he doesn't say that explicitly, and probably doesn't even think that. His discussion of natural selection and evolution is so loose, equivocal, and as you point out even contradictory, who knows what he believes. Point being, no matter who's the good guy or bad guy, he uses very specious reasoning to build a sophisticated theory about how the world is ordered, the very kind of specious logic used in racist thinking everywhere.
His discussion of New Guineans is used to directly refute the claim that Eurasians were more advanced due to intelligence - New Guineans are just as intelligent (or more intelligent in his opinion) as Europeans, but did not develop advanced technology, thus intelligence cannot be the determining factor in technological development. He rejects intelligence as a determining factor, and spends the rest of the book after the preface explaining how influence of geography is much more convincing causal factor.
How you reach the conclusion that this reinforces racist thinking, particularly when he explicitly states that the geographic explanation he offers in Guns, Germs, and Steel is meant as a refutation to race-based explanations, is beyond me.
You should read Why Nations Fail as a comparison. It's an incredible book. It starts off mentioning GGS with the gist being not that Diamond is wrong per-se, but "it's a little bit more complicated than that". The authors then proceed to put forward a thesis centering around how political institutions define economic institutions which define what opportunities the people have available to them and that massively affects outcomes. Fittingly the book is basically about how the Anna Karerina Principle played out throughout history during every political regime change. Most were simply that. Change. It was the confluence of a number of unique and necessary things all in the same place and same time that finally gave birth to a democratic society.
Really incredible book.
I need to get around to reading my copy of GGS. I think the geography hypothesis has some degree of validity to it, but certainly not a complete explanation.
Family psychologists say it's false -- there's only a handful of problems that families are unhappy about. But there are millions surprising ways happy families are happy.
It's hard to understand how family as an abstract entity can be happy. If we don't have some metric of happiness for one, how can we add it up?
If abstract happiness existed it would be incomprehensible, like consciousness in China brain[1] thought experiment. Or such happiness would be observable for all but this particular family, i.e. show-off happiness. The latter is a punchline in Dostoevsky books.
I mean family is a structure built on top of real human beings. Social structures are real while everybody trusts in them, same as how any belief system works, money, etc. Even the definitions of family are different in various cultures.
What it means in real life, the happiness of family is not a sum of everyone's, but some sort of compromise.
When I worked in a research lab with some finicky bespoke equipment, I remember the boss making a related point. If our experimental device is made up of N components that are each 95% reliable, that sounds pretty good... until you start thinking about how big N is, and how the overall reliability is only 0.95^N. It doesn’t take long before you’re only getting data one day a week or whatever.
Imagine you have 20 experiments, each with a 5% chance that the null hypothesis shows a positive result, then there is a good chance that one of them will be positive even if null is true for all of them. That experiment is bunk but you say p=0.05 so all is good!
The Swiss cheese model applies for redundant checks, the Anna Karerina principle applies for cumulative checks. Or, if you get success if any of your components works, you get the Swiss cheese model, if you get success only if all your components work, you get the Anna Karerina principle.
On a practical note, when I need to write code for a platform (I usually write for Apple platforms), there's usually a dozen ways to do something, but only one or two ways to do it right.
I just went through this today, as I was selecting the best way to trap and pipe stdout.
My experience is that with software there's very often no correct way to do it. It seems to me that most of engineering is simply deciding which compromises are acceptable to you today. Or picking something, and shoring up the leaks yourself.
Various impossibility theorems say otherwise, eg. the CAP theorem for distributed computing, the scalability trilemma for blockchains, Arrow's impossibility theorem for voting systems. Many times you have mutually incompatible constraints that are both desirable, and the best you can do is pick a point on the continuum that's satisfactory to the particular subset of users you wish to serve.
The (highly cost-ineffective) Right Thing is to invest enough in network infrastructure that there are no network partitions.
I don't know about the other examples off the top of my head, but they're probably similar (and similarly impractical).
Anyone can design a bridge that won't fall down; engineering is the process of figuring out which corners you can cut and still have the bridge just barely not fall down.
It's still a distributed system if clients connect to the closest server, speed of light latency between servers exists, and the system deals with consensus between servers, even if the design assumes network partitions never happen.
My favorite one of these is metastability[1]. Any time you have a crossing clock domain the fact that your ASIC works is based on statistics and not hard guarantee.
The issue I happen to be working on right now is: the operating system returns an incorrect value, for a particular function call, in many cases.
I'm reminded of an article I once read about determining whether a large number is prime. There are probabilistic algorithms that are much faster than actually checking every possible factor, and surprisingly accurate. Also, if you run any algorithm long enough, it's susceptible to bit flips (due to a cosmic ray or bad RAM or whatever). Together, this means that beyond some point, you're more likely to get the right answer with the probabilistic test than a perfect algorithm.
What's the "one right way" to deal with a bug in the OS? Sure, in the worst case, I could reimplement that entire subsystem, if I had infinite time and money ... but it's a large enough subsystem, odds are I'd introduce even more bugs in any replacement.
That's the tricky side of engineering. Sometimes "not cost-effective" is so expensive it defeats the entire purpose of whatever you're building. You very quickly get into massless elephants and frictionless inclined planes, and that doesn't help anyone.
I don't believe in "one right way" except in the simplest cases. As Harvey Pekar said, "Ordinary life is pretty complex stuff."
> The issue I happen to be working on right now is: the operating system returns an incorrect value, for a particular function call, in many cases.
Yes! Great example. So the right way is to fix the bug.
The bug is not in your source code and fixing a bug in someone else's source code that you rely on is not cost effective because, in the words of Bob Dylan, "everything is broken."
It's just a pithy quote that is obviously false and yet very difficult to unlearn. People with different values can be equally happy. There are tons of objectively crazy people who are happy, and even a few who are rich. There lots of miserable people with "perfect lives". There are many approaches to learning something. There are many ways to contribute to a successful team. There are lots of good programming languages, and lots of different ways to solve problems.
I'm not suggesting that there is no such thing as happiness or that wrong can't be distinguished from right, I'm just pointing out that you'll probably go through life being very judgmental and probably unhappy if you really believe Tolstoy's quip.
You could call this the "maze principle": there is one way to get out of the maze, and all other routes are dead ends. (Actually, there's often more than one way to get out, just like not all "happy families" are the same; it's just much harder to find another correct route than it is to run into a dead end)
"Correlations, Risk and Crisis: From Physiology to Finance"
Abstract
> We study the dynamics of correlation and variance in systems under the load of environmental factors. A universal effect in ensembles of similar systems under the load of similar factors is described: in crisis, typically, even before obvious symptoms of crisis appear, correlation increases, and, at the same time, variance (and volatility) increases too. This effect is supported by many experiments and observations of groups of humans, mice, trees, grassy plants, and on financial time series. A general approach to the explanation of the effect through dynamics of individual adaptation of similar non-interactive individuals to a similar system of external factors is developed. Qualitatively, this approach follows Selye’s idea about adaptation energy.
Did you read a particular translation? It's a book I've always intended to read and I started trying to read Constance Garnett's translation recently, but the writing felt awkward and it turned me off pretty quickly. It felt like I was reading poorly written fanfiction or a book intended for young adults with limited vocabularies.
An example from one of the first few pages:
"There happened to him at that instant what does happen to people when they are unexpectedly caught in something very disgraceful. He did not succeed in adapting his face to the position in which he was placed towards his wife by the discovery of his fault."
I've never read an English translation of any classic Russian literature that didn't require re-reading of at least 10% of the sentences to figure out the meaning.
On the one hand, it makes reading such books feel like an arduous undertaking. It took me about a year and a half to finish Anna Karenina, and I just gave up on The Brothers Karamazov.
On the other hand, using two dozen words when five could suffice has the effect of drawing out each moment of the story, giving it a texture and flavor that would not otherwise be possible.
I found the matching paragraph in my 1918 Maude translation:
"At that moment there had happened to him what happens to most people when unexpectedly caught in some shameful act: he had not had time to assume an expression suitable to the position in which he stood toward his wife now that his guilt was discovered."
I wonder if this could be applied to success of frameworks or programming languages. (Of course, you'd have to define "success" as something relatively binary which might be hard to do.)
I must use this quote at least once a month with various clients, in an attempt to awaken them to the source and solution for their delivery performance problems
aka the weakest link: because a chain breaks if any link breaks. (Being more mechanically-minded, I prefer this less literate metaphor.)
I haven't read AK; from context, is that really what Tolstoy meant? Without context, the well-known quote could be read as happy families are alike in that they are happy, since, in practice, happy families are not identical.
Or rather, I have a idea of starting a MOOP company - Massive Open Online Psychology.
By monitoring a household, listening to conversations (interruptions, time to respond, aggression etc) and even vision (physical contact etc) it should be possible to see where we all are on some scale of "good interpersonal relations"
Reason: the easiest framework that allows modeling "the resulting random variable R is zero if any of the incoming variables vₙ is zero" is the product formulation,
R = Πₙ vₙ
which by taking the logarithm of the individual variables becomes a standard central-limit-theorem prediction; as a result success in such contexts follows a log-normal distribution[1], which is long-tailed, even if the individual factors fₙ = log vₙ do not look very normal.
This then leads to a growth pattern called Gibrat’s Law[2].
[1]: https://en.wikipedia.org/wiki/Log-normal_distribution [2]: https://en.wikipedia.org/wiki/Gibrat%27s_law