I worked at Demon Internet in the late 1990s, and our “homepages” service ran on thttpd. My colleagues had hacked an earlier version to support mass virtual hosting - there were tens of thousands of homepages web sites at a time when that was a lot.
On the front end we had a few cacheing reverse proxies running Squid on FreeBSD, like an early CDN (we had nodes in London and New York). The FreeBSD kernel was hacked to support configuring IP addresses by the CIDR block instead of individually. When Homepages was set up, the HTTP Host: header was not yet well supported, so it needed an IP address per web site. The Squid boxes had 96k addresses each (a /16 and two /18s) and the network was configured to spread the load across them.
Squid was modified to translate the server IP address into a Host header as necessary, so the back end did not need many IP addresses. There was one box running thttpd, one box running an FTP server for uploads, and a NetApp NFS storage server. There were hacks in thttpd to shard the web site names across multiple directories to avoid performance problems with very large directories.
We made some effort to get the modifications upstream, tho thttpd had diverged a lot by the time we sent our code to Jef. But, IIRC thttpd’s vhost setup has some resemblance to the way Demon did things. (This is also why Apache httpd mod_vhost_alias is so weird.) And these days, firewalls can be configured to do the CIDR IP address trick, so there is no need for kernel hacking.
That’s a sad story. Mental illness is such a tragedy. The author clearly has a great deal of technical depth shining through which makes it all the more tragic.
This is becoming a scary story thread. This reads like a threat that could actually be carried out. It could end up like the Ebay/EcommerceBytes case or worse.
I guess it's funny in retrospect. I sell my own software and have to deal with these "confidently incorrect" people all too often. Seems like they are either mentally ill or have some kind of sociopathic personality. I'm thinking about quitting the game because of it.
I guess the "smart" thing to do would be to pay someone to deal with these people, but I really don't want to subject anyone else to the abuse. Not to mention that having another person on the payroll shifts the calculus of whether or not it is even worth it.
I thought I'd get used to it. But, after 10 years, it still makes my fight or flight response kick in. Every time.
At least I get paid to take the abuse. I can't imagine having to deal with it for an open source project. I guess it explains why so many open source project maintainers seem so testy.
Thanks for letting me air my grievances. I feel a little better :)
Oh, I have the SysAdmin style of humor because you either laugh or cry. I've sold software and experienced that fun angle to customers. My favorite is still the customer who got confused and blamed me for another vendor's software. It was an interesting e-mail chain. Being a SysAdmin for a number of years has lead to me seeing some of the worst in people.
I used to think these people are mentally ill or sociopaths too, but I think I have come to a much simpler explanation. Steve Jobs was correct about computers being a bicycle for the mind, but I think we need to look at the negative part of that statement too. Computers just emphasis what is already there and a cost of the strengthening / speed is the removal of societal filters.
We grow up learning a bunch of societal filters that we use to take our raw thoughts and make them acceptable to the world around us. Most people get this right. At our hearts, most of us are good people and don't want to cause harm.
Sadly, most people don't really write e-mail, tweets, or posts to another person. They write it to some image in their head where the basic societal filters are no longer operating. Communication is dehumanized and you get the raw person. Sadly, many of our fellow humans are not very pleasant without those learned filters. In fact, they are rather primitive scum. Add to that the voices that say you don't have to be nice or even be humane to "them" for some value of "them" that is not "us" or "me". I really wish there were some fix, but I guess its AH from here on out.
The scale goes from “less moving parts” to “more focused”; the optimal point varies, but I don’t think you can get away from the fact that those are in opposition. Of course, the usual vernacular meaning of “more focused” is “does less things I don’t need”, but that’s not unrelated: because everyone needs a different subset of things, the more things a given piece of software needs the larger, on average, the proportion of those that you don’t need; so running more pieces of (less feature-rich) software seems necessary in order for having less stuff you don’t need or understand overall.
(In the specific case of TLS, I get additional warm fuzzies from being sure that, however screwed up the web server is, it cannot be confused[1] into revealing the secret keys when it does not have access to them in the first place. I don’t know to which extent this is actually important, though. A factotum[2]-like approach is a compromise that gives the web server the ephemeral keys in exchange for not having to pass the entirety of the traffic through the terminator, but I’m not aware of any practical implementations except for the one Akamai, disgustingly, patented[3] 10 years after the actual invention.)
> warm fuzzies from being sure that, however screwed up the web server is, it cannot be confused[1] into revealing the secret keys when it does not have access to them in the first place.
On the front end we had a few cacheing reverse proxies running Squid on FreeBSD, like an early CDN (we had nodes in London and New York). The FreeBSD kernel was hacked to support configuring IP addresses by the CIDR block instead of individually. When Homepages was set up, the HTTP Host: header was not yet well supported, so it needed an IP address per web site. The Squid boxes had 96k addresses each (a /16 and two /18s) and the network was configured to spread the load across them.
Squid was modified to translate the server IP address into a Host header as necessary, so the back end did not need many IP addresses. There was one box running thttpd, one box running an FTP server for uploads, and a NetApp NFS storage server. There were hacks in thttpd to shard the web site names across multiple directories to avoid performance problems with very large directories.
We made some effort to get the modifications upstream, tho thttpd had diverged a lot by the time we sent our code to Jef. But, IIRC thttpd’s vhost setup has some resemblance to the way Demon did things. (This is also why Apache httpd mod_vhost_alias is so weird.) And these days, firewalls can be configured to do the CIDR IP address trick, so there is no need for kernel hacking.
(edited to add) Here's an old blog post on the topic, with the links I left out of this comment https://dotat.at/@/2012-09-25-large-scale-ip-based-virtual-h...