Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I didn't know they were strained. Probably best to block them so they don't spend resources on unimportant things. You can find a few UA strings here https://udger.com/resources/ua-list/bot-detail?bot=archive.o... Ultimately, it looks like they do respect the `archive.org_bot` string so if you want to help the strained organization, probably best to block it as well.

You can also save them some space, it appears, by uploading a file on your website asking them not to retain it and then sending an email to them. See https://webmasters.stackexchange.com/a/128352

For now, I just blocked the bot, but when I have some time maybe I'll ask them to delete the data.

Hopefully, with a little community effort we can reduce the strain on them so they can spend their limited resources reasonably.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: