100 MB is indeed small but there are privacy concerns with storing hundreds of thousands of lines of history. Do you really want every command you've typed for the last several years to be visible to anyone with physical access to your box? It could contain clear-text passwords to other machines.
I'd be happy with 100 MB as a maximum size but there should also be time-based limits, like remove anything from history that is older than a few months.
Do you really want every command you've typed for the last several years
Yes.
visible to anyone with physical access to your box
Maybe not. That's an auth problem, not a reason to throw away your archives. I keep all my email even though there's probably something in there embarrassing to me given some audience.
Just FYI, You can hide sensitive commands (like those that require you to type your password in plain!) in bash, by simply adding an space at the beginning (after adding `export HISTCONTROL=ignorespace` in your bash_profile, of course). I don't know about fish yet.
I'd be happy with 100 MB as a maximum size but there should also be time-based limits, like remove anything from history that is older than a few months.