Not necessarily. I have no v4 on my desktop and I'm not hitting a lot of issues. It only becomes an issue if you don't implement some form of backwards compatibility.

I occasionally find websites that are broken and don't load, but that only happens rarely and it's not exactly a problem that's exclusive to v6.