I figure that's up to the voters. Personally I upvote things I disagree with all the time, provided the comment presents a compelling argument worth considering. As for downvotes, it's never due to disagreement for me - only in times I find a comment to be unnecessarily negative, personal, or the opposite of "a compelling argument worth considering".
I don't care if I agree with someone. I just want them to provide more information and add some value to the discussion. If the information is invalid, I just do what I can to provide counter resources.
I also generally try to approach downvotes the way you're saying as well, though that brings us back to downvotes being "mini flags". I don't know that I disagree with that.
Mostly just bringing up that upvotes and downvotes aren't really defined. They represent the general unspoken opinion of the people voting, which I think is fine.
AFAIK upvoting a story is the only way to save it in your profile. I believe that this 'bookmarking' side-effect can induce a good measure of the value of a comment as well (one would have to kill the personal comments history page for maximum effect, though).
