Almost all sites that have comments and user moderation concentrate on the comments and up voting / down voting them. Normally, when dealing with people, I don't remember the individual quote that made me think they were brilliant / a troll, I wrap that up into my sense of them. If I'm flipping channels and see someone who has struck me as a brilliant commentator, I stop based on the visual cue of their face. Names are kind of hard (is this the guy who called me a $%$% or was he the one who really knows python?).

I guess I wonder if the same thing keeps happening on "comment moderation" sites, isn't it time to look at the ways your view could be based on your (not the group's) opinion of your fellow commentors? I don't have a technical suggestion, but I will probably think a lot on it.