Yeah, this sounds like as much of an issue as the legal risk to me. Imagine if you reject someone and they blog about the letter you sent them and the interview process, and the post makes Hacker News or Reddit.

Now you have to decide whether to fight in public with someone you didn't hire, normally bad form, or say nothing.