Are you saying that working with a very skilled person is worth sacrificing your ethical integrity? I would not hesitate for a second to deny working on things I consider against my personal beliefs even if there was both money and fame at stake.
To use the good old reductio ad absurdum, would you be willing to kill a random stranger if it meant you would get to work with Albert Einstein?
You'd have missed out on the opportunity to work with Feynman if you'd declined work at the Manhattan Project.