How is rejection and not commenting any different from not commenting? It sounds like you want to be excused from the social contract (of saying thank you for a bad gift) while not being accountable for your behavior (removing your name from the gift via rejection).

Sorry, but that is in fact the kind of behavior that, if most people did it, would make exchanges impossible. Which is why it is rude.