Currently, any man could enter the women's competition in most sports and in this way, take a space that should belong to a woman. There's your extra barrier. There's nothing "extra scary" required. Just any random man who'd rather compete in the women's than the men's.