Not wanting to date a trans person is fine, no one is owed a date. The way it is often expressed can be transphobic and the reasoning for the preference is sometimes based on transphobic assumptions.

Nobody is saying you have to date someone, just that you shouldn't be an asshole about it and invalidate them or the people who would date them. Just look at the places in this thread where a lesbian who would date a trans woman was told she must be bi.