Nefarious? no. Said in ignorance of the subtext? yea.
Forget the fact that you, yourself personally were making an off-hand comment, do you believe comments like that can carry the subtext I see in it?
Personally I see it (not you specifically being some evil villian trying to impose beauty ideals on all men so no need for the eye rolling defence mode) as the sort of comments that are used to keep everyone adhering to the mainstream ideal.
It's not really any different from someone saying "as far as I know, any woman who is a real woman, wants to stay at home and be a mum". Meaning women who don't want to, well they are not real women. And straight men who don't fancy latinas (which is an ideal in my face all the time...one I have used to my advantage

) aren't proper straight men.
Again, not saying this is what motivated your comment or your wording, but if you can't examine your wording and only fall back on a personal defence mode, what real progress is any of this thread going to make?
The way I see it is if someone wants to interpret what I said in a way I didn't mean it, that's on them, not me. If someone is gonna make their decision on who they find attractive based on some offhand comment from a guy on the internet because they think they need to do it to be a "real man"-- well, I can't fuckin help that nor do I see it as my responsibility to censor my words so as to avoid having it misinterpreted/misapplied by impressionable people.
Is it so hard for people to step back and ask themselves whether they feed the fire? whether they can think about off-hand comments and where they (potentially) lead to?
Again, I do not regard this as an ethical responsibility, at least not in this case. I made an offhand observation, without malice or prejudice, if someone chooses to interpret that in a negative way that is on them solely.
I will listen to any potential arguments you may have for why this would be my ethical responsibility, but right now, I don't see it.