What do you mean, you don't? You're not trying are you?
But what exactly does this mean? Male and female bodies differ mainly in proportion. Ignoring the fact that the average height for women is something like four inches less than it is for men, the ideal shape for a male body is a 'V' shape, and the ideal for a female body is Jessica Rabbit - or something along those lines. We can ignore differences in height, because few people would claim a six foot tall model is unattractive because she is significantly higher than most women. Indeed, being tall is quite often appealing.
So, typically, men have broad shoulders and get progressively narrower as they reach the ground. However much added flesh it may be hidden with, the male human skeleton follows this rule fairly closely. This is why when buying clothes, TVs usually have to buy a top that is a size bigger than the bottom. In addition, the female upper torso is proportionately shorter than the male. This means that, compared to a woman of similar height, a man will have longer arms and a taller chest, but shorter legs. Don't dispair though - because the average male is taller than the average female, he usually has legs that are reasonably long (hence the claim that TVs 'always have gorgeous legs'), but also has an over-tall chest (boo, hiss).
The claimed ideal for a woman is to have a chest and hip size that roughly match (hips being a couple of inches larger), and a waist size ten inches less than that. I say claimed because this is a very old ideal that was already hard to attain and has become harder as the average shape of women has altered during the last century. Diet, health and a whole load of other factors have made it unusual for a woman to be 36-26-38, but certainly the visually oriented male brain still finds proportions like these attractive.
The other main differences centre around protuberances. No, not those, but male joints and the adams apple basically tend to stick out. Women of course don't even have an adams apple and also have much smoother knees, ankles and elbows which means they should swim faster and have a higher terminal velocity if thrown out of a 'plane.
I'm afraid I can't work out facial differences. Maybe someone out there can help on these?