It's an unspoken assumption that goes unchallenged because it's easier for people targeted by these stereotypes to distance themselves from "nerds" than to challenge these assumptions.