Every word that you can think of for people with low intelligence is charged -- stupid, idiotic, foolish. Everyone nods in approval when a senator intones that people with third-world skills will have to be content with third-world wages. Unsaid is the notion that stupid people should leave the world for the rest of us.
It is perfectly all right to make fun of stupid people. Think about how easy it is for you to say that you can not stand fools. You could not say the same thing about fat people for example. Yet, intelligence is at least as genetically determined as obesity (far more in my opinion).
There is nothing wrong with choosing the best people for a given task. However, when society starts to align itself in such a way that non-smart people have no recourse, it is unfair. It will be a sad world, a world with only alphas and betas.