A problem with open borders is the vast majority of competent people leaving their impoverished countries.
Those countries need competent people the most.
Also seeing some terrible results from mass immigration in Sweden right now due to not being prepared to take on this many immigrants.
Our system isn't made to deal with it, and I can imagine that problem arising again for anyone else with open borders if a nearby impoverished country experiences some sort of horror. (ISIS, tsunami, floods, war, etc.)