Refugees Immigration is always a scary proposition because it upsets the democratic balance, especially when the immigrants have differing ideologies. The response from the current residents is usually aimed at preserving that balance, either through limiting the immigration or enforcing some sort of assimilation mechanism, like language, dress, or behavior enforcement. There's no question that each side misstates the consequences, with one side screaming the world is coming to an end and the other side screaming it's just racism. The truth is that the significance of the immigration will depend upon the actual number and how prevalent they'll be in the specific area where you live. There's no question that concentrated areas of immigrants have changed the socioeconomic and political structure of certain areas, leading to migrations within the country.
If you are the US, with a population exceeding 100 million where mobility is fairly easy, and a history of immigration since its founding, it might be easier to deal with than in smaller, more homogenous countries.
A significant part of the OP relates to the overstatement of the number of refugees, which seems to tacitly accept the fact that the raw number of immigrants is relevant in terms of maintaining a certain way of life. That is, it concedes (at least to some extent) that immigrants are bad, but then offers consolation by saying that the nation is big enough to absorb it, so stop worrying.
Let's simply admit that not all immigrants offer something positive for a nation and that a nation is not being racist or nationalistic when it protects its borders. A basic tenant of being a sovereign nation is that it be able to protect its borders. There is no reason to be apologetic about that as long as the prosperity of the nation is truly being considered, as opposed to simply eliminating people based upon ethnicity or another irrelevant consideration.
The Netherlands, for example, is more prosperous, more fair, more progressive, and more enlightened than pre-ISIS Syria, and it has every reason not to want many of those Syrian values carried over by the immigrants made part of the fabric of the Netherlands. That's not racist or nationalistic. That's just trying to protect the good thing you have. You will control the fears of the current residents by accepting that some of their concerns are valid and offering them assurance (through the passage of laws if necessary) that they and their children will be able to continue to live in the same comfortable, safe environment. Telling them that they're over-reacting will only identify you as a political opponent who they will feel motivated to weaken in the next political election.