Who are the 1%? It's also a psychological issue. Despite diminishing marginal utility, why do billionaires spend so much of their time on work? Yes, they may not work hard as a slave on a fishing boat off the coast of SouthEast Asia, but they do spend many hours a day on work. Why bother running your company and attending Wall Street meetings? Why not just permanently retire after you accumulate your first 50 million in the bank and enjoy the rest of your decades on Earth on your hobbies or something? For most citizens who work because they want to satisfy a certain level of consumption and leisure, it's difficult to fathom the pointless but enormous amount of time and energy spent on making more money.
The reason is that once you're that rich, accumulation of money becomes about status competition. Conspicuous consumption is a kind of addiction. Being very rich is not just bad for everyone else, it's also bad for rich people themselves. Being very rich is inherently a bad thing for human beings. It’s something that society needs to eliminate, just like a disease.
I think the economic externalities to the rest of society is the main reason why we shouldn't allow concentrated wealth to exist, but I think it's a worthwhile additional critique.