Infinity It's also fine to have a philosophical stance that there are no abstract objects. But being true to that stance then requires eschewing even everyday locutions about mathematics and everyday thinking about many things. For that matter, at least for me, the use of language in thought and communications is to provide frameworks for dealing with so-called concrete experience, not merely to remark that one observes the so-called concretes.
Mathematics does not pretend to be isomorphic with all the concrete objects and particles of science. Nor that mathematics is a factual report about concretes. Rather, mathematics provides an idealized framework that we can choose to use in different ways, including providing an axiomatization for the formulas we do use for the sciences. Mathematics is an armature for knowledge about concretes; it is not supposed to be itself a report of those concretes. The armature is not itself the things you put in it.
And the way I understand - other mileages may vary - frameworks, whether mathematical, philosophical or conceptual in any field of study, is that they should facilitate fluid thinking and communicating, and to avoid, if possible, having to stretch oneself in contortions such as having to grasp for convoluted expressions to avoid saying the word 'object' in an utterly natural way when talking about things such as numbers, or to have to eschew the economy of conceptualizing numbers as things rather than to commit to imagining that a number is born and dies, off and on and off and on, every time someone thinks of it and then stops thinking of it or that there even is no 'it' they are thinking of but only physical events in a brain, or that, wait, what is the notion of 'event' anyway without abstraction?
On the other hand, if one wants to try to think of mathematics and formulate it and communicate it but without reference to abstractions or abstract objects, I say have at it. But that doesn't make everybody else wrong for thinking of numbers as things and saying such ordinary things as "the sum of two and two is four". And especially classical mathematics is not crippled by the mere wish of a crank, without a concrete proposed alternative, that there is an unannounced, unarticulated physicalist replacement.
I find it a crude notion that each mathematical mention must correspond to represent each, every and any of the concretes and particles that are themselves present to us mentally as constructs in a conceptual framework. A framework is not an assertion, and its value is being able to conceptually and/or practically cope with or predict experience. Such frameworks may be preferred or not in how well they conceptually and/or practically cope with or predict experience, but also in the satisfaction derived from the conceptual order and beauty they provide. When I am confused by too many facts all at once, or about, for example, how things work, I am relieved of that confusion by a framework that allows me to put that experience in order, to process it. I may be confused by the behaviors of other people, for example. But then I may posit such things as traits, goals, etc. I don't posit that those are concrete things. They are abstractions, they are posited as a conceptual armature so that a person's actions don't appear to me as a random jumble but rather my framework allows me to think of those actions in a narrative and to make predictions about them on that basis. When I there are of numbers mentioned, I don't have to think of them as popping in and out of existence each time they are mentioned or not, but rather I have an armature in which numbers don't do that. And when I there are a lot of numbers involving some problem, either conceptual or practical, that I want to solve, I have a system of principles about numbers that allows me to find the answers I want. That system is an abstract armature, not a concrete thing. It's not required that each concept, each abstraction itself corresponds to a particular concrete.
Meanwhile, maybe there is a way, but I don't know of it, to avoid that thought and language themselves presuppose that 'object', 'thing', 'entity', 'is', 'exists', etc. are basic and that explication of them cannot be done without invoking them anyway. When I say "What is that thing in the sink?" I presuppose even the concept, which itself is an abstraction, that there are things, that concretes are things, and even the notion of 'concrete' is an abstraction. And I don't see anyone who can talk about experience ('experience' also an abstraction) without eventually invoking utter abstractions such as 'object' and 'is', whether referring to abstractions or concretes.