I think what's strange about this problem is that the setup makes human beings helpless before the implacable necessity of mathematics, and that's the wrong story to tell.
In so many cases, it is the use of mathematics that enables us to identify problems, clarify them, and solve them. And there is immense creativity here ― which is why I gestured at the invention of rational numbers.
In real life, a case like this is more likely to play out this way: you've got these 23 thingamabobs, and there's talk of splitting them three ways. You say, "Won't work," and someone less numerate than you says, "Well, let's just try." As they fail, with a puzzled look, they say, "Wait, I messed up somewhere. Let me start over." You will want to explain to them that it's impossible, because 23 is not only not a multiple of 3, it's a frickin' prime.
What's of primary interest here is that you, because of your relative expertise in mathematics, understand the situation better than the person who, even after trying and failing several times, still believes it might be possible.
But I'll address what the philosophers want to say. First, of course this scenario only makes sense given the relative durability of the thingamabobs across the sort of time scales we're interested in, and they have to be such that we can reliably distinguish them and count them. Our faculties must persist too. On and on. Absolutely there are prosaic physical qualities of the situation assumed.
All of those physical factors are also presumed in the case where we have 24 thingamabobs. In that case, if the divider-up failed,
you would be the one to say, "You must have messed up, because it definitely
can be done."
There's a disjunction in there right? In the first case, with 23, it was "Either I messed up, or it's impossible"; in the second, with 24, it's "Either it's impossible, or I messed up." Same thing, but with a different expectation. For the first sort of situation, it is sometimes much easier to determine that the task was impossible, than to confirm that no mistake was made in any attempt. In the second case we rely, again, on it being easier to confirm the possibility (of evenly dividing 24 by 3) than to figure out where you went wrong. Same thing again!
But because we short-circuit the disjunction differently, we're actually using it in slightly different ways. In the first case, you're discharged of responsibility for your performance because the task is impossible ― for all we know or care, you did mess up, but that's not
why you failed. In the second case, we know it's
not impossible, so you must have messed up; here we do judge your performance, and your mistake
is why you failed. (We need a little more here actually: some guarantee that an algorithm exists, some cap on its complexity, our ability to implement it, and so on. You might still be off the hook.)
There are several options we pass by in such reasoning: we say, it's impossible, thus you needn't or shouldn't try, not that you cannot try or must not try; we say, it's possible, thus you can succeed and maybe ought to succeed, not that you must succeed.
Now come back to "why". Given 23 thingamabobs, does mathematics guarantee failure? No. It guarantees only the conditional, if you try then you will fail. (As Simpson noted, "Can't win, don't try.") Given 24 thingamabobs, does mathematics guarantee success? No, of course not, not even if you try.
So mathematics cannot compel you to succeed or to fail, but it does play a role in how we judge performance.