Why are More Deaths Worse Than One? (Against Taurek) Thanks, everyone, for your input. This discussion has been helpful.
If anyone is interested, I find the linked paper by Tom Dougherty to be a rather persuasive paper on why the numbers should count.
It has been a common belief for many of us that because of the moral indifference of the situation (that is, that all individuals are equally needy), it seems only rational then to look beyond the loss-to-persons and toward the loss-of-persons.
This paper, in my opinion, does a good job of analyzing our intuitions towards saving the greater number without appealing to consequentialism. Ultimately, Dougherty looks at our rational want to satisfy as many of our altruistic ends as possible. He believes as moral beings, we have a certain moral attitude to want to do so. His paper then claims that we ought to satisfy as many of these ends as possible if we are to act rationally.
This is a rather short and incomplete summary of his argument so I suggest reading the paper! It's a quick read!
https://philpapers.org/archive/DOURNA.pdf