It's certainly common these days to treat set theory as fundamental, and for kids to learn naïve set theory, and I agree that's useful. But you didn't learn ZFC in elementary school and weren't taught anything about alternative axiomatizations or independence.
What were you taught then? A lot of the mathematics people learn in classrooms is definitions and techniques. This is what we mean when we say .., this is how it works, this is what you do. Questions about whether those definitions, which support those techniques, are "good" just don't arise. And that continues to be true for much of mathematics.
It's one of the curiosities of set theory that now and then people do worry about whether the axioms are "good", not just in having the usual mathematical virtues of being powerful enough to do the job but not more powerful than needed, but in the sense of "natural". The axioms are supposed to be like Euclid's old axioms, just spelling out our intuitions clearly. Of course there was a massive failure there relatively early on with the axiom of comprehension and Russell's paradox. We teach kids you can make a subset of "all the blue ones", but we don't tell them there are rough waters ahead if they think they can always do that sort of thing.
Maybe this is what I'm trying to say: children are not actually being taught foundations, and not even really being taught set theory as they are taught other mathematics --- definitions and techniques. What they're being taught is an
application of something they already know, that things can be grouped together, and they can be grouped together according to rules. In order to apply this basic intuition, it gets tidied up and even formalized a bit (though not much at this stage). But the idea is that sets are not introduced the way, say, tangents are later: here's the definition, it's just a thing, and we promise it'll turn out to be interesting. They're expected to nearly understand sets already, but not to realize just how much they can do with them.
That last part -- what you can do with sets -- might turn out to be all of mathematics, but not in practice, not by a long shot. No one proves theorems starting from ZFC, and certainly no one does calculations that way. There's a sense in which the difference between calculus before the development of set theory and after is just a change in notation.
I guess the question that's left is something like this: does our ability to express all of mathematics in the notation of set theory mean that set theory
is the foundation of mathematics? Both answers to that are tempting, but perhaps that's because it's a bad question. There is no single thing that is set theory, in that sense; there are various competing ways of axiomatizing our intuitions, all of which are adequate to doing mathematics (and you actually need less than ZFC I believe to do most math).
One last example: having later learned about cartesian coordinates, you can readily think of a line as a set of points defined by a linear equation, an infinite set. But that's not how you were taught what a line is; you were taught that it's "straight". When you learn that y = mx + b produces a line, that feels like a result, not a definition, because you already know what a line is, just as you already knew what sets are.
On the one hand, I think I agree with Turing about contradictions mattering, but on the other hand it does seem clear to me that practice and intuition is the foundation of theory not the other way around, and you don't really need the theory, even when it comes to mathematics, insofar as foundations counts as the theory, to practice. Which is not to say that it can't be helpful. Maybe it's just that mathematics makes it clear there are at least two approaches to theorizing: one to justify what you're already doing, but one that is expected to feed back into practice. A whole lot of mathematicians do the latter without ever bothering about the former, starting from when, as tots, they learn about sets for the latter reason much more than the former.