3 The logical picture of the facts is the thought.
3.001 “An atomic fact is thinkable”—means: we can imagine it.
3.01 The totality of true thoughts is a picture of the world.
3.02 The thought contains the possibility of the state of affairs which it thinks.
What is thinkable is also possible.
3.03 We cannot think anything unlogical, for otherwise we should have to think unlogically.
3.031 It used to be said that God could create everything, except what was contrary to the laws of logic. The truth is, we could not say of an “unlogical” world how it would look.
3.032 To present in language anything which “contradicts logic” is as impossible as in geometry to present by its co-ordinates a figure which contradicts the laws of space; or to give the co-ordinates of a point which does not exist.
3.0321 We could present spatially an atomic fact which contradicted the laws of physics, but not one which contradicted the laws of geometry.
3.04 An a priori true thought would be one whose possibility guaranteed its truth.
3.05 We could only know a priori that a thought is true if its truth was to be recognized from the thought itself (without an object of comparison).
Here, he connects the thought with the logical picture.
Therefore, from the above, it may become apparent how Wittgenstein defines the logical, logic: anything we can think of, anything that is conceivable. If, for example, we can think of a flying unicorn, then this flying unicorn is logical, being an image [3, 3.001]. If, however, this image we formed with our mind, with our thought, is true - meaning it corresponds to reality - then we say that this image is a part of the world, not just an image, but an image
of the world [3.01]. Anything we can think of is logical because in our thought lies the possibility of what we are thinking, regardless of whether it is simultaneously true; there is, according to Wittgenstein, a dimension between the possible and the true: for anything we have the ability to form an image, that is a logical possibility, part of the logic of the world [3.02], while the truth of this image is something else, unrelated to logic (which he discusses later). [Although not so unrelated in the end, since the image was formed through logic].
Therefore, we cannot think of anything illogical because then we would have to think illogically [3.03]. Consequently, what we call "illogical" does not exist; everything is logical, as possibilities of our thought. And a "non-logical" or "illogical" world is something we cannot even conceive. So the propositions of Tractatus 3.03 and 3.031, "It was once said that God can create everything except what would be contrary to the laws of logic. We could not say, therefore, what an 'illogical' world would look like," are essentially tautologies, as he will later say that all logical propositions are, propositions of logic. Why is it a tautology? Because he says that anything that appears logical to us is also logical. And that even God, as we imagine Him, would have to submit to the laws of logic, as they appear to us.
But there is also a problem when we say that something - a proposition - is a tautology because from this tautological proposition, no information can be extracted about the world. So, if all possible logical propositions are indeed tautologies, this tells us nothing about what is logical, what logic is, and how it is defined. If, for example, in response to the question "what is logical?" the answer is "everything is logical" - since we cannot think illogically - then what conclusion can one draw about logic? No conclusion can be drawn, except one: that logic cannot be defined through logic itself, and if we want, for some reason [why?], to find out what it is, then we must turn elsewhere, to other means.
Anyway, the Tractatus does not directly deal with the definition of logic, although it makes some useful observations about it; nor does it talk about what is true. Instead, it provides the theory needed for one to be able to properly understand what is being said to them, as well as what they themselves are saying, when they speak clearly, and when they make mistakes, whether they are in or out of tune, like with music. I think that what he's saying is equivalent to music theory, but now applied to the theory of language/logic. Challenging things, for sure, but perhaps with a little help from our friends, we can make some progress.
What would you think if I sang out of tune,
Would you stand up and walk out on me?
Lend me your ears and I'll sing you a song,
And I'll try not to sing out of key.
However, the fact that we can think of something does not necessarily mean that this something is true; therefore, a comparison between the image we formed with our thought and reality will be needed to determine its truth. The possibility of what we are thinking falls under logic [3.02, 3.03].