Does everything have a start? The problem with asking for something to have a start is that the argument might always be circular: there will always be a need for a "prime mover", at least with much of how we understand the world now. People can argue space-time as an emergent phenomona, quantum fluctuations in the void, but none of that ever really solves the problem of explaining how those things themselves came to be. Some will say this is beyond the limits of our understanding, and perhaps that is the correct approach.
However, there are other views as well: existence may very well be a brute fact, without beginning, without end, an eternal wheel. In that sense, perhaps Time itself is the one thing that requires no start (though this does take a very simple view of Time or Change as being something which exists independently of any observer).
However, a contrary view is that worldly existence may also be an illusion, in which no Time or causality exists in-itself, but is just a by-product of the complexity of the present reality we eternally exist in (e.g., block theory of time, boltzmann brain). In this view, the past and future are not real, time does not actually exist, and we exist in a static universe where nothing ever happens because time does not actually exist, we are just deceived into believing it does by how our mind is constructed in this reality of the "present moment" in which we are stuck like a broken record that does not realize it is playing the same moment over and over. If you remove time, you eliminate the need for a start, for causality, because if time does not exist, nothing ever passes or comes into existence, everything just is as it always has been and always will be. You need not ask for a "how" or a "start" in a timeless universe since without time, nothing ever happens, things just exist. There is no need for someone to put them there, for you would need time in order for that to happen and time never was, nor will it ever be, a feature of noumenal reality in this theory.
Having said that, I still find a timeless theory a bitter pill to swallow and would like to believe our experience of time has some validity but I think many assumptions we have of basic reality, especially related to quantum reality and specifically, entanglement, will need to be better understood in order to pave a new theory that will make everyone go "ah-hah! How could I have not seen that before?".
The answer may very well be out there, I just think we are not there quite yet (unless of course the timeless interpretation is real and we are stuck in Plato's cave admiring the shadows).