What I was getting at in my previous posts is that the real "you" exists outside of physical reality. It is very incoherent to assume that very uniform process of biological development would produce this singularity.
...
Let me ask you the exact same question, but with a slight difference.
You have a computer. It has a bunch of programs you installed, a bunch of files you created or downloaded, and all the settings are set up to be the way you want them to be. That is *your* computer, among the entire universe of *not your* computers.
Let's look at the moment in time 2 hours before you booted your computer for the very first time.
At this point the Universe has only been producing *not-your* computers and the Earth is populated completely by *not-your* computers. Now what is it in the physical/biological process that warrants that in 2 hours *your* computer will be booted and running for the first time? Why doesn't the Universe just continue producing *not-your* computers as it always has done. Can you imagine the computer you call yours will be born in 2 hours and it will be just another *not-your* computer with its own software and personalized settings?
^^^ That is how I read your question. And the answer is, it actually is the body, and my experiences through, it that make me *me*. Nothing more. Moreover, the *you* and *not you* property isn't all that drastic or binary. My arm is me. If it gets severed, it is no longer me. For disabled people, their wheelchairs and prosthetics are very much a part of *them*. If I have a cancerous growth, it's *me* but not really, because I don't want it to be me, and it's genetic code isn't exactly me. The only *you* that matters is the physical brain, just as the only *your computer* that matters is the physical location of 0s and 1s that store your files and settings and make your computer yours.
The stuff you're talking about is secondary. You can only call something *yours* (your computer, your arm, your body, your brain) when you have *you* in the first place.
You argue that a particular experience is what makes you *you*, but the truth is *you* is the one who receives the experience, who lives through it. The real *you* is a thing in itself, all the stuff that you are trying to attach to it is just that - an attachment. This includes your body, your memory, your experience, even your complete personality.
Just sit quietly and try to imagine a different past for yourself, imagine you made some important decision in the past in a different way. Your experience of life and your memory of it to this point would be different, but you would still be *you* and you know that. It's empiric, it's part of observable Universe and any theory, simple or otherwise, would need to cope with that at some point.
I've given away all the keys now, it's up to a curious and persisting mind to play with them and see which one opens the door.