...everything is you and you aren't aware of the vast majority of it.
However, this seems exceedingly counterproductive definition of the word "you" to me. I would rather define not me as that which I am not directly aware of. Or at the very least something which isn't integral part of that which is my core consciousness. Me is that which is close to me, not me is that which is far from me.
This is a kind of quantifiable quantity. I can think of a N number of thing in t amount of time I can't expect to be able to do at arbitrary time of my choosing. However, I cannot think of any thoughts I could not think of (kind of obviously). Of course I might still be limited by the lack imagination when it comes to my own thoughts, or even be only a product of a deterministic processes and as such possess no "true" imagination, but never the less I experience the feeling that I can think of any thoughts I can imagine at any time I desire.
Additionally, I can see my consciousness existing perfectly well without experiencing any dreams. Similarly, I can see it existing perfectly well without other people. However, I cannot see it existing without my thoughts and experiences. Even though my existence might in the end become empty and meaningless should all the people vanish from the world and should I be unable to find meaning in solitary endeavours like exploring the remaining universe let alone the case where no universe would be left to experience anymore and I would only be stuck in perfect void with only my limited memories.
It seems trivial to me there is a world external to me, it's simply the way language is used. To claim otherwise would appear to me only as dysfunction and incoherent semantics counterproductive to the purpose of language.
It also seems trivial to me that we cannot have absolute knowledge of anything short of our own existence and we shouldn't expect to. This is simply the nature of information. All knowledge itself seems to be subject to this same kind of continuum of distance as this internal and external world discussion. Nothing particularly surprising, unknown or problematic in my opinion.
To certain extent there doesn't seem to be any point in trying to draw a well defined line between individual and the world as everything interacts with everything. The wave function of a single electron covers the whole universe. It is still reasonable to discuss degrees of separation, isolation, distance, time etc as near and far is not equal. This is true even if it is a discussion of physics or simply thought.
It is my impression that this sort of discussion would be a good starting point for analyzing consciousness as well. I'm not convinced philosophical zombie for example is a meaningful concept, or if it is then we should perhaps all consider ourselves philosophical zombies. For example if there is a mechanistical explanation to human mind which perfectly explains all observations then there doesn't seem to be any room left for unknown factors. If some unknown "essence" would be added on top of the equation then we would still necessarily be reduced only to prisoners in our own bodies.
However, that is already much more subtle point than my point about solipsism. Solipsism I would consider trivial and unproblematic aspect of human experience whereas I might still be open to the idea that there might still be some fundamental questions left about consciousness.