No, let's start again:
Everything around us that we can see, the trees, the earth, the stars, the sun; they are all physical things. They were all caused by a cause. I can't think of a physical thing which has not been caused by something else, so I infer that all physical things must have a cause.
Because all physical things have a cause, there would have to be an infinite regress of causes, since "everything physical has a cause". However, there would not be a point at which we could say "...And this was the first physical thing that there ever was and which did not have a cause" because I have already inferred that all physical things have a cause. So having an infinite regress has not solved the problem. You could keep asking "...And what caused that cause?"
Yet there must be something that started off the chain. As I have demonstrated above, it cannot be anything physical, since all physical things have been caused themselves. Therefore, I infer the existence of an extra-physical dimension. The cause must be found in this. And the cause that is found in this dimension must be uncaused.
Questions?
In the first part I bolded, you are implying that the property of an element is also the property of a set of those elements.
You are jumping from: "every physical event has a cause" to "the set of all physical events has a cause" as if such set MUST be a physical event itself.
Such a big assumption requires a hypothesis because it's not inferred from anything.
It would be like saying: since every natural number is limited, then the set of all natural numbers MUST be limited.
Now, let's take for granted that the set of all physical events is also a physical event itself.
Then, yes, it must have a non-physical cause.
But you pulled out of nowhere that such a non-physical cause MUST be uncaused.
Why MUST it be uncaused, exactly?
Even if we assume that the "set of all physical events" is a physical event (and, thus, requires a cause)... You cannot logically say that "the set of all physical events + some non-physical events" is a physical event at all, so you cannot infer that there is NOT an infinite regression of non-physical causes.
And that doesn't work not even if you claim that "the set of all physical events and non-physical events is a physical event" because, then, you would have that "all physical events + their supposed uncaused cause" is a physical event itself. And it would be uncaused. Which contradicts hypothesis #1 that all physical events have a cause.
The very thing you highlighted in bold explains why the set of all physical things must be caused.
Because the physical universe must be caused by an uncaused cause in order for the current existence of anything physical to make sense, I hypothesize that there is an extra-physical uncaused cause.