Note that I used the word 'freedom'. I have no idea what a 'right' might be except for things that might be established by some kind of law and I would assume that making a law would not make something either right or wrong.
So in the simple sense of 'freedom', if I sit somewhere on a public beach, I 'infringe someone else's freedom' to sit in the same place. If I don't sit there, they would be free to sit there. Because I do, they aren't and so their freedom is infringed.
A 'right' in this context may simply be taken to be an asset/privilage that cannot be confiscated/abridged without justification. And of course, just because something is a law doesn't make it morally correct.
Someone else doesn't have the freedom to forcibly remove you from that place, just as someone else doesn't have the freedom to steal your property.
In this case, since the place in question is not the property of this individual, they have no more right to it than you do, and therefore to remove you from it constitutes an unjustified use of force.
Their freedom is limited by your being in this place, but their freedom is also limited by their inability to, for example, take your property without your consent. Their freedom (I use 'freedom' here in the general sense) is 'infringed,' but their right is not as they have no claim to that spot on the beach, as it's not their property.
Simply put, their freedom is limited, as freedom in the general sense is necessarily limited, but their rights are not infringed.