What would you do if you were me?

I know this is kind of stupid and everything and I'm overreacting.

But,whatever.
On Friday I was heading to my Language Arts class,1st Period.
We don't have assigned seats in that class but everybody sits in the same spot they always do so it kind of seems like assigned seats.
Anyways,I go to my locker every morning and wait to walk with my friends to that class,so we're usually the last people there.

There was a guy sitting in my seat and I shrugged at my friends as I walked over to him and asked if the teacher had changed his seat.
He said No and I told him I usually sit there so I wanted to know why he moved and if I could sit where I usually sit.
He brushed me off and yelled at my to go sit over at his old seat.
Class was going to start so I went over to that seat,but I was completely distracted since he sits in the back of the class by all the annoying kids who are distracting.
The seat I used to be in that he is in now was in the front of the class with people who weren't distracting and I learned a lot up in the front.

So my question is,what should I do,or if you were in this situation what would you do?

I don't know if I should just suck it up or sit back in the class or try to get earlier to class and get my seat back,but yeah..

The teacher noticed that we switched.
She also scolded him and said he might go back to his old seat if he doesn't stop talking to the person behind him. So I'm unsure if I should wait for that to happen.
I think my teacher thinks that we agreed to switch maybe?

Thanks everyone for answering! <3