IIRC, it was because the tiller for the rudder (with which you steer) was on the right side. (Styrbord in Swedish = steer board) As to why you have port in English, I have no idea. It's babord in Swedish, from bakbord (back board) as when steering, the left side was behind you.
Edit: Apparently it's port because you'd dock with the left side to the port as otherwise you'd crush the rudder, which again was on the right.
My grandfather was a bomber pilot and he always remembered it by picturing himself flying north along the east coast, where all the ports would be on the left.
Still doesn't seem necessary. Your orientation doesn't matter if you always take the direction as if you were in the drivers seat (captains chair). It's the same with cars, left side is drivers side, right side is passenger (unless you're in one of those backwards countries, then left would be passenger, right would be driver, but it's still the same side of the vehicle).
Historically, boats had rudders (or “steer boards”) along the side of the ship instead of in the middle like we’d see them today. It was always on the right side of the boat, so to avoid smashing your rudder into the dock, you’d dock your ship on the other side. That means it was always the left side that literally faced the port, while starboard faced out into open water to protect that side from damage.
Also, port is red and Starboard is green. Usually boats will colour the sheets to match. It helps to identify the lines when someone tells you to ease the port sheet.
I remember being young taking my boaters safety course and having to take this section over three times. Why does left and right stop existing once you are on a floating pos?
Because left and right can be interpreted as direction from your point of view. Port and starboard are always understood as left and right of the vessel you are currently in.
In performance arts, stage left and stage right perform the same function as well. "Stage" directions are always from the perspective of a performer, so that referring to right/left is absolute.