It's a community treasure hunting game. A person hides something, then posts GPS coordinates and/or clues to lead you there. Often the hidden item is a sealed container so people can leave log books or unique trinkets. Some of the trinkets have serial numbers you can use to see where they've traveled.
I haven't done it in years, but I remember it being a lot of fun. Hunting for caches led me to a bunch of interesting new places that I would never have explored otherwise.
Yeah, it's a shame when a container is abandoned and essentially becomes trash -- in my experience, it's rare for that to happen, though.
On the other hand, there's 'Cache In Trash Out' events (CITOs): Geocaching events where people gather to pick up litter, remove invasive species, plant trees, or otherwise help the environment. They're pretty popluar in my local area. So I wouldn't say geocaching has a net negative impact as far as littering goes!