Depending on the public restrooms, I tend to wash my hands before touching things downstairs. I shower and I launder; I don't use my junk to touch everything through the day like I do my hands. As far as I'm concerned, downstairs is one of the cleanest parts of my body and I like to keep it that way, so washing hands after the fact is a near redundant action for my personal hygiene and a social courtesy for others'.
I have no idea where that bathroom attendant has been, which is why I always use a manservant to tuck me back in and zip me up. It's the only way to be sure.