They're out there. The Venn diagram of people still choosing IRC (as opposed to being forced to use it b/c that's where the community is) is probably just a circle.
I was a big XMPP user back in the day, but because of the lack of multi-device message syncing and the really shoddy state of encryption, I wandered away. Plus, using XML for the protocol really geeked me out. XML is a document format, and per the spec, to be well-formed it needs to have an open and matching close tag. Jabber hacked around this by making a sort of infinite document - you get the open tag, but never the close tag - and it just felt really icky.
I understand a lot of these things have since been addressed. I don't know if XMPP still uses that bastardized version of quasi-XML without a close tag. But other things have come along that I like more. About 6 months ago I started running a client on my desktop again, but like you, nobody I knew was still using it, and nobody new was advertising it as their connection info, so... yeah. After a few months, I stopped running the client.