Do you need a domain name if you are hosting a Lemmy instance, or will it work fine with just an ip-address + port (e.g. <username>@<ip-address>:<port>)?
It requires an HTTPS connection, and certificates can only be obtained for domain names. So yes, pretty much.
Also consider than IPs can change, even if you're using a hosting provider. Domain names makes changing the IP much easier.
Domain names can be obtained for as cheap as $3/year for the xyz TLD. If you can't pay for anything, there's also free services that can let you get a subdomain, like noip.com, afraid.org, azote.org.
The reason being that federation means other instances send you things. It's not pull-only, or else you could likely get away with private instances sitting behind NAT. But since activitypub involves publishing to inboxes from source to destination, they need some way to reach you. And since we want to validate that connection and that some external authority can vouch for ita ownership, we use TLS Certs with the DNS hostname that matches your server name.
SSL certificates for IP addresses are possible; but they require you to outright own the IP(-range). Some large organizations do. So for individuals it's rather unheard of, but it's technically achievable.
Well, I just learned something, but what does "control" the IP mean? If they are only validating a single address via http then presumably you could just use an Amazon elastic IP as long as it resolves. I doubt that letsencrypt will support that but I would be interested to know. If they do then yeah, you could presumably set up the instance using the IP as the name, but I don't know why you would want to. Apart from the fact that it would be hard to remember, could change at some point, screwing things up, it might work. I suggest OP do the necessary and report back accordingly.