Is it possible to make WoL unicast work indefinitely?

I am trying to set up my homelab to boot whenever it is being accessed. I set up wake on LAN for unicast, and it works fine for a while after shutting down the system. It stops working after a few hours of the server being turned off, though.

From what I'm gathering online, the issue is that my router uses an ARP cache. Before that gets flushed, it still works. However, it seems like my router does not allow me to configure a static ARP entry (using a ISP provided router). I already set a static IP for my server in the router, but it still won't work a few hours.

Is there any way to make this work? I just want to allow a friend of mine to boot my server whenever they try to access a service on it.