What happens when an instance starts needing more resources than its original server?
What happens when an instance starts needing more resources than its original server?
Is there a way that users could cohost an instance? If there isn't, I think that would be a great feature to implement. Having a situation where instances slow down or even occasionally disconnect will be hard on any community. I feel like it could eventually cause people to prefer using instances owned by large and well funded groups instead of passionate users.
I think this is going outside the realm of self-hosting and moreso into actually creating a server architecture. All servers would need to use the same database, so you'd want likely as its own server a database server, caches on the front-end servers so popular things aren't queried for the same info again and again.
I've never set up anything like this, so this is just me trying to think of how I'd throw it together, I'm sure there's a bunch of async problems I've not even considered how to tackle, and even having the DB be offsite from either of the front-end servers would be less than ideal.
I suppose you could have the DB in one of the servers, but then that one now has the same frontend-load as the other while it also is the only one doing DB queries, so the load's not really being distributed properly. 🫠
6I think storing the DB in multiple places is the most important part of this. There could even be different levels of cohosting. You would ideally always have at least two servers hosting the full DB going all the way back in time. Those would be the most important to implement just to have an always available service. Then you could have people that might not have enough resources to hold everything just host all of the most recent and high-demand data. Some advanced server architecture will for sure be needed if this model is going to succeed. It just isn't possible to do everything on a single server. As far as processing load from running the server, I can't imagine that will be much of a problem compared to actually storing the data. I am sure a lot of this is all problems that have already been solved and old solutions could be used. Maybe something similar to the torrenting system, with just a few controller servers.
2
At that point it's easier to run your own instance, I guess? As it stands now, it's not trivial to scale neither the db storage (postgres) nor the backend (lemmy).
I don’t think that users’ donating parts of compute is a way to go, honestly. You’ll have to think about bad actors (such users will effectively be instance admins for some subset of data), and it might quickly deteriorate into the weird crypto world (aka let's use blockchain as a storage because no one can be trusted to really count the upvotes).
Unfortunately, it's a very tricky issue to solve. I'd say donating to your instance so that its operators have enough finances to support it is the way.
2... creating our own capability to make cloud computing infrastructure basically...? Yea that'd be kinda cool. Would piss Amazon off though, we might have our hands full at the moment. Drawing the scrutiny of another tech giant this early might be ... unwise.
1Amazon is definitely not the only one doing cloud infrastructure. I really don't think we'd be pissing anyone off.
3Probably, but it depends. Only a fool would ignore an easy opportunity to prevent the entry of a new competitor. It's not like we're some big company with real monetary power of our own. We're easy to strangle in about a million different ways, just more tech dweeb stuff that only internet losers should like. Or so it could be easily spun.
1