myersguy @ myersguy @lemmy.simpl.website

Posts

2

Comments

269

Joined

2 yr. ago

I've seen this happening a lot, so I would assume there are some issues federating comments.

Wow, that is surprisingly not bad given the size of the instance!

Interesting indeed. I spent some time combing through the DB and pictrs folder to try to figure it out, but I'm at a loss so far.

Again, my knowledge of the Lemmy codebase is very small, and we could possibly host the monolith in microservices style. The point I am making is this (when it comes to scaling monolith vs microservices):

If the federation logic were split out, we could configure it to run on super tiny docker instances on Google Cloud or AWS. Any time we needed it to, it would autoscale to handle the traffic. The configuration for these dockers could be super minimal memory, no storage, and multiple weak CPU cores. This would be super affordable while still being able to handle as much traffic from federation as we ask it to. One of the cool things with Google Cloud Run is that it handles load balancing between docker containers for you (just point the federation traffic at the necessary URL)

IF Lemmy has things like background services, scheduled tasks, etc, this would significantly muddy the water (we would need each service to be able to handle being run on a multitude of instances, or we would need to be able to disable each one instance by instance). And if we just scaled by spinning up more instances of Lemmy, we would also need to ensure that only federation traffic is heading to the weaker instances that we spun up for such purpose, or we would need to ensure that each spun up instance has enough resources to handle federation traffic along with the main application.

I feel like I need to state once more: I don't necessarily think Lemmy needs to move to microservices. Only that scaling monolith vs Microservices is not necessarily the same.

I'd be less concerned with memory (of which Lemmy seems to use very little), and much more concerned with CPU core count. I touched on it in my other comment, but I don't understand how a few cores is supposed to handle the ridiculous number of federation workers people are setting their instances to.

Well, I'm going to start by repeating that I don't necessarily agree that it being monolithic is necessarily a problem right now.

The immediate thought in my mind would be all of the federation logic. That's where all of the instances seem to be lagging behind, and it seems the common fix is "just increase the workers to one billion". Apparently that does something meaningful, but the developer in me wants to know how a few cores can put so many workers to use.

Spinning federation off into a microservice means you could deploy it on something like Cloud Run or AWS ECS, and have it autoscale as the workload demands it. Seems like a pretty prime candidate to me.

I'm going to have to disagree. I just remoted in to check it out. I have 2560x1440 resolution images weighing 2.9 MiB in there. It's pretty clear that these are full size images.

I'm not so sure this is the case. The images folder on my single user instance is over 1gb, and I haven't posted any images.

I disagree that it being a monolith is immediately a problem, but also

In fact you scale a monolith the same way you scale micro services.

This is just not true. With microservices, it is easy to scale out individual services to multiple instances as demand requires them. Hosting a fleet of entire Lemmy instances is far more expensive than just small slices of it that may require the additional processing power.

Yes, sex is a lot like sweeteners in that respect.

Abstinence is the safest way to have sex.

I understand you are in other arguments in this thread. This wasn't meant to be one of them. I just got a kick out of you using "10 years", "nowadays" etc for a game that is 20 years old.

It's the same energy as these kinds of memes

We’re 2024 already?

So we are. That kind of makes the 10 year statement a little more silly, no?

Half Life 2 is just shy of being 20 years old.

Anyways, I enjoyed how old you made me feel with this comment 😅

if you have been gaming 10 years and only play AAA.

Half Life 2: A 2013 classic.

I was super confused thinking those tweets were in response to this news story.

This. No diss to Endeavour, but Arch is just as easy using Archinstall