LocalLLaMA @sh.itjust.works fubarx @lemmy.world 1w ago I built a private Al mini-cluster with Framework Desktop
I'm new to this, but some advice seems to be use vulkan to get stuff to work. in your spare time one day, look at rocm.
I appreciate that! I'm just trying to recap the authors results for people that might be interested in the bottom line.It seems that the beowulf architecture still has too much overhead
TLDW: Tried huge open models and got 0.7 tokens/s. Seems clustering tools aren't ready yet.
I'm new to this, but some advice seems to be use vulkan to get stuff to work. in your spare time one day, look at rocm.
I appreciate that! I'm just trying to recap the authors results for people that might be interested in the bottom line.
It seems that the beowulf architecture still has too much overhead