Alpaca Simplifies Running Advanced AI Language Models on Linux
Alpaca Simplifies Running Advanced AI Language Models on Linux
Also recommended (I use the Flatpak version): GPT4All
- Website: https://www.nomic.ai/gpt4all
- Source: https://github.com/nomic-ai/gpt4all
- Flatpak: https://flathub.org/apps/io.gpt4all.gpt4all
And no, this has nothing to do with ChatGPT. It can download different AI models from HuggingFace and run them on CPU or GPU.
19I actually found GPT4ALL through looking into Kompute (Vulkan Compute), and it led me to question why anyone would bother with ROCm or OpenCL at all.
11I run models like Stable Diffusion and Llama with ROCm but models like RealESRGAN for upscaling or Rife for interpolation with Tencents Vulkan thingy (forgot what it's called) and that's far easier. Would be cool if LLMs and stuff could just be run with Vulkan too.
5OpenCL is needed for me for non AI stuff, so that Darktable (an image program) can use my GPU; which is much faster. But for AI? No idea how they compare, as I did not use it for that purpose. ROCm itself also is troubling...
Do you have the new Llama 3.1 8B Instruct 128k model? It's quite slow on my GPU (I have a weak beginner class GPU with 8GB, but plan to upgrade). To the point its almost as slow as my CPU. I've read complains in the Github tracker from others too and wonder if its an issue with AMD cards. BTW the previous model Llama 3.0 8B Instruct is miles faster.
2
Bookmarking it to check this out after work
... I should really go through these bookmarks one day
4I have a separate "ToDo" bookmark folder with temporary content, that I want to look in the near future. And for things I am looking into in near future, the pages are already in the browser open as tabs and loaded everytime I start the browser (but in an unloaded state until I click it).
... I also should really go through these bookmarks and tabs one day.^^
3My bookmarks are competing with my unplayed steam library.
3
I'm happy with Open WebUI
3While downloading models, the progress bar is getting decreased sometimes, like from 11% it'll go back to 10%. Wired.
2Literally the average experience working with ML tools. All of it feels so hacked together and barely functional.
2
I've used LM Studio for a while now. It's pretty good!
2