I've started uploading quants of exllama v2 models, taking requests
https:// huggingface.co /bartowski I've started uploading quants of exllama v2 models, taking requests
Finally got a nice script going that automates most of the process. Uploads will all be same format, with each bit per weight going into its own branch.
the first two I did don't have great READMEs but the rest will look like this one: https://huggingface.co/bartowski/Mistral-7B-claude-chat-exl2
Also taking recommendations on anything you want to see included in readme or quant levels
0
comments