• Hi guest! As you can see, the new Wizard Forums has been revived, and we are glad to have you visiting our site! However, it would be really helpful, both to you and us, if you registered on our website! Registering allows you to see all posts, and make posts yourself, which would be great if you could share your knowledge and opinions with us! You could also make posts to ask questions!

Koboldcpp can run AI Large Language Models Locally

KjEno186

KjEno186

Site Staff
Staff member
Jr. Sentinel
Joined
Apr 9, 2022
Messages
891
Reaction score
2,380
Awards
11
Koboldcpp will run llama3 and is backward compatible to llama1. Version 1.66 is just out today. It is easy to install on Windows, Linux, and MacOS. I simply download the Linux executable, put it in my /home directory, give it permission to run (in user space), and launch it via a .desktop file I made. It's likely similar in Windows. You just download the single .exe file, put it somewhere convenient, and click to run it. You can even compile the program yourself because it is open-source software. Aside from versions which enable NVidia CUDA acceleration, there is a branch for AMD (ROCm), Vulcan compute, and CPU-only modes of operation. Find tons of GGUF models on Hugging Face.

Please, Log in or Register to view URLs content!

And a very active Discord community:

Please, Log in or Register to view URLs content!

(Discord link will expire within 7 days of this initial post.)
 
You must log in or register to reply here.
Top