Generally you dont have to change much besides the Presets and GPU Layers. Launching with no command line arguments displays a GUI containing a subset of configurable settings.If you're not on windows, then run the script KoboldCpp.py after compiling the libraries. exe, and then connect with Kobold or Kobold Lite. To run, execute koboldcpp.exe or drag and drop your quantized ggml_model.bin file onto the.Weights are not included, you can use the official llama.cpp quantize.exe to generate them from your official weight files (or download them from other places such as TheBloke's Huggingface.You can also rebuild it yourself with the provided makefiles and scripts. Windows binaries are provided in the form of koboldcpp.exe, which is a pyinstaller wrapper for a few.It's a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios and everything Kobold and Kobold Lite have to offer. KoboldCpp is an easy-to-use AI text-generation software for GGML models.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |