I've been comparing against TextGen and llama.cpp while I port to LocalAI and have been surprised by what's happening over the API, even with the defaults and jinja. It's been a fair reminder not to eschew familiarizing myself with the repos.
I've been comparing against TextGen and llama.cpp while I port to LocalAI and have been surprised by what's happening over the API, even with the defaults and jinja. It's been a fair reminder not to eschew familiarizing myself with the repos.
Nothing beats the classic of figuring out something yourself with your brain, but I also like dictating to LLMs a stream of consciousness with what I'm interested in (while forcing it to NOT give any answers or opinions), and getting back file names it suggests I look at and explore.
Modern frontier LLMs can still be used as rubber ducks, and it's a great.