can you recommend a setup with ollama and a cli tool? Do you know if I need a licence for Claude if I only use my own local LLM?
can you recommend a setup with ollama and a cli tool? Do you know if I need a licence for Claude if I only use my own local LLM?
You must try GLM4.7 and KimiK2.5 !
I also highly suggest OpenCode. You'll get the same Claude Code vibe.
If your computer is not beefy enough to run them locally, Synthetic is a bless when it comes to providing these models, their team is responsive, no downtime or any issue for the last 6 months.
Full list of models provided : https://dev.synthetic.new/docs/api/models
Referal link if you're interested in trying it for free, and discount for the first month : https://synthetic.new/?referral=kwjqga9QYoUgpZV
What are your needs/constraints (hardware constraints definitely a big one)?
The one I mentioned called continue.dev [1] is easy to try out and see if it meets your needs.
Hitting local models with it should be very easy (it calls APIs at a specific port)
[1] - https://github.com/continuedev/continue
I've also made decent experiences with continue, at least for autocomplete. The UI wants you to set up an account, but you can just ignore that and configure ollama in the config file
For a full claude code replacement I'd go with opencode instead, but good models for that are something you run in your company's basement, not at home
we recently added a `launch` command to Ollama, so you can set up tools like Claude Code easily: https://ollama.com/blog/launch
tldr; `ollama launch claude`
glm-4.7-flash is a nice local model for this sort of thing if you have a machine that can run it
I have been using glm-4.7 a bunch today and it’s actually pretty good.
I set up a bot on 4claw and although it’s kinda slow, it took twenty minutes to load 3 subs and 5 posts from each then comment on interesting ones.
It actually managed to correctly use the api via curl though at one point it got a little stuck as it didn’t escape its json.
I’m going to run it for a few days but very impressed so for for such a small model.