I've been working on a more unixy version of his tool I call llcat. Composable, stateless, agnostic, and generic:
https://github.com/day50-dev/llcat
It might help things get closer..
It's under 2 days old and it's already really fundamentally changing how I do things.
Also for edge running look into the LFM 2.5 class of models: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct
I love this concept. Looks great, I will definitely check it out.
Please use it and give me feedback. I'm going to give a lightning talk on it tonight at sfvlug