My first query:
> Make it a little warmer in here.
The reply:
> "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.",
Maybe I'm doing it wrong?
Tested your example, the confidence is 0. In smart home context, I can think of an application where the low confidence answers can be forwarded to cloud, whereas the vast majority generic queries solved locally, if the confidence is reliable enough. The response is quite fast by the way.
But this isn’t a query that should need to be forwarded to the cloud for acting on!
I don't know, the back of my napkin says you'll need consensus of at least three servers in order to complete the command.
It is a 14MB model. You don’t need servers to run it. Check your napkin well.
I am sorely dissappointed that the state of the internet has become such that my sarcasm was not only possibly but actually missed.
Mine too! I was not serious. I was just answering your napkin joke. But when I reread now, it is not that obvious. I guess with text you can pass only so much of emotions.
Considering this is a 14MB model running almost in a microcontroller, I am fine with such ‘ambigiuous’ queries cannot be handled, as long as the model confidence score is accurate. By the way, I did not test this model thoroughly. I am speculating on the potential of a small model like this. I don’t know if this one is good enough or not.
Try asking it to set the thermostat to a value. It's a very small model, there's not much reasoning capability
Language models were supposed to be more flexible than Siri.
Otherwise why bother with an LM? 14MB can fit a lot of regexes for fixed sentences.
It's not a conversational model. It's meant as a local tool calling model.
Yes, I think OP understands that. What he and many others in this thread are trying to understand is what makes this model useful.
Turning a voice command into a tool call should be self evidently useful, being able to do that on a small embedded device is the novelty here. In this theoretical example, the thermostat is hosting the model on device and would use no external services.
I confused by the dispatch model. Tool calls typically need some reasonability to be deterministic and, more importantly, predictable in response (o/w GIGO). Why would the thermostat need to interpret a voice command that the node capturing the voice command would not interpret instead?
The node capturing the voice command could be the thermostat. From my understanding they are targeting very small devices.
So that could be a master home automation node, but why not also a single purpose device? I can think of more bad examples than I can good ones, but maybe I am doing some soldering and I need my soldering iron turned up a bit; my hands are full, so doing that by voice would be useful enough. Something I would never link up to a big AI model or home automation network, but could be useful to control by voice.
If it's something that can be burnt directly into a chip and shipped with the products for cheap, maybe that's a more pragmatic way to get AI into small devices (see taalas for a much bigger model doing that, althoug not yet cheap).
Oh I agree that's not unreasonable. I wonder about the harnessing heft required to make it feasible though. If I say I'd like it a bit warmer, a tool can deterministically bump a few degrees while an LLM might bump it 10C. So do we limit the tool's range?
That's exactly what they were testing and what it failed at
Yes, and it seems the tool called here was the thermostat, to set the mode literally the opposite of what was requested.
"Make it warmer"
tool that cools room called
but wouldn't it need to know the current temp for that to work? my understanding was that the model is best at tool calling
In what world would this make any sense at any temperature?
> 'warmer' implies need for cooling
"It has gotten warmer" could imply a desire to return to homeostasis.
Now you are just making stuff up. The original query was "Make it a little warmer in here". Just because you can come up with some sort of context where the word warmer could mean that you want it cooler does not mean it fits in this scenario which is exactly what people are using LLMs for - to actually understand that context, if it can't do that it's worthless