These are small enough that you can run them entirely in the browser https://huggingface.co/spaces/webml-community/ternary-bonsai...
Remember to clear the downloaded weights afterward.
Like the last model, it's amazing they work as well as they do. Use it for any longer task and they fall apart spectacularly and in interesting ways.
So you have some fun examples?
Sadly crashing on Pixel 9 Pro, but I guess phone GPU with 16GB RAM total wouldnt be enough anyway.
To be fair, the GPU in pixel 9 and 10 is pretty bad.
I really hope they stop using PowerVR in pixel 11.
3/16 GiB of RAM on the P9Pro is dedicated towards on-device AI models, which WebGPU probably can't access.
Runs on my 16GB M2 Air (firefox). ~7 tok/s
It should be though.
Even native AI gallery uses smaller Gemma models. I guess whatever Chrome is using for WebGPU compute on Android is just adding too much overhead.