I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what I get up to see what sort of correlations it finds. Give a coding agent access to a data lake of events and let it build up its own analytics tooling to extract and draw out information from that data, and present it to me as daily/weekly/monthly summaries.
That's definitely doable. Planning similar except more webscraping / newsfeed / monitoring like.
I've got 3x SBCs that can run the Gemma 4 26B MoE on NPU. Around 4W extra power, 3 tokens a second...so that can hammer away at tasks 24/7 without moving the needle on electricity bill
I wonder if some investment firms are already doing this internally at a large scale. (Probably.)
They are - I’ve seen it.
They just use APIs though. There is very little interest within them to do the model engineering and inference in house.
This was along my lines of thinking at one point as well. Though I'm now more interested in having it experiment autonomously on my software projects overnight.