I wonder when/if we’ll see Gemini beating Fable and Astra. Last year I would have confidently bet Google will overtake the others just because they have the data, the hardware (TPUs) and a fat advertising money pipe and yet they are still behind. Anyone anonymous at Google want to hint when Gemini 4 will be out?
As an "everyday mans AI" I'd say 3.8 Flash definitely already has. Smart enough for the vast swath of people, and only slightly eeked out by Astra(Max) on vision capabilities, like the kind of "Point your camera at something and ask questions" that non-tech people like to do. It's crazy fast and very compute light, so not getting bogged down constantly.
I can't think of a better general purpose model than 3.8 flash right now. It also writes more naturally than the other big models too.
I find it really good and up-to-date, like within an hour of current events or website updates that it can reference. Astra is a step up for complex stuff, but I'm not going to use up all my tokens to ask it the specs of a 2023 MacBook pro then wait 3 minutes only to get a tome with a complete breakdown of the Mac OS platform and including such things as the supply side dynamics driving the ram size choices and pricing at that time.
I was thinking about coding specifically. Also, see all these math and physics breakthroughs, it's usually not Gemini but Fable and Astra.
I think 3.7 Flash is very good at coding. I have access to both that and Opus 5. Opus is only slightly better IMO and it's much more frustrating to read.
I think it's good at basic coding and is cheaper and faster. Trying it vs Fable 5.1 I don't see it being even close capability-wise. What I mean by basic coding is "write a script to parse this data" and answering some questions based on the code. It does that well and answers fast. However, when it comes to planning and thinking through a more complicated design problem it's just not there yet.
It's not even close. Some benchmarks show 3.8 Flash as being close to Astra (DeepSWE v1.1), but they are mostly "bench-maxed". Newer benchmarks like Terminal-Bench 4.0 show Astra at 57.9% and 3.8 Flash at 19.1%. For anyone who has tried to code with Gemini models, there is no contest here.
Astra wins handily on every other benchmark, too. Research, science, 3D, visual, Humanity's Last Exam, etc.
Not only that, the output of tps is high. So not only can it handle most mid-level tasks perfectly fine, but can do it with speed!
Yeah its good. Reasonably priced too (at current prices, if they do raise them in January I would stop recommending it). 3.7/3.8 were good releases.
To me it's a good replacement for search engines. I ask it things like 'If redshifting destroys energy ala Noether, than how can we say that time is reversible or that entropy will find an equilibrium?' and it will not only explain, but make nice interactive diagram/toys to help. A regular search engine would have taken me hours to find an answer.
However, if I want it to DO something then Gemini is in absolute last place. I don't trust it for anything more than renaming files that I don't care about very much or extracting data (though it's too expensive for data extraction at scale).
I’ve mapped my iPhone Action Button directly in to Google App AI Mode (which is Gemini 3.8 but faster inference than the Gemini App, due to harness)
It’s for this exact kind of scenario where a random question pops in to my head.
Plus, it’s the most grounded by real live data of all the chatbots.
Silicon Valley people are majorly sleeping on Google Search AI Mode.
Yeah +1 to this
People use text with LLMs but it's great to have a high fidelity "analyze this image"
I’m wondering if they even see a coding agent as a valuable prize. It’s a competitive market in a race to the bottom economically, hard to establish consistent differentiation and virtually zero switching cost for customers.
I think they’ve made a shrewd move in focusing on search integration and everyday users (Gemini app) vs software power users. They have their corner and nobody is really competing with them, plus it feeds directly into their existing revenue stream.
OpenAI/Anthropic are being valued at 25/50% of Alphabet respectively in secondary markets.
Maybe everybody is wrong about how valuable these companies will be, but atm it looks very dumb to not be competitive in coding.
Google is not being dumb or making some 4D strategic move by not going after coding capabilities.
Occam's Razor is overwhelmingly that they just don't have the organisational capability to capture this market. If they did then they absolutely would have.
> I wonder when/if we’ll see Gemini beating Fable and Astra
Google has essentially give up the race for the frontier. The gap is widening very fast. Their strategy appears to be to focus on niche areas like voice, small models for on-device (see their deal with Apple), and specialised models for tasks like search, which are very inference efficient.
Anecdata, but I've been using gemini personally instead of what I used regular google searches for the better part. Even in car chat/search and some lighter and not so light research (if it's not heavy on technicals). It's good enough that I use it nonstop like that. I wouldn't trust it for coding at all - switching between fable and now astra.
Google in a sense won (me over) like that. I also expected them to brute force their way into everything and dominate. This is how it played out though. Image generation is great as well, but ChatGPT one is more lenient on copyright and nannying - for example when my kid asks me to "take a photo of him and Sonic". Gemini cops out either because of the kid or Sonic, disappointing us both, but ChatGPT can be.. persuaded.
I just use Grok for anything "controversial" like that. I actually love that different LLMs have their niches. In a typical week I might use all of Claude, Gemini, and Grok, for different kinds of asks.
Considering how far backwards Google has gone since the release of the 3.x models, the brain drain they've let happen, the lacklustre software ecosystem and their track record with product management ... I would say they're more likely to drop trying to compete at the frontier and try to focus on something else instead.
For me 3.8 has been good enough that I don’t think I’ll be extending my Claude subscription.
For coding models? I don't think Google is motivated to fight in that market. There's no incentive for them.
Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models? They would just be competition.
Famously, Google just eventually discards almost all businesses that don't have the same fire hose of revenue that ads does. Selling coding plans isn't something they are going to want to do.
Google is clearly motivated to make better search and information finding tools and stuff that will ultimately drive users through their existing search/ads/youtube ecosystem. That's really why they're in Android, that's why they do Chrome. Everything else with them is a sideshow.
Google is also full of beancounters obsessed with data centre quota and resourcing. Even massively profitable ads projects have to justify and fight for it. (Source: used to work there).
I can't think of anything less resource & revenue sensible than providing outside parties access to your TPUs for the purpose of letting them write stuff which could just end up competing with you.
Yes, maybe as part of their cloud business, selling token access could be useful money. But I doubt they'd tune it for coding.
> Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models? They would just be competition.
Bragging rights to say they have a SOTA model. I guess that was more like Google of 10 years ago with moonshot projects. Nowadays, yeah, perhaps if it's not helping sell ads, it doesn't make sense.
Google did it AlphaGo, AlphaFold etc for bragging rights, now let frontier lab do their thing.
It would be utter stupidity if they keep competing with hyper agile frontier labs. They sensibly moving to big infra provider and that's good niche.
> .. perhaps if it's not helping sell ads,
Perhaps cloud business is also a thing which is making big revenue
You say "keep competing"... the hyper agile frontier labs and Google/MSFT are practically one and the same via circular financing arrangements. Competition on the frontier is kinda a farce, what matters is who comes out of the modern-day Darien scheme as a winner.
At worst Google just need to keep the lights on until the frontier labs file for bankruptcy under the weight of that >$1tn debt they've built up paying for FAANG's data centers. Then Google walks away from all of this with the the best remaining models, best R&D lab, and several hundred billion dollars in infrastructure on their balance sheet.
> Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models?
Catch is, even Googlers internally do not have access to top-tier models. (or did not until recently, when apparently Claude was made accessible to the SWEs internally).
Googlers now have internal access to frontier models.
Funny enough, after trying it, I went back to G3.8
Yes, agreed, 3.8 with thinking is quite good.
Also: Google has like a 20% stake in Anthropic, and a very fat cloud partnership
I have a shared understanding for what you're asking, but semantic versioning doesn't even make sense for AI models
these are essentially monthly releases, the dated releases that Deepseek and Qwen do make more sense
If Google didnt have their ad buisness theiy'd be out by now. They're like BlackBerry and Nokia at this point almost.
Not sure what your comment mean in the context of parent's comment.
As opposed to what, them not having it and burning money that isn't their instead like openai and anthropic? At least Google is feeding itself instead of having to create a bubble to stay alive
Google spent most of their cash, they are now taking loans for data centers too. They recorded their first quarter of negative cash flows ever
Google has in excess of $121 billion of cash (& equivalents), net of total debt.
Big rich companies take on debt for reasons that are sometimes inscrutable from the outside. Recently, they have been borrowing for ~5%, about a half point above what the US government gets for 10-year Treasuries.
Apple has been financing operations with debt for a number of years as part of a complex optimization plan.
No, Google is not broke.
There are a lot of accounting shenanigans going on in Big Ai, financial reports are misleading, eg Meta "building" a data center through a shell company and then renting it to themselves. It does wonders for the looks of the books
Some expert wall street analysts discussing what they found and how they dissect things, have a healthy skepticism of Big Ai
https://www.youtube.com/watch?v=YrJzjC4kKCY
In a quarter with record breaking income of over $100 billion, they also had negative $5.9 billion in free cash flow from their "bank account", because the people selling them chips and building them data centers want to be paid, while the investment itself is depreciated over time.
If my quick search is correct, Google is sitting on a quarter trillion dollars in cash and marketable securities. They could keep doing the negative cash flow thing at this scale for another decade.
Have you seen Google's balance sheet?
Which is still a better position that the others? My point is you can't consider that a bad thing if you think it's ok for their competitors in the field. And if you don't and your judge them equally, then at least Google has its own cash glow and could turn the gas off at any point to go back to printing money while they have not choice.
And YouTube, and Cloud, and Play Store, and Waymo, not to mention that they could coast on their Anthropic and SpaceX stakes if they didn't have any of the above.
Maps, Waymo, TPUs, YouTube, Docs, GMail, Cloud, Android, Chrome, Photos...
They remind me of Kodak inventing the digital camera and sitting on it to preserve their film business.