I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought
One line of evolution seems to be toward some form of latent-space reasoning, as in [1]. Natural language seems like a relatively low-bandwidth channel for intermediate reasoning.
[1] https://github.com/sapientinc/HRM-Text
What does ‘latent’ mean in this context?
"Latent" means it (the space) is there all the time but usually not exposed or apparent. Latent space is a high-dimensional space that is not legible to humans, so normally reasoning gets constrained to progress in tokens instead so that it can be evaluated.
I think https://arxiv.org/abs/2412.06769 is a good description of the premise of reasoning in latent space, although https://arxiv.org/abs/2604.15726 argues it's already what really happens.
So, does "reasoning in latent space" mean that it does/will not use tokens (e.g. using CoT etc.)?
It needs to argue with itself to extract most of the knowledge embedded in the weights into the context. Asking it to synthesize ideas directly in a single go is simply unreasonable. And MoE models need to walk multiple experts to extract all the knowledge on top of that. So you need to give them the reasoning trace to first spill all the associations into.
It “thinks out loud” to populate its token space. Asking it to shortcut risks truncating that process.
Yeah. It's "thinking" in absurd massive vectors. It needs to assess a couple to weigh out. That's the compression. That's the nature. It looks ridiculous when thinking traces render out such simple statements ('reassessing ..') but I expect this is far deeper an assessment than it can fully reflect to us on, and I expect its a huge part of their thinking.
I was pretty happy with Depseek Pro in Opencode util I discovered I can see the thinking trace by clicking on the "thinking..." communicate. All those seemingly unnecessary "but wait" messages are frustrating to read. But at least to some extent it's just model taking time thinking through the problem, and the trace produced doesn't have to be representative of what happens internally: https://arxiv.org/abs/2404.15758
It is MoE. It needs to engage multiple experts when the problem is complex or unclear. So you naturally see more of those simply as a primitive it learns to use to page in more diverse set of weights. Remember that each token is just 6 experts out of 256. So it literally needs to tell its router that it needs a different set the next time.
And this memory control primitive leaks into the reasoning chain, because it has no other channel for it available and we do not know how to train any other channel.
On the flip side, it tends to converge quickly, roughly proportional to the actual difficulty / clarity of the task.
So-called "caveman" thinking attempts to address this.
The important part of "actually wait, I really need to XYZ" is just "XYZ".
The model can attend to just "do XYZ" and produce almost the same vector modifications as full verbose "reasoning".
I don't think that's true. If a context contains a statement followed by something opposing that statement, that will confuse the model. So "actually wait, I really need to" is there to signal that the previous thinking may be flawed and that what follows is a new attempt.
It's good to remember that LLMs have no more state then what they can derive from the context up til any point. So if that context is hard to interpret, that will reduce effectiveness.
As long as the model itself understands that when it says "foo bar" in its own reasoning trace it means "pursue theory foo; no, this didn't lead anywhere, let's backtrack and pursue theory bar", where's the loss in clarity?
I hated it at first too...
Now though I'm considering all the hidden "thinking" in the models layers that happens for each token output. It is a wild amount of waste! We just can't see it.
This kind of stupid excessive computation is fundamentally how these models are so good.
One day hopefully not so soon someone smart or a foundation model will come up with a more efficient architecture. That's when things get really scary.
Chain-of-thought output shouldn't be taken literally. The tokens are a substrate for computation, not necessarily evidence that the model is wasting time and electricity by gratuitously second-guessing itself over and over.
You can see evidence of this phenomenon in models dating back to the OG Deepseek R1. It was common to see the model talk itself out of the correct solution in the <thinking> block, or fail to reach it at all, only to produce a correct answer in the response. And vice versa; it was also common to see it reason its way to the right answer and then fail to follow through in the response.
I am interested however in why fine-tuning on reasoning traces of a frontier model is such an effective way of improving an (open-weight) base model. See e.g. https://huggingface.co/hesamation/Qwen3.6-35B-A3B-Claude-4.6...
I can see the reasoning being a substrate for computation, but in which space should we interpret this computation to be happening? The vector representations of individual tokens are completely different (and even the way the reasoning traces are broken up into tokens will be pretty different) between Qwen and Claude e.g.. The only way I can see this being effective (which it is) is thus that we SHOULD interpret the model to be "computing" in natural language and thus we can indeed take the chain-of-though somewhat literally.
The Deepseek R1 behaviour you describe is from a model from january last year, are you sure this is not pathological behaviour rather than an indication of the reasoning not needing to be taken literally?
I do however agree with the point that it is not necessarily a dead-end. That Qwen loops almost at an OCD-like level, but retains accuracy on the times it does answer, shows that. Yes ideally it loops less, but I am for now happy to accept that this is what it takes to run models locally. At least it is available for our inspection.
Not the person you replied to, but I think a more accurate description of the reasoning we see is proof of effort, not necessarily great insight into how the reasoning is occurring. For the most part, researchers currently describe the intent and motives (in however one may define them for LLMs) as black boxes right now. Even the mechanics of the cognitive process is not well understood. Depending on the model and harness, the thinking will often look like gibberish. I suspect they've invested considerable effort into presenting thinking as a reasonable approximation of what they imagine it to be. Claude and OpenAI have also begun encouraging multi-step problem solving (or the models themselves decide this), and we can see their more accurate responses at the conclusion of each phase.
Fine tuning or post-training is effectively biasing certain outcomes: making them more likely to occur. This comes with trade-offs. A coding LLM will bias technical language, which would harm a model for general use.
This opens a really interesting field of research. Our brains use specialised regions because specialisation turned out to be the most energy efficient method for biological compute. It might also be the best performant. We don't want to activate 100% of our prefrontal cortex to breath. What a stupendous waste of the organ. I think we see incredible advancements in model clusters in the future, using specialised models for specialised tasks. We have the appearance of this today in some harnesses, but they are shallow imitations. The real innovation will be low-cost, accurate routing. Existing solutions are woefully inadequate for many reasons.
Hardcode their "thoughts" in your agents.md... But they might still reason through it anyway.