They're 60 years old. The amount of training data on them is endless. Books, textbooks, videos, blog posts. The problem is when you want you do something that doesn't have 60 years of freely available documentation of their functions and applications

I'm having it help design a 68k computer similar in spirit to the original Mac (the spirit being a tightly coupled video subsystem that time-shares the CPU bus), but updated with more modern peripherals, like PS/2 and SD cards. It's got the design more or less done, but the routing will be a nightmare. I'm not ready to just gamble on it having gotten everything right, so I will be doing a thorough design review myself and re-deriving all the timing analysis.

>without an actual understanding of what they're doing.

At what point do you start to question your assumptions that are causing you so much cognitive dissonance?

But to answer your question: to predict the next token really well you just have to model the world. Think of it like this, a simple statistical model might say "when token A is seen respond with token B". The next step will add conditions, "...respond with token B unless X has been seen, then respond with Y". Add a few billion more of these contexual clauses and you have a sequence of logical rules that indirectly model the relevant processes in the world.

“Next-token prediction” describes the output format, not the computation required to choose each token. During training, models develop internal representations of concepts, constraints, possible futures, and algorithms.

The PCB agent also writes circuit code, runs simulations, reads failures, and revises the design. It isn’t one-shot autocomplete.

Astra and Fable are already hard to square with “mere autocomplete.” We may be (really) close to AGI, and token-by-token generation certainly doesn’t rule out subjective experience (I think we should at least treat that as an open question).

Great videos: https://www.youtube.com/watch?v=D8GOeCFFby4

https://www.youtube.com/watch?v=Bj9BD2D3DzA

https://www.youtube.com/watch?v=l6DKRf-fAAM

https://www.youtube.com/watch?v=GlYgs6v2YfU

My 2¢:

When google trained a neural net on Go moves, using some text notation for them, with no other vocabulary of any kind, just predict the next go move, they noticed a representation of a Go board had essentially formed in the network, all on its own. It had never “seen” a go board, or had one explained, but they could map neuron states to go board squares pretty much 1:1.

I truly think that LLM’s with hundreds of billions of parameters in their neural networks have all kinds of hidden “models” of things that arise from the simple act of predicting tokens. We’ve seen that the hidden layers in their networks model all sorts of program execution state for instance, when they’re working on coding tasks.

“Predict the next token” is a way of shaping/reshaping the neural network until it actually develops models of the things you’re giving it. Like the go board example. And I would wager that it has a compounding effect: once you have some useful models in the network, they can unlock the creation of other models, and so on.

Look up "mechanistic interpretability" in the context of LLMs. The next token prediction machinery is just a foundation for a higher order learned structure that appears to encode specific concepts, regardless of input language.

The analogy to humans is that the human brain is "just atoms bouncing around", but there's unquestionably something "more" going on that just that.

"my brain hurts trying to understand how that can be"

Well, we all are, some are just more used to it by now and take the magic for granted.

My simple explanation, those neural networks save lot's of patterns of data, and that pattern can represent an image, a code snippet, a poem, or well ... description of a circuit board. And especially the text variant, LLM's - did copy all from us - so obviously they sound like humans, when they internally debate how to do something as this is what is in their trainings data how humans sound, when doing similar tasks.

But really understanding it? Not sure if there is a single person on earth who does.

> Yet, it's all the most likely token and my brain hurts trying to understand how that can be.

You and everyone else. That's the great mystery of transformer architectures as applied to language.

To be clear though, they're only good at schematic capture, which is very much a textual representation. Most of the data basically boils down to netlists, which are a text based format mapping connections between abstract pins that only later map to physical copper. The actual schematic portion is for human consumption and LLMs don't need to produce those to be useful.

Where LLMs completely break down is the next step, PCB routing. That's an NP-complete research problem that's been ongoing for decades without much progress. I've had some fun playing with using LLMs to better specify DRC rules in Altium so that the "classical" algorithms are more usable, but at the end of the day their geometric intuition is nonexistent.

Now go read Blindsight and enjoy the mental crisis.

type “shai next-token” and then “transformers learn shortcuts to automata” into arxiv and prepare to be blown away

humans when a machine better than them at spotting patterns appears:

You fell for the stochastic parrot meme and next token over simplification. That's the explanation.

They do, https://jlcpcb.com/parts