Genuine Q about word optimization/token density:
If we create a stripped-down vocabulary with greater token density to use less resources and to resolve ambiguities earlier in the semantic process, aren't we creating NEWSPEAK and dragging along the worst aspects of it? The ambiguity and multi-valence of words is what creates more connections between words, increases the directionality of associations, and expands the potential subtlety and depth of meaning. By paring down (or requiring verifiability) we make it harder to say certain things, or at least make it harder to unintentionally say something that makes MORE or DEEPER sense than what we intended. If the token density becomes extreme, you're left with something like a calculator.
Maybe this is the ultimate path toward better coding? But the worse path toward better genuine thinking?
This is one of the dangers. AI boosters would say that humans already do this compression and it was accelerated by mass media and then the internet, and that model memory + context can be broad enough that compared to human capabilities the opportunities for depth and variability are even greater. But I think we know which way this optimization usually goes. Even the notion of a "fine-tune for subtlety" is a contradiction.
I don't think so. It's pretty clear that LLMs use the higher level layers for reasoning, so a bit of logorrhea very possibly enriches the result quality.
Nah reducing token length means that we're just reducing English down towards a programming language like a nice demi-glace