> LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).
Now that a larger proportion of text is AI, I'm appreciating human-written text more and more; this got a good chuckle out of me.
>and whatever the hell Angular is
I've definitely seen LLMs compose jokes with this structure a lot where "whatever the hell ___ is" is in parenthesis. Not saying this is AI generated tho
I wouldn’t be so sure that that’s human text
I can reliably get my custom models to output a 95% perfect replication of my tone and voice
As somebody who attempted that recently, I seriously doubt that percentage :D
Maybe you just have a voice that's, shall we say, in sparse latent space? ;)
New insult dropped!
I have an insane amount of writing output as training data going back a loooong time so it all comes down to how much training data do you have
Many of us can do it on just our hacker news comments (been there, done that).
What do you mean by your custom models? Do you mean custom prompting?
Guessing - finetuned? Or LoRA?
EDIT: Nevermind, see below. I would definitely go with a LoRA though.
That’s if I was going to build my own model on myself, which I’m not planning on doing at this point
Couple things:
I have a “style guide” markdown that is pretty expansive
I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps
Then I will manually clean up that last 5% after a half dozen rounds
That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output
It’s been three years of tuning I’d suspect at this point
How did you train your style guide, though?
I have a somewhat expansive style guide, and it still reeks of Claude.
My cleanup is much closer to 40%, which still makes it worth the effort, but sometimes when speed matters, I'll apologize for the slop.
I've recently been using gemini argon as a voice judge. so it goes claude ( with style guide) -> voice judge with argon -> claude adjusts. I do that in a few loops before it hard stops if it can't reconcile, and pushes to me for review.