There has been a relatively strong local model community since the days of llama-3. IMO, the number of non-technical people interested in qlora, adapters, parameter fine tuning etc is remarkable.
There has been a relatively strong local model community since the days of llama-3. IMO, the number of non-technical people interested in qlora, adapters, parameter fine tuning etc is remarkable.
that was when RAM and GPUs were cheap. Now that is dying, too.
Only people who have no concerns about privacy would give up on it entirely.
But yes, it has been a test of just how few tokens you can get by on.