You don't need a datacenter to train a 30B model. Further, the rapid trend of increasing efficiency and decreasing model size for a given level of capabilities means that what is possible to do without a datacenter will continue to increase. Presumably performance on the level of the current frontier of AI would be achieved in short order and the ceiling would only continue to advance from there. Ergo I expect such an approach to regulation would prove entirely self defeating.
Remember, as I mentioned earlier the human brain only consumes on the order of 20 watts and fits in a handbag. Would you have us destroy all chip fabs? Ban all biomedical and genetic research? How far are you imagining this butlerian jihad would go?
> Presumably performance on the level of the current frontier of AI would be achieved in short order and the ceiling would only continue to advance from there
That's an enormous presumption! You're saying that even in the theoretical case that frontier research is halted but efficiency isn't, we could do better then the frontier and reach world-changing AI in 30b parameters at home-scale labs?! If that's true then we can just give up now: the world as you know it is going to end in around a decade and billions are going to die, there's nothing we can do. But I don't think that's true. Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards.
If you're right then I concede. It doesn't matter what we do, regulations or not. But if I'm right then regulation can do something and in theory help bring a better future.
> frontier research is halted but efficiency isn't
Why are you treating those as if they're separate things?
> Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards.
This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth.
The frontier labs build massive models because if you're competing and you have a lot of cash and brute force is a viable option then it's easy and predictable. But the fundamental research itself doesn't in general require scale (certainly not entire datacenters) and models at any given capability level keep shrinking.
I keep repeating myself at this point but the human brain is on the order of 20 watts. That's a fraction of a single datacenter GPU! So again, would you have us destroy all chip fabs and ban all biomedical research?