I wish we had some kind of distributed training capability... Like Folding@home, but for LLMs.
See the recent advance of DiLoCo at Nous Research and Prime Intellect.
See the recent advance of DiLoCo at Nous Research and Prime Intellect.