I've been wondering if OpenAI make Luna artificially cheap to get people into their eco system.
I think it's great and hope the price can stay the same.
I've been wondering if OpenAI make Luna artificially cheap to get people into their eco system.
I think it's great and hope the price can stay the same.
I think Luna might be just small enough to provide some kind of stepwise improvement in how it is hosted.
Going from 81GB of weights to 79GB of weights can mean a 50% reduction in GPU capacity required.
If you can fit a model in just one GPU (or rack) as opposed to across an entire datacenter, the latency gains can be substantial too. If you can reduce token latency by half, that would double the amount of customers you could support.