My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
8x RTX PRO 6000 or 4x Spark? Or 1x M5 Ultra 512GB.
The model is theoretically FP8, but really internally its mostly FP4 already, so there won't be a cut-in-half-but-almost-just-as-good quant coming for this one.
a beefy pc with at least 20 GPUs
8x RTX PRO 6000 or 4x Spark? Or 1x M5 Ultra 512GB.
The model is theoretically FP8, but really internally its mostly FP4 already, so there won't be a cut-in-half-but-almost-just-as-good quant coming for this one.
a beefy pc with at least 20 GPUs