curious why the HF pill (on the right) always has inaccurate values
I believe it's because this model is natively fp8 (for the most part), and that display struggles native quants.
I noticed the same, and I wonder as well.
I suspect they are calculating something in the weights or config, I see it pretty consistently with quants
I believe it's because this model is natively fp8 (for the most part), and that display struggles native quants.
I noticed the same, and I wonder as well.
I suspect they are calculating something in the weights or config, I see it pretty consistently with quants