Why did you use 1-bit quantization vs 3-bit quantization?
It looks like the 3-bit requires 90 GB[1] which, I imagine, would fit within the DGX Spark's 128GB of unified memory.
Why did you use 1-bit quantization vs 3-bit quantization?
It looks like the 3-bit requires 90 GB[1] which, I imagine, would fit within the DGX Spark's 128GB of unified memory.