Parameter count is not everything.
20b parameters * 1.5 bits per parameter is just 30 billion bits, about 3.75gb
a full 20b fp16 is about 40GB.
I find it weird how a smaller model still produces decent text, except it bullshits all the way.
Parameter count is not everything.
20b parameters * 1.5 bits per parameter is just 30 billion bits, about 3.75gb
a full 20b fp16 is about 40GB.
I find it weird how a smaller model still produces decent text, except it bullshits all the way.