The relevant freedom is the freedom to opt-out.
It's much harder to opt-out of a government than a privately-crafted social scoring system. But some become so large that you can't de-facto opt-out, not without significant consequences to your quality of life... And that becomes a problem.
As an exercise, can you construct a version of private social credit that supports opt-out and isn't dystopian? I posit that any such system would interpret and opt-out as an effective negative score, heavily disincentivizing that option, and making it de-facto mandatory.