> it would just be another signal to fingerprint: "Oh, your browser is returning random values? Must be a Tor browser user".

You'd have to fingerprint the browser first to determine that the "random values" were indeed coming from it.