I think the point is that it's practically impossible to correctly perform RLHF in open domains, so comparisons simply can't happen.