It's no guarantee, but it's a positive indicator of trustworthiness if a codebase is open source.

I don't have hard numbers on this, but in my experience it's pretty rare for an open source codebase to contain malware. Few malicious actors are bold enough to publish the source of their malware. The exception that springs to mind is source-based supply chain attacks, such as publishing malicious Python code to Python's pip package-manager.

You have a valid point that a binary might not correspond to the supposed source code, but I think this is quite uncommon.