The most common mistake you'll get here is people running tests, then making a small change later, then having tests fail unexpectedly. Even accidentally. This has happened to me before with regular CI, and often hit me during an automerge I fully expected to pass.
That and having local changes to either uncommitted files, or something external like a library version that works here but will fail on deployment.