You can see it in action. I have a M1 Ultra Mac Studio, an insanely powerful machine, and when building open source software, actual compilation flies but the autonomy step crawls because IIT has to build test binaries to test OS features and notarization slows that down dramatically.
Notarization is completely optional when building any OSS software on a Mac, and not part of any default build process I know. A Mac can sign builds for running locally, a process which is fast, completely local, and does require building test binaries or anything like that. Even a Mac building for an iPhone in developer mode has a local cert it can use, and doesn't require notarization.
Notarization is only needed when distributing binaries to others. Personally I do it once a month for the Mac app I distribute.