Some languages make this harder than others, and languages that require expensive compilation step will certainly make it hard, while e.g. interpreted languages that allows dynamic reloading of code can potentially make it easy - allowing preloading of the tests and reloading of the modified code.

If you also don't expect necessarily running the entire test suite, but just a subset of tests that are, say, labelled to test a specific function only without expensive setup, it'd potentially be viable.

You can also ignore running it on every keypress with some extra work:

- Keypresses that don't change the token sequence (e.g. because you're editing a comment) does not require re-running any tests. - Keypresses that results in a syntactically invalid file does not require re-running any tests, just marking the error.

I think it'd be an interesting experiment to have editing rather than file save trigger a test-suite watcher. My own editor syncronises the file state to a server process that other processes can observe, so if I wanted to I could wire a watcher up to re-tokenize an edited line and trigger the test suite (the caveat being I'd need to deal with the file state not being on the file system) when the state changes instead of just on save. It already retokenizes the line for syntax highlighting anyway.