I like the idea but that seems complex to put in place and would risk degrading the perfs.

You can test this prompt yourself elsewhere, you will notice that you get sensibly the same experience.