This is an active area of research to inject humility into llms in order to create some kind of knowledge boundary. You can look this paper from nouswise https://arxiv.org/html/2604.17843v1 and the product build on top it to try the humility.