There's some really interesting stuff we've been looking into on the Matrix side to solve this - e.g. https://github.com/asonnino/arke aka https://eprint.iacr.org/2023/1218 or https://martin.kleppmann.com/2024/07/05/pudding-user-discove....
Meanwhile, Matrix for now does support hashed contact lookup, although few clients implement it given the privacy considerations at https://spec.matrix.org/unstable/identity-service-api/#secur...
Yeah you're doing a lot better job on the privacy side than signal is IMO.
Especially just being able to run my own service will be priceless when something like chatcontrol eventually makes it through. Signal can only comply or leave, but they'll never manage to kill all the matrix servers around.