Stuff like this is what doctors should use instead of blasting patient data to a AI scribe who knows where. Im pretty sure that is illegal but it still takes place and I am pretty sure one of the reasons for the push to solve FHE...but its obviously way more practical and private to just do the processing locally.
In the US, HIPAA requires companies to sign a BAA to handle any protected information. That's pretty much the only thing stopping data from being shared. That is why, for example, a company website might use Freshpaint or a similar service instead of Google Analytics, as Google will absolutely not sign a BAA for the Analytics platform.
Your data is still collected by a third party analytics company, but they contractually bear all the legal responsibility for maintaining compliance. AI scribe companies are much the same