What threw me off was the (possibly misunderstood) suggestion for minimizing the generalization bound over the prior after the data has been incorporated.
What threw me off was the (possibly misunderstood) suggestion for minimizing the generalization bound over the prior after the data has been incorporated.
Ah, sorry for the misunderstanding, I can see how my comment reads that way. That is done in the Gaussian process context, not in my first example, and yes, it's a dirty idea, but you can justify it using differential privacy arguments (basically you are optimizing few parameters and these do not have full interaction with the data).
Yeah, I had read one of your parallel comments and understood what you had meant. Differential privacy is a good formulation (well, the only one I know) to deal with the peeking problem in general.