You raise a good point.

I had not considered the pedagogical value of starting with PAC Bayesian bounds. It will be interesting to see ML courses that take that route.

Similar issues show up in traditional statics when considering multi-hypothesis testing. I wonder if betting/gambling over hypotheses might be a more accessible way to convey these ideas to new students.