I assume the choice of phrase "bitter lesson" is intentional irony (since the original concept is that you get better results by just scaling up and not trying to be clever with domain-specific knowledge)?
I assume the choice of phrase "bitter lesson" is intentional irony (since the original concept is that you get better results by just scaling up and not trying to be clever with domain-specific knowledge)?
I assume that the bitterness of the bitter lesson is not for engineers but for subject matter experts. I can only imagine how it would feel to discover that your decades of hard-earned expertise don't amount to a whole lot when it comes to domain-specific ML modeling, compared to simply throwing more compute at the problem.
Maybe the ultimate bitter lesson is that entropy always wins in the end.
[dead]