it's as simple as precisely describing "common sense"

Yes, that very simple task :D

But I also wonder if that is actually an equivalent.

I think of "common sense" as think you should know to function well in your current society.

I really don't know what "casual knowledge" would mean. In my head it's some kind of low stakes knowledge for everyday life (but more 'useful' than trivia).

Maybe the order is "common sense", "casual knowledge" and trivia?