How do you know that caffeine was the cause?

This of course cannot be generalized, but withdrawal is quite noticeable for personal well-being in a positive way.