This reminds me of how I learned about Zipf's law. 19 years ago, I spent the summer after my sophomore year in a research lab at UC Santa Cruz as part of an NSF-funded summer research program for undergrads. My project that summer was to investigate index design for the Ceph distributed file system in order to enable faster file search. This was a hot topic at the time, especially with Spotlight being introduced to Mac OS X Tiger, and we wanted to see how to enable similar functionality for a distributed file system. I learned about Zipf's law, which states that there is a power law distribution of word frequencies in many natural languages. I remember downloading the Project Gutenberg corpus and writing a Python script where I stemmed the words and computed the frequencies of the stemmed words. The resulting distribution confirmed Zipf's law.

[deleted]