Consider the source of the idiom: rolling your own cigarettes.

Which involves taking some rolling papers, a pouch of loose tobacco (or whatever), and perhaps a little device if you're rich. As opposed to manufactured cigarettes, you're just doing some manual assembly for the end-product.

You don't need to cultivate the plants or pulp any trees to roll your own.