How much of gardening is dealing with bugs?

It depends what on what you're growing.

With roses, a significant % of manual time is spent fighting aphids