Because beside some skill needed in going fast during descends at 70-80-90km/h without dying (which is not easy but not extremely difficult either), a cyclist is basically an engine. Most other sports need physical fitness (speed, stamina, strength, endurance etc) AND coordination skills, and the latter is not easy to improve chemically.
I could agree with this. You do need some physical gifts as far as muscular endurance beyond the capacity of most but after that, its a very limited set of movements performed over and over again for hours. Plus a massive amount of will power and pain endurance. No amount of chemicals will turn even most gifted people into an NFL athlete.