Large delays in audio/video aren't condusive to interactive applications. But chat up someone who plays a pipe organ; they often play in ensembles and have to deal with serious latency (but probably they just worry about hitting their notes in time to be on time and the ensemble has to match up with them)
Great point. I'd love to see a chart for respective "attack" values for classical instruments. My experience with organs is only smaller electric organs like the Hammond B-3 so it's not really as much of a factor there.