With enough speakers coupled on the order of the wavelength of the sound (and for most frequencies, these seem like they will be), you can use beamforming to aim different sounds in different directions from a single source, with speakers facing in only one direction.

For an extreme example of this, refer to the Sphere, where they can target sounds at individual audience members from any arbitrary direction using speakers in the surround display.