I think we often "think" they're responding to the sound, when what they are really responding to is visual. Unlike the speed of sound, there is no delay in the speed of light. Anything suddenly moving fast, like bow limbs, can sure get a deer's attention.
And, sometimes, just at that micro-second of the release, that's also just when a deer gets a whiff of human scent. Sometimes deer suddenly bolt even when they hear or see nothing.