Note that sitting in front of a speaker setup playing a carefully selected recording is not the same as the scenario above. We do not yet have the technology to fool someone walking by a room, especially for orchestral works or even an acoustic drum kit.
But I have experienced exactly this. Not an orchestra, but a recording of a band. I guess you can split hairs about the definition of "fool", but I have been fooled as you describe.
