I don’t think it’s the fuel pump. Does the ventilation system change in any way when this happens?
I’m suspecting something controlled by vacuum is making the sound, and on many cars the ventilation system has vacuum-controlled devices which can sometimes make the sort of noise you describe when they move.