It very well may be your turbo or the turbo dump valve. WHen you let off throttle after being on it (and under boost) for a while when you let off the pedal the turbo system is supposed to “dump” the high pressure so that it doesnt slam shut the throttle valve…if i had boost in the system when you let off the pedal the pressure will slam into the closing throttle door. YOu may not be properly venting this pressure out via the dump valve this will also cause the pressure wave to back up into the turbo…damaging the compressor wheel…making it slam into the body of the turbo.
Do this…Pull off the pressure tube from the turbo to the engine…then see if you can put your fingers on the compressor wheel…check for looseness in the turbo shaft. If it is loose enough with side to side play it may be abe to contact the turbo housing if it is contacting the turbo housing and will self destruct violently very soon if continued to be used. CHeck this…it is very important and will leave you stranded if the turbo seizes up. CHeck it NOW.