Low, rumbling sound when the car is in idle

So for the last week my Civic has been making a low, rumbling sound (almost like an overly powerful bass system) and vibrating whenever the car is idling. The noise is most pronounced when the car is started but continues when I am driving, but only when I take my foot off of the gas. Whenever I accelerate, the noise stops. Also, the noise is very faint from outside of the car, but very pronounced from inside. What could be causing this?

Check your exhaust,heat shield. A scarier guess would be transmission related

Alright so I can hear the sound most pronounced (from outside of the car) coming from the exhaust pipe but other than that it sounds like it is coming from the whole car in general (or whole exhaust system) or something. I can also see the pipe vibrating slightly but thought that this was normal. How would I narrow down the problem further? Checking to see if it is loose?