07 Manual Honda Civic Rumbling

I have a 2007 manual/stick shift Honda Civic that we bought brand new. Just lately, it has started making this rumbling noise when I turn on the car. It started when the weather started to cool down, but I live in New Orleans so our definition of cool isn’t as cool as usual. It feels like a vibration all over the car and it’s really loud inside the car, but you can’t really hear it when you’re outside of the car. It completely stops when I accelerate in gear, but when I’m changing gears or if I just coast without my feet on any pedals while the car is in gear, it makes a rattlng noise (could be one of the struts or mounts is getting worn, but it doesn’t sound like a broken strut) accompanied with the rumbling that I hear when I turn the car on. Then as soon as I stop, whether in or out of gear, it does the rumbling thing until it warms up. Anyone have any idea of what this may be? An idling issue? Exhaust issue? I don’t have money to throw around on pricey mechanic diagnostic fees.

Maybe an exhaust system heat shield is loose and rattling.