Driver side mirror came out of its support and wobbles

While driving my wife went over a hump and a plant branch hit the mirror. The mirror got cracks and I have ordered a replacement mirror. The mirror box came out of its vertical support loose. I understand that if I can remove the mirror holder I can put the holder back. How do I take the mirror holder out? should I use much force? The replacement mirror - how do I put it on the holder? Any help will be appreciated. Thanks a lot. The model year is 2005 and the car is a DX Sedan.

Your description is not helping much. Can you post a link to a picture of what you are talking about?