Whats in the radiator

we had a shop replace a thermostat in oct 10, last week our truck started running hot so we put in about 3 qt of 50/50 coolant. it returned to normal temp the next day we noticed some foam coming out of the overflow. when we opened the hood it looked like a brownish/red color and is a thick foam. When taking it back to the garage they cannot tell what it is. They have tried to flush it out and say they put in 3qt of fluid and only get out 1qt. It can be chunky at times. Anyone have any idea of what it could be? Really need some help on this one.

Brownish/reddish color leads me to think the transmission cooler in the radiator is leaking and allowing transmission fluid to leak into the radiator. Check the radiator fluid for level and contamination.