The key thing is a function of “luck” and “cold”. But at the starter - not the key. Go to shop. Mechanic will find the problem quickly and explain what’s going on. You will most likely need a new starter. Usually the gear on the starter is worn down. He’s the guy that dances with the ring gear. On cars with very high mileage, it can be the ring gear that’s worn out. That will be bad news and expensive. But again, you’re probably due for a new starter - that’s routine and not that expensive. But take care of this now. If you keep going on luck, you risk damage to the ring gear.