How worried should I be?

I think this may be a vision problem. Have you gotten new glasses, or maybe switched from glasses to contacts, or vice-versa? Are you experiencing vision problems in just one eye?
Your problem seems to be with your depth perception, which enables your brain to judge distance based on the information provided from both of your eyes. The loss of vision in one eye is the most obvious cause of loss of depth perception, but something as simple as a new prescription for eyeglasses etc. can cause an adjustment period for your brain to interpret the signals it is receiving.

I should add, to answer your question, the tire is taking a beating, and you may break the bead on the wheel that seals the tire to wheel. At five miles per hour you are probably not doing much damage otherwise. But if you can’t correctly judge the distance between the front of your car and a curb, you will have the same problem eventually with a pedestrian or a child waiting for the school bus or a cyclist , and so on. The fact that this problem occurs on the passenger side only might mean something. If you do not wear glasses at all and have never been to an opthamologist, go right away. If you have one, see him or her right away and ask about this problem.

Yeah, I agree. Once will ruin a tire or cause damage. We covered how to make right turns 45 years ago in driver training. Just take a look at what you are doing and alter your technique a little. Either you are pulling too close to the curb when turning, or turning the wheel too quickly before moving into the roadway. You should be moving out into the roadway some and then turn the wheel to make the turn. (how come this window won’t expand???)