Leaving the key on didn’t damage the starter. Because the starter wasn’t operating.
But if banging on the starter gets it to start, there may be a problem with that new starter. Because that’s what you do to get an old/worn out starter to function.
Tester