You may be experiencing pre-ignition. This can happen when hot spots develop inside the cylinder. Hot spots can be areas where carbon has built up, hot valves, or the like that get so hot they can ignite the gas mixture before the sparkplug does. The mixture explodes at the wrong time, as if the timing was off. This can create the symptoms that you are experiencing and explains why it only happens when the engine is subject to being overheated.