The problem might with the blower motor itself.
When blower motors become worn out, they can stop and start at any time. Turn the blower speed to high, then reach under the passenger side of the dash and tap on the blower motor with the handle of a screwdriver. if this gets the blower motor operating again replace the blower motor.
Tester