Air Conditioner Only Works When It Wants To

When I press the AC button and turn on the fan, the fan will blow but the compressor will not start, so the fan just blows warm air. Very occasionally when I press the AC button I will hear the compressor start and then it will blow cold air, but usually only for 10 minutes or so. When the AC works it is really cold, but it doesn’t work very often. Is it the switch that is bad or the AC unit? Any help would be great!