I’d have to guess that your air vent wasn’t making noise, although you may have heard it from there. Based on the description, I’d guess that your blower fan is on its way out or getting clogged up with debris.
You didn’t say what kind of car this is, but the blower fan is usually stuffed up into the dash down under the glove box. If you or anyone you know is reasonably handy and has some basic tools, such as a socket set, they are not that difficult to remove. It mostly requires some acrobatics for getting down under the dash. If you pull it out you may find gunk all down in there (leaves and pine needles and stuff), and if you clean it all up you might be good to go. Or it is on its way to seizing up and burning out in which case you’d install a new one.
Of course, I’m just guessing based on your description. So knowing where the blower fan is can help you confirm the source of the noise before messing around with it.