It’s not the fuse. If it were, the passenger side switch wouldn’t work either.
I tend to suspect the switch itself is bad. It’s likely wired such that everything routes through the driver’s side, which means any other failure in that circuit would fail both the passenger and driver side switch. That the passenger side does still work suggests that the driver side switch isn’t making contact when you switch it.
Pull the door panel, and check for obvious things like frayed/broken wires or wires that aren’t completely plugged in. If you don’t find anything, then unhook the plug that feeds the driver side switch, and get out your multimeter. Set it to measure resistance (Ω) and put one lead on each side of the switch contact. The reading should be 1 because the switch is open. Then operate the switch. If resistance doesn’t drop below 1, then you’ve confirmed a bad switch.