Volvo XC90 2004 does not start after not being driven for a few days

Volvo XC90 does not start after being parked and not driven for a few days. The starter cranks normally. If I keep on trying, eventually the battery drains and needs to be recharged. Then, after 2-3 hours of recharging the battery and I try starting the car again, it starts on the very first try, no problem.
I’m thinking it’s a fuel supply problem, but what exactly?!?

The problem might be with the anti drain-back valve on the fuel pump.

To determine if it is, the next time the vehicle sits and you go to start it, turn the ignition switch on so the dash lights come on for two seconds, and then turn the ignition switch off.

Repeat this a half dozen times and then try starting the engine.

If the engine starts right up, that’s the problem.

Tester

Thanks a lot, I’ll be sure to try this out next time.

Tried the approach suggested by Tester. The test was not 100% conclusive, so I \ installed a back valve on the rear of the fuel filter to insure this was not the problem. It seemed to have solved the problem when I tried starting the car after it sat for 3 days. Then when I tried starting the car this morning the problem came back. Not sure what to look for next. Any ideas?