The Scion xB seems like the ideal sort of solution. I don't know the deck height, but they seem really low. They are also supposed to be pretty fuel efficient.
Of course, there are reasons lots of bands use old full-size vans. They are cheap and can carry lots of stuff. My neighbor has a Ford E150 with some aluminum load ramps. He buys and sells antique furniture, and is in his 70s or 80s. A hand truck and load ramp solves lots of your issues. You can get the load ramp even for a smaller van, but it might be hard to move around inside compared to a full-size van.
My wife just bought a used conversion van that she's going to use for her chocolate shop business. I'm currently looking at loading and storage options. I don't know if it would help you, but there is a slide-out platform available for vans that can be installed in either rear or side doors. Here is an example http://www.bedslide.com/products/st_carpeted.html
I have seen others with different options. Perhaps something like this would help?