I would stick with the standard 16" wheels/tires. You’d have to be an experienced race car driver to detect the handling difference (if any) with the 17" combo, plus the larger the wheel diameter, the more expensive it is to replace tires.
All the negatives you listed in your post are true.