Other than the restricted visibility, which I’ve have to judge yay or nay w/a test drive, personally I wouldn’t be overly concerned about the downsides myself. A stiff rides comes from stronger springs, which should make the car hold corners better. I drive with windows down, so the cabin is always noisy. Idle vibrations and seats? … no car is perfect. Downsides I’d care about is if the car had to be in the shop frequently or for months at a time b/c nobody could diagnose the problem and/or nobody knew how to fix it.
OTOH, competing cars are closer to perfect than the Fiat 500X. USA Today ranks the Fiat 500X last of the 17 subcompact SUVs they reviewed. The bottom of the barrel in this group in order from best (?) to worst is:
MINI Cooper Countryman
Chevy Trax
Jeep Renegade
Mitsubishi Outlander Sport
Fiat 500X
No good reason to choose the Fiat, unless you’re hooked by the looks. Not like you’ll find many, I only found 10 near me, 2016-2018. It’s a twin of the Renegade, another bottom-dweller.
Closest dealer still shows a pair of 2021 500X’s in inventory for about $21,000. They’ve mainly been an Alfa Romeo dealer the last few years anyways. Same showroom and lot.