You are a victim of tradition, and folklore. Historically, all US cars were 6 or 8 cylinder, and the only 4s were cheap imports that did not last long.
Then came the VW Beetle, and 4 cyl. Volvos, and the truth emerged that the number of cylinders and engine size did not really matter; it was the quality and design that made these engines last.
In foreign markets, 8 cylinder cars are rare, and 4 cylinders are the vast majority. I rode in a Toyota Corolla taxi in Asia with had a 4 cyl. diesel with nearly 1,000,000 miles on the original engine. Volvos routinely exceed 500,000 miles on their 4 cyl. engines.
Domestically we just had a post from a forgetful Neon owner who went 270,000 miles when he broke his timing belt, which should have been replaced much earlier. The car was running great up to that point.
People who drop dead of heart attacks are seldom the small skinny type. It’s the big muscular ones who stopped excercising!! Car behavior is very similar.
The only exception to this is if you are pulling a trailer; then you need good horsepower and torque top get performance and long life.
American cars used in urban settings wear out their engines prematurily from starting and stopping, since they don’t warm up properly. A smaller engine would actually last longer, since it gets a good workout.