Unless squeezing every possible mpg out of the vehicle is the prime concern for the OP, I agree that he will likely be happier with the larger engine.
When he is accelerating onto an interstate highway, or when he is climbing hills, he will almost surely regret settling for the 4-cylinder engine, instead of paying a bit more for the six. Yes, the larger engine will provide more…fun…but there is no denying that there are instances where having superior acceleration is a safety factor.
And, if noise levels in the car are of any concern to the OP, he should be aware that, because the six engine is turning at lower RPMs at highway speeds, the cabin will be much quieter than with the 4-cylinder engine.
Many years ago, I decided to apply my mother’s wisdom (“There are no pockets in burial shrouds”), and to enjoy my money while I still can. Thus, I spent the extra money for the larger engine on my last two cars, and I have never regretted it.