Your mechanic’s advice is sort of the automotive equivalent of someone telling you that you shouldn’t eat celery if you have red hair. Ergo–incorrect, and somewhat bizarre.
Are you doing any harm to your engine? No
Are you paying a premium price for something that you likely don’t need? Yes
Unless your part of the country is typically exposed to extremely low temperatures (sub-20 degrees Fahrenheit) in the winter, there is really no advantage to synthetic oil. If you live in more temperate areas of the US, conventional motor oil in the viscosity specified by Toyota is all you need. Also, be sure that you don’t exceed 5k miles between oil changes–whether you use conventional or synthetic oil.