Yep. Have the fluid exchanged. Do it again in 10,000 miles like @Mustangman said.
Your second mechanic was correct that the transmissions on the 2004-2006 TLs were more prone to failure than normal, but incorrect that it has anything to do with having replaced the fluid. It’s actually the 3rd and 4th clutch transmission fluid pressure switches that were problematic. Their job is to reduce line pressure momentarily during shifts to make the shifts smoother. But they tend to get worse over time and choke off fluid flow when they aren’t supposed to, which eventually wrecks the transmissions. A lot of owners of that era TL will replace them every 50,000 miles.