Here is where the difference between the mfr's maintenance schedule and a proper maintenance schedule may differ.
No matter what extended intervals might be listed in a maintenance schedule, the reality is that trans fluid needs to be changed every 3 yrs/30k miles (whichever comes first) if you want to avoid premature trans failure. When the trans is not serviced on that type of schedule, trans failure can take place any time after ~90k miles, and is pretty much of a sure thing after...let's say...120k miles.
Ideally, the trans fluid (and filter) should have been changed 3 times already.
Has that been done?
The problem is that many manufacturers, in an effort to make their vehicles look relatively maintenance-free, have dropped this important service from their maintenance schedule. So, if you did not service the trans as described above and this service was not listed in the Lexus maintenance schedule, you are not the one to blame.
On the other hand, if that service procedure is listed in the maintenance schedule and you tried to economize by skipping it, then you should look in the mirror in order to view the culprit.
This is not unique to Toyota/Lexus, so most car owners who follow a mfr's maintenance schedule that fails to list trans service every 3 yrs/30k miles will have a similar experience at some point.
All of that being said, I suggest that you take the car to an independent transmission shop for evaluation.
DO NOT go to AAMCO, Lee Myles, Cottman, Mr. Transmission, or any other chain unless you want to be told that you need a new transmission (whether you really do or not). Additionally, those chain outfits are frequently known for poor workmanship and for failure to stand behind their warranties when their overhauled transmission fails. You will be more likely to get an honest diagnosis, good workmanship, and a fair price from an indy trans shop.