If you do decide to change the tranny fluid and filter, (not sure if Toyota has a drain plug and remote filter) make sure the pan is dropped so as to check for metal shavings or small metal pieces.
Wait a sec. You said the vehicle was regularly serviced. Did that include tranny service too?
Brown colored fluid usually means burnt. (from overheating)