Sounds like your pinion bearings are going bad. They can “feel” OK and still make noise.
Its gotta come back apart and be assembled correctly, with new bearings and a new crush sleeve. Maybe even the carrier bearings, too, if the misplaced caps have damaged them. You should be able to see mis-matched caps on the carrier bearing races - the wear pattern will not be even.