is it really necessary to change the brake fluid every 3 years regardless of mileage?
No it isn’t necessary, but it isn’t a bad thing to do either. If your fluid looks really dirty, consider it.
Even if it doesn’t look dirty, you should change it.
After 3 years, it is likely to be diluted with moisture, even if it looks relatively clean.