Can I just check the battery myself with a multimeter?
No…
A car/truck starter will draw 300-600amps OR MORE of power from the battery when it’s trying to start. Running a radio or turning on the lights MIGHT draw 10 amps. So it’s a very good possibility it’s still the starter.
Try jumping the truck with jumper cables or putting the battery on a charger.
The battery may also be fine…but the alternator isn’t working properly.