This really depends on your plans for the car. If you are going to sell it or trade it in fairly soon, say in 6 to 12 months, then the used engine makes sense. If you want to keep it for 2 years or more then you go for the rebuild.
The used motor will have similar miles on it to the motor that just threw the rod. Rebuilt motors have lots of new parts, especially parts that wear like bearings, piston rings, and such. Some rebuilts even have better parts than OEM if there were weaknesses in the original design that have been recognized and improved upon.
All this means a properly rebuilt motor should be a very strong bet to significantly outlast the used motor.