Continued use will make more parts in the steering fail. The longer you drive it like that the more expensive the repair will be.
And then of course, if you cause an accident and someone finds out you were driving with compromised steering, the legal problems will make your power steering problems look like peanuts.