[backend] do not simply remove the worker building data

Removing it can make the dispatcher abort jobs with "no compliant
workers found". Instead, move the worker data to "down" for
a short amount of time. This is not optimal, but introducing a new
"resting" state seems a bit overkill...