A date has been set for a former housing office to be demolished after it has become a magnet for vandals and antisocial behaviour.