Sandstone Brick Wall

This is the only background wall that makes a house less attractive to move into. A room built with it still passes as valid housing, but the game picks it far less often when deciding where a new arrival should live, so a town walled with it fills up at a crawl.

That cuts both ways. If you have built a desert town and cannot work out why nobody has shown up, swap the interior walls for anything else and they will. If you want a room to stay empty, for storage or because you are saving it for someone you intend to assign by hand, this does the job without any extra work.