what passive means
Passive is the behavior every town NPC and most critters run. They walk in whatever direction they feel like, hop over holes rather than falling in, get shoved around by knockback, and burn in lava like anything else. What they never do is chase you or attack. For a town NPC the behavior also comes with damage immunity from the player, so you can swing a sword through the Merchant all day and nothing happens.
they are not invulnerable, just immune to you
Enemies kill town NPCs, lava kills them, and a fall into a pit kills them. During an invasion or a Blood Moon they will happily walk out the front door and into the fight, which is why a town that survives is usually one built with closed doors, a solid floor and no exposed lava under it.
There are deliberate exceptions to the player immunity. Equipping the Guide Voodoo Doll or the Clothier Voodoo Doll makes the Guide or the Clothier killable by your own attacks, which is not a curiosity but a required step: dropping the Guide's doll into lava is how the Wall of Flesh gets summoned. Rotten Egg and the Flymeal are the other things that hurt an NPC despite the rule.
critters wander off
Critters use the same walking and hole-jumping logic, and that is exactly why they escape. Anything you want to keep, catch with a Bug Net and place inside a room with a real floor and real walls rather than an open pen, since a passive walker will find any gap and jump it. Water and lava will kill a caged critter as fast as an enemy would.
The Goldfish is the exception that surprises people. It only behaves passively while it is raining. On a dry day it is running a different behavior entirely and will come at you.
designing around it
Because these NPCs move on their own, placement is a suggestion rather than a rule. If you want a specific one where you can find them, build the housing so their walk loop is short: a flat floor a dozen tiles wide, doors on both ends, nothing to fall off. Vertical stacks of houses work fine, but a resident who wanders down a hole between floors will not climb back up on their own.
For the ones you actually need mid-fight, the Nurse especially, put the house at the arena rather than trusting them to be where you left them.