he is a townsfolk in a bad state
Throwing Purification Powder at him turns him into the Tax Collector, and that is his entire purpose. Once it happens he stops spawning in that world for good. The Clentaminator does nothing here despite doing the same job everywhere else, so bring the actual powder rather than assuming the upgrade covers it. If the Tax Collector is killed later he returns on his own and you never have to repeat this.
the usual way to fail
He counts as an enemy, which means your own build will kill him before you get a chance to convert him. Minions pick him as a target with no input from you, and the Chlorophyte set bonus fires at him from across the room. Unsummon everything and change armor before you go looking, or you will stand there watching your reward die at range.
finding him
Only one can exist at a time and he is uncommon, so this is patrolling rather than hunting. If you already built a bridge across the underworld for the Wall of Flesh, walking it is the easiest sweep you have. The audio cue beats the visual one: he is the only thing down there that opens and closes doors, so a door swinging somewhere off screen means he is close. A Lifeform Analyzer flags him as well.
do it early
Get this done before you kill a mechanical boss. Afterward the underworld starts producing Red Devils and Lava Bats, and simply surviving long enough to line up a powder throw becomes a project of its own. The tidy approach is to carry powder into the Wall of Flesh fight and sweep for him straight afterward. If no house is free when you convert him he waits down there until you build one, and if the underworld kills him before that he only comes back once a room exists.
small things
He has no banner, so the Tally Counter never shows a count for him, yet the bestiary tracks kills separately and fills in normally. The chains and the money lending afterlife make him a fairly direct nod to Jacob Marley. There was also a stretch where he could pick up coins in Expert, so a purified one would arrive as a sparkling Tax Collector who dropped money when killed.