Forbidden Fragment

Sand Elementals are the only source, and they only turn up in a Desert while a Sandstorm is blowing after the world has gone into Hardmode. That makes the whole material weather gated: you cannot go and get some on demand, you have to be ready when the storm starts.

Build a flat platform over the desert surface ahead of time so you have somewhere to stand and something to run along. Elementals teleport around you and fling sand tornadoes rather than closing in, so a weapon with reach or homing is worth far more here than anything you have to swing.

The armor set is the real reason to farm it, and that set is aimed at a summoner who also casts, so it is a dead end for a pure melee or ranged character. If that is you, gather only enough for the Sandstorm in a Bottle upgrade and the decorative blocks, then leave the desert alone.