Vanilla: I would say only the Vex, Ghast, or Illusionist. One deals too much damage early game and is difficult to hit in its natural habitat, the other causes too much environmental damage for my liking and can see you from far away, and the last one needs a rework to be fun/balanced. Blindless + being shot at from four sides is not fun.
Modded: Too many to name due to the high amount of poorly designed mods out there. However, some modded mobs that I abhor greatly have to be any "Dweller" mob, Siren Head from Box of Horrors, every mob from Scape and Run: Parasites (Especially Assimilated Enderman and the 5 tiers of bosses,) Phayres from Fungal Infection: Spore, Crimson Mosquito, Froststalker, Tiger, Murmur, Seagull, Bison, Skelewag, Basalt Golem, and Komodo Dragon from Alex's Mobs, and Yiga Clan Member from this one Legend of Zelda Breath of the Wild mod all for various reasons.
