I'm someone who started getting the itch only somewhat recently, but grew up with (and unfortunately sold) a lot of my NES, SNES, and PS1 stuff a long time ago.
One thing I've noticed though, is that everyone's definition of "retro" varies a bit. I don't have any interest in systems like Atari, Colecovision, NES, or N64, for instance, but have spent a lot of time figuring out which NeoGeo AES, PS1, PS2, Dreamcast, and Saturn games to hunt down, particularly Japanese imports. If I could afford it/had space, I'd be going for PCBs of Cave shooters, too. Point being, there's no singular "retro scene" in place, just a whole bunch of different niche markets, and supply/demand moves like the tides.
A lot of the ridiculous pricing is just down to certain series/games developing a particular reputation over time, along with sometimes low print runs (Ogre Battle on SNES, Valkyrie Profile on PS1, tons of Atlus games, etc), and sellers keep those prices jacked up because people are willing to pay that much. I don't like seeing so many $150+ sellers out there, but I also generally won't pay more than around $80 for anything.