Pick a system that you already enjoy and means something to you. Don't make a turbo grafx a purchase early on if you have never played one but just think it looks cool. This is a good way to blow a lot of money, get burnt out, and become resentful.
Buy the common classic games at first. "Mario World for SNES" "Sonic for Genesis" These defined the system for a reason and will keep the love for the system alive after you buy some stinkers or have trouble finding the game you want. Getting a solid collection early on helps majorly later on.
Stay away from buying "rare/expensive" games. If you buy stadium events for $5 at a flea market, that's cool. But don't start a change jar for Dinosaur Peak from when you start collecting the NES. When I see someone's "epic" Saturn collection that is 10 games, three of which are Panzer Dragoon Saga, Burning Rangers, and Shining Force 3... I see a Saturn collection that will be up for sale in a year.
The above are my suggestions after I see people rush in and burn themselves out. In short, stay away from the glam.