My personal experiences. First most of my PS1 collection was bought straight off the shelf when it was current. Alot of my PS2 crap was the same thing so no help there.
Ebay: Ive bought quite a lot of stuff from Ebay to be honest. If you watch auctions and stick to a limit, eventually you will win one.
Amazon: I dont use it as much cause nothing is negotiable. I did however send a guy a message once and got him to part out a game to me and it was worth it on both ends.
Thrift stores: Its all a lucky random there. Sometimes Ill see SMB for 9.99 and then there are times Ive found (for example) Die Hard for $1.80, Mt Dew limited halo for $6.99, etc
Garage sales: I personally have never bought anything from a garage sale. I have heard of amazing scores though. I just never seem to be on the receiving end of it.
Craigslist: Ive gotten stuff from there but most people in my area think that craigslist prices should be Ebay + 10%. Its really annoying. I did meet one couple however in my area who just find stuff in storage and I buy it from them. So far Ive gotten a top loader NES, Snow bros, and motor city patrol for $50 total.
Local stores: There is a good side and a bad side to them. Most of them are knowledgable enough on the games that the prices are really not a score. But sometimes their system doesnt get updated and the current high price game goes for next to nothing. My best find was TMNT tournament fighters for $10.95 (their system only recognized the SNES version).
Other forums: Most of my collection has come from Nintendo age/Sega age/here. A lot of collectors (myself included) really prefer to deal with other collectors. I have gotten a lot of great deals so far.