What I hate about game collecting? It has a lot to do with hexen's problem really. And that is: Resellers.
I hate those bastards, especially the ones that actually brag about ripping people off. Not that all resellers are bad off course (there's one in a nearby flea market that I don't have too many problems with beyond a few slightly overpriced games and the fact that he knows the ebay value of rarer games), but most of them are just terrible. There's this one guy at the crappier local Flea Market who will buy games from another reseller that already overprices his stuff, and he'll overprice them even more. For example, he'll sell Conker's Bad Fur Day and Super Mario RPG for 80-90$ each which is just crazy, and he says that people will pay those insane prices because he tells them that these games are rare (despite him always having multiple copies of them...). The moment I decided to never buy from him(as if the previous situationwasn't enough) was when he told a kid that Final Fantasy Dissidia on PSP was rare and tried selling it for 20$ (in terrible shape), when, at the time, it was 10$ in stores, NEW.
As a close second, I hate all those people selling retro games on the local Kijiji/Craigslist, selling for around the same price as the bad resellers at local flea markets, sometimes more. For a completely random example, there's one time I found Flintstones: Surprise at Dinosaur Peak on there... the guy tried selling it for 400$, claiming it had "sentimental value" for him... And it seems everyone finds terrible excuses to price games high. Let's just put it this way: finding NES systems with no games included for 100$ isn't rare.
So, basically, the part I hate about collecting games is the actual "going around and finding games" element... it seriously sucks to even find games around here, so I HAVE to resort to online more often than not since there are basically no good sources for games nearby, and online is almost always a better deal (specifically jjgames.com, they have good sales every month or 2 if you sign up to their newsletter).