The first thing that comes to mind is the Neo Geo Pocket Color. I think it is due to when I got it, which was recently. The lack of a back-light screen is what kills it for me. I know it can be modded to have one by using an Advance SP screen, but I don't want to go through that trouble. Now if I had gotten this when it came out, things might be different. For example, I liked the Gameboy when it was new, even though I also had a Game Gear. Concerning portables, only the Game Gear and Game Boy Advance SP (to a lesser extent, the Nomad but I don't use) are the ones for me. The original Gameboy is fine but it gets a pass because you can play the games on either the Advance SP or the Super GameBoy/Color add-on for SNES.
I'm not considering computers like the Timex Sinclair 1000, which I was trying to get for some time only to be disappointed by it and its capabilities once I got one.