Now, I could be wrong in what I'm about to say, but this is all what I've observed. Take it with a grain of salt.
By your definition of fragmented, I'd say yes, it is super fragmented. Easy to see why too, because everyone likes different things. Similarly, everyone places different emphasis on how much and how often they game. There's no real rhyme or reason aside from that, but that's fine. There are people like my brother who plays games only a few times per week, if that, and when they do it's always some variation of a FPS. I also met a good friend at college who only really plays Pokemon games and not much else. He even plays competitively, and he's super passionate about the series, but he's not a big gamer aside from that.
Then you have people like me, and I'm sure many others on this site. I couldn't stand to limit myself to one genre or one game or series, and a lot of the fun I have gaming is from playing many things, finding lesser known games, looking for inventive ways developers crossed two or more genres, whatever. Seeing all kinds of things in gaming is what I enjoy. For me being a self-proclaimed collector, I don't have a specific direction. I find good games and I buy them. And of course, I have favorites. I prefer action-heavy games, turn-based strategy RPGs, games that offer difficult but fair challenge, and games which out heavy emphasis on great level design. I'm not as big a fan of open world sandboxes, branching narratives, or overly elaborate stories, though sometimes I'll play one and absolutely love it. Variety is my enjoyment.
So yeah, it really is as simple as seeing what each person prefers in a game and how they enjoy them. Nothing wrong with it.