I've had both the PS1 and N64 since around their release, so I experienced both during their peak, but I got the N64 at a time where I was already way into the PS1, so, while I enjoyed it, I really didn't gave it much attention. I was, however, deep into Pokémon Stadium and used to host tourneys, as well as would play StarFox 64 not-stop for better scores.
Overall, I like the N64 and personally have no problems whatsoever with the controller. I think the console is overrated, though.