The Expansion Pak is required for a few games. Donkey Kong 64, The Legend of Zelda: Majora's Mask, and Perfect Dark are the ones that come to mind. Perfect dark will come on, but the single player isn't accessible.
It also improves the performance of several games like Spider-Man or Rogue Squadron. In those, it removes some of that N64 fog and ups the resolution. In Gauntlet Legends and Hydro Thunder, the Expansion Pak allows 3 and 4 player multiplayer.
It's not that important for just playing the hits, but it does make the game better and I always have an Expansion Pak in my N64.