I haven't actually played a 3DO more than a few minutes, so don't take these to heart. However, I had a 3DO on my wishlist for a long-time, so I've researched into it's library a bit.
That said, these are the ones I see with the best reviews and being brought up the most:
-Alone in the Dark 1 & 2
-Gex
-The Horde
-Myst
-Need For Speed
-Road Rash
-Star Control II
-Super Street Fighter II Turbo
-Wing Commander III
-Wolfenstein 3-D