Other than RPGs, most of the games on PS1 are cheap. And even some of those can be cheap. There are a few exceptions, of course. But for the most part, you'll find it to be a very affordable system to collect for. And if basic graphics don't bother you much, there's actually a lot of innovative gameplay to be found in this era.
My recommendations for cheap, good PS1 games:
Action/Adventure:
-Castlevania: Symphony of the Night
-Fighting Force
-Medievil
-Mega Man X 4-6
-Metal Slug X
-Resident Evil 1-3
-Spider-Man 1 & 2
-Tenchu: Stealth Assassins
Light-Gun Shooters:
-Area 51
-Time Crisis
Platformers:
-Crash Bandicoot 1-3
-Gex
-Rayman
-Spyro the Dragon 1-3
RPGs:
-Brave Fencer Musashi
-King's Field 1 & 2
-Legend of Legaia
-Xenogears
Shoot-Em Ups:
-Apocalypse
-Darius Gaiden
-Raiden Project
-Strikers 1945
-Thunder Force V
Other:
-Jet Moto series (jet-ski racing)
-Road Rash 3-D (mixture of motorcycle-racing and combat)
-Tony Hawk's Pro Skater 1-4 (skateboarding)
-Twisted Metal 1-4 (vehicular-combat)
-Vigilante 8 & V8: 2nd Offense (vehicular-combat)
-WWF Smackdown 1 & 2 (wrestling)