You are more likely to get in trouble for distributing carts full of ROM's, or buying from a company that engages in such practices. I've never heard of anyone getting in trouble for downloading ROM's, especially in the last 10 years.
They're not gonna go after someone who downloads 25 year old arcade ROM's, simply bc there are no legal means to buy the product ( w/o huge markup on the secondary market) and there's hardly much of a benefit to do so compared to people who illegally download music or movies.
The laws are specifically against illegally sharing the material. This is why sites get shut down.
The few individuals who have been sued or what have you, the charges were actually about the distribution, not downloading of the materials. This specifically refers to public demonstration, distribution (regardless of sale or gift), or other methods that potentially damage a copyright holders business or profitability.
A lot of the ROM sites like to say you can only download it if you own the game physically (This is technically still ilegal) and that if you do not, you should only be downloading the game for trial like purposes to decide if you want to buy the game and thus delete the download within 24 hours (Also, still ilegal).
The fact of the matter is that REALLY, Nintendo is the only company that tries to pursue these matters. Usually a simple cease and desist. Sony usually only does it for distribution of the BIOS for the hardware, but I don't know if they still do that. I know for a while GBA games were really hard to find because Nintendo was constantly having sites shutdown or threatened to be shut down.