There is a difference and anyone who says they've ridden for years (as in more than a decade on a motorcycle) without ear plugs and hasn't lost an ounce of hearing are either lying or really don't know what they've lost.
People also don't always understand hearing loss as well. Its not necessarily that you become deaf. I know I've lost some hearing over the years. I was a big fan of loud music (still am) but found myself on many occasions at a club or concert standing right in front of a speaker blasting way too much wattage. I can hear volume just fine. What I've lost is the ability to make out some detail, particularly where there is some background noise. I end up having to make people repeat themselves a lot in certain situations.
People think, well my bike isn't loud and I wear a helmet so I'm okay. Fact is, even on stock exhaust, the dB levels reach at least 100dBs or more at highway speeds. The bigger culprit is actually wind noise and numerous studies have shown, and proven, this. The only exception to this is the Schuberth C3, which Schuberth claims to be only 84dBs (82dBs for the Pro) at 60mph, but it has been tested in wind tunnels and it has varied between 85dBs-91dBs depending on the position. It is a sport touring helmet and in the upright position, at 60mph, it varied between 85-88dBs. Excellent all things considering.
It should be noted that how sound is measured is important. 100 dB measured 12 inches (which is probably like 3 feet from the rider's head) from the exhaust opening is vastly different than 100 dB measured right at the ear canal.
Some helmets certainly can be better than others. Mine is fairly loud, but if I shrug my shoulders up, I can seal the bottom of my helmet and the wind noise becomes very small. But its really impractical to ride like that. I have the Nexx XR1R and it supposedly tested well for noise in reviews, but I get a ton of wind noise. I suppose it just seals poorly against my neck in my case.
I will add that I rode with the new triple flange sleeves on the SE215's this morning and they worked great. Not as good as the ear plugs I used, but more than adequately. Didn't have any issues keeping them in my ears while putting my helmet on. Streamed a little NPR during the ride as well to see if I can hear voices clearly and it was totally clear.