Why do some folks accuse people with hearing disabilities of "selective hearing"?

Since my time in military service I suffer from hearing disability, not very bad, but enough to be a nagging annoyance in many ways. The saddest thing is the accusation by some people, especially in the family, that I had "selective hearing". It is NOT like that. I frequently face this question: "How come if you don't hear good you still complain if someone is too noisy in the house?"
Indeed, even with a bad hearing, when I want/need to fall asleep, I am annoyed by noises that might be going on in the house, since I'm not totally deaf.
Also, when the circumstances are right and very little background noise is going on, I can hear everything out of a conversation. And this fact is not understood by people with normal hearing....there I face another question: "You just hear what you wanna hear, do you?"
Than again, sometimes I have to ask several times what someone said and those replies can be very hurtful since no one seems to be even willing to understand my problem.
Anyone w/. clues?

I'm sorry for your hearing problems. Have you checking with an audiologist to see if there is anything they can do to help?
I had to smile a bit when I read your question asking about "selective hearing". I used to say my husband had selective hearing but then I realized that with him, it's actually simply selective ANSWERING! He hears when I tell him dinner is ready but not when I ask him to let the dogs out. My kids seem to have this problem too. Hmmm..
But seriously, this is affecting your life and your interaction with people, you should check into your options, perhaps for a hearing aid of some kind. There are different levels of hearing loss, some people lose the ability to hear lower or higher sounds and have no problem hearing in between. My own mother says that she can hear most things just fine but has a hard time hearing a persons voice because it comes across sounding muffled. Please take care of yourself and check with a specialist. I hope you can find a solution to improve your hearing. Good luck!

Perhaps they mean it as a joke? I know women often like to say that ALL men have selective hearing?

Because some people with hearing problems are sensitive to loud noises. Also, people with hearing problems may only hear part of what is being said to them so that is why they are accused of this. Explain to your family that you are not ignoring them and if you seem like you are they need to repeat what they are saying to you.

It's a standing joke, don't take it seriously, tell them to p*ss off. The problem is that you hear certain frequencies more than others. That explains the night time. The same is with conversations, if there's too much going on, you can't follow because one frequency drowns out the other. My husband suffers from that.