I think you're taking the words too literally. The adult women could be aunts, guardians, sisters, adoptive mothers, biological mothers, etc. We can always seek out ways to find words to be unfit in a sentence, but, the purpose of speaking is to communicate. Normal people will understand that 'moms' in this case means any adult who in this scenario is representing a child in some capacity. It's shorthand.
Sure, the speaker could have included all possibilities when addressing the audience, etc., but why? We know what they mean, they didn't intend to exclude you in any way and as an adult you can process the exceptions (moms, the bathrooms are there) you can figure out where the men's toilet is.
If we concentrate on people's speech patterns, we can always find faults. There are better things to spend your energy on. We could also embark on pluralization vs apostrophe too, it's a noble cause, but we all know what people mean.
I think OP is well aware that the speaker isn't specifically addressing moms and only moms. But even knowing that, it can make you feel less included, which is in itself unpleasant, even if on a more rational level you know that you are included.
I've been in these situations and I really don't feel excluded at all by casual language. I'm sure I could, if I wanted to find cause, but I don't because I understand the subsurface meaning of what people are saying.
Sure, the speaker could have included all possibilities when addressing the audience, etc., but why? We know what they mean, they didn't intend to exclude you in any way and as an adult you can process the exceptions (moms, the bathrooms are there) you can figure out where the men's toilet is.
If we concentrate on people's speech patterns, we can always find faults. There are better things to spend your energy on. We could also embark on pluralization vs apostrophe too, it's a noble cause, but we all know what people mean.