Everyone stating that gambling will affect someone's image in the society so this is actually happened with you?
For me it happened once why are you gambling its a sin in our life I replied its my wish I can do what ever I wanted.
Everything that we are doing will surely affect our reputation or image in society. As for gambling, there are different beliefs about it, for some it's okay, but for others, it's not. If you are a gambler, you shouldn't bother yourself with the opinion of others. As long as you know that you are not violating laws or doing anything bad that may hurt or cause a negative effect on the people around you, especially those who are close to you.
Gambling is a personal decision. You are using your own money. You have a control on what to do with it. If you are using someone else money or if gambling becomes your addiction, then don't be surprised by the negative comments you might hear. But if you are sure to yourself that you are handling your finances well, then I think there's nothing wrong about it.
Personally, I don't always involve myself in gambling. It's not because I see it as a sinful activity, but it's just my choice. Like what you have written, we can do whatever we have wanted.