If you have worked to be wealthy or lucky to be one, you should see it as a social responsibility to help the less privileged people in the society. If I may ask, do you feel comfortable driving past several beggars on the street when you have billions of money in your account that you or your entire family may not be able to exhaust in the next 100 years?
Think about this seriously. With the so much money in the world, poverty can greatly be reduced if not totally eradicated.
All that is necessary is the distribution of this wealth.
There is enough to go round.
Note:
You must not necessarily give physical cash or money directly in order to help the poor or the less privileged. You can set up an empowerment scheme where they could be empowered or work to earn a living for themselves. It is like teaching them how to fish instead of offering them the fish directly to eat. I guess it is best that way.
But if you are already paying so much in taxes in which you are effectively giving half of your time and life to the government so they can solve those problems, and they do not do it then it's entirely their responsibility and you should not feel responsible to try to do even more, if you want to do it you are welcome to do it but there should not be any moral obligation.