Middle class and the rich gamble, that is true. But you have to consider the bulk of the gambling volume in gross USD probably come from people living in developing countries like Nigeria and gambling the little money they can afford to lose, in order to try to get a jackpot and change dramatically their living situation. There is a good reason there is more gambling proliferation in countries and neighborhoods which are considerably poorer then compared to others, it is because those who are the least educated about math and the laws of chances are more likely to become gamblers in the long run and hence, lose money to both casinos and betting agencies (other bettors).
Working and investing is harder than gambling but that is what asset managers do, they do not gamble with the money of their costumers, because they know how chances work on casinos, if anything, an asset manager would rather use capital to start a casino but never to gamble on one.