I agree with your first sentence of gambling being where we spend money, but the popular notion with people is that gambling is a means to earn money, and some even go as far as taking gambling as means to become rich quick which have oftentimes end them in frustrating outcome and result.
Gambling shouldn't be taken as anything other than fun because once you take away to the fun from gambling, you end up with frustration since the house edge is always there to put the casino at an advantage over the players.
Well, having the mindset of gambling as fun is the best, but still, you may at the same time not deny yourself of making the money in there if you have the opportunity, and the opportunity will not even come if you do not give it all that it takes. For me, the only difference here should be the mindset we have in trying to make the money in gambling, if the mindset is bad, then everything could be bad and can lead to desperation and frustration. But if the mindset is good, no matter the outcome, it will not hurt much even as the gambler is not taking it as a do-or-die affair, it is never a must to earn through gambling.
Whether we like it or not as well, gambling can't be pretended as an avenue to have fun without the temptation of making money, and if I may ask, how many people are even having fun in gambling or taking it as a fun-filled avenue? If there are, the percentage will be so little. This means that most people are gambling for the money and we can't eliminate the fact that so many people are professionals too and are making money through it. This is why I always advocate sports betting to make gambling easier and still override some of what you alleged gambling and the house of. But when it comes to the casino itself, then you could be mostly right.