I don't think so. If we will have a cashless world, the more we become slave by other people because they can hold or freeze our bank accounts even if we will not give our password to them. Why? It's because they can hack our accounts without any hesitation. Unlike when we still have money, we still have the authority of controlling it because for example our bank accounts will be hacked by someone, the bank owners are responsible to retrieve it so we can get back what we lost, but if it's already cashless, we can't get our share anymore and worst we can't eat or we can't have our freedom anymore if we will not follow what the highest authority will command us to do.
The only reason why the banks give you back your money if your bank accounts gets hacked is because they have unlimited resource of fiat money. Yes, the hackers will be the modern thieves. But regardless of what type of monetary system that we have, be it bitcoin or fiat or cashless society, there will always be opportunists looking for people to take advantage of. Nothing really has changed once cashless world has achieved in terms of theft. The only solution is to be prepared for the different attacks that can be done to us and to counter that attack. Be aware of these attacks and think like the thief so you wouldn't have the constant fear that things can be taken away from you any time.
Secure your funds enough, and you don't have to feat the cashless world. The whole world is a game and a lot of people are playing, you better know the techniques of people to manipulate you into giving them the advantage that they want.
It will always depend on how much risk a person can handle. Just like what you have said, there will always be bad people who will do bad things in order to get money. Bottomline is not only we need to be wise enough in using our money but also how much risk we are willing to take about our finances. Just like true investors always say "Don't put your eggs in one basket".