Good point this is one of the bad sides to fully integrate cashless society. There will be problem you point in maybe biggest one, how will they do it. Every system is not perfect but cashless society maybe primary goals could be to be perfect in some way. Also if this cashless society would be on internet payment or online, that would be even bigger problem cuz not all places in the world are cover will net. Still from this there are more problems then any good solution for this.
I guess that the negative implications for a cashless society outperforms the positive ones. While it may take a few years for it to become a reality, this could speed up from doing so as internet access becomes available to anyone in the world.
To reach a large scale, I believe that at some point in the future the internet would be distributed to any region worldwide via the use of satellites to transmit the signal for it, just like it is done with TVs nowadays.
When a solution for a worldwide internet comes in, a fully cashless society would come into effect enabling governments to have full control and access over our financial information, eliminating our rights for privacy, but greatly reducing criminal activity such as money laundering due to the complete transparency such system would have if powered by a blockchain.
Right now, all of the technologies required for this become a reality are available, and it wouldn't be surprising if governments begin using implants, along with AI, IoT, and the blockchain to achieve not only a cashless society, but a fully autonomous and transparent society giving them the full control over our most sensitive information whenever personal, medical, or financial ones.
Nevertheless, before this becomes a reality, we should be prepared to embrace it in the future. Just my thoughts