You only need fill the wallet *once* though for it to be exchanged an infinite number of times. eg: look at a casasius coin-- once it is bought by somone there it no stopping how many times it can be physically exchanged between different people before someone put it back into a different wallet a new transaction on the network.
Also, you only need of regular totals of off-chain wallets to make any amount you wish to exchanged: eg- you can have .... 0.001, 0.002, 0.005 0.1, 0.2 0.5, 1, 2, 5, 10, 20, 50, 100BTC,.... totals in the wallets and you exchange the key-pairs of the required numbers of wallets to make up the any amount you wish (ie: similar to how regular denomination coins/notes work in real like currency)
You could also create a paper note with a private key inside. Like printing a QR code on the outside of a sealed envelope. So how do you get someone else to trust it? You would have to get the vendor to 1) trust you, which would likely require identifcation; 2) trust that the currency is legitimate, i.e. not counterfit; and 3) trust that the mint that created it was trustworthy.
Casasius is a well trusted community member, but even his coins are not universally well regarded. If he can make one, someone else can fake it. Eventually this is going to happen.
Furthermore, loading your wallet to exchange still requires on-network transactions. It's impossible to always avoid that, no matter how many off-network trades you can do.
Yes, you need to trust someone when you exchange *physically*, but when you exchange electronically you only need to trust a computer (and of course whoever wrote the program-- but with open source with is not such an issue).
Such electronic transfers in the real world already exist, but currently require special hardware-- eg: smart transport cards for public transport where you load money onto the card. This technology is becoming available on desktop computers-- it is being installed on the desktop to prevent piracy: eg, remote attestation, secure booting, etc..
As regards the fact that it still needs an on-network transaction to load the wallet-- it only needs to be done *once*. Once it is off-network, it can stay off-network forever and still be exchanged.