Question is what's the best strategy to now go to the desired end state:
Singlesig (offline-wallet) --> Multisig (offline-wallet)
In that case, you don't really have any option other than sending the UTXOs from one wallet to the other and paying the transaction fee each time.
Why do you want to move the coins to a multi-sig wallet? Is your offline wallet no longer secure? Is leaving the coins where they are not an option?
If you do still want to send them across, then you can either wait until fees are lower or just bite the bullet and pay the fees. You will have to spread the transactions out over a period of time so they are not obviously linked together. You could potentially consolidate two or three of the UTXOs together each time to save on fees if you are happy with the privacy implications of doing so.
albert0bsd had the same question. I believe OP wants to ameliorate his setup and I don't believe it is a bad idea. But since two members of this forum found this curious, I can't help but ask, why? I mean don't you think it is a good idea to move the coins to a better setup? Of course provided that the multisig vault will be properly set up. Fully airgapped, different devices, proper backups etc.
Yes! a properly set up singlesig wallet, where the seed phrase is produced from a permanently airgapped device, where the backups are stored offline in separate locations, where you add a strong passphrase that provides >128 bits of entropy is safe.
But, a multisig vault can have some advantages compared to the singlesig wallet:
1. In terms of safety, I reckon that both wallets are impossible to get compromised with brute-force.
2. In terms of theft, then the multisig vault is safer because the attacker will find one backup and they have no clue that you have set a multisig vault with another wallet. They will probably think of bruteforcing the passphrase though.
3. In terms of physical disaster, multisig and singlesig are equal if you have backed up the signers properly.
4. In terms of usage, signing a transaction with a multisig is much easier, because you can create a PSBT with the 1st cosigner and then sign with the rest of them, bringing the PSBT to the rest of the signers. However, with singlesig you need to get the passphrase where the device is, unless you remember it, but this is highly unlikely with a proper random passphrase.
5. In terms of fees, provided that we talk about the same amount of inputs and outputs, singlesig is cheaper.
Having said all that, I want to make clear that multisig creates a false sense of security. Multisig is in general safer, indeed, but you must know what you do and avoid obvious mistakes. Multisig is not a panacea.
Supposing that OP knows all the dangers and follows all the necessary measures, then, according to the list above, I think moving to a multisig is not a bad idea.
But if the only motivation is security, OP must know that a proper singlesig wallet with a strong random passphrase is safe enough.