You can't get rid of the prompt.
Ledger wallets bitcoin chrome app is the UI for both chains, BTC and BCH.
You will be able to access both chains with any of those two apps opened (BTC/BCH). But you only will be able to spend from the chain, whose app you have opened on the nano S.
As far as im concerned there is no way to 'skip' this prompt when interacting with the chrome app.
You could switch to electrum[1] and use this as your interface. The security stays the same. The private keys don't leave your device and you have to manually approve transactions.
[1] https://electrum.org/#home