Would it be possible to, say sell bitcoins in exchange for CCs and require the other party to enter into legally binding arbitration? Would this work?
It would probably be a breach of your merchant conditions. It's the credit card provider who sets the merchant and customer conditions regarding chargebacks, not the merchant or the customer.
Another problem is that the person using the credit card isn't going to be the credit card-holder in a lot of chargeback cases. The card details are going to have been used fraudulently and you're not going to have the details of the person who actually bought (and most likely immediately sold) the Bitcoins from you. Even if the person using the card had agreed to arbitration, the actual card-holder wouldn't be bound by that agreement.
Also, arbitration is not free. It's something you'd need to cost into your overheads. There's a high level of credit card fraud in "card not present" transactions so you could reasonably expect a lot of chargebacks. You'd be the one paying for arbitration and the onus would be on you to prove that the transaction was legitimate - how on earth would you go about that?
