As far as I have experienced, using the XAPO card for purchase at merchant's POS is free as long as your card's currency matches the POS terminal's currency. In this case the only drawback can be the exchange rate that XAPO uses to convert your bitcoins to fiat on the fly. But I'm sure that if you count your time and your comfort, you can choose the card instead of some other exchanges like localbitcoin and other exchanges available to get the necessary fiat for your BTC.
I think XAPO knows this as well this is why they're setting their exchange rates the way they're doing it now.
I got the card quickly, activated it on the Xapo site, loaded it with some bits and used it at a local store no issues and no fees in the same currency.
HOWEVER, the selling exchange rate in my experience is ~5% lower than the market which is substantial. I emailed Xapo to see how exactly they're getting the debit card exchange rate.
Just do a check here on the forum, there is a official Xapo services thread, in there they explained how the whole exchange rate thing works. The fees are included in there as well so it not the actual exchange rate. IIRC they were working on having the information display correctly, i.e. properly itemised, on the website when you make a transaction.
Will see if I can find the post and post the link here.
EDIT: Actually found it quickly, was first item on the search. See this post from Xapo
https://bitcointalksearch.org/topic/m.12758123