If BitcoinXT becomes the majority and then something needs to be changed, it'll take months to move between soft or hard forks. It'll be planned and we'll know the level of consensus amongst nodes months in advance.
In short, there's no easy option for bitcoin devs to just make a fickle or odd change to the network without a chance for other devs to review and users to reject.
As Far as I am aware there are only 2 developers that are giving approval to the blocksize update in XT, while at least 5 developers have to come to consensus with Bitcoin core. XT could certainly continue merging contributions made from the ~304 developers who work on core, but there would be a new precedent set that we should ignore the consensus process with BIP on core and simply fork the main repository if we cannot get agreement.
Personally, I see nothing wrong with Gavin and Hearn's ability or right to fork core as that is how open source works and ultimately the miners and full nodes will decide which fork to use. I do find it revealing that Gavin decided to discuss the XT solution to bypass consensus before he submitted a BIP proposal. He must have expected disagreement and is using XT to leverage for a change to core.
From the looks of it we may see a few developers walk away when consensus is reached which will be bad for everyone because all of them have made great contributions in the past.
I look forward to adjustments in the proposed BIP's or new ones that we all can rally behind. We need to start listening to each other a bit more and addressing each others concerns. Perhaps if Gavin's proposal could be merged with Garziks and there are other changes proposed that address some of the concerns with centralization or giving to much control to the miners.