My understanding is that decision making power in this case very much rests with the developers. The consensus so far seems that Segregated Witness will be proposed as a soft fork when the BIP is published. Mining pools, exchanges and processors have no direct say. All that is needed for the BIP to be merged is agreement among the lead developers. That would be Van der Laan, Gavin Andresen, Jeff Garzik, Gregory Maxwell and Pieter Wuille. Correct me if I missed a name or am in error.
This will be very true if you add Btc guild's owner and dial back time to 2013, where they successfully prevented an accident fork by commanding more than 60% of the hash power and forced the chain go back to their desired direction
And today? Btc guild does not even exist, large mining pools from China have dominated hash power distribution, and even inside the core devs there are large disagreement
The core devs are losing dominance because bitcoin is open source. Suppose that a group of large mining pools are not satisfied with the solution proposed by core devs, they could easily deploy their own R&D people and make a version that they prefer, and because they have dominant hashing power, when they roll out the change, their fork will be the strongest chain immediately. Of course they could not persuade others to join them, and they risk being abandoned by the other pools and exchanges, but having so much hash power definitely can affect lots of things
This was not the case during early days of bitcoin when mining pools were just a couple of hobbyists, they didn't have resources to implement such large code change, they only listen to core devs (I still remember the deepbit pool refused to upgrade to any latest version and stayed at 0.3 for a very long time
That could be the earliest case of mining pool not following core devs)
Fortunately, chinese mining pools are not very interested in codes, so core devs still hold a lot of saying power, but when there is a disagreement inside core devs, we have already seen on BIP101, miners have veto right
Let me quote Bitmain’s Pan Zhibiao's word here:
“There are many technical solutions. Every solution has pros and cons. The miners are put on a pedestal to be a jury. Right now there is no lawyer on both side. We need a lawyer, we need more discussion, more evidence. ”
I happened recall the movie "Divergent", there are 5 factions, each doing its own job, maintaining a balanced society, hopefully we can reach that status of balance here