It will be a problem for little while.
Bitcoin miners aren't stupid, even Bitmain will realize that huge price increase will help them much more than exploiting ASICBoost or AntBleed of whatever they have hidden within their ASICS.
We have first sign of Chinese mining pools interested in switching sides - ViaBTC recently hold a poll on their twitter. They asked what community think of SegWit activation. SegWit received 83% votes.
Here is why I think otherwise:
Jihad Wu and Roger Ver have an army of brainwashed people in /r/btc that will follow anything they say.
If they finally give up and accept segwit, their entire community will collapse within itself and they will be no longer seen as "saviors of bitcoin" by those idiots.
Having a bunch of people aligned within your agenda is a very useful power. They know that they will lose this, if they give up and adopt segwit. So they have something to lose.
Sure they will gain acceptance by the rest but they will always be seen as people you can't trust by everyone else outside of the /r/btc community.
And while all of these game theories interact and create drama, LTC has all the technology needed to deal with mainstream adoption.
It is then reasonable to buy and hold LTC for the time being.