This is true, and in particular few countries are in dominance. Why this is happening? All are due to the revenue generated out of cricket. If cricket is taken to Olympics and to other World level meets, automatically there'll be more countries participation. This will make ICC loss its control. Another important thing, few countries that are into dominance will also suffer as more countries make fast progress.
As far as I understand, these are the concerns raised by the BCCI and ICC on Olympic inclusion:
1. Foreigners won't be allowed to represent. Olympics and Commonwealth Games can be participated only by citizens of a particular country
2. Stringent testing against performance enhancing drugs.
3. Cricket board need to report to the national Olympic association for matters related to Olympic Games
4. No automatic qualification. All the teams need to participate in qualifier tournament
5. For the duration of Olympics, the cricket calendar maybe blocked due to unavailability of players
6. Revenue distribution will be decided by the Olympic committee.