Olympics is a place where the best athletes around the world meet once every 4 years. Despite football being the most popular sport in the planet, many of the star players have participated in the Games. Cristiano Ronaldo represented Portugal in 2004, and Lionel Messi took part in the 2008 edition. If we take the case of basketball, NBA legends such as Shaquille O'Neal and LeBron James have been part of Olympic squads. Cricket is being the odd one, having refused to take part in this competition till now.
It is supposed to be a great honor for any player to take part in the Olympics as well. But the problem is, as you said, "cricket has been the odd one". And it is not because the players do not want to play in the Olympics. It is because the cricket authority, the ICC does not want cricket to be included in the Olympics.
I also think that some cricket boards also do not want cricket to be included in the Olympics to some extent. As a sports body, the ICC should want which is the absolute best for the game. But they just want to think about money. That's why cricket took so long to be included in the Olympics. And even when it is being included in the Olympics it is only six teams play in the Olympics. Which is not a great sign. This means the sports body and certain cricket boards still do not want cricket to be included in the Olympics. They are just doing it because they have to.