Inoue is fighting in a stadium with 50k capacity. Smaller fighters just don’t draw those kinds of crowds anywhere else in the world. Inoue has fought outside of Japan several times. It is not a matter of not wanting to fight outside of his home, that’s just where it makes the most financial sense to stage his fights. For example, we can see that the Garcia vs. Haney fight is struggling to sell tickets because neither fighter has a large fanbase in New York.
He is with the top Rank, I'm sure Bob can market him in the US but I guess his camp just want to protect his legacy as having a pound for pound like Inoue is not something they enjoyed most of the time, I mean I haven't known a boxer before from Japan who is as accomplished as Inoue now.
He is co-promoted by another Japanese manager, but yes he is with Top Rank but I don't think that Arum are forcing him to go and box in the US unlike Manny Pacquiao. In this stage he could really reach what Manny had done and even more. Tyson himself says that Inoue is better than Manny, but he refuses to fight in the US so we won't see how Americans are going to embrace him.
And most likely why he is not in the top list of pound for pound, although that list could be debatable. But unlike Manny during his prime and winning in the US, he become #1 for years. Maybe the camp of Inoue will go to the US once they go up in weight class at 126 lbs.