Propagation times and orphan rates would pretty much guarantee that very large blocks would be orphaned.
No thrifty pool operator would make large blocks anyway beyond a few experiments: it just would not be worth the orphan risk unless those large blocks are full of high fee tx (and even then)
There are already regularly nearly empty and fully empty blocks because of propagation and orphan issues.
IMHO the block size debate is like counting angels on a pinhead: it is rather pointless unless propagation & orphan issues are fixed.
If we look at it in Fairlight, fairglu , we will understand that removing the limit altogether (or placing it too high) creates an incentive to do just that (fix propagation & orphan issues), which in turn allows the network to consolidate around high-bandwidth lanes leaving the home-based demographic out of the picture, while Bitcoin becomes nothing but a Dead Ringer to its true self.
Oh, and by the way, truth doesn't need a BIP.