I thought measure of work is based on the difficulty of the hash, which
has nothing to do with how many transactions are in the block.
If the network consensus was that 5 MB blocks are valid, and a pool were to mine 3 consecutive 4.5 MB blocks, these blocks were to properly propagate throughout the network, then all nodes that are in receipt of these blocks should mine on top of the 3rd 4.5 MB block received. Under your proposal, a miner could ignore those blocks if they arbitrary thought those blocks was too large and could start mining on top of a chain that is not the most cumulatively difficult chain