I don't understand how your algorithm will give the entity with higher hashpower more profit by mining blocks that give less BTC per block, that doesn't make any sense to me.
Because in so doing, the miner increases the penalty pool, and will reclaim it in his future blocks.
Why wouldn't the bigger miner mine smaller blocks and get the higher BTC per block, like your example was showing?
Because if he does, the penalty pool will be smaller, and he will get less BTC per block. If nobody creates supersized blocks, we go back to scenario 1. The big miner, like all other miners, will get only about 4.50 BTC per block. When the big miner creates supersized blocks, he gets 4.68 BTC per block.
On the other hand, if it really is more profitable to mine larger blocks, then why wouldn't the smaller hashpower nodes do it?
Because if a small miner creates a large block, the effect on the penalty pool is not as meaningful as the effect the big miner has.
All miners want to maximize their (tx fees - penalty) + collection. Collection is equal to the average penalty paid by all miners. A small miner plays only a small part on this average, so collection is constant as far as he is concerned, so he more or less wants to maximize just (tx fees - penalty), which happens at around 6000 txs. If he goes further, he will decrease his (tx fees - penalty) with barely any effect on collection.
A big miner has a big effect on the average. As he creates blocks larger than 6000, (tx fees - penalty) decreases but collection increases at a faster rate. Above 7250 txs, the gain in collection doesn't justify the drop in (tx fees - penalty).
if this is true;
Reward per block for 1% miner: 5943 * 0.0006885 + 3.2223 - 0.4589 = 6.8552 BTC (more than in scenario 1)
Reward per block for 90% miner: 7251 * 0.0006885 + 3.2223 - 3.5294 = 4.68521 BTC (less than 1% miners in this scenario; more than the miners in scenario 1).
Why isn't the 90% miner simply including 5943 transactions like the 1% miner is, in order to get about 2 BTC more per block?
Because if he starts creating 5943-tx blocks, the penalty pool will be smaller, and so will the amount he collects per block. See scenario 1
here - if the big miner behaves as the small miners do, it's the same as if they're all small miners. He will get just 4.50 BTC per block, instead of 4.68. He can't have the cake and eat it too - not pay a penalty, but expect to reclaim it...
I'll say again - the default state is where everyone just maximizes their own (tx fees - penalty). A big miner can afford to build bigger blocks than that because he knows he can reclaim 90% of the penalty, so it doesn't bother him that much. In so doing, he increases his profit per block - but he increases the profit of the miners who don't do this even more.
At some point, the increase in penalty outweighs the increase in tx fees + collection, so he stops there. Whether one or more of these summands is more than 2 BTC is irrelevant, it's their sum that counts.
The problem is, regardless of what the collection is in the pool, the highest profit per block is always greater by simply mining at the target block size.
If the major nodes have driven up the BTC in the collection pool to provide greater profits on the whole, then the first big miner that stops contributing to the pool will essentially drain the pool by mining blocks at the target block size. In the end, trying to maximize profits by relying your competitors to "play fair" won't work.
I'm not making any assumptions about fair play, reciprocity, collaboration or any such thing. All miners are behaving selfishly and without regard to the other miners. However, I am assuming each miner expects to continue mining, at a similar hashrate portion to his current (or, alternatively, he bases his calculations on his future anticipated portion). The big miner will create supersized blocks simply because he know he'll be around to reclaim some of the penalty later. So he doesn't maximize (tx fees - penalty), he maximizes (tx fees - 0.1*penalty), which happens at larger blocks, with a steeper marginal penalty.
If there are several big miners, they should all create supersized blocks, and then every miner will enjoy the supersized blocks of the others. But the reason for each individual miner to create supersized blocks is
not that he hopes others will do the same. It's simply because it's more profitable for him to do so, regardless of what the others do. This is not the prisoner's dilemma, where there is a conflict between selfish desires and the greater good.
Of course, it
is possible to miners to try and band up in a cartel, creating blocks even larger than what is selfishly optimal for each of them individually. But such a structure will not be stable.