JeromeL
This proposal does seem to help resolve the economic problems assosiated with the maximum blocksize issue. However it also appears to potentially undermine the core longest chain rule consensus mechanism. If blocks have different difficulty targets, which is the longest chain?
There is another potential solution I have proposed, which avoids this, which involves a seperate “mini proof of work” for each transaction, as well as the traditional proof of work for the whole block. However, if this is implemented then more powerful miners have a comparitive advantage over smaller miners and mining centralisation would be encouraged dramatically.
Although I think we are on the right track and this type of thinking could hopefully lead us somehwere useful.
Topic: Individual Block Difficulty Based on Block Size (Read 4688 times)
Sorry for reviving this topic but it's still pretty topical regarding the max block size discussions.
How would this blend with the rule : in case of conflicting chains, the chain with the maximum proof of work wins ?
This looks like a clever way to link the maximum block size to transaction fees.
How would this blend with the rule : in case of conflicting chains, the chain with the maximum proof of work wins ?
This looks like a clever way to link the maximum block size to transaction fees.
That's not clear at all.
https://gist.github.com/oleganza/8cc921e48f396515c6d6
We need, at a minimum, a majority of *active* hashrate to not "attack" the network.
You're trying to accomplish more than what is possible.The best achievable guarantee for mining is: "ought to find it more profitable to play by the rules."
There is no technical solution available to counteract attacks by highly-resources entities who do not care about profit.
The only thing we can do is to make Bitcoin so valuable that the opportunity cost of not playing by the rules is high enough to deter such attackers.
We need most of the planet's double-sha256 capability devoted to Bitcoin mining instead of doing something else.
As long as that condition holds, the actual hash rate isn't very important.
As long as that condition holds, the actual hash rate isn't very important.
That's not clear at all. We need, at a minimum, a majority of *active* hashrate to not "attack" the network.
Your economic theories aside, there is very little consensus on how we will achieve this.
I'm an optimist. I'd like to think that regardless of block size, a million people will donate a little bit of hashing on their USB sticks at a loss leading to an extremely hard to censor network. The other equilibrium aren't nearly so clean.
Mining is extremely important and core to Bitcoin. Let me try to describe some of the desirable characteristics of Bitcoin mining, which are often overlooked:
There is significant potential for weaknesses in mining incentives which could adversely impact some of the desirable characteristics of mining. In order for miners to keep doing the above, they need to be incentivised. Mining is crucial for determining the longest chain.
Maybe this is right, but even if most of the world’s hash rate is devoted to Bitcoin mining as there are not many other uses, if miners cannot make contributions to their marginal costs, why would they care that they help build the longest chain?
I appreciate I could be wrong here, maybe I have too much of a short time mindset when assessing Bitcoin game theory. As Mike Hearn recently said "Miners are not incentivised to earn the most money in the next block possible. They are incentivised to maximise their return on investment" [in the long run].
- Miners are always hashing, using significant real world resources, 24 hours and day, 365 days a year
- Miners always ensure they are working on the block which they think is in their own financial interest and are constantly evaluating which block to mine on
- Miners can change their mind about which block to mine on anytime they like and are therefore always contributing the determination of the longest chain
There is significant potential for weaknesses in mining incentives which could adversely impact some of the desirable characteristics of mining. In order for miners to keep doing the above, they need to be incentivised. Mining is crucial for determining the longest chain.
We need most of the planet's double-sha256 capability devoted to Bitcoin mining instead of doing something else.
As long as that condition holds, the actual hash rate isn't very important.
As long as that condition holds, the actual hash rate isn't very important.
Maybe this is right, but even if most of the world’s hash rate is devoted to Bitcoin mining as there are not many other uses, if miners cannot make contributions to their marginal costs, why would they care that they help build the longest chain?
I appreciate I could be wrong here, maybe I have too much of a short time mindset when assessing Bitcoin game theory. As Mike Hearn recently said "Miners are not incentivised to earn the most money in the next block possible. They are incentivised to maximise their return on investment" [in the long run].
I think we could do a lot to mitigate the second (see https://gist.github.com/gavinandresen/630d4a6c24ac6144482a for a partly-baked idea).
There is contradiction between Bitcoin's use as a settlement system and its unconstrained ability to revise its history (reorg).
The ability of revision will have to be constrained, as society will not accept a "technical subtlety" to override settlement of contracts that are considered final with "reasonable care".
Settlement cycles in the current banking system repeat typically a few times a day. Bitcoin would not be deemed suitable in the same context without a similar frequency of final settlement. This means reorgs that would revise the history more than a few hours back would not be accepted, at least not automatically. This makes me think that automated Bitcoin reorgs should be limited to 6-72 blocks.
We need most of the planet's double-sha256 capability devoted to Bitcoin mining instead of doing something else.
As long as that condition holds, the actual hash rate isn't very important.
As long as that condition holds, the actual hash rate isn't very important.
I think there's a large amount of cargo culting going on regarding the hash rate rather than useful analysis of threat models, attacker capabilities, and exactly what proof of work accomplishes.
I agree.
My guess is that we will end up with a very secure system with a modest amount of hashing in the future, because PoW hashing does three things:
1) Gives a steady 10-minute 'heartbeat' that limits how quickly new coins are produced
2) Makes it expensive to successfully double-spend confirmed transactions
3) Makes it expensive to censor transactions
The first becomes less important over time as the block subsidy halves.
I think we could do a lot to mitigate the second (see https://gist.github.com/gavinandresen/630d4a6c24ac6144482a for a partly-baked idea).
And I think the third might be mitigated naturally as we scale up and optimize the information sent across the network (there will be strong incentives to create "boring" blocks that don't include or exclude transactions everybody else is excluding or including).
The key is all those fees will just be going towards non-hashing. Security of network will most likely die with an infinite blocksize unless a minimum fee is imposed by miners via soft-fork.
There's a lot more to security than just hash rate.I think there's a large amount of cargo culting going on regarding the hash rate rather than useful analysis of threat models, attacker capabilities, and exactly what proof of work accomplishes.
stuff
The key is all those fees will just be going towards non-hashing. Security of network will most likely die with an infinite blocksize unless a minimum fee is imposed by miners via soft-fork.
I think Bitcoin with and without subsidy work very differently and have very different incentives and equilibriums.
The idea of block difficulty based on block size is interesting (and old). It is one of several ideas of how to regulate, smooth or create a market we know very little yet, since we have our subsidy and we''ll have it for a long time.
I suggest nothing is done before we actually get into the no-subsidy trouble, if we ever get into it.
I recall some other similar ideas I had in the past:
- The Co-var fee restriction: https://bitcointalksearch.org/topic/a-clean-solution-to-all-bitcoin-problems-satoshidice-block-size-future-fees-147124
- Spreading the fee of transactions over future blocks (the first miner gets only a % of the fee, the following gets another %, in geometrical or linear steps)
- Creating an open market where miners FIRST announce their fee/kilobyte (e.g. in the coinbase field) and then they are bound to that price (they cannot include transactions of lower fee/kilobyte). This requires also miners pre-announcing an pseudo-identity.
- Pre-announcing transactions in blocks so everyone has the same view of the market (a global tx pool). Transactions would be included in a special part for additional data in the block and they won't be "executed". The miner who first publish the transaction would take 50% of the fees when the tx is executed. A following miner would specify the transactions to execute (by tx-ids to save space) and claim the rest 50% of the fees. Non-executed transactions would be dropped from the global tx pool after a fixed number of blocks.
Maybe anyone wants to dig into them, or combine them with the dynamic difficulty idea, and see if they worth a math paper or a trash.
Best regards, Sergio
The idea of block difficulty based on block size is interesting (and old). It is one of several ideas of how to regulate, smooth or create a market we know very little yet, since we have our subsidy and we''ll have it for a long time.
I suggest nothing is done before we actually get into the no-subsidy trouble, if we ever get into it.
I recall some other similar ideas I had in the past:
- The Co-var fee restriction: https://bitcointalksearch.org/topic/a-clean-solution-to-all-bitcoin-problems-satoshidice-block-size-future-fees-147124
- Spreading the fee of transactions over future blocks (the first miner gets only a % of the fee, the following gets another %, in geometrical or linear steps)
- Creating an open market where miners FIRST announce their fee/kilobyte (e.g. in the coinbase field) and then they are bound to that price (they cannot include transactions of lower fee/kilobyte). This requires also miners pre-announcing an pseudo-identity.
- Pre-announcing transactions in blocks so everyone has the same view of the market (a global tx pool). Transactions would be included in a special part for additional data in the block and they won't be "executed". The miner who first publish the transaction would take 50% of the fees when the tx is executed. A following miner would specify the transactions to execute (by tx-ids to save space) and claim the rest 50% of the fees. Non-executed transactions would be dropped from the global tx pool after a fixed number of blocks.
Maybe anyone wants to dig into them, or combine them with the dynamic difficulty idea, and see if they worth a math paper or a trash.
Best regards, Sergio
I think you are trying to solve a different problem: I think you are trying to ensure that "enough" fees are paid to secure the network as the block subsidy goes away. Yes?
I think this is the problem foolish_austrian is trying to solve, yes.
We used to have a tragedy-of-the-commons situation with zero-fee transactions, but we solved that by rate-limiting them based on priority. And we have a working market for zero-fee transactions (see the graph here).
If you have a chance, please could you explain the graph in a bit more detail as I am not sure I understand it.
There is no tragedy-of-the-commons race to zero transaction fees, because miners do not have infinite bandwidth, memory or CPU to accept and validate transactions.
Gavin, you may well be right here, I do not know. Although, I still think there is some risk of a tragedy-of-the-commons race to zero/lower transaction fees scenario, you and others have convinced me this is smaller than I thought. I think the theory assumes that the marginal cost of bandwidth, memory and CPU for each additional transaction added to a block is becomes very low such that there is no longer any relevant marginal cost. The argument for this is similar to the reasons you put forward for the increase in a block size limit being ok, which is that the costs of these may decline exponentially over time.
Thinking about the above further, I don't think the race to zero is necessarily the issue, what matters is the overall mining cost curve and that there is sufficient differential marginal cost of adding transactions for different miners. If all miners have the same marginal cost of adding transactions to blocks, e.g. zero, then I think there is a risk that a race to the bottom type scenario could occur, however unlikely one may think this is. foolish_austrian's proposal is interesting as it ensures miners have a different marginal cost of adding transactions, although it causes many other problems.
I advocate keeping a block size limit not because it directly solves the differential marginal cost problem, but as an alternative mechanism to ensure fees are high enough such that we dont run into this potential issue.
Interesting idea, but I'm afraid I disagree with your premise.
There is no tragedy-of-the-commons race to zero transaction fees, because miners do not have infinite bandwidth, memory or CPU to accept and validate transactions.
We used to have a tragedy-of-the-commons situation with zero-fee transactions, but we solved that by rate-limiting them based on priority. And we have a working market for zero-fee transactions (see the graph here).
Assuming network bandwidth is the eventual bottleneck, and assuming there is demand for transactions to fill the available network-wide bandwidth (even if that demand is transaction spammers), nodes will start dropping transactions before they relay them. Prioritizing them based on fee paid and dropping the lowest fee/kb transactions will result naturally in a working market for fee-paying transactions.
As justusranvier points out, off-the-blockchain deals between transaction creators and miners doesn't change that logic, because low-fee transactions that are not broadcast break the O(1) block propagation assumption and have a direct cost to the miner.
I think you are trying to solve a different problem: I think you are trying to ensure that "enough" fees are paid to secure the network as the block subsidy goes away. Yes?
There is no tragedy-of-the-commons race to zero transaction fees, because miners do not have infinite bandwidth, memory or CPU to accept and validate transactions.
We used to have a tragedy-of-the-commons situation with zero-fee transactions, but we solved that by rate-limiting them based on priority. And we have a working market for zero-fee transactions (see the graph here).
Assuming network bandwidth is the eventual bottleneck, and assuming there is demand for transactions to fill the available network-wide bandwidth (even if that demand is transaction spammers), nodes will start dropping transactions before they relay them. Prioritizing them based on fee paid and dropping the lowest fee/kb transactions will result naturally in a working market for fee-paying transactions.
As justusranvier points out, off-the-blockchain deals between transaction creators and miners doesn't change that logic, because low-fee transactions that are not broadcast break the O(1) block propagation assumption and have a direct cost to the miner.
I think you are trying to solve a different problem: I think you are trying to ensure that "enough" fees are paid to secure the network as the block subsidy goes away. Yes?
I think you're misunderstanding the problem as this isn't about transaction selection. The issue is, should someone who is mining a more difficult block (say, 2x the normal difficulty) continue mining against an older block even after hearing about a 1x new block? To a naïve first-order approximation, they stand to benefit from doing so because a 2x block would beat out the 1x block and the would be able to steal the fees of the transactions in the 1x block. Of course there are a lot of assumptions wrapped up in that naïve assessment, nor is it clear that it is a reflectively stable outcome (if the 1x miners knew you would do this, how would that change their strategy? how would that change of strategy affect the profitability of this attack? etc.)
jonny1000's comment raises an interesting point.
According to the selected polynomial, we have very different ratios of normalized difficulties between smallest and biggest blocks (around x6 in first model, around x20 for n=100/p=0.99).
Am I wrong if I say that, with the latter, a chain of 6 (or 7, ...) smalls blocks should be considered as less secure, since a single big block could orphan them ?
Usually, people consider a transaction "safe" after 6 blocks. With this model, we really should think in term of work produced after the block embedding the tx.
I guess the choice of the polynomial plays an important role to mitigate this effect.
I think the biggest risk is the complexity factor.
I have clearly underestimated this statement.
It isn't just smaller blocks it is larger blocks as well.
Say miner A is targeting a block 2x the average and miner B solves a 1x block that takes most of the highest fee txns from miner A. Since miner A knows that when it solves its block it can orphan miner B block it may be optimal for miner A to continue building on its own chain. Depending on its share of the network the optimal choice may be to attempt orphan block B by a size just marginally larger than B (say difficulty 1.0000000001) and since txn will be included in a fee descending order that my recover 90%+ of the fees. The analysis is already complex and we are just considering miners looking to optimize revenue (at the detriment of network security). When you start to consider an active attacker is becomes even more complex.
If the txn an attacker wishes to double spend is included in a block with a large amount of fees the security of that confirmation goes down. The attacker can solve and broadcast a smaller containing only the double spend and thus free up the fee revenue which may encourage otherwise legitimate miners to build off the attack chain in order to collect the fee revenue for themselves.
Say miner A is targeting a block 2x the average and miner B solves a 1x block that takes most of the highest fee txns from miner A. Since miner A knows that when it solves its block it can orphan miner B block it may be optimal for miner A to continue building on its own chain. Depending on its share of the network the optimal choice may be to attempt orphan block B by a size just marginally larger than B (say difficulty 1.0000000001) and since txn will be included in a fee descending order that my recover 90%+ of the fees. The analysis is already complex and we are just considering miners looking to optimize revenue (at the detriment of network security). When you start to consider an active attacker is becomes even more complex.
If the txn an attacker wishes to double spend is included in a block with a large amount of fees the security of that confirmation goes down. The attacker can solve and broadcast a smaller containing only the double spend and thus free up the fee revenue which may encourage otherwise legitimate miners to build off the attack chain in order to collect the fee revenue for themselves.
I see, yes I misunderstood the problem.
If a miner sees a block 1x the network average, it would have just cleared out the most profitable transactions. While waiting for the transaction pool to refill, miners should continue to mine those same transactions, ignoring the solved block. If they find a block that is slightly smaller, then they can reveal it to the network. The network will look at the second block, and because it leaves a larger pool of transactions, they would should all switch to the second block and ignore the first block, since their blocks will be more profitable mining on top of the second smaller block. Actually, after every block there would be a short time period where everyone would profit by trying to orphan the known block by producing one slightly smaller...
If a miner sees a block 1x the network average, it would have just cleared out the most profitable transactions. While waiting for the transaction pool to refill, miners should continue to mine those same transactions, ignoring the solved block. If they find a block that is slightly smaller, then they can reveal it to the network. The network will look at the second block, and because it leaves a larger pool of transactions, they would should all switch to the second block and ignore the first block, since their blocks will be more profitable mining on top of the second smaller block. Actually, after every block there would be a short time period where everyone would profit by trying to orphan the known block by producing one slightly smaller...
I think you're misunderstanding the problem as this isn't about transaction selection. The issue is, should someone who is mining a more difficult block (say, 2x the normal difficulty) continue mining against an older block even after hearing about a 1x new block? To a naïve first-order approximation, they stand to benefit from doing so because a 2x block would beat out the 1x block and the would be able to steal the fees of the transactions in the 1x block. Of course there are a lot of assumptions wrapped up in that naïve assessment, nor is it clear that it is a reflectively stable outcome (if the 1x miners knew you would do this, how would that change their strategy? how would that change of strategy affect the profitability of this attack? etc.)
Yes, but it is not immediately obvious what effect that has on mining incentives. Someone needs to actually work out the possible mining strategies here.
I started writing a simple simulation that generates a distribution of miners with differing efficiency. As I was programming the behavior of the miners, I started by assuming miners will act rationally and always try and maximize their profit, and from there I plan on trying to have individual miners behavior differently and exploit the network.
An interesting consequence of them action rationally is that if the expectation value of their profit per hash E
I think it should be easy to prove that a rational miner will always mine the largest fee per kb transactions. I would think this proof already exists in a general form somewhere. Let S=[s1, s2, s3...] such that s(n) > s(n-1). Let P = S xor s1 so that P = [s2, s3, s4...] and take S-P you get s1, which means S is larger than P by s1.... errrrrrrrr, did I just prove the axiom of an ordered set? Anyway, someone could probably do this better than me, but the intuitive statement is that if you remove highest fee per kb transactions, you always lower the mean fee per transaction.
For proving a Nash equilibrium exists, I had to look this up, and it seems like an interesting problem. While I think I can say I have a conceptual understanding, the proof is probably beyond my expertise in game theory or my ability to generate rigorous proofs (which is like 5 lectures on open-courseware and what I gleam from my wife - she's a math teacher).
Yes, but it is not immediately obvious what effect that has on mining incentives. Someone needs to actually work out the possible mining strategies here.
Jump to: