How much slower is a big block propagating than a small block?

cunicula

legendary

Activity: 1050

Merit: 1003

Quote from: Gavin Andresen link=topic=324972.msg3514227#msg3514227

I am much more worried about transaction fees right now than "selfish mining". That is why I've been working on better fee estimation in the reference implementation.

Good to see you have your priorities straight. Algorithmic fee adjusment is a very difficult problem. Consider getting input from economists. I'm a know it all, but I recognize this problem as much too difficult for me. I'm worried that it may be impossible to prevent supply from stepping in to fake demand (mining their own txn ans paying fees to themself). The only leverage you have is the miners' loss of fee revenue due to the sacrifice in block space.

I expect you need a very clever mathematical economist armed with rigorous proofs and stuff. Without rigor, the problem is going to be too complex to grasp. (Too many things are going on simultaneously to use intuition.)

Gavin Andresen

legendary

Activity: 1652

Merit: 2316

Chief Scientist

Quote from: Sergio_Demian_Lerner on November 05, 2013, 09:27:04 AM

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.

Is this correct?

Yes.

I am much more worried about transaction fees right now than "selfish mining". That is why I've been working on better fee estimation in the reference implementation.

I agree with Mike-- we need better measurement, and we need to make "full-match merkle block relaying" the default to drive that 80ms number way down.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Right now, it's more important that you avoid requesting a block from one of the vbitcoin nodes

I imagine it's how blockchain picked up the 268083 as an orphan before receiving 268082

re: bigger blocks. i'm sure most of these pools are on good connections, so it's probably not much of an issue. if you're solo mining on limited bandwidth, then, yeah, you'd want to reduce the size.

cunicula

legendary

Activity: 1050

Merit: 1003

Quote from: Sergio_Demian_Lerner on November 05, 2013, 09:27:04 AM

Quote from: Gavin Andresen on November 04, 2013, 07:38:50 PM

See:
http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

Quote

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay: sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
http://www.verizonenterprise.com/about/network/latency/

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.

Is this correct?

This works for an atomisitc miner, I think.
As pool size increases, however, I think that orphan costs decrease and eventully become orphan benefits. Once you are at say 60%, the failure of your blocks to propagate is going to mean that you get more thqn 60% of block reward.

justusranvier

legendary

Activity: 1400

Merit: 1013

Quote from: justusranvier on November 04, 2013, 11:42:52 PM

1) Make the nonce long enough that the extraNonce field is no longer needed in the coinbase transaction.

2) Now it's possible for miners to broadcast their Merkle tree as soon as they start hashing (10 minutes on average before they finish)

3) When they find a valid hash, now they only need to broadcast the block header because the rest of the network has (usually) already received and validated the Merkle tree.

4) Block header propagation is very fast and not dependent of the size of the blocks.

Using this model, miners can create large Merkle trees without worrying as much about orphans, because they get to broadcast them in parallel with searching for a valid hash. Probably miners would want to limit their Merkle tree to the amount of data they can transmit in 2 minutes to make it likely that the rest of the network has received and validated it before they finish hashing.

This is a much more efficient use of bandwidth and allows final header propagation to be extremely rapid and independent of the number of transactions in a block.

Also, it may have positive implications for the Selfish Miner attack.

Mike Hearn

legendary

Activity: 1526

Merit: 1134

Blocks can be easily sent as full-match merkleblocks. It just wasn't a priority to do that yet.

I'd like to see real-time metrics on block propagation and such. I suspect the picture is a lot more complicated than a fixed 80msec per kb (transferring data isn't that slow and modern nodes are much faster at verifying transactions than they used to be).

cunicula

legendary

Activity: 1050

Merit: 1003

Quote from: DannyHamilton on November 05, 2013, 09:05:15 AM

Quote from: cunicula on November 05, 2013, 09:03:22 AM

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

Wouldn't the forgone transaction fees become the orphan cost?

If the other txn fees are zero, then the orphan cost is also zero.
You only begin to get positive txn fees when you run up against a block size constraint.
Here, I guess gavin is saying that there is an intrinsic block size constraint even without a coded block size limit.
Running up against this constraint will force people to pay fees.
The constraint also encourages larger pools. Orphan cost will decrease in pool size.

Taken as a whole, this seems like a bad thing to me.

Sergio_Demian_Lerner

hero member

Activity: 555

Merit: 654

Quote from: Gavin Andresen on November 04, 2013, 07:38:50 PM

See:
http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

Quote

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay: sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
http://www.verizonenterprise.com/about/network/latency/

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.

Is this correct?

DannyHamilton

legendary

Activity: 3528

Merit: 4945

Quote from: cunicula on November 05, 2013, 09:03:22 AM

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

Wouldn't the forgone transaction fees become the orphan cost?

cunicula

legendary

Activity: 1050

Merit: 1003

Quote from: Gavin Andresen on November 04, 2013, 07:38:50 PM

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

Gavin Andresen

legendary

Activity: 1652

Merit: 2316

Chief Scientist

See:
http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

Quote

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay: sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
http://www.verizonenterprise.com/about/network/latency/

SebastianJu

legendary

Activity: 2674

Merit: 1083

Legendary Escrow Service - Tip Jar in Profile

Quote from: gmaxwell on November 04, 2013, 05:18:02 PM

It takes longer to transmit more data, it takes longer to run more checksig operations (the latter reduced somewhat by caching, if the transactions had already been public). Thats it.

So its inevitable that big blocks, that contain all transactions, are slower and have the risk of being orphaned? I dont think that is good for the network.

Cant this be speed up anyhow? There is no small datapart that can be checked beforehand so its known that the block is correct. Then the incoming block could be streamed already while incoming to the next nodes without the full block proof. Nothing like that possible?

gmaxwell

staff

Activity: 4326

Merit: 8951

It takes longer to transmit more data, it takes longer to run more checksig operations (the latter reduced somewhat by caching, if the transactions had already been public). Thats it.

SebastianJu

legendary

Activity: 2674

Merit: 1083

Legendary Escrow Service - Tip Jar in Profile

There is a tip that not including transactions in a block will help propagate the block faster because of the size. Even though i dont know that this was done already it looks like many miners exclude transactions with no and low fee, probably for that reason.

Now my question is... is the slower propagation of big blocks only because it takes a bit longer to transfer the block or is there another waiting time until that block is checked for some reason. So that it might take longer to check if that block is valid until it spreads.

I mean nothing holds a miner from, for example only include transactions he gets a decent fee from. If this helps him propagating faster it might be a good move for him. But it wouldnt for the network.

So is there a way to make it more fair somehow so that big blocks have the same speed of propagation (on the assumption they are similiar connected to the network) like small blocks? A found block has to be transferred in full, checked in full and sent to other nodes in full again without a chance of speeding it up? I guess sending a block forward before it was fully received and proofed might mean a security risk of ddosing the network.

Topic: How much slower is a big block propagating than a small block? (Read 1628 times)