Slowing down block propagation - page 2.

2112

legendary

Activity: 2128

Merit: 1073

Quote from: DeathAndTaxes on March 03, 2015, 10:53:01 PM

Out of process or not, you can't have O(1) block propagation time IBLT unless the entity you are broadcasting a block to already had the transactions. IBLT can't make up data that is not known, it can only allow other nodes to determine which subset of memory pool are included in the block. The relay network doesn't change that dynamic in the slightest.

You again start writing with a style and confidence similar to Mircea Popescu. I mean the absolute value is similar and the difference is in the sign/direction.

Quite obviously: transaction propagation (1) and block header propagation (2) are two different things and there's no point to keep conflating the two. The respectable and not underhanded miner will propagate transactions first before propagating the discovered block headers. For the (1) part the average propagation time is in the order of 5 minutes, only in the (2) part every millisecond matters and directly affects the probability of orphans.

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Out of process or not, you can't have O(1) block propagation time with IBLT unless the entity you are broadcasting a block to already had the transactions. IBLT can't make up data that is not known, it can only allow other nodes to determine which subset of memory pool are included in the block. The relay network doesn't change that dynamic in the slightest.

2112

legendary

Activity: 2128

Merit: 1073

Quote from: DeathAndTaxes on March 03, 2015, 08:51:49 PM

This is a misnomer. With IBLT the block propagation time is only O(1) is all other nodes also contain that transaction. A miner has no way to of knowing if all nodes contain all transactions as if the txn volume exceeds the bandwidth availability or other resources of a given node then it will drop some transactions. The smart miner would not take all txns especially not those with an almost zero fee but instead would try to estimate the resources of the entire network and choose the highest paying subset of txns that are likely to be in the memory pool of a majority of the nodes.

So yes there is still an 'additional cost' for including additional txns. If the txn included isn't known by all nodes then it will have to be relayed directly and that increases block propagation time and orphan cost.

I think you are presuming that the miners are still using the legacy network protocol as implemented in the core client. I think it is already obsolete and anyone who mines uses the new "relay network" protocol implemented out-of-process by Matt Corallo.

https://bitcointalksearch.org/topic/how-and-why-pools-and-all-miners-should-use-the-relay-network-766190

I'm going to assume that your analysis is obsolete purely because for anyone really mining (pools or large farms) the new protocol is the way to go.

And even if the new "relay protocol" isn't going to get included in the core client it is a no-brainer that any large scale progressive-thinking miners should have private connections between themselves, possibly secured with IPsec. Ideally they should have a multicasting backbone established amongst themselves (kinda like NASDAQ extranet), but I'm too cynical to believe that they are technically capable of that.

onemorexmr

sr. member

Activity: 252

Merit: 251

i see two problems here.

Quote from: DeathAndTaxes on March 03, 2015, 08:51:49 PM

If the txn included isn't known by all nodes then it will have to be relayed directly and that increases block propagation time and orphan cost.

as you stated a miner does not know which transactions are known by other nodes. so he does not have a choice: he must publish all transactions in his block.
and if i understand you correctly he must do this fast (a priori?) otherwise he risks an orphan (because other miners cant build upon his block).

^ if that is true all is fine: but thats definitely not O(1).

Quote from: DeathAndTaxes on March 03, 2015, 08:51:49 PM

A miner has no way to of knowing if all nodes contain all transactions as if the txn volume exceeds the bandwidth availability or other resources of a given node then it will drop some transactions.

if - because of bandwith - not all transactions reach a node the node cant decide which one to take. maybe an upstream proxy could be developed but that sounds a little strange to me.

it gets worse, because different parts of the network will see and keep different transactions.

did i misunderstand you?

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: onemorexmr on March 03, 2015, 08:05:31 PM

its just rational to take them all: they have already received and validated them, putting them in a block does not add additional costs.

This is a misnomer. With IBLT the block propagation time is only O(1) is all other nodes also contain that transaction. A miner has no way to of knowing if all nodes contain all transactions as if the txn volume exceeds the bandwidth availability or other resources of a given node then it will drop some transactions. The smart miner would not take all txns especially not those with an almost zero fee but instead would try to estimate the resources of the entire network and choose the highest paying subset of txns that are likely to be in the memory pool of a majority of the nodes.

So yes there is still an 'additional cost' for including additional txns. If the txn included isn't known by all nodes then it will have to be relayed directly and that increases block propagation time and orphan cost.

onemorexmr

sr. member

Activity: 252

Merit: 251

Quote from: 2112 on March 03, 2015, 07:31:51 PM

Go back and read the Microsoft's "Red Balloons" paper. This has been discussed here and elsewhere many times.

http://research.microsoft.com/apps/pubs/?id=156072
http://research.microsoft.com/en-us/projects/socialalgs/bitcoin-red-balloons.aspx

ok, add an incentive for nodes to broadcast transactions is nice, and solves the problem partly.

IMHO the other part is, that if miners are not punished for bigger blocks it means that they will put all transactions with fees in any block which leads to the problem that is not possible to build a fee-market.

its just rational to take them all: they have already received and validated them, putting them in a block does not add additional costs.

2112

legendary

Activity: 2128

Merit: 1073

Go back and read the Microsoft's "Red Balloons" paper. This has been discussed here and elsewhere many times.

onemorexmr

sr. member

Activity: 252

Merit: 251

i know this sounds ridiculous, but let me explain why i think it may be useful.

i am nearly convinced now that a big block size together with 1BLT is a threat for bitcoin as miners dont have any incentive to make smaller blocks.

so i am thinking about ways to incentive them.

an easy solution could be that a block needs to be transmitted in full to be accepted - the question is, if this is enforceable between miners (sadly i think it isnt).

btw... not heard much about headers first. is the following scenario possible?

miner A mines a block with a few transactions he has crafted himself.
he published a double-spent-attempting transactions shortly before he found the block.

miner B receives the second transaction from miner A
miner B receives the new block header and starts mining
- as he does not have the first transaction he tries to include the second one

^ this should raise miner-B's orphan rate.

an easy way to mitigate this would be to mine empty blocks until you have all transactions from the previous block. but this would lead to more empty blocks - raising the blocksize-scarcity even more

i am a little lost right now...

Topic: Slowing down block propagation - page 2. (Read 3669 times)