Topic: Really not understanding the Bitcoin XT thing... (Read 5820 times)

Not really. It follows the same growth pattern, not necessarily the same rate. A linear growth will always surpass a logarithmic growth, regardless of the rate. Once we are using the same functions, (k log(x)), suddenly all that matters is the rate (k). There is no indication we would be on that border exactly, we could be way below or way above it. What do the law of physics have to do with block space demand again? Note that I am not arguing its effect on supply.


The only way this can happen is if a miner is purposefully creating his own tx and not emitting them publicly. Mempool hit rate is just another propagation system, with time it will always tend towards 100%. The scenario you describe where the mempool hit rate constantly hovers at 95% if only possible if a miner is purposefully creating transactions at a pace and withholding them. That makes no sense as it increases his propagation time (reduces his profit) and motivates other miners to eventually ignore him. On contrary, I would speculate all miners will make a point of sticking to well propagated transactions particularly to suppress their orphan rate.


What makes you believe that information cannot be communicated pre solution? Also look at IBLT implementation. The intent is to give other miners the rules you build your own blocks with. There is no reason a miner propagating an IBLT should stray from the pattern it describes. Again, miners have a clear incentive in fast relay networks to use as predictable a tx set as possible. That means ignore transactions that are not fully propagated and stick to your IBLT.


No I only need 51%. The rest of the network will get steamrolled out of business pronto. Part of your paper relies on how rational miners should account for propagation time in their tx set choice (the model you propose is naive but let's keep it at that). Now somehow you are arguing 2 things to me:

1) You are suggesting rational miners won't take every opportunity to reduce propagation time.

You state yourself that a high propagation time will result in lower revenue. Rational miners are motivated by profit, why would they stay away from any coding gain?

2) You are suggesting irrational miners have a stake in this game.

Clearly they don't, their only fate is to get eaten alive by rational miners, or bleed money from an external source ad nauseum (or until that source depletes I guess).


You always have to propagate the block header and your coinbase tx at solution annoucement. But there is no indication we have to stick to a network that has to propagate every other bit of information in a block along with the solution. We have solutions being developed that allow miners to only ever have to emit the header and coinbase tx. The Nash equilibrium will keep all miners doing just right that as long as propagation time inversely affects profit (read forever).

So the amount of data emitted is static and the network bandwidth will keep growing until we reach the physical limit of the carrier. Therefor the propagation time is tending towards 0 (remember math 101, tending is not reaching). So we do have a situation where blocks will propagate at insignificant speed compared to the block emission period, without the poles inverting.


No I purposefully omitted it, replacing it with ellipses to signify that it was an excerpt from a sentence or paragraph, as common citation etiquette dictates.

Correct.  Even with coding gain, the result holds.  


If the information required to be sent grows logarithmically with block size, then the market would be exactly on the border between healthy and non-healthy.  

But this is a bold claim that requires further justification.  Even doing as you suggest, I cannot see how the information won't be linear with the block size.  For example, you said you still need to communicate "any TXs that you know aren't part of other's mempool".  I agree.  But if we imagine that 95% of the transactions are common between mempools, then the number of missing TXs still grows linear with block size.  

Furthermore, you still need to communicate the order in which the transactions are sorted in a block (even if all the TXs were common between mempool).  Communicating the order of a list of transactions requires an amount of information linear in the number of transactions in that list.  

Lastly, even if you can come up with a scheme to do what you're suggesting (and I don't think you can), you still need to get 100% of the contributing hash power to go along with it.  If only 90% does, the fee market remains healthy.  

I think you are looking for ways around a Law of Physics here.  The only way the market will not be healthy (according to the definition in my paper) is if exactly zero information about the transactions included in a block is communicated with the block solution announcement and this holds for every block solution announcement.


You missed the rest of the sentence .  I think I agree now that you point it out the word "full" is ambiguous and should be changed, however.

Ok, this post is long enough already so I'll leave the last half of your response for later...

The whole point of "anti friction" measures like relay networks and IBLT is to communicate as much data as possible about a block "out of band" (while the block is still being mined). This doesn't simply mean it takes less bytes to communicate a block, it also means the bytes sent to bytes used ratio won't necessarily grow linearly anymore.

If you have to communicate as many bytes as you use in a block, then your analysis stands. If a coding gain can be achieved in the form of lossless, flat % compression, then ratio sent/used will remain roughly static, describing a linear growth. Again the model will stand.

If however the only data you have to transmit at propagation time is what you cannot communicate out of band, i.e. block header, coinbase transaction and any tx that you know aren't part of other's mempool, then the sent/used ratio will grow logarithmically, which is the same as saying that propagation speed (Note speed, not time. Higher speed means faster propagation) will increase exponentially with block size. This would put us in the case you describe as the "non-existent market".


Maybe organic or self sustained are better wordings. Semantics are relevant in a research paper.


This is from the introduction:


The full cost to post transactions is not explicitly defined so it is reasonable to expect it means "the cost to maintain proper difficulty" (which is largely accepted as the main purpose of transaction fees as inflation diminishes), and this is the assumption I was operating on until I reached page 7.


If there exists a market condition where a miner's profitability per hash increases with total hash rate, then there is a centralizing pressure on mining, as larger miners will out earn their competition. Either we're talking pools and hash rate providers will centralize around a single large pool, or we're talking actual miners and the largest of them all will win the "arms race" by default.

If you doubt propagation time can have this effect, then consider that Q* grows with a miner's hash rate, as they have to propagate their solution to an decreasing portion of the network to have it validated. After all, they only need to propagate to a cumulated 51% of the network's hash rate.


If block size limit is a production quota, then coinbase reward is minimum wage or government grants. Both are externalities and both have devastating effects on an economy. Yet both are currently in place in the Bitcoin network (and every alt coin out there has a mix of both), and the only true centralizing force at the moment is the heavily subsidized cost of electricity in certain parts of the world. Clearly the macro economics of classical commodity markets do not apply (at least as is) to crypto-currencies.


And yet that situation is defined outside of your model: some transaction emitters would mine their own transactions, and others will pay them to mine theirs. If your model cannot come to this conclusion then it is at least lacking something. Do you realize that according to your results, no one would ever mine on the testnet chain?

Well written post.

The paper shows that as long as a non-zero amount of information about the transactions included in a block is communicated (on average) during block solution propagation, then the fee market will be healthy, according to the paper's definition of a healthy market. This assumption about information propagation has held over the entire history of bitcoin, will hold if the entire network uses the Corallo relay network, or adopts any implementation of IBLTs I can imagine.  The fee market exists.  

In hindsight, perhaps I should have chosen a word other than "healthy" that would have been less controversial.  Nevertheless, I define precisely what I mean by a heathy market in Section 7.  This figure illustrates it:



A healthy market is defined as one in which a rational miner will be incentivized to produce a finite block.  In other words, I show that block space behaves as a normal economic commodity (both the laws of supply and demand hold).  

You could argue that the market is "not healthy" (using a different definition) because the equilibrium block size in a free market would result in some negative externality (centralization--although I don't buy that).  And then you would propose that a quota on the production of block space needs to be put into place, for some greater good.  However, before we go down the road of production quotas, we should take a long look at other economies where they've been implemented.  The most famous is of course the USSR, but even in Canada we have quotas on the production of eggs.  These were originally implemented to ensure that farmers could earn a living wage and the public could enjoy a reliable supply of eggs; however, they now serve to keep small famers out of the commercial egg market and result in all sorts of lobbying and palm greasing (and higher prices for consumers).  He who controls the quota wields the power to pick winners and losers...



The model simple no longer applies as R/T -> 0.  It's neither right nor wrong.  It's sort of like an equation where you have 0 divided by 0.  It's undefined.  You need to evaluate the limit.  

I don't know if fees will be enough to secure Bitcoin 50 years down the road.  My work said very little about this.  That's the topic for a future paper.  

This debate has become academic.  The network has already chosen a path, and XT ain't it. BIP 100 looks like the winner, and it also looks like a reasonable compromise; introducing a miner voting variable, and setting a 'soft' block limit based upon the lower bound of the three center quintelles of the vote distribution.  Looks like it will function similar to how the target difficulty system works, but with a 12 week period (instead of two) and a 12 week delay period.  It also has a factor of 2 max change limit, just like the target difficulty system; with a 32 meg upper hard limit.

i don't think its a matter of becoming feasible, its a matter of making it the primary data source. I.e., if blocks increase in size, bitcoin shifts from the concept of the groundlayer of cryptocurrency architecture and into the "all transactions on the primary blockchain" architecture. The more and more data leaves the primary blockchain, the less valuable it becomes.

re: second bold - once you introduce monetization into this equation, the lack of an intuitive link just transforms into another data mining problem.

hence probably why confidential transactions are in the works, which are somewhat dependent on the adoption of sidechains, which is somewhat dependent on the blocksize not changing. And if CT make their way into general adoptions (and I hope they do), you can say goodbye to data harvesting from the blockchain.

Man, dependencies are even a PITA when compiling reality!

I am pretending it would happen with large propagation times, so we are not contradicting each other.


I am not doubting the value of data mining financial transaction logs, but you would have to prove Hearn is trying to modify the network in a way that it becomes feasible on the blockchain, and that such way can remain private to a few select people in order to turn a profit from this method first hand. After all, this dataset is public and there is no intuitive link between addresses and individuals.


I like Ken

Possibly open a book on what their sfII character of preference is  Unless they're one of those "wildcard" assholes    ( )

aaaae lookout, checkout the Street Fighter II challenge between kanoi and goatpig.
one half of the ck bitcoin mining software limited  joint venture versus Armory Technologies engineer.
Bitcoin software engineering's finest battle to the death. This will definately be in
Bitcoin the movie.  

It's incendiary one way or another.

To get a second opinion, find someone who has worked in dynamic display ads and ask them how much they love data.  High value data like precise transaction logs probably has a broad range of opportunities for monetization which go far beyond simply spamming people with ads, but I suspect that just that alone would be enough to support a very robust monetary framework infrastructure and reason to work hard on gaining the largest footprint possible.

This doesn't happen.

May well in some future scenario, but certainly not now.

You affirm this because you are not considering the compound effect of the significant network "friction" the example assumes, as well the new incentive the orphan race creates for M and m.

Let's go back to the example in my previous post: M as the large miner, m as the small miner, N as the neutral miner, and significant propagation time. What you are stating is that in the case of a race between M and m, it doesn't matter to N which block it mines off of, it only matters that both blocks extend the last known top. After all, you are only "wasting" hash rate if you mine on top of a block below the max height.

Or are you? Clearly there is no incentive to ignore a valid solution if the only thing you are going after is the coinbase reward + average fees. But what if there was an incentive? Say someone emits a ZC paying 1000 BTC to address A with a small fee, and as soon as this address is mined, emits a transaction B that spends the same TxOuts to address B, but this time paying a 50BTC fee. Wouldn't it be in every miner's benefit (but the one which mined the block with A) to orphan that last block?

Not saying this what's happening in the orphan race per se, or that it is good practice to accept a 1000 BTC payment after a single confirmation, but what stands is that there can be incentives to actively orphan a block. And in the case of an orphan race, it is a coinbase reward + fees riding in the balance.

The proposition goes as follow: in the case of a race between M and m, M is expected to recruit over 51% of network the hash rate behind its block simply because M > m. Now we have 2 miner groups, where G mines off of M's block and g mine's of m's block. There are 2 outcomes:

1) G finds a solution first for height H+1, validating M's block at height H. At this point m has lost the race and its block reward, but this result is irrelevant to others miners in g.
2) g finds a solution first for height H+1. However, M knows it can propagate its solutions faster than m, so there is a window in which M can actively try to orphan g's block and still propagate its solution for H+1 to 51% of the network faster than the g miner can. M has an incentive to do so, which is to save its reward of H.

What does that mean for N?

a) If N is part of G and finds a solution to block H+1 first, there is virtually no chance this solution will get orphaned. After all, N is guaranteed M will back this solution, so there is at least that much extra hash rate guaranteed, which reduces the propagation time of N's solution. If m can't compete M's against propagation time, there is no reason to believe it can compete against M + N.

b) If N is part of g and finds a solution to block H+1, it knows M will attempt to orphan this block for a short period of time, and may succeed. M will try to orphan N's block simply because a window exists in which it is more profitable for M than to start work right away on top of N's block.

So N has no chance of getting orphaned if it is part of G, and a non zero chance to get actively orphaned if it's part of g. What motive does N have to be part of g? What motive does N have to not be part of G? If one solution has a quantifiable risk and the other doesn't, all of this for the same reward, why pick the risky one? Keep in mind that for N, there is no cost to switching from one solution to the other. N can start mining on top of m and switch to M later at no loss.

Point being, this situation can take place without the need of a 51% cartel.
You could argue that M is a cartel. But it doesn't need be 51% to present that active orphaning threat to smaller miners.
You could argue that over 51% of the network using the same relay network is a cartel, but then again their motivation to join the relay network is not to actively orphan smaller pools but to reduce their own orphan risk. There is no agreement that other members of the relay network will support your block in a race, but there is a strong indication that they will get your solution faster than they will get one from miners outside of the relay network.
You could argue that a mining pool is in fact a cartel in its own right, but this is only evidence that any barrier to entry will promote the formation of cartels (after all a solo miner is always guaranteed to reduce his propagation time by joining a pool, and gets other benefits on top of it).

"But I've never seen blocks emitted so close one another"

Far from me to doubt your figures, but the reality is that the 1MB block cap prevents propagation time from exploding upwards. The point isn't so much that concurrent block solutions are propagated within mere seconds of each but rather the average propagation time based on position in the network topology and hash rate. If 20MB blocks were to bump your propagation time to 30 sec, you would be clearly be a victim of this mechanic.

"But it would never reach 30 sec"

30 sec is a bloated value for the sake of this example but is it really all that unrealistic? I think that currently, ~60% of the network hash rate is in China. They naturally propagate fast to each other, and slowly to the rest of the world. How big does the average block have to be in the absence of fast relay networks for Chinese miners to receive it in an average 30 sec? Do we really want to try and find out?

"But no miners submit themselves to this logic atm"

That's mostly because the low block size hard cap prevents network propagation from getting out of hand. If the hard cap goes away with the current network topology, eventually some miners will start exploiting this aggressive orphaning scheme and the rest of the network will have to go along. And there already is a group in a prime setup for this purpose: Chinese miners.

"But cartels!"

While I am trying to demonstrate this particular scheme does not require a 51% miner/cartel, the scheme on its own is only enabled by long propagation times. This characteristic of the mining network creates imbalance and modifies the rules of the game in way that promotes the formation of cartels, the same way the heavily subsidizes cost of electricity in China is, in my opinion, the main factor for the current state of the mining market (where Chinese miners dwarf everybody else).


This ties back to the original topic, and the question that birthed this entire argument:


That's outright disturbing if this presentation stands true. Otherwise it's kind of incendiary. I don't want to judge before I see the actual proposal. Surely someone knows about it.

Large pools are made up of individual miners.  If a pool does something people generally don't like, miners will stop mining on that pool.

It's extremely frustrating, even if somewhat amusing, when economists continually fail to take real world data into account for their on-paper-it-sounds-good theories.

So for all these years that Bitcoin's been trucking along working beautifully and gaining more and more mining power while never coming close to the cap, suddenly now when it threatens that system and prevent users from being able to make transactions, suddenly now you think it's a good idea to let that happen, even though Bitcoin's been working great without an effective cap for years?

Bitcoin's usefulness relies on people USING it.  You think clogging it up so that it becomes more expensive and less useful to users is the best long term approach to scaling and/or maintaining Bitcoin's value?

Jesus Fucking Christ, you guys are either stupid, insane, or short-term financially motivated.  There's just no nicer way to say it.

That ignores co-location propagation advantages though. I would suggest that the miner should choose a solution emitted by one of the other miners in their own group to accelerate propagation even if they are the smaller miner.

This is not how Bitcoin propogation is supposed to work.  If this is part of the XT proposal, it would destroy bitcoin, by centralizing more than can be mitigated with a fee market anyway.  Such a change would render nodes no longer peers, as larger pools would be superior.  This would break the 51% rule.

Ah, I see.  Not a student of economics.  You just completely lost me with this one, and since I don't have the time to consider every point of view, you also just earned a spot in my ignore list.

The business intelligence value of knowing who made what transactions alone is easily enough to pay for operating the network.  Especially when one has access to 'big data' processing facilities and a big network footprint already.  Much higher value than knowing something about people's search terms or scanning their e-mail and cloud storage contents.  I would expect to see users (to use a more kind descriptor) get 'cash back' when things are ramped up.

I have little doubt that Mike is keenly aware of this principle, and I suppose that he didn't really want to use it as a sales pitch so he cobbled together some fairly questionable 'mining assurance contract' scheme or whatever it was.

You never receive 2 at the same time.
It will 100% always be at different times, one after the other.

OK a pool has X%, then they only need 51% - X% actively switching to their block instead continuing on a current block at the same height to help them win an orphan race they would expect, on average, to lose.
That's still a 51% cartel.
However you word that, you need collusion to get other pools to consider doing that.

Blocks are rarely if ever very close together.
I don't know when I last saw 2 less than 2 seconds apart in my logs ... so it's so rare that it's basically next to irrelevant.

My logs are millisecond accurate, I've changed cgminer and ckpool is the master gits to be ms accurate, all my logging scripts are ms accurate and I modify the relay to be ms accurate ... though the current commit in the relay that shows the block time with 1 second accuracy - I created the pull
The main reason for saying this is that I've done monitoring of all sorts of stuff on the network against my pool (that I know most pools haven't even looked at ...)

The reality is that if you start mining on a later, same height block, you will be mining on the losing block most of the time you do that.
Yes there may be some rare cases where that isn't the case, but a very high % of the time you will be mining on the losing fork.
BUT ...
Now does that matter which fork you mine on? No. Since all that matters is who finds the next block to decide the fork.
Bitcoin core's rule is ludicrously simple and works.
If you are building on a block at height X, and another valid block appears at height greater than X, start building on the new block. Anything else, ignore it.
It doesn't matter which (valid) fork that new block is on, just switch to it.
So yes you could choose to help some other pool confirm it's 1s, 10s, 100s late block, if you choose to, but you wont gain anything on the bitcoin network by doing that ... without a cartel setup where they say they will do that for you also.

See what matters is getting your own block confirmed, not some other block.
So if you send out a block to the network, ALL you are hoping is that the majority of the network is building on it.
Except in the rare case when some other pools sends out a block around the same time, you don't have to worry.
If on the other hand, some other pool did send out a block about the same time, you CERTAINLY do not want to switch away from building on your block.

As for block propagation and orphan rates ... well ... my little less than 1% pool has pretty good orphan rates ...
Who knows if that is due to a small sample (2 out of 317 blocks in 11 months) ... ... or what it is