Topic: are there any actual stats on chain reorgs, by depth? (Read 2028 times)

thanks for the confirmation. One additional nicety would be if the setback limit divides evenly into 2000. I guess it could be 8, but that doesnt conveniently add up to 2000.

My plan is to have smaller bundles of 100 and 10, so at most there would be 19 mini-bundles of 100, 9 microbundles of 10 and 9 blocks for a total of 37 things to search through

As far as my node data, there have been only 2 instances of a reorg over 10 blocks, both due to significant unique events. 1 instance of a normal 6-block reorg. 2 instances of a 3 block reorg. 8 instances of a 2 block reorg. And lots of 1 block reorgs. 4 or more block reorgs are *extremely* rare. The standard client considers anything under 6 blocks to be unconfirmed. That seems like a good number to use for your purposes too.

Are you seriously saying that since A -> B, if C -> B then C -> A can be concluded?

Like: you wake up in the morning. criminals wake up in the morning. Therefore you are a criminal.
or: you breathe air. criminals breathe air. Therefore you are a criminal.

what is the name for this logic, I dont know about it and want to find out more. I know about transitive https://en.wikipedia.org/wiki/Transitive_relation when A -> B and B -> C that A -> C, but the reverse, this is quite novel. I guess if people operated as strict field elements with specific properties of reverse transivitism(?), maybe there is some basis for this, but last I checked I wasnt a finite field element. Are you?

What is a bit frustrating is that you mix in some highly intelligent stuff with total nonsense, but I am just supposed to accept it all. otherwise I am like a fraudster or mobster? I dont ask for any respect that I dont deserve. If I make a logic error or math error, then point it out. make fun that I am a bad calculator, fine. But this nonsense attack by implication that is supported by absurd "logic", well maybe you are the one that needs to go to school? Maybe I need to be your teacher about some things.

If you cant find any flaws with the logic or math, then you could just say that or be silent about it.

James

P.S. I have done most of my learning by using google to find relevant pages, so there are likely gaps in my knowledge as compared to someone who spent many years learning all the vital things that are learned in schools.

I don't know. What I do know is that fraudster and gangsters (more generally common criminals) have exceptional sensitivity to being given "no respect" (cue Marlon Brando in The Godfather). Of course, correlation is not causation. But see how DeathAndTaxes reacted to the mention of money orders in 2012. I think that even he didn't know at that time what scam he's going to pull of years later.

https://bitcointalksearch.org/topic/m.1036760

Your reaction to the mention of high-school-level science curriculum is very similar.

Again, time will show.

Thanks! This really helps me find the math error in my analysis. I was missing the storks.

so we just need to wait stork number of blocks and that will be the optimal for mountain climbing.

I guess you think satoshi was an idiot too, as he said something about 10 blocks. Or was he committing fraud or whatever this crockpottery is.

ad hominem attacks vs. historical results with some actual math

And what exactly is this fraud you imply I am conducting? That using a hard coded lookup table (generated from p2p network and validated) instead of a DB is more efficient. Is that my supposed fraud?

James

Take mountain climbing as a hobby. Your main adversary will be then impersonal gravity. But you'll learn to deal with reality and how to overcome obstacles. Or you'll get yourself killed.

I'm actually proud of getting called troll by now know fraudsters like DeathAndTaxes or shtylman. On you the question is still open: are you leaning more towards harmless crackpottery or towards willful fraud? We'll see.

This is public forum. I make post useful to the global audience of the readers, even if they aren't useful to you or any particular poster of any thread.

Your math error is called: GIGO https://en.wikipedia.org/wiki/Garbage_in%2C_garbage_out .

The mistake you are making is common enough that there's lots of educational texts regarding the "correlation vs. causation" issue. When I was in school profs used to refer to an excellent joke paper written by some Scandinavian scientists about presence of storks and childbirth rates in Scandinavia. I'm not sure if it was ever translated to English.

Edit: More seriously: many law schools offer "personal development" classes on how to deal with adversity in a public courtroom. Apparently it is a common problem for even excellent students that were home schooled or went to religious schools.

I dropped out of kindergarten, I couldnt handle being shamed and put in the corner when I asked questions I wasnt supposed to.

Rather than disprove how my analysis that 10 blocks is sufficient, you state that my analysis is bogus. So I should set it to 100 blocks? 1000 blocks? Some random number? That is your recommendation? Or maybe I should go to DBMS school since apparently without a DBMS Phd nothing I write has any validity.

What does my education have to do with the analysis that 10 blocks appears to be sufficient? It might make others question your objectivity if you keep taking every chance to try to make me look stupid. Or you dont mind if people just categorize you as some type of troll? I would think a smart person would want his point of view appreciated and not just dismissed as troll blather.

I appreciated your post about the getchaintips RPC, that was useful. Maybe you can stick to making useful posts? Like math based analysis, which is what I tried to do. Did I make any math errors?

If the statistical model is dominated in the real world by non-PoW factors, then that is certainly an important issue and might need to be factored in. If so, what should the setback be? would 1 year be enough time to be sure?

James

You seem to have an excellent command of English for a non-native speaker.

At the same time you seem to lack basic scientific education at the level of high school, where typically the "correlation vs. causation" is explained.

In a previous thread you've exhibited 0/10 knowledge of basics of DBMS.

I have no idea where you've received your education (home schooling? religious school?) but I haven't used any harsh words. I've used standard sentences that would be said by a teacher to a completely unprepared student face-to-face in any normal school. I'm not going to even delve into further scholastic experience like participating in competitive sports or military training.

Why are you always using such harsh words? Maybe I am an idiot and you are super genius that is always right about everything.

I specifically limited my question to issues unrelated to software versions. just trying to understand things one piece at a time.

Now monsterer is saying it is always the same, you say it is probably going to increase. But I agree with you about the non PoW factors most likely dominating and since I am always wrong, does that mean you are wrong too?

James

Your analysis is bogus. Your are mistaking correlation with causation.

The long term changes in the orphan rates are caused not by changes in hash rate but by changes in the number and connectivity of the mining pools. The number of pools decreased and the connectivity of the remaining pools improved. In particular Matt Corallo's Relay Network caused decrease in the orphan races.

The probability of reorgs probably going to increase in the future due to the bugs in the competing implementations, due to DDoS attacks on the mining pool's infrastructure and due to Matt Corallo finally abandoning the maintenance of his Relay Network.

The difficulty increase is just an innocent bystander.

Sure, theoretically, but I dont like to just assume that the theory is precisely followed by reality.

My concern is with the practical. When did the ASIC's come out? That is a nonlinear increase in hashrate per amount of capital.

Moving forward is it likely to have the same hashrate improvements was was possible a couple years ago. Not sure on the state of ASIC geometries, but if already the smallest practical geometry is being used, then it cant really get much more cost effective. Larger, yes, but with proportional cost for the silicon. Cleverer, yes, but possibly most of the low hanging fruit is already done

Not sure the state of mining at block 225K, but given those factors it is possible that things are still exactly the same, but I am skeptical about that. What are the currently envisioned improvements in hashrate efficiency?

Actually instead of just guessing, lets look at https://blockchain.info/charts/hash-rate?timespan=all&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address=

the alltime chart shows a definite slowdown of hashrate increase, so that does not seem the same to me. Granted from one block to the next, not much changes, but the discontinuity of new tech is what would create the temporary ability for a bigger reorg.

01/01/2010 18:15:05,0.006997468320659102
01/01/2011 18:15:05,128.16206770778084      18315x 2010
01/01/2012 18:15:05,8591.40080069935          67x 2011
01/01/2013 18:15:05,24291.440498714688      2.82x 2012
01/01/2014 18:15:05,11740802.089632047      483x 2013
01/01/2015 18:15:05,335365290.0920246       28.56x 2014
01/01/2016 18:15:05,697129166.4058079       2x 2015

As things went from CPU -> GPU -> FPGA -> ASIC and as market price is changing dramatically we see the change in hashrate vary from 18315x to 2x, so it does not seem safe to assume that things are the same. The 2014 gain was probably skewed by the bull market peak price bringing in a lot of new capital, but maybe that is around the same time ASIC made a big leap?

The most likely way for an attack against bitcoin PoW seems to me for the creator of advanced tech to have a disproportionate hashrate per capital available, especially when efficiency gains are measured in the orders of magnitude. However, once the orders of magnitude technology gains are already done, it becomes a matter of much more "incremental" improvements of doubling per year.

At least that is what I had guessed and the numbers indicate something similar to that. One way to look at this is back in 2013 what was the likelihood of a 500x hashrate increase? Was it really such a remote possibility? And what is the chance of a 500x hashrate increase now?

Is there really any chance that next year we see 1,122,825 * 500 terahashes? or even 10x.
Yes I know the difficulty adjusts, but that is every 2016 blocks, there is a window of time where a technical leap increases the chance of larger reorg, but as time passes I think this chance is less and less.

James

It was the same then as it is now - back then, network power was less than it is now, so although difficulty was lower than it is now, blocks were still 10 minutes each.

hopefully a few more long time nodes can post and we can get a much clearer idea.

just having the second node confirm the 6 block reorg changes that from probable isolated node issue to a likely global reorg. and it wasnt that long ago, so the hashrate required would have been a lot. maybe it was during the DDOs attack?

Standard PC running the default Linux client with basic cable modem connectivity. It's online probably 95% of the time, which would mean it could have missed a few reorgs. There could be an interesting story about that 6 block reorg. That does appear to be the biggest naturally occurring reorg.

Fantastic!! thanks for this invaluable history, 2012 is quite a ways back. Also reorgs from bugs/new versions are different from "normal" reorgs caused by propagation

and the hashrate was a small fraction back around block 225K. Not sure, but I think it is harder to reorg now vs then? If only since we now have ASIC so the tech leaps in hashrate from CPU to GPU to FPGA to ASIC is already done.

It seems vast majority are just 1 deep.
around 10 2 deep
just a few 3 deep

the 6 block reorg seems to be an outlier along with the 30. All the talk about eventual consensus, but with bitcoin it looks like 99.999% of the time, within 3 blocks it is in consensus

unless your node is special.

with propagation of around 15 seconds though, it seems other than at the edge of one block, it would take three fast blocks to require a 3 block reorg. Maybe that 6 block reorg was an actual attack? it seems way out of bounds and both your node and 2112's has it so it is unlikely that is it some local connectivity issue

since nowadays the miners are all probably doing their best to push the new blocks as fast as possible, that combined with the nextgen ASIC probably not going to be much more than 2x increase, maybe as little as 5 blocks would be enough, but there isnt much difference between 5 and 10, so probably best to setback 10 blocks

Extremely! This is closely related to the odds of pulling off a double-spend 10 blocks deep. D&T explained nicedly at https://bitcoin.stackexchange.com/questions/1170/why-is-6-the-number-of-confirms-that-is-considered-secure.

The worst was the overflow bug. That would have been around 53. https://en.bitcoin.it/wiki/Value_overflow_incident

My node only goes back to 2012, so I missed out on the overflow fork. But I have the next big one. My node saw the .7/.8 fork at a reorg of 30:



Feel free to analyze my data at http://pastebin.com/LZxst5vD.

Has anybody seen a 10 block reorg in the last year?

It sure seems like a good value for a backstop. The cost being a few minutes recalc and offline time if a bigger reorg happens

what was max depth with the one of 165 branches?

The results I posted were from a portable laptop that had seen multiple ISPs but runs only occasionally. On the other laptop that had seen only one ISP but runs more often I had 165 branches instead of 15.

You'll need way more nodes to have meaningful statistics.