are there any actual stats on chain reorgs, by depth? - page 2.

jl777

legendary

Activity: 1176

Merit: 1134

Quote from: 2112 on March 18, 2016, 05:03:25 PM

Quote from: jl777 on March 18, 2016, 04:54:43 PM

very nice! didnt know it listed all reorgs.
so if I read that right since block 395487, the max depth reorg has been: 1

if so, does that mean a node that has been constantly running for 1 year would be needed to run getchaintips, to get the entire year's history of reorgs?

It seems 10 blocks at current hashrate would be very rare

James

More or less. But remember that the reorganizations aren't really globally identical. You'll really want several long-running nodes on several ISPs to have meaningful statistics.

Also, testnet has way more and longer reorgs, very good for testing the handling of the reorgs.

reorgs are totally dependent on each node, yes. but if your node is seeing max reorg depth of 1, i would think any node that is actually connected to the internet and not being flooded, would not be seeing reorgs of 10 blocks

For testing I can just manually force a reorg, I just wanted to calibrate things with the right initial offset using current network characteristics

3 seems too low, but 10 should be at most a monthly occurance, which is fine for a several minute recalc. But if anybody has more extensive data or much worse reorgs, please post!

it could be 2112's node is special

2112

legendary

Activity: 2128

Merit: 1073

Quote from: jl777 on March 18, 2016, 04:54:43 PM

very nice! didnt know it listed all reorgs.
so if I read that right since block 395487, the max depth reorg has been: 1

if so, does that mean a node that has been constantly running for 1 year would be needed to run getchaintips, to get the entire year's history of reorgs?

It seems 10 blocks at current hashrate would be very rare

James

More or less. But remember that the reorganizations aren't really globally identical. You'll really want several long-running nodes on several ISPs to have meaningful statistics.

Also, testnet has way more and longer reorgs, very good for testing the handling of the reorgs.

jl777

legendary

Activity: 1176

Merit: 1134

Quote from: 2112 on March 18, 2016, 04:44:36 PM

The official Bitcoin client has "getchaintips" command that gives you the output like this:

Code:

[
    {
        "height" : 403263,
        "hash" : "00000000000000000210fee63635c72da771c6b136a63f20ff9d058ce778618d",
        "branchlen" : 0,
        "status" : "active"
    },
    {
        "height" : 402922,
        "hash" : "00000000000000000692c289cc4a4e9726018d6a030ad10c84f70adc2e920597",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 402751,
        "hash" : "00000000000000000180755378324c16847b2f49af428c763b8ff411d51d7767",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 402723,
        "hash" : "000000000000000000263e1722b119e2547a4167b2f9d5bfc89f0d9dc28e0e18",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399879,
        "hash" : "0000000000000000005642fd14f92213a8ff68aa5071588f84dd2f1c61e93d0e",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399706,
        "hash" : "000000000000000006aef254935a76c8b321519dcd86d4f5f5092f51ef374251",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399332,
        "hash" : "000000000000000006972f107e48524ce3e01752c24a282beafc7b046f29f221",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399247,
        "hash" : "00000000000000000228b92c2c2c8a91673c45f6aa10fb9acabfb587f73ccc81",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399138,
        "hash" : "00000000000000000382da0f1d9cb0bffc6edbbdd53ba8f87877ffe11263100d",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399098,
        "hash" : "000000000000000001bb9eac7a0684d2d0481b9a7fe74568ada10ada5734c857",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 398487,
        "hash" : "0000000000000000008fb4567640f05fddc79d28fcb0601a187b8e5cb46cb84c",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 398217,
        "hash" : "0000000000000000078f1ed2154a4db3d7cfa5615c584585c07e3f53ddba6aa8",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 398205,
        "hash" : "0000000000000000067886752a9524c8043341b5adc4100fbe11e7e45af1020d",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 397486,
        "hash" : "00000000000000000158e2c73ded7c24f0087514d646ae59ff017347bfff4c15",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 395487,
        "hash" : "000000000000000002782e988a2da3aa252752f3accae8624242527ca99de06e",
        "branchlen" : 1,
        "status" : "headers-only"
    }
]

very nice! didnt know it listed all reorgs.
so if I read that right since block 395487, the max depth reorg has been: 1

if so, does that mean a node that has been constantly running for 1 year would be needed to run getchaintips, to get the entire year's history of reorgs?

It seems 10 blocks at current hashrate would be very rare

James

2112

legendary

Activity: 2128

Merit: 1073

The official Bitcoin client has "getchaintips" command that gives you the output like this:

Code:

[
    {
        "height" : 403263,
        "hash" : "00000000000000000210fee63635c72da771c6b136a63f20ff9d058ce778618d",
        "branchlen" : 0,
        "status" : "active"
    },
    {
        "height" : 402922,
        "hash" : "00000000000000000692c289cc4a4e9726018d6a030ad10c84f70adc2e920597",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 402751,
        "hash" : "00000000000000000180755378324c16847b2f49af428c763b8ff411d51d7767",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 402723,
        "hash" : "000000000000000000263e1722b119e2547a4167b2f9d5bfc89f0d9dc28e0e18",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399879,
        "hash" : "0000000000000000005642fd14f92213a8ff68aa5071588f84dd2f1c61e93d0e",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399706,
        "hash" : "000000000000000006aef254935a76c8b321519dcd86d4f5f5092f51ef374251",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399332,
        "hash" : "000000000000000006972f107e48524ce3e01752c24a282beafc7b046f29f221",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399247,
        "hash" : "00000000000000000228b92c2c2c8a91673c45f6aa10fb9acabfb587f73ccc81",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 399138,
        "hash" : "00000000000000000382da0f1d9cb0bffc6edbbdd53ba8f87877ffe11263100d",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 399098,
        "hash" : "000000000000000001bb9eac7a0684d2d0481b9a7fe74568ada10ada5734c857",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 398487,
        "hash" : "0000000000000000008fb4567640f05fddc79d28fcb0601a187b8e5cb46cb84c",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 398217,
        "hash" : "0000000000000000078f1ed2154a4db3d7cfa5615c584585c07e3f53ddba6aa8",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 398205,
        "hash" : "0000000000000000067886752a9524c8043341b5adc4100fbe11e7e45af1020d",
        "branchlen" : 1,
        "status" : "valid-headers"
    },
    {
        "height" : 397486,
        "hash" : "00000000000000000158e2c73ded7c24f0087514d646ae59ff017347bfff4c15",
        "branchlen" : 1,
        "status" : "valid-fork"
    },
    {
        "height" : 395487,
        "hash" : "000000000000000002782e988a2da3aa252752f3accae8624242527ca99de06e",
        "branchlen" : 1,
        "status" : "headers-only"
    }
]

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

~~Although I don't think Bitcoin Core has stats on reorgs,~~ it does keep all of the valid blocks it receives. You could scan through the blk*.dat files and see if there were any chain reorgs.

Edit: I stand corrected:

Quote from: 2112 on March 18, 2016, 04:44:36 PM

The official Bitcoin client has "getchaintips" command

jl777

legendary

Activity: 1176

Merit: 1134

I know the odds for a deep reorg are small, but I was hoping some website is keeping stats on how often the small reorgs are happening. I guess it would be relative to a specific node, unless there is some internal bitcoind stats that can be queried across all peers.

I am assuming that 10 block deep reorg is very rare, but wanted to get statistical confirmation.

It would incur a several minute recalculation cost to have to regenerate the data for all the blocks after the reorg, so dont want to set this at something that happens every day. But no sense to set it so it would be expected to happen once a year.

The shorter the delay, the faster things will work for realtime queries as it minimizes the amount of data that needs to be scanned that is not in the parallel format. Though, now I think about it, maybe I can make a special case for smaller bundle at the end.

Regardless, of the realtime calc speed, I currently delay creation of a bundle until 10 blocks after the last block of that bundle and wanted to know what is the probability that it will need to be regenerated base on real world stats

James

Topic: are there any actual stats on chain reorgs, by depth? - page 2. (Read 2028 times)