Topic: Difficulty adjustment needs modifying - page 2. (Read 10435 times)

Oh forgot about that

Anyway, again the problem with the current is it isn't accurate at all - it's based on the past 2 weeks which means it is always wrong except when the network hash rate is unchanging.
Even though 3 days has a high (but not extreme) SD, the actual result over time shouldn't be that bad IMO
... and again in situations like now, it will speed up the diff change and help retard the major negative effects the 14 day delay is currently having

  432 would put it pretty high up on the curve there. Maybe 432 blocks sooner verses every 432 would keep us from being too far off. Its not as big of an issue coming back below 1mil difficulty. I think the problem lies, and probably what the thinking is for wanting it where it is, would be for the 10mil+ diff where a few points would make a much bigger impact on the deviation.

From Maaku's post on page 1;

OK I've been thinking about this more again.

I think the simplest solution, without the issues of having 2 different calculations interfering with each other, is to make the 2016 block diff recalc a 432 block diff recalc
Yep drop it from ~14 days to ~3 days.

Points:

If some bot net can mess up the diff recalc for 1.5 days of a short 3 day diff recalc
I really can't see them having all that much more of a problem also reaching 7 days of a 14 day diff recalc.
It's 4.7 times longer - not even an order of magnitude more.
It also means, however, that the amount of time the current difficulty will be extended due to a bot difficulty attack is quite obviously smaller for a smaller diff recalc

The difficulty is not adjusted based on the current network, it's based on the past network - since we cannot know what the current network really is.
It also is an estimate simply calculated from the past block generation rate.
This guarantees it will always be late to change.
Is that there by desire, or due to consideration of the issue with calculating it?

The last diff change extended 2 days longer that the expected 14 days.
I wonder how long this current one will go ...

If some statistics pro wants to work out and compare the expected standard deviation of 432 blocks vs 2016 blocks that would probably simplify a yes/no opinion of this.

  I sure hope you don't really mean that, Mr. Anderson.  I like to take a slightly more optimistic stance on it and attribute such a loss in hashing power in the near future to 'something much more serious wrong with the present state of the community at large', rather than Bitcoin itself. You and everyone else who have put in so much time and effort to build what is in concept nearly flawless certainly deserve more optimism. I'd like to think a massive loss in hash any time soon would be more of a 'weeding' out of all the crap. And potentially, could afford many who do see the value of the protocol a more stable environment to rebuild in.. just my 2 bitcents worth.


  Yea, kano, we are down to 14% drop now and if the current little trend continues into the next difficulty, Ddosing aside, we are looking at a 25-40% for the next one. Which is fine save for the potential for it being a bloody 4 week+ change time. :/  There is however another variable that may or may not enter the equation. That is our lil botnet buddies, if they decide to actually find a place to nest and mine. They are capable of about 3T worth on their own from best I can be calcumaltin'..  Interesting timess ahead, none the less.


  Again, I must agree with Kano that atleast some discussion should be had amongst those that know more than I about the 'What If' scenario of losing a massive chunk of hash early on in a dif change......
 

I've just run it again (and updated it at the link http://tradebtc.net/diffcalc.php )

Back to last difficulty: -14.91%
-432 blocks (~3 days): -31.78%
-144 blocks (~1 day): -45.75%

Worst figure around -100 blocks: -82 blocks: -59.94%

So I guess the following means
there is ALMOST something seriously wrong with bitcoin now ...
(1 day worth of hashing suggests we might be half way to 90%)

Aye, I caught your drift, m8. sittin at 13% now. ;p 149184 13/10/2011 22:55   the time is about what it was a few days ago when all the large pools had 24h+ of super bad luck. It has the same effect.

Judging by the last 10 blocks being back up a bit from a few hours ago (btc guild atleast mining again), I think the attacks are losing some steam. 

  The only thing they will accomplish doing such petty things to multiple places will be getting their botnet busted quicker than usual by 'the man'.

Hmm - but I was thinking more: how long it might extend out for the remaining ~100 blocks.

Aye, showing about 12% here now. I would think with so few blocks left the effect will be very minimal. Mainly due to the affected pool ops working their butts off to counter the attacks.

Hmm - more than 24 hours late on the diff change and still more than 100 blocks to go
(so that means a guaranteed drop of at least 11%)

... and DeepBit died about an hour ago ...
(Edit: and Slush and some of BTCGuild)

I wonder what will happen to the trailing end of this difficulty ...
(start time was "22:40:40 27-Sep-2011 UTC")

My point had nothing to do with the pools themselves. I was refering to the individul users who hold the majority of the hash power. If even a small number of them decide to flip the switch, it will hurt.....

And yes, many will keep going no matter what. I am one of those. But how much hash power do 'we' hold?

Well the extra data after the adjustment is somewhat meaningless since it is a different difficulty,
but I decided to go with the simple calculation of just do 2017 blocks and mark 3 of interest.
In this case when we are close to the end, the data before the diff adjustment is pretty useless, but soon after a diff adjustment, it is worth looking at.
Anyway, only really those three % values are of real interest, the rest is just here to work out the numbers or run on a bit after the end.

I've just run it again (and updated it at the link)

Back to last difficulty: -12.67%
-432 blocks (~3 days): -23.06%
-144 blocks (~1 day): -37.09%

Interesting that -144 seems to be the lower limit and it's rising a bit since then
(though it's hard to tell for sure since the figures get very unreliable as you get closer to 1)

But earlier today I had some scary numbers:
back to last diff=-12.77% ~3days=-24.61% ~1day=-34.91% worst around 100 is -99=-51.84%

Free electricity for mining is about using generated heat for other purposes.  

AKA. Winter

@Kano , Very nice stats, Shouldn't your average restart at zero after adjustments ? , block 1802   147167

Very few people have free electricity.

At what point do we start to care ? If 85% of hashing power was to disappear overnight it would take an hours on average between blocks and we would have 3 month to explain to the user-base that it is not the end of the world yet.

If BTCs were to drop at 10c usd we would still have a fairly large % of population with free electricity that would not stop mining.


Pool centralization does not affect transaction speed nor difficulty change. If a big pool was to disappear it wouldn't take half a day for 2/3 of it's miners to move elsewhere.

 After looking at the top 5 pools(not counting 'other') today to get average user counts it was pretty obvious that a HUGE chunk of the hash power is in a very small percentage of hands. That said, I am in very strong aggreeance with you on this!

  Hopefully we will get lucky and those that may decide to switch off will do so during the last 1/4 of a 2016 chunk before a diff change. That would have the least amount of negative impact versus the more likely scenario that they would switch off right after dif change. 

Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
Hmm firstly, 2weeks is ~17 hours from now but still 300 blocks to go ...

Counting back to last change the new diff would be -11.48%

However:
 432 blocks (~3 days) is -16.78%
 144 blocks (~1 day) is -25.48%

(and -84 blocks is -30.37% <- yeah I chose that one on purpose - the highest one close to 100)

Again, I really think this needs to be addressed BEFORE it happens ... and if this past week is any indication of the near future ...

OK luke-jr I shouldn't have agreed with you about the time in a block.

I just ran another block table generation (not at the link but these are in the old link anyway) and it shows this:

148111   04:04:32 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   -73m 00s   9m 38.15s   +3.64%
148110   05:17:32 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   84m 01s   10m 01.40s   -0.23%
148109   03:53:31 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   13m 25s   10m 02.45s   -0.41%

Yes of course 148110 is Eligius - more than a whole hour?!?

If you look through my table probably every time you find a block with a negative time it is after an Eligius block
True for the first 3 and easy to check the block look at the coinbase for the word Eligius, or a whole bunch of outs in the generation transaction.

If the diff change occurred on block 148110, that one block would affect the calculation by extending the average by more than 2 seconds thus making the difficulty lower (yeah I said higher - fixed ...

I guess this is what was being referred to before?

ahhh, must need sleep. Also did not see you were running backwords. ;/  Why not from bottom up?
edited my last post a bit to fix my half ass attempt at maths...


hehe, I am fairly certain even if I keep missing the rest that my bump idea will fix that. Just do it in reverse, since I for whatever dumb reason thought you were calculating from older first. so instead of it dropping .50s, drop it .52s

My running average is simply the average of all block generation times from the top down to the line it's shown on.
The first one is of course the time to generate the last block, the 2nd is the average of 1 and 2 etc ...
So you can see what the average is at any point running back from 'now'
(now being block 148231 23:57:28 5-Oct-2011 UTC - in the past)
I can regenerate it (it's just php - and takes - as you can read - almost 4 minutes - to do 2017 getblockbycount's to my bitcoind)

The standard difficulty calculation is only interested in the blocks from calculation time back to the last difficulty change,
so I've made the averages and % relate to that.

Yeah I guess the 2016 line isn't really accurate since it crossed the diff change boundary,
ignore it

Sorry, did not see the table before. I was wondering where you got your %'s from.

 Couple of 'errors', well, questions really. Where are you getting your starting "running average" figure from?

  And, wouldn't the running average time be more accurate if at the point of difficulty change you bumped it up or down by the same % that difficulty adjusted. Being that in the case of it going down
the 'average' time to solve should be less so the penalty in adjustment % would be greater. Its a sloppy fix, and not being a smarty guy like you peeps I can't offer up a fancy, terminology laced explanation, but give it a try and see. ;p

i.e. 147167 - 147168 should consider the difficulty adjustment, instead it appears to be the same as 147166 - 147167
147168 22:47:04 27-Sep-2011 UTC 0x1a09ee5d (1689334.4045971) 6m 24s 10m 53.95s -8.99%
147167 22:40:40 27-Sep-2011 UTC 0x1a098ea5 (1755425.3203287) 2m 00s 10m 53.45s -8.91%
147166 22:38:40 27-Sep-2011 UTC 0x1a098ea5 (1755425.3203287) 1m 58s 10m 52.95s -8.83%


Should we not bump the running time average from 10m 53.95s up by .37647003433787% to 11m 18.57s ?

Edit; very sloppy on my part. Would like to see the actual spreadsheet to see how you are doing the math. I know the time is lacking the needed adjustment for dif change but am not sure about it being fixed by bumping at the 'running average'. We would want to add the % change I suggested into the formula you are using for running average before it actually calculates the new running average time. Which if my brain isn't completly fried would only be adding the .3%~ to the .50s, the expected calculated change in time. So instead of 50s up that it currently is, it would be .5188s added to 53.45s or a new current running average time of 10m 53.97s

Very trivial when I look at it like that, but across a few more dif changes it would begin to add up if not corrected for. imho.