Author

Topic: Gold collapsing. Bitcoin UP. - page 133. (Read 2032266 times)

legendary
Activity: 2044
Merit: 1005
July 09, 2015, 02:43:52 AM
Let's face it If we want adoption by a cb we need more tx/s however we can solve other probs first before raising limit to efficiently come to that conclusion.
legendary
Activity: 1162
Merit: 1007
July 09, 2015, 12:37:00 AM
Yes, but the ongoing spam shows clearly that all full nodes are handling the traffic just fine. We were told that they would crash and burn from overloaded memory. Not true.

Wait, wasn't that exactly Hearn's main argument in his "the sky is falling if we don't increase the blocksize" blog post?  So why do you see the fact that it did not prove true so far as supporting your position of increasing the block size?

It provided evidence that the network can handle the increased load, and it confirmed that people do get frustrated by the confirmation delays. 
legendary
Activity: 1135
Merit: 1166
July 09, 2015, 12:28:52 AM
Yes, but the ongoing spam shows clearly that all full nodes are handling the traffic just fine. We were told that they would crash and burn from overloaded memory. Not true.

Wait, wasn't that exactly Hearn's main argument in his "the sky is falling if we don't increase the blocksize" blog post?  So why do you see the fact that it did not prove true so far as supporting your position of increasing the block size?
legendary
Activity: 4760
Merit: 1283
July 08, 2015, 11:47:58 PM
Spammer only spends money 1x during lifetime, but miner has to keep bigger disk-space forever (2 hard disc consume more electricity than single one)

OK.  Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers:

1.  The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent.
2.  The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.


I'm sorry, I have no estimates. I know that bloat-chain is not solution  There are better solutions how to organize blockchain.

My full node just crashed ... all data are lost ... downloading 6 years 25 weeks.  ... hmm I hope that it is only me and there will remain some copy of blockchain. :-)  (not joking)

Don't worry.  According to Mike Hearn Bitcoin can survive just fine with 4 or 6 (forgot which) copies of the blockchain worldwide.

legendary
Activity: 1764
Merit: 1002
July 08, 2015, 11:27:03 PM
When miner's mempool exceeds block size by some backlog factor, they need to send out a message to the peers they are connected to saying "don't propagate transactions to me with smaller than X fee per kilobyte until further notice". This will mitigate the impact that spam-that-can-never-be-added-to-a-block can have on the network, but it won't entirely eliminate it, because as the mempool backlog reduces, the miner will need to back off the notice to some unknown lower fee threshold. The miner could guess but it will never be 100% accurate. So spam will get verified and rejected. But perhaps those estimates can be close enough that the losses the spammers takes in fees for transactions it sends that do get into a block will be greater than the gains it values from effective spamming (what ever those gains are  Huh). Appears to me the issue can be fixed.

Afaik, the reason incoming transactions are verified before it is known whether they will be in a new block, is because it is considered spam to propagate (send out again to another peer) invalid transactions that would be rejected and never appear in a block.

gmax already told us to use minrelaytxfee.
sr. member
Activity: 420
Merit: 262
July 08, 2015, 11:08:36 PM
When miner's mempool exceeds block size by some backlog factor, they need to send out a message to the peers they are connected to saying "don't propagate transactions to me with smaller than X fee per kilobyte until further notice". This will mitigate the impact that spam-that-can-never-be-added-to-a-block can have on the network, but it won't entirely eliminate it, because as the mempool backlog reduces, the miner will need to back off the notice to some unknown lower fee threshold. The miner could guess but it will never be 100% accurate. So spam will get verified and rejected. But perhaps those estimates can be close enough that the losses the spammers takes in fees for transactions it sends that do get into a block will be greater than the gains it values from effective spamming (what ever those gains are  Huh). Appears to me the issue can be fixed.

Afaik, the reason incoming transactions are verified before it is known whether they will be in a new block, is because it is considered spam to propagate (send out again to another peer) invalid transactions that would be rejected and never appear in a block.
legendary
Activity: 1764
Merit: 1002
July 08, 2015, 10:59:27 PM
all the yearly silver and mining stocks have that sick top left to bottom right chart configuration. this is a horrible leading indicator for both gold and stocks:

silver futures:



SLW:



PAAS:



silverbox's GPL:

legendary
Activity: 1764
Merit: 1002
July 08, 2015, 09:55:42 PM
I just want to highlight the comments here from bitcoin-dev today, which seem to indicate that these spamming attacks would have been 5.5x more expensive if a dust threshold software change had not been made. Perhaps there are deeper insights someone else has on this...?

Quote
16:53   wangchun   why can those spam get confirmed. 0.00001 BTC vout below dust threshold right?
16:54   phantomcircuit   wangchun, iirc the dust threshold is 546 satoshis
16:54   wangchun   not 5460 satoshis? changed?
16:54   aschildbach   wangchun: Yes it was cut by 10 a few months ago.

i don't think it really matters, does it?

since most of the mined blocks are filled with real demand trying to get into blocks with or without spamming, the only difference being that with spamming the real demand has to pay significantly higher fees to get in, the cost to fill up the rest of the block with spam is minimal as is the minimal cost to bloat the mempool to extraordinary levels (94K 0confs as i write).  i say minimal cost for the mempool b/c most of it gets turned over after 24h deletion allowing that spam to be recycled w/o having to pay.
legendary
Activity: 1764
Merit: 1002
July 08, 2015, 09:12:12 PM
The point I'm trying to make is that if CreateNewBlock is super-linear in mempool size, then it would not be surprising to see more empty blocks (what Cypher was calling "defensive blocks") when mempool swells (the miners are mining on an empty block for longer while waiting for CreateNewBlock to finish).  This was Cypher's point from the very beginning that many people, including myself, were suggesting was probably not the case!  

that is exactly what i was saying w/o knowing anything about CNB.  it was just intuitive that constructing a new block would take longer for a large mempool so i deduced that a miner might decide "f*ck it" that he doesn't want to not only spend additional time validating a large incoming block but also not want to spend additional time creating a large new block so instead just automatically switch to 0 block mining upon arrival of a large block.  the thing none of us anticipated was that he would not validate that incoming large block at all and switch to SPV mining while >50% of the network would also be SPV mining (Discus, Antpool, BTCGuild) and trigger an invalid SPV longest chain.

that's some wild shit going on.

given this newly recognized scenario, i still conclude however that this is all a result of the block size limit being hit continuously by attacking spammers and secondarily intentionally causing a bloated mempool which then causes these unanticipated defensive strategies and more intentionally, user disruption.
legendary
Activity: 1078
Merit: 1006
100 satoshis -> ISO code
July 08, 2015, 08:25:48 PM
I just want to highlight the comments here from bitcoin-dev today, which seem to indicate that these spamming attacks would have been 5.5x more expensive if a dust threshold software change had not been made. Perhaps there are deeper insights someone else has on this...?

Quote
16:53   wangchun   why can those spam get confirmed. 0.00001 BTC vout below dust threshold right?
16:54   phantomcircuit   wangchun, iirc the dust threshold is 546 satoshis
16:54   wangchun   not 5460 satoshis? changed?
16:54   aschildbach   wangchun: Yes it was cut by 10 a few months ago.
legendary
Activity: 1372
Merit: 1000
July 08, 2015, 07:24:26 PM
Spammer only spends money 1x during lifetime, but miner has to keep bigger disk-space forever (2 hard disc consume more electricity than single one)

OK.  Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers:

1.  The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent.
2.  The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.



I'm sorry, I have no estimates. I know that bloat-chain is not solution  There are better solutions how to organize blockchain.

My full node just crashed ... all data are lost ... downloading 6 years 25 weeks.  ... hmm I hope that it is only me and there will remain some copy of blockchain. :-)  (not joking)

your data management systems need updating if you lost the data. Bitcoin is a $3B network and if you dont have an invested interest in keeping the blockchain to prove you own X coin or care enough to manage the responsibility you shouldn't be dictating how we should use it. not everyone who can't backup data or doesn't want to should be running a node. I have friends who dont trust digital photos because they trust paper copies more. I'm surprised to find you fit the profile.

I also think you're not weighting the cost and benefits correctly, the cost of 2 hard disc consuming more electricity than single one, is an inconsequential cost at the scale of executing this same attack with an 8MB block. (hell its inconsequential at the cost of the attack on the 1MB block.)

limiting block space is not going to save anyone except you a little bandwidth because you dont back up.

legendary
Activity: 1162
Merit: 1007
July 08, 2015, 07:19:04 PM
OK.  Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers:
1.  The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent.
2.  The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.
I may be missing the context as this thread is high volume and I've not read any of the backlog...

But for a full verifying node, the on-going cost cost of 1GB of additional transactions with all outputs spent is 0; all the cost related to that 1GB of data is related to the bandwidth to get it to you and the verification cost, and for short term storage until its burried, after that it need not be stored.
The cost for unspent is some non-zero number which depends on your estimation of storage costs.

This thread can be hard to follow if you're not following it all the time!  

The question was in reference to a debate I was having with Odalv about these "order of magnitude" estimates shown in this table.  I was suggesting that, under the conditions considered in the table, it is cheaper for miners to write the spam to the Blockchain and more costly for the spammer, than continually rejecting it:



Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from?  If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.  
Yep, I already pointed that out to you specifically! It's superlinear in the mempool size (well, ignoring caching)  But thats unrelated to f2pool/antpool and the other SPV miners, as they're not ever calling createnewblock in that case, as they're mining without even validating.   One can mine on a validated chain with no transactions while waiting for createnewblock (which is what eligius does, for example).  

Sorry, yes I know you explained that.  The point I'm trying to make is that if CreateNewBlock is super-linear in mempool size, then it would not be surprising to see more empty blocks (what Cypher was calling "defensive blocks") when mempool swells (the miners are mining on an empty block for longer while waiting for CreateNewBlock to finish).  This was Cypher's point from the very beginning that many people, including myself, were suggesting was probably not the case!  

Furthermore, how can f2pool/antpool mine a non-empty block without calling createnewblock?
legendary
Activity: 2156
Merit: 1072
Crypto is the separation of Power and State.
July 08, 2015, 07:04:40 PM
Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from?  If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.  
Yep, I already pointed that out to you specifically! It's superlinear in the mempool size (well, ignoring caching)  But thats unrelated to f2pool/antpool and the other SPV miners, as they're not ever calling createnewblock in that case, as they're mining without even validating.   One can mine on a validated chain with no transactions while waiting for createnewblock (which is what eligius does, for example).  I also pointed out that this is trivially optimizable, but no one has bothered previously.

Wait, how is an empty block created without calling createnewblock?  Is there a createemptyblock thingy or what?
legendary
Activity: 4760
Merit: 1283
July 08, 2015, 06:38:03 PM
...
I may be missing the context as this thread is high volume and I've not read any of the backlog...

I wonder if I can get you to touch on my question here if you could:

  https://bitcointalksearch.org/topic/m.11817437

staff
Activity: 4284
Merit: 8808
July 08, 2015, 06:26:59 PM
OK.  Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers:
1.  The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent.
2.  The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.
I may be missing the context as this thread is high volume and I've not read any of the backlog...

But for a full verifying node, the on-going cost cost of 1GB of additional transactions with all outputs spent is 0; all the cost related to that 1GB of data is related to the bandwidth to get it to you and the verification cost, and for short term storage until its burried, after that it need not be stored.
The cost for unspent is some non-zero number which depends on your estimation of storage costs.


Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from?  If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.  
Yep, I already pointed that out to you specifically! It's superlinear in the mempool size (well, ignoring caching)  But thats unrelated to f2pool/antpool and the other SPV miners, as they're not ever calling createnewblock in that case, as they're mining without even validating.   One can mine on a validated chain with no transactions while waiting for createnewblock (which is what eligius does, for example).  I also pointed out that this is trivially optimizable, but no one has bothered previously.

legendary
Activity: 2156
Merit: 1072
Crypto is the separation of Power and State.
legendary
Activity: 1414
Merit: 1000
July 08, 2015, 05:10:31 PM
Spammer only spends money 1x during lifetime, but miner has to keep bigger disk-space forever (2 hard disc consume more electricity than single one)

OK.  Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers:

1.  The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent.
2.  The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.


I want to say. (English is not my mother tongue)

The more adoption the more spammers.  ( if only 0.1% is malicious then it is 1,000,000 spammers for every 1,000,000,000 adopters )
The more full nodes the more upgrade of whole network will cost.

=> result: increasing block size can kill bitcoin
 b/c
 a) spamming is cheaper (goes to zero)
 b) and mining costs more (goes to infinity)
legendary
Activity: 2156
Merit: 1072
Crypto is the separation of Power and State.
July 08, 2015, 05:00:20 PM
Mempool is full of tx waiting to be (slowly, thanks to completely unoptimized Createnewblock) processed into new blocks.

Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from?  If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.

 Undecided  I think gmax already (indirectly?) answered your question here:

I expect correlation between empty blocks and mempool size-- though not for the reason you were expecting here: Createnewblock takes a long time, easily as much as 100ms,  as it sorts the mempool multiple times-- and no one has bothered optimizing this at all becuase the standard mining software will mine empty blocks while it waits for the new transaction list. So work generated in the first hundred milliseconds or so after a new block will usually be empty. (Of course miners stay on the initial work they got for a much loonger time than 100ms).

This is, however, unrelated to SPV mining-- in that case everything is still verified. As many people have pointed out (even in this thread) the interesting thing here isn't empty blocks, its the mining on an invalid chain.

but I'll attempt to unpack and contextualize his response...  Tongue


CNB execution time depends on CPU/RAM/SSD speed and mempool size, both of which are adjustable user defined values.

The theoretical "more empty blocks when mempool swells" effect only dominates when the previous block validates faster than the "100ms" maximum CNB takes to make the next one; according to gmax avg. validation times are "16-37sec" (for f2p/ant).

IOW, CNB while 'slow' in the relative sense it is unoptimized, is much (>1600-3700 times) faster than avg. blocks' verification times.

So, (if I've not failed spectacularly in my amateur hour gmax impersonation) Frap.doc's hunch only applies after empty or nearly empty, very small, tiny little blocks.

As block become larger (non-tiny/empty), verification time increases quadratically.  Absent any threshold effects, I'd expect CNB performance to be ~linear with mempool size (which, again, is adjustable, unlike previous blocksize).

Larger blocks are thus self-defeating (IE encounter negative marginal return) because their quadratically-scaling verification time greatly exceeds CNB's (perhaps linearly-scaling but in any case much faster) execution time, ensuring more empty blocks are produced exactly when network throughput/load needs non-empty blocks the most.
legendary
Activity: 1162
Merit: 1007
July 08, 2015, 03:50:15 PM
Mempool is full of tx waiting to be (slowly, thanks to completely unoptimized Createnewblock) processed into new blocks.

Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from?  If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.  

legendary
Activity: 2156
Merit: 1072
Crypto is the separation of Power and State.
July 08, 2015, 03:17:33 PM
Mempool is full of tx waiting to be (slowly, thanks to completely unoptimized Createnewblock) processed into new blocks.

The blocks are then propagated (slowly thanks to limited/expensive upstream bandwidth), and (eventually, thanks to slow ECDSA and quadratic scaling) validated by the receivers.

All that slowness you're referring to results in the spam being deleted after 24 hours allowing recycling of the fees while still disrupting usage.

What network disruption are we talking about here?

What usage "disruption" are we talking about here?  I fear you may be exaggerating again, as is your wont.

Fees rising from ~2 to ~5 cents is mere adjustment, not disruption.  If you're using a fixed-fee wallet/service, that's not BTC's fault.

The spam being deleted is a good thing.  It doesn't need to (and shouldn't) be included on the One True Eternal Blockchain, to be slowly propagated and validated and stored by every node until the end of Bitcoin Time.

The 24-hour deletion/resubmission cycle gives mine/pool ops/devs time to adjust their spam filters accordingly.
Jump to: