Topic: Blocks are [not] full. What's the plan? - page 7. (Read 14343 times)

Then you haven't sold anything.

The "critical window" for computing orphan losses is the time between when a miners finds a block AND the entire network is building the next block on top of that.  That includes the time to relay and verify the node, and all miners to switch to the next block.

During the "critical window" if another miner finds and broadcasts a competing block the network will be split.

Announcing an announcement of a block would solve absolutely nothing.  There are solution to significantly reduce the block broadcast latency but that isn't it.

Nope.  The miners would still get to work on the first one they received.  Then, if they get the announced block within 15 seconds, they'll switch to the first block they heard about.  But if they don't get the announced block (and if it's a fakeout message they won't) then they don't change what they're doing at all.  Even if that includes *FINDING* a new block before the announced block arrives, in which case the announced block *will* become an orphan.

Not sure If I got it properly... Correct me if I didn't: So anybody broadcasting fake announcements would be able to waste 15s of the miners willing to trust the fake announcement?

Here's a protocol design that makes block size *almost* irrelevant to orphan costs.  Could be a reduction in orphan costs of 3 orders of magnitude or more - unfortunately it comes with a cost in bandwidth.

Consider a new protocol message, with semantics "HEY!  Somebody found this new block!" which is tiny for rapid propagation, issued from the miner immediately before he issues the block.  

Clients who are hip to it can propagate it (way faster than the block) before they even get done reading the block, let alone checking it and deciding to send it on.  And clients who don't know what the heck it is can drop it on the floor until their software is updated.  If even one client in five knows about the new message, it'll still propagate faster than a block.

Anyway, you'd then expect clients who hear about one block, then receive another, to hash on/possibly extend the first received only until the first one they heard about arrives, then switch.  Assuming it arrives in fifteen seconds or less.  If it doesn't then the first one they got is the one they work on.

That gives a new block the *effective* propagation time of a very small message, and gives it all the clients it can broadcast to in the next fifteen seconds (which, really, ought to be all of them).

You'll still get the occasional orphan block when two different "announce" messages go into the network at the same time, but this should reduce the incidence of orphan blocks dramatically.  

Welp, you're not the only one. I did the same thing in another topic, and wound up calculating (wrongly) on several 'fee' questions.

Thanks man, very neat info! In particular the net.cpp sleep time trick, never heard about or considered that... Do you see any downside to lowering the timer in net.cpp with your conf, or it's just all better?

I had blockchain stored in ramfs (which requires removing all the diskspace checks from the source)...  also the 600+ connections...  also 1gbit connections, so after a block hit I'd see something like 2000kb-25mb-70mb-30mb-5000kb  in dstat

but probably most important was lowering the sleep timers in net.cpp

none of the verification steps were removed, aside from it checking available disk space

Wow, they were relaying like crazy... 
Did you have any particular tuning (I saw the question raised on bitcoin-dev list, but didn't see any answer) to reduce checking or make any relevant step faster?

the top 3 on here

https://blockchain.info/hub-nodes

plus one other top secret one!

(remember 129.132.xx.xx is evil)

How many did you shut-down?

I killed all my bitcoind nodes (@ approx 5:00PM CST).  I doubt it'll have any effect whatsoever on propagation, but I can't help but be a bit curious...

Anyone know what timezone the daily charts on http://bitcoinstats.com/network/propagation/ are based on?

Miners already do this. Some pools will even trigger longpolls to trigger miners to get work early if enough new txn come in (or at least eligius did in the past, I'm not sure if they still bother).

Technically 4b can be completed by changing the tx set.  Essentially you just need to make any change to the blockheader and changing the tx set works for that.

It probably doesn't make sense to do it on each new transaction and there are methods for pool workers to make a change locally (like incrementing the timestamp) which saves a round trip to the server but  any change to the blockheader (including changing the tx set works).  Pools are highly customized but I would assume they strive for a balance between reacting to changes in the memory pool and having miners make timestamp changes locally to reduce server load.

Computing a new tx set may seem like a lot of work but the tx are already validated before they are in the memory pool so you are just looking at 2x the # of transactions in SHA-2 hashes.  Even for a large block a single CPU core can do that in a millisecond (or less).

I have always wondered about this:

Would it be possible in step 4b) instead of modifying the header do the following

Add some more transactions to the block that have come in since the last header creation
Do the step needed now that the new transactions are in the block
Zero the nonce
Continue

Or is that too much work?

It seems that they may be able to pick up some good paying transactions into the block "on the fly" that way and the new transactions will modify the header for the new nonce cycle.

Miners don't add tx to a block after they find one.

Simplified mining process
1) Miners select tx from the memory pool for inclusion in a new block (the number, method, and criteria is up to the miner)
2) Miners construct a merkle tree from the tx in step #1
3) Miners create a block header with the merkle tree, prior block hash, version, timestamp, and nonce
4) Miner will hash the blockheader.  If it is a solution go to step 5.
4a) Miner will increment nonce and go to step 4.  If nonce range is exhausted go to step 4b
4b) Miner will modify another element in blockheader, reset nonce range back to 0 and go to step 4
5) Miner has found a block.  They broadcast it to peers.

So once a miner finds a block they are done.  All the work for deciding tx comes BEFORE solving a block.

If a block gets orphaned, it means another viable block was added to the chain while the miner was adding transactions to the orphan block, right? Does that mean the miner doesn't get neither the 25 coins nor the transaction fees? If so, what's the point of adding bigger transaction fees as a safeguard against having the block orphaned?

We may use total bitcoin day destroyed as the tie-break for a fork. I guess Satoshi had a similar idea?

The first rule of 10001 club is we don't talk about 10001 club.

My transaction did not require a fee. It was a whole bitcoin more than a day old. But I was trusted not to double-spend, and receiver didn't need it right away.

So we waited 28 hours for it to get included in a block.

Next time, I'm adding a 0.00010001 fee.

One day of mining on a 2 GH/s USB stick takes care of my future transaction fees for a long time. (also, 1 gamble on 98% on any of those dice games.)