Topic: High orphan rate and long confirmation time discussion - page 2. (Read 9907 times)

https://blockchain.info/
slush just hoovered up over 900 txns

the unconfirmed number is very dynamic, sort of depends on whether pools accepting lots of txns get a run or those limiting txns get a run of blocks
looking every few days doesn't give a good representation

869 Unconfirmed Transactions now

Hmm, no orphans for over 24 hours (since 185346) according to blockchain.
Well at least some good news even if it is unrelated to any code change

... jinx ...

That low?    It was over 9,000 unconfirmed just a few days ago...

lol we did have a nice run while I slept
currently 704 Unconfirmed Transaction as I post, was ~300 when I woke up

We need Ozcoin to hit a block to clear the backlog.   

No it is target difficulty.  Generally speaking longest == more difficult.  The reason why the network checks difficulty and not raw length is because otherwise an attacker could exploit the dynamic difficulty and pull off a 51% attack with <51% of hashing power.  For most intents and purposes outside of possible attack longest == most difficult but the code only looks at the sum of the chain's block target difficulty to determine the best chain.

~1000 isn't rare at the moment.
I restarted my bitcoind 12 hours ago - but the highest I've got since then was only 1338 - so not as bad as some days.
... 1 less and it would have been 1337

1200 Unconfirmed transactions as I post this.   

I concur with TheHarbinger. I wish forum admins would install the SMF Bookmark Mod, which provides the same user experience as "subscribing" to a thread.

I had gotten the impression somewhere that "most difficult chain wins" meant the hash difficulties, not target difficulties. If that is not the case, the only time that there would be a more difficult chain would be on difficulty changes - i.e. if you had received two block 183456s (a block with a difficulty recalc), one may have a more difficult target calculated by its miner. A unique hack that might make your recalc block win an orphan race (once in a million) would be to round up the difficulty recalc within the limits of network acceptance. Outside of that, if the only policy Bitcoin miners use is "first new block received", orphan prevention would seem to be just a race to publication then, that you tell all other miners about your block before they find their own or receive a block from another pool.

longpoll is not the issue, but merely a symptom of other issues. Fix those and longpoll will still suffice... that and I'd be really pissed if I had to support something else after spending months getting longpoll working a treat in cgminer.

An effort to create a bitcoin backbone is now underway...

^ This +1. Any smart pool should be blasting its blocks out across as many nodes as it is able to as quickly as possible. One way to do this is to manually connect to several reliable super nodes.

... implement a protocol rather than use something designed for web servers ...

I agree, longpoll notification is slow. I wonder if using a lightweight message queuing system like zeromq would do a better job instead of the http based long polling.

Chain proof-of-work is calculated based on the hash target, so if you get another block at the same height there is no benefit to keeping the one with "the smaller hash".

Maybe if you receive a second block solution, keeping the block that removes the most transactions from the memory pool would be the right "good for the entire ecosystem" policy.  That way even if small blocks propagate slightly faster that might be offset if a larger, slower block was found.  (but making new-block-flooding independent of the size of the block is an even better solution, and that shouldn't be too hard to implement)

Creating a semi-trusted backbone of connections to other pools/miners so your new blocks propagate quickly is a good idea.

The first poster discovering the client-block-receive to miner-long-poll-receive delay time reveals another factor unrelated to the Bitcoin network block propagation speed - the time it takes for a pool to transmit a new block to all miners. In my past experience, depending on the number of pool miners, the number of pool servers, and the pool software and it's delays communicating with Bitcoin, going through the list of miners and sending each a LP response can take a significant amount of time, generally four, but up to ten seconds from the first to the last miner being notified. This from my own research profiling pool LP times as a predictor of who found the block.

If large block propagation was a significant factor, you would expect to see pools, more highly connected and on faster connections, sending your miner new block notification before your Bitcoin client gets it through the P2P network. That it is the opposite indicates instead the internal pool latency in pushing new work to all miners.


One could make the argument that if your pool and a 0-tx mystery miner both generated a block at the same time, the much smaller block could propagate to miners first and be built upon first with another fast block find. The chief optimization, if you are dedicated to creating larger blocks than other miners may be creating, would be to ensure all your pool servers are highly connected to other pools, and communication of new blocks to other pools' known IPs takes priority. Pools should publish and share IPs of bitcoind's with each other (best it be a closed/trusted list to prevent DDOS facilitation), creating a high-priority sub-network to lower orphans for all.

Another coding optimization would be if you receive a second block solution, to immediately discard the first and push out the newer one if it has a significantly smaller hash, of course still keeping track of both as valid work from miners.

I would caution people to only check this page when you really need to - listing the current 2342781 byte backlog of transactions makes for 7.7MB of HTML.

There are two parts to slowing down network propagation:
1) Sending crap repeatedly
2) Wasting time before sending it

It does seem rather odd that (even after I mentioned it) people are still debating what is actually the problem and looking for answers ...
Is it really that hard to put some debug in bitcoind and send a big block from one place to another and see the results at both ends?
Repeat and rinse.
Is this sort of testing beyond the level of bitcoin devs?

I think he meant that should be a fairly urgent update in order to reduce the problem for pool owners.

It's mostly the rest of the network that needs to upgrade for this to be of any benefit.