as you know, even Gavin talks about this memory problem from UTXO. and yes, i read the Reddit thread that resulted in which you participated and i'm aware that UTXO can be dynamically cached according to needs.
http://gavinandresen.ninja/utxo-uhohGavin was insufficently precise. There is a reddit thread is full of people calling gavin a fool (
) for saying "memory" when he should have been saying fast storage.
https://twitter.com/petertoddbtc/status/596710423094788097Why do you think it's prudent to argue this with me?
i'm not really arguing about this with you. you said UTXO is not in memory. i'm saying it depends on how fast a node wants to verify tx's via the dynamic caching setting they choose which does get stored in memory.
Let me chime in hear quickly, because I think Greg and I are talking about slightly different things. My model was considering the time between the first moment that a pool could begin hashing on a blockheader, and when the previous block had been processed, a new non-empty block template constructed, and the hashers re-assigned to work on this non-empty block.
It looks like this time, empirically, is 15 sec (F2Pool) and 30 sec (AntPool), based on these
estimates.
Here I suspect you're suffering from an excess of empiracisism without adequately devling into the mechenism. You can directly measure that time time from input to minable on an actual node under your control and will observe the time is hundreds of times faster than your estimate. Why? Miners don't magically know when their pool has new work, they'll get work in the first milliseconds and then grind on it some time before submitting returning work. Even if the pool long polls them, it takes time to replace work. So what I suspect you're actually measuring there is the latency of the mining process... which is consistent with what we've expirenced with P2Pool (5-20 second latencies from ASIC miners are common).
I noted you posted a result of a classification, did you run the same data through a simple logistic regression with prior size as the treatment? The intercept in the model would be interesting.
But indeed, these conversations have been conflating several seperate issues (latency vs throughput, etc.). Tricky to avoid that since they're all relevant.
but you haven't verified that f2pool or Antpool has increased their minrelaytxfee have you to minimize their mempool?
I have, they'd previously cranked it down, and were producing small blocks and were flamed in public. They've since turned it back up.
remember, this whole mempool discussion was based off you responding to Peter's mathematics post the other day where you argued that the block verification times were only 80ms for a 250 kB block b/c tx's had been pre-verified after being passed around to all nodes across the network and didn't require re-verification by miners on the relay network and was therefore a refutation of his hypothesis of increasing block verification times (16-37sec on avg) leading to SPV mining.
As PeterR points out, they only need to wait for verification to actually verify (which they're not doing today), though they may have to wait longer to include transactions---- though I point out thats not fundimental e.g. no matter how big the backlog is you can produce a template sufficient to completely fill a block while doing no more work than handling a mempool of twice the maximum block size. (by using a tiered mempool, though no one has bothered to implement this yet-- no one has even been complaining about how long createnewblock takes, due to the ability to produce empty blocks without skipping transactions).