CreateNewBlock too slow (especially while bitcoin is under tx spam attack)

m0mchil

full member

Activity: 171

Merit: 127

Here is a temporary workaround for pool operators http://github.com/m0mchil/bitcoin/tree/poolmode

I renamed CreateNewBlock to ProcessTransactions and call it on every new block or incoming transaction. If less than 60 seconds elapsed since last execution it does nothing. Then CreateNewBlock simply provides most recent snapshot of processed transactions. The bottleneck is moved from RPC thread to network thread.

Of course this will be obsolete when actual problem is fixed. As I understand it, now orphan transactions get verified (ECDSA) when their parents become available.

ArtForz

sr. member

Activity: 406

Merit: 257

Yes, only once every 60 seconds.
note that CreateNewBlock holds cs_main and cs_mapTransactions while doing its thing.
So it not only blocks RPC for that time, but also processing and sending of p2p messages and ... really, pretty much everything.
On a decent box with 1500 cached tx, it takes 1.7 seconds.
In extreme cases (lots of cached tx, slow cpu, low free memory so a decent % of reads actually have to hit disk), well north of 10 seconds.
And it's entirely avoidable.

Hal

vip

Activity: 314

Merit: 4276

Looking at the getwork code, it should only call CreateNewBlock() every minute or so:

Code:

if (pindexPrev != pindexBest ||
   (nTransactionsUpdated != nTransactionsUpdatedLast && GetTime() - nStart > 60))
   {
[...]
   nTransactionsUpdatedLast = nTransactionsUpdated;
   pindexPrev = pindexBest;
   nStart = GetTime();

   // Create new block
   pblock = CreateNewBlock(reservekey);

This will call CreateNewBlock() either on a new block coming in, or on a new transaction if it's been 60 seconds since the last call. This shouldn't affect every getwork call, unless it's actually taking 60 seconds to do CreateNewBlock(), which would be remarkable.

molecular

donator

Activity: 2772

Merit: 1019

I'm describing the problem here from my personal perspective, bitcoind running on my desktop, which has a slow atom cpu and is frequently under various other load conditions), poclbm running on dedicated miner with a 5970.

Problem is this: after running bitoind for a while, I see "Problems communicating with bitcoin RPC, data: None" messages on the miner. The freuqency of these increases over time (presumably with size of bitcoind's transaction cache). Getwork interval is at 5s.

It has been suggested to used git-version with -limitfreerelay=1. I tried that and it might have taken some pressure, but the problem still occurs.

After a couple of hours my miner doesn't do any work any more, because getwork continuously fails.

I'm quoting from #bitcoin-dev to elaborate the problem and possible workarounds and/or solutions:

Quote from: #bitcoin-dev

ArtForz, am running into slight getwork troubles at 66 tx cache size already? does that make sense?
nope
molecular: what's the trouble? getwork taking long time to return?
do you have a really slow disk or high I/O load?
yes
getwork takes several seconds
at some point it takes longer the my getwork interval (currently 5s)
I have slowish cpu (atom)
yeah, that might do it
which is also under load from other shit since it's my desktop

this made sense to me, ArtForz kept analysing:

Quote from: #bitcoin-dev

can you check if it's actually pegging the CPU?
because here it seemed to be more I/O than CPU bound
I can't see it using cpu in htop. trying to verify that
there's some iotop app? what's it called?
iotop ?

I'm emerging iotop on my desktop, while the chatter continues:

Quote from: #bitcoin-dev

what stalls getwork is CreateNewBlock
and I suspect *that* is more I/O than cpu bound
I don't really know why though, it doesnt *look* like it does lots of I/O
CreateNewBlock is in O(n) with n == ?
yes
ArtForz, why do you think IO might be the problem?
it's all in RAM

Quote from: #bitcoin-dev

yes, it does a fopen/fseek/fread/fclose for blk0001.dat
so that's probably what slows it down
(if the input is from a tx thats already in a block)
and it does a lookup in blkindex DB for every call, too
"if (!txPrev.ReadFromDisk(txdb, txin.prevout, txindex))" <- you mean this, tcatm?
molecular: yep
yeah, that sounds like it might cause slowdowns, especially if you dont have enough free memory to keep blk0001 cached
how does that code find the transaction in blk0001.dat? is there an index for txhash -> blkhash?
yes
blkindex.dat for txhash->offset

Quote from: #bitcoin-dev

urrr... why the F are we not caching that stuff?
slush, workaround will relieve the pain, but in the end we should fix stuff in bitcoin
it's not like tx can magically appear in blocks while no new block comes along
I think ArtForz might've just identified the root of the problem?
could be...
can't think of a elegant way to work around it though
thats... really weird

Quote from: #bitcoin-dev

well, we do 2 things really with that prev tx
1. check if it's in a block (can be cached between block updates)
2. if it is, get the value of the output referred to (same thing)

I'm stopping here and backposting link to #bitcoin-dev...

Topic: CreateNewBlock too slow (especially while bitcoin is under tx spam attack) (Read 3811 times)