[solved/paid][1btc reward] PHP RPC commands freezing bitcoin

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: dooglus on August 08, 2012, 07:13:45 PM

Quote from: dooglus on August 08, 2012, 06:59:25 PM

I just noticed bitcoin-qt is using 100% CPU. It's still responsive, but it's a start...

That was probably a red herring. I was running 'bitcoin-qt -noconnect' to stop it connecting to any peers. That seems to be what causes the 100% CPU usage, whether I use your wallet or mine.

Yeah - that was unrelated. Giving bitcoin-qt the -noconnect flag causes the CPU to go to 100% for everyone, I guess. I reported it, and provided a fix:

https://github.com/bitcoin/bitcoin/issues/1664

So I'm back to not being able to reproduce your problem. I doubt that broadcasting the transactions to the network would cause a problem, but it's possible. I guess I'd like to try with one of your bigger wallets once you're able to check them out.

drakahn

hero member

Activity: 504

Merit: 500

Quote from: dooglus on August 08, 2012, 07:13:45 PM

Quote from: dooglus on August 08, 2012, 06:59:25 PM

I just noticed bitcoin-qt is using 100% CPU. It's still responsive, but it's a start...

That was probably a red herring. I was running 'bitcoin-qt -noconnect' to stop it connecting to any peers. That seems to be what causes the 100% CPU usage, whether I use your wallet or mine.

could broadcasting unconfirmed transactions add to it?
[~3-6 per minute ~10 minutes per block, so ~30-60 transactions some up to 90kb being broadcast]

though i'm doing the same now

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: dooglus on August 08, 2012, 06:59:25 PM

I just noticed bitcoin-qt is using 100% CPU. It's still responsive, but it's a start...

That was probably a red herring. I was running 'bitcoin-qt -noconnect' to stop it connecting to any peers. That seems to be what causes the 100% CPU usage, whether I use your wallet or mine.

dooglus

legendary

Activity: 2940

Merit: 1333

I just noticed bitcoin-qt is using 100% CPU. It's still responsive, but it's a start...

drakahn

hero member

Activity: 504

Merit: 500

Quote from: dooglus on August 08, 2012, 05:58:12 PM

Quote from: drakahn on July 25, 2012, 03:45:56 AM

Okay so I'm trying to make use of the PHP martingale bot that's bouncing around, and after a while the bitcoin client freezes and jsonRPC fails to connect (the client stays frozen and won't start until the whole computer is restarted) it doesn't seem to be connected to anything specific, just after a while (30 transactions on one wallet, 1800 on another) cpu use of the client hits 25% and it freezes

bitcoind freezes then crashes

I'm in windows 7 with the latest client from bitcoin.org

Anyone know how to stop this?

edit : it seems to happen when trying to use the rpc at the same time as bitcoin is downloading a block

OK, so I'm unable to reproduce this behaviour.

When I try to create a transaction that spends most of the satoshis, it very quickly selects the right coins, but then takes about 6 seconds signing all the inputs, and then fails because the maximum transaction size is apparently 100kb:

Code:

static const unsigned int MAX_BLOCK_SIZE = 1000000;
static const unsigned int MAX_BLOCK_SIZE_GEN = MAX_BLOCK_SIZE/2;

if (nBytes >= MAX_BLOCK_SIZE_GEN/5)
return false;

That puts a hard limit of something like 555 inputs on a transaction, although the coin selection code makes no attempt to keep the set of selected inputs to 555 or less.

So in summary: when you try to spend more than 555 inputs at a time it fails to create the transaction after taking a relatively long time to sign all the inputs, but I'm not yet seeing and crashes or long freezes. Any idea what I might do to provoke the failures you were seeing?

I'm really not sure, at first i thought the new blocks coming in had something to do with it, it may have been a factor, i assume that the 100kb size limit is the problem though, 6 seconds would be some of the ones in the client, (some of them minutes) and the early 'warning sign' ones using RPC

A thing i noticed is it would get worse with every bet, i may have sent you a wallet that i stopped early so was just 'slowing down' instead of freezing and making rpc fail (i chose the smallest one first and then corrupted my blockchain yadda yadda yadda so just sent that one)

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: drakahn on July 25, 2012, 03:45:56 AM

Okay so I'm trying to make use of the PHP martingale bot that's bouncing around, and after a while the bitcoin client freezes and jsonRPC fails to connect (the client stays frozen and won't start until the whole computer is restarted) it doesn't seem to be connected to anything specific, just after a while (30 transactions on one wallet, 1800 on another) cpu use of the client hits 25% and it freezes

bitcoind freezes then crashes

I'm in windows 7 with the latest client from bitcoin.org

Anyone know how to stop this?

edit : it seems to happen when trying to use the rpc at the same time as bitcoin is downloading a block

OK, so I'm unable to reproduce this behaviour.

When I try to create a transaction that spends most of the satoshis, it very quickly selects the right coins, but then takes about 6 seconds signing all the inputs, and then fails because the maximum transaction size is apparently 100kb:

Code:

static const unsigned int MAX_BLOCK_SIZE = 1000000;
static const unsigned int MAX_BLOCK_SIZE_GEN = MAX_BLOCK_SIZE/2;

if (nBytes >= MAX_BLOCK_SIZE_GEN/5)
return false;

That puts a hard limit of something like 555 inputs on a transaction, although the coin selection code makes no attempt to keep the set of selected inputs to 555 or less.

So in summary: when you try to spend more than 555 inputs at a time it fails to create the transaction after taking a relatively long time to sign all the inputs, but I'm not yet seeing and crashes or long freezes. Any idea what I might do to provoke the failures you were seeing?

drakahn

hero member

Activity: 504

Merit: 500

Quote from: gweedo on August 08, 2012, 05:07:35 PM

Quote from: drakahn on August 08, 2012, 04:58:01 PM

Quote from: gweedo on August 08, 2012, 04:55:49 PM

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

the issue isn't bitcoind, it is windows, should have used linux

fyi it froze slightly quicker in ubuntu, but at least used 100% cpu instead of just one core

still should have used linux, aka Gentoo or Centos, but if this is a big bug, is it reported? Cause I am using the latest bitcoind and I never have this problem on my VPS and I get ~1000 php rpc calls. I use hardly any CPU.

Since removing the clients ability to spend single satoshi inputs i have not had a problem, i'm not sure exactly what the problem is/was, but i think the single satoshi inputs are/were a big part of it, RPC froze the client (timeout set to 600 but it never waited that long before throwing cannot connect) and the client spat "transaction create failed" (After freezing for a few seconds to a few minutes) at me if i tried to spend "the wrong amount" of bitcoins (if it failed i tried more or less and it would eventually work)- the biggest transaction i saw was ~90kb, perhaps at a certain size the client can't do it? a better error would be "transaction too big" then... >.> but i am not sure

gweedo

legendary

Activity: 1498

Merit: 1000

Quote from: drakahn on August 08, 2012, 04:58:01 PM

Quote from: gweedo on August 08, 2012, 04:55:49 PM

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

the issue isn't bitcoind, it is windows, should have used linux

fyi it froze slightly quicker in ubuntu, but at least used 100% cpu instead of just one core

still should have used linux, aka Gentoo or Centos, but if this is a big bug, is it reported? Cause I am using the latest bitcoind and I never have this problem on my VPS and I get ~1000 php rpc calls. I use hardly any CPU.

NRF

sr. member

Activity: 279

Merit: 250

Quote from: grue on August 08, 2012, 04:49:54 PM

now that's what i call shotgun debugging!

It's Windows Tongue

. Its the only sort of debugging I know that works with that OS.

Many a time some friend of family member asks my about a problem they are having with there windows box and the first thing I ask is "Have you switched it off and on yet?". Works 90% of the time Tongue

.

drakahn

hero member

Activity: 504

Merit: 500

Quote from: gweedo on August 08, 2012, 04:55:49 PM

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

the issue isn't bitcoind, it is windows, should have used linux

fyi it froze slightly quicker in ubuntu, but at least used 100% cpu instead of just one core

although building probably would have been easier in linux with gitian, after the vm taking too long i just decided to make it work in windows, and it was interesting to say the least

gweedo

legendary

Activity: 1498

Merit: 1000

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

the issue isn't bitcoind, it is windows, should have used linux

grue

legendary

Activity: 2058

Merit: 1452

Quote from: NRF on July 30, 2012, 12:20:39 AM

Try this;-

1. Shut everything down.

2. Disable your firewall.

3. Disable your Virus protection.

4. Go to User Account Control Settings and set it all the way to the bottom

5. Reboot (making sure that none of the above start automatically)

6. Start Bitcoind as administrator (right click on the Icon and "Run as administrator")

7. Start your process that invokes the PHP script (right click on the Icon and "Run as administrator")

8. See if it is fixed!!!

At that point, if it is not fixed restart all of the above and reboot. If it is fixed work backwards up the list until it is broken again, then yell "Aha, that's the problem". ;P

now that's what i call shotgun debugging!

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: dooglus on August 08, 2012, 01:16:42 PM

Do you have a wallet which still causes the hangups and crashes but which is empty enough to share with me? I'd like to be able to reproduce the problem without having to make thousands of SatoshiDice bets to get there...

I have a copy of your wallet. It contains:

  * 2121 addresses which have been used and are currently empty,
  * 1484 which contain a single satoshi,
  * 1 address with 8 separate single satoshis, and
  * 1 address with a single 48508 satoshi coin.

I'll see if I can get the client to hang when I ask it to select coins for a transaction.

drakahn

hero member

Activity: 504

Merit: 500

Quote from: scribe on August 08, 2012, 01:25:34 PM

Great to see you got a solve/workaround, would never have found that :-)

Even if i had of had the idea, i would never have found where in the source to change, i don't know the language and a lot of it is plain greek to me, hell making it build in windows was a mix of other people's stories and deleting everything under macos, linux or !windows in the .pro file

The source looks easy enough to follow, like php, but... there is a lot of it, and i have a feeling to know one part of it i would need to know most of it...

What i know of php i know from changing other people work, the same will probably hold true for bitcoin, next on my list is to make a client that lists the single satoshi's as a separate balance, and has a button/text box that will make a transaction with N single satoshi's - but i'm not stressing too much on it, just looking through source every now and then following variables to see what they do

Eventually computer programming will be done by thought, and there will be no need to "know how" to do it.... either that or computer programming will be done by computers, and they will have no need for us... except as entertainment...

note to self : add 'boredom' to future skynet

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Just waiting on his bitcoin address

1 BTC received. Thanks.

scribe

sr. member

Activity: 295

Merit: 250

Great to see you got a solve/workaround, would never have found that :-)

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: drakahn on August 08, 2012, 12:37:37 PM

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

I PM'ed drakahn my address after reading the other thread, then caught up on this thread. I guess I may as well post the contents of the PM here:

Quote from: dooglus on August 08, 2012, 06:09:03 PM

Hi.

I'm glad my little change worked for you.

My address is: 1Up2upZ1xo4tBoq86GLYywwRmshzJc7tA

I hadn't seen your thread offering the bounty, but would have been interested to look into the problem.

Do you have a wallet which still causes the hangups and crashes but which is empty enough to share with me? I'd like to be able to reproduce the problem without having to make thousands of SatoshiDice bets to get there...

drakahn

hero member

Activity: 504

Merit: 500

Yes, the the satoshi dice thread https://bitcointalksearch.org/topic/m.1086135

Dooglus suggested a change in the clients source and after all else failed i built that (took me hours to make bitcoin build in windows) and it seems the problem is solved, if it ignores the single satoshi's it doesn't freeze (well hasn't in the last 24 hours at least)

Just waiting on his bitcoin address

ErebusBat

hero member

Activity: 560

Merit: 500

I am the one who knocks

Did you get this solveD?

drakahn

hero member

Activity: 504

Merit: 500

okay so i want to use sendmany to be able to control where the return tx get sent to, but it freezes fast using sendmany

changing back to sendtoaddress it lasts about 8 hours (1200 transactions this last time) but pays out a lot in fee's then freezes leaving me with all the losses all over the place
http://blockchain.info/tx-index/14167011/3c4ce78154144dcc3b420ec1f86bbf0944ea171dac8e6b35261e6485e51ff14c
to organise, and often when organising the client throws "Failed to create transaction" which i suspect is related to the bot failing

at least it died while it was ahead this time.

drakahn

hero member

Activity: 504

Merit: 500

Quote from: scribe on July 30, 2012, 06:18:38 AM

Did you try it with the "sendtoaddress" line of code commented out? Trying to work out if it's the RPC calls generally or creating the transactions that causes the problem...

with that part commented out (i use sendmany, but same thing) it doesn't do anything but check the balance over and over and i haven't left it long enough to freeze, however it does freeze on checking balance half of the time when running normally, not just creating a transaction, and it does seem to "build up" (but calm down for a couple of minutes if a block comes in)

Is there a limit to the number of unconfirmed transactions? perhaps the amount of unconfirmed sent and received transactions are causing this?

I've only had one last overnight (1800 transactions, the first one) all the others after that (new wallets and blockchains) freeze between 30 and 200 transactions

scribe

sr. member

Activity: 295

Merit: 250

Did you try it with the "sendtoaddress" line of code commented out? Trying to work out if it's the RPC calls generally or creating the transactions that causes the problem...

drakahn

hero member

Activity: 504

Merit: 500

Quote from: NRF on July 30, 2012, 05:38:24 AM

Odd,

Assuming you did the shut off everything thing I suggested the next thing I would do is download the blockchain from scratch and a new wallet.

I use jsonRPCClient and jsonRPCServer classes alot and have never experienced the same issues.

I've tried on a couple of computers, both as administrator, with and without firewalls, nothing else running, fresh install, blockchain and wallet a few times on each

do you use 0.6.3 ?

NRF

sr. member

Activity: 279

Merit: 250

Odd,

Assuming you did the shut off everything thing I suggested the next thing I would do is download the blockchain from scratch and a new wallet.

I use jsonRPCClient and jsonRPCServer classes alot and have never experienced the same issues.

drakahn

hero member

Activity: 504

Merit: 500

Nothing I've tried so far has fixed it, after scribes suggestion still resulted in the client freezing I tried a python version, it seemed to freeze quicker than php, however does seem to follow the rpc timeout rules so is still going, just very slowly instead of dying

I've tried the php version on two computers (both win7 64) and the python on one, i guess next step is trying either linux (not really the best option but i could make it work) or go through old versions of the client until one doesn't freeze (or crash)

unless there is something else I could try

NRF

sr. member

Activity: 279

Merit: 250

Try this;-

1. Shut everything down.

2. Disable your firewall.

3. Disable your Virus protection.

4. Go to User Account Control Settings and set it all the way to the bottom

5. Reboot (making sure that none of the above start automatically)

6. Start Bitcoind as administrator (right click on the Icon and "Run as administrator")

7. Start your process that invokes the PHP script (right click on the Icon and "Run as administrator")

8. See if it is fixed!!!

At that point, if it is not fixed restart all of the above and reboot. If it is fixed work backwards up the list until it is broken again, then yell "Aha, that's the problem". ;P

drakahn

hero member

Activity: 504

Merit: 500

Quote from: ?? on ??

Quote from: drakahn on July 30, 2012, 12:00:39 AM

It seems any RPC call (ATM machine... damnit) that tries while bitcoin is frozen fails (nothing to connect to)
not too sure it has anything to do with the blocks now, it freezes in between them too
I am on a quad core, PHP is up to date and Is there a 64 bit windows version?

Quote from: scribe on July 29, 2012, 04:24:53 PM

Sounds like some kind of race condition or lock condition maybe, possibly caused by too many things trying to access the same resource. The jsonRPCClient code doesn't seem to close it's stream neatly, but don't know if that would matter. You could try adding "fclose($fp);" inside jsonRPCClient.php, after the line "$response = json_decode($response,true);"
More extreme version of the above: Can you edit the script to destroy the jsonRPCClient object and construct a new one each loop? Might help to totally reset any open connections.

I've pregenerated a few million keys in the keypool and its been running, when it freezes again I will try these....

there is your answer a few million keys in the keypool, is alot, i think a good number is 10,000 should be good

... I'm not sure you are following along... I generated the millions of keys to try and stop it from freezing, the keypool is back on default now but i have millions of keys, so one less thing for the client to do that could cause a freeze by locking out rpc, until i've used up the keys, but that should be a few days at least, and then i can try the suggestions from scribe and hope for the best

other things i have done that haven't worked
-Increased BdB cache to 1gb
-Increased/decreased number of connections
-A 30 second delay between RPC's

drakahn

hero member

Activity: 504

Merit: 500

It seems any RPC call (ATM machine... damnit) that tries while bitcoin is frozen fails (nothing to connect to)
not too sure it has anything to do with the blocks now, it freezes in between them too
I am on a quad core, PHP is up to date and Is there a 64 bit windows version?

Quote from: scribe on July 29, 2012, 04:24:53 PM

Sounds like some kind of race condition or lock condition maybe, possibly caused by too many things trying to access the same resource. The jsonRPCClient code doesn't seem to close it's stream neatly, but don't know if that would matter. You could try adding "fclose($fp);" inside jsonRPCClient.php, after the line "$response = json_decode($response,true);"
More extreme version of the above: Can you edit the script to destroy the jsonRPCClient object and construct a new one each loop? Might help to totally reset any open connections.

I've pregenerated a few million keys in the keypool and its been running, when it freezes again I will try these....

scribe

sr. member

Activity: 295

Merit: 250

Not one I can test here, but some ideas to try:

Sounds like some kind of race condition or lock condition maybe, possibly caused by too many things trying to access the same resource. The jsonRPCClient code doesn't seem to close it's stream neatly, but don't know if that would matter. You could try adding "fclose($fp);" inside jsonRPCClient.php, after the line "$response = json_decode($response,true);"
More extreme version of the above: Can you edit the script to destroy the jsonRPCClient object and construct a new one each loop? Might help to totally reset any open connections.
Can you isolate which RPC call is causing it by commenting out the sendtoaddress line? It might be the effect of many RPC calls, or one specific one.
Are you running the right bitcoind version for your PC, i.e. 32-bit vs 64-bit? Any improvement if you "downgrade" to 32-bit if you're running 64-bit?
Alternatively, is your PHP version up to date?
25% is a weird number - are you running a quad core and maybe the bitcoind hits a CPU limit in one core?

drakahn

hero member

Activity: 504

Merit: 500

Quote from: ?? on ??

Quote from: drakahn on July 26, 2012, 03:57:47 AM

This last run went for 60 rolls of the dice and then froze, it seems it freezes when bitcoin gets a new block at the same time as trying to send a transaction, and was actually using 150mb of ram, not under 100mb ... but still plenty free

is there a way to check for a block before trying to send a transaction?

what version did you try updating?

i have 0.6.3 the latest on bitcoin.org, thats in the OP...

I'm really not sure its tied to new blocks any more, it seems to just randomly freeze on an api call

drakahn

hero member

Activity: 504

Merit: 500

This last run went for 60 rolls of the dice and then froze, it seems it freezes when bitcoin gets a new block at the same time as trying to send a transaction, and was actually using 150mb of ram, not under 100mb ... but still plenty free

is there a way to check for a block before trying to send a transaction?

drakahn

hero member

Activity: 504

Merit: 500

bitcoind just crashes, bitcoin-qt freezes at 25% cpu use

Still freezes if i use sendmany

bitcoin uses under 100mb of ram at the point it freezes, over 3gb free

drakahn

hero member

Activity: 504

Merit: 500

I have it all set up, but it freezes after a few transactions, thats the part i need working

drakahn

hero member

Activity: 504

Merit: 500

I'll add a 1 bitcoin reward for information leading to a working php => bitcoin rpc setup in windows 7 for me

drakahn

hero member

Activity: 504

Merit: 500

Okay so I'm trying to make use of the PHP martingale bot that's bouncing around, and after a while the bitcoin client freezes and jsonRPC fails to connect (the client stays frozen and won't start until the whole computer is restarted) it doesn't seem to be connected to anything specific, just after a while (30 transactions on one wallet, 1800 on another) cpu use of the client hits 25% and it freezes

bitcoind freezes then crashes

I'm in windows 7 with the latest client from bitcoin.org

Anyone know how to stop this?

edit : it seems to happen when trying to use the rpc at the same time as bitcoin is downloading a block

Topic: [solved/paid][1btc reward] PHP RPC commands freezing bitcoin (Read 4333 times)