Author

Topic: Help with Bitcoin JSON RPC - more SLOOOWness (Read 3500 times)

legendary
Activity: 1498
Merit: 1000
Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.

If you're successful it probably be too late. Not one bitcoin company uses a stock bitcoind anymore. The software has become too focused on the end user, which is important for bitcoin-qt but not bitcoind.

Well, how long did it take you to make the JSON/Account Management features?

Maybe I could make some sort of open source project, its truly a pain. Maybe just for me, because I'm bad, but whatever.

I don't use the account management I use MySQL and it makes it easier. But it only took me a week to get the features I needed.
legendary
Activity: 1498
Merit: 1000
Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.

If you're successful it probably be too late. Not one bitcoin company uses a stock bitcoind anymore. The software has become too focused on the end user, which is important for bitcoin-qt but not bitcoind.
full member
Activity: 168
Merit: 100
Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.

If you're successful it probably be too late. Not one bitcoin company uses a stock bitcoind anymore. The software has become too focused on the end user, which is important for bitcoin-qt but not bitcoind.

Well, how long did it take you to make the JSON/Account Management features?

Maybe I could make some sort of open source project, its truly a pain. Maybe just for me, because I'm bad, but whatever.
full member
Activity: 168
Merit: 100
Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.
full member
Activity: 168
Merit: 100
Does BitcoinJ also have account management / JSON availability?
full member
Activity: 168
Merit: 100
How big is your wallet?

getbalance() is O(N) where N is the total number of transactions in the wallet. (actually, it can be even worse than O(N) if you have long chains of unconfirmed transactions in your wallet)

"Patches welcome" to fix that, I expected one of the big bitcoind-using services would submit a performance fix patch long ago.



Oh great, so getbalance basically cant be scaled at all?
legendary
Activity: 1652
Merit: 2311
Chief Scientist
How big is your wallet?

getbalance() is O(N) where N is the total number of transactions in the wallet. (actually, it can be even worse than O(N) if you have long chains of unconfirmed transactions in your wallet)

"Patches welcome" to fix that, I expected one of the big bitcoind-using services would submit a performance fix patch long ago.

full member
Activity: 309
Merit: 100
Have you tried running your script locally on the bitcoind Node? without knowing your network setup, Id suggest try doing that a few hundred / thousand times and see if you have any problems.

If you do the test, and no problems occur, immediately run the Remote script and see if any problems come up.


If problems occur locally, then its bitcoind / your server. If problems don't occur locally but happen remotely, then there is a problem between your bitoind server and your remote host.


That could be 1 or more things that could cause that. As suggested above, Firewall would be one of them, but could also be ISP problems on either side, rate limiting, etc.

If you find the problem, I would be interested in knowing.


kosta
legendary
Activity: 1498
Merit: 1000
Sounds like firewall is making it timeout.
full member
Activity: 168
Merit: 100
A bit more information, a list of response times for the following code:

Quote
$time = microtime(true);
require_once 'jsonRPCClient.php';
$bitcoin = new jsonRPCClient('http://bitcoinrpc:mypassword@myserver:8332/');
echo $bitcoin->getbalance("kazu");
echo "
";
echo $save = microtime(true)-$time;
echo "
";
file_put_contents('time.txt', $save . PHP_EOL, FILE_APPEND);
?>

0.12215495109558
0.12762403488159
0.12906503677368
0.12794995307922
0.145015001297
0.12692308425903
0.13107013702393
0.13046789169312
0.12596988677979
0.12823700904846
0.12685894966125
0.12688183784485
0.13011193275452
0.13094592094421
0.1299901008606
0.12974500656128
0.13020300865173
31.179399013519
0.12966895103455
0.13000798225403
0.1297299861908
0.13064789772034
0.14454817771912
0.12939882278442
0.12828493118286
31.177670001984
0.12911486625671
0.13026690483093
0.12602400779724
0.129154920578
0.12882518768311
0.12818312644958
0.12817215919495
15.141080856323
0.16296315193176
0.12952899932861
0.19870591163635
0.16739010810852
0.12964200973511
0.16857099533081
0.13083600997925
31.268596887589

There was 1 or 2 timeouts in that process as well.
full member
Activity: 168
Merit: 100
This is your local wannabe-server-guy and as usual I've managed to screw things up again. After realizing that bitcoind was working very slow I found a server with a much faster disk IO I loaded bitcoind on there, and boom, instant good response time!

Quote
time bitcoind getinfo
{
    "version" : 80100,
    "protocolversion" : 70001,
    "walletversion" : 60000,
    "balance" : 0.00000000,
    "blocks" : 244761,
    "connections" : 8,
    "proxy" : "",
    "difficulty" : 21335329.11398300,
    "testnet" : false,
    "keypoololdest" : 1374707237,
    "keypoolsize" : 101,
    "paytxfee" : 0.00000000,
    "errors" : ""
}

real    0m0.006s
user    0m0.004s
sys     0m0.000s

After quickly thanking the internet for helping me with that I'd like to ask another favor. The server running bitcoind here can't be my webhost for reasons. So I have done an rpcallowip allowing connection from a remote PHP host. And, wonderfully enough, it works much of the time, with response times about 0.14-ish. Which is fine by me. However, sometimes I get:

Quote
Warning: fopen(http://...@myserver:8332/): failed to open stream: Connection timed out in /stuff/jsonRPCClient.php.

Fatal error: Uncaught exception 'Exception' with message 'Unable to connect to http://bitcoinrpc:mypassword@myserver:8332/' in myfile Stack trace: #0 /myfile: jsonRPCClient->__call('listtransaction...', Array) #1 mypath: jsonRPCClient->listtransactions('kazu') #2 mypath: include('file.php') #3 {main} thrown in /stuff/jsonRPCClient.php on line 140
Listtransactions happens to be the first call I make. This error comes after waiting a while. To test, if this was a legitimate issue, I tried telnetting. Now, I'm not 100% sure this means that it should be working working, but I did:

time telnet myserver:8332

and repeated it 100 times, and got back

telnet: could not resolve myserver:8332/telnet: System error

with an average response time of 0.016 seconds.

So is the issue with my bitcoind-machine's internet, or does the telnet experiment imply that its a problem with my client? What should I do?

EDIT: Just tested, I get "telnet: could not resolve server:8332/telnet: System error" even for servers that aren't listening. So, I tried portscanning. I get "port open" for on my server on 8332 in a number of milliseconds. What gives?
Jump to: