Topic: 2 btc for debugging short python script with json-rpc and bitcoin (Read 2236 times)
thanks
August 27, 2011, 06:43:55 PM
Thanks, techwtf. Just sent the bitcoins your way. BTW, I realized there's absolutely no reason why I need to run this cron job from the machine with my web server. All my other cron jobs need to be located on the web server, but there's no reason this one can't be located on the machine with the bitcoind daemon. So I don't have to worry about the SSL (still sent the extra btc for that, however).
In any case, techwtf's script is the only Python script to dump wallet contents to an address on the web that I'm aware of that will work with Python 2.7. Components of the various JSON-RPC libraries are broke on Python 2.7. So hopefully this will benefit someone else.
In any case, techwtf's script is the only Python script to dump wallet contents to an address on the web that I'm aware of that will work with Python 2.7. Components of the various JSON-RPC libraries are broke on Python 2.7. So hopefully this will benefit someone else.
August 27, 2011, 02:13:17 PM
btw, I dont really sure what is needed to verify a self signed SSL connection.
you may also have to set pycurl.SSLCERTPASSWD, pycurl.SSLKEY, pycurl.SSLKEYPASSWD.
you may also have to set pycurl.SSLCERTPASSWD, pycurl.SSLKEY, pycurl.SSLKEYPASSWD.
August 27, 2011, 02:06:43 PM
1L8x5DkbfpRag5PZz23bQtdUfTtqVnp3hA
GAE has some strict SSL checking support, embedded in its urlfetch service. you may need a working SSL cert signed by some certificate authority to make sure a fake cert is not used.
self signed cert can be checked by curl(pycurl), using pycurl.SSLCERT.
I've not tested with ssl enabled environment. have tested without ssl.
ps: you can use ssh tunnel like ssh -L 8332:127.0.0.1:8332, so no more ssl needed XD
---
import pycurl, json, StringIO
amount = 1
address = '------'
postdata = json.dumps({"method":"sendtoaddress","params":[address,1.23456789],"id":1}).replace('1.23456789', '%.8f' % (amount))
rpcuri = 'http://user:pass@localhost:8332/'
b = StringIO.StringIO()
c = pycurl.Curl()
c.setopt(pycurl.URL, rpcuri)
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.NOSIGNAL, 1)
c.setopt(pycurl.POSTFIELDS, postdata)
#c.setopt(pycurl.SSLCERT, "/path/to/cert/file") # FIXME
c.setopt(pycurl.WRITEFUNCTION, b.write)
try:
c.perform()
print b.getvalue()
except pycurl.error, e:
print str(e)
c.close()
GAE has some strict SSL checking support, embedded in its urlfetch service. you may need a working SSL cert signed by some certificate authority to make sure a fake cert is not used.
self signed cert can be checked by curl(pycurl), using pycurl.SSLCERT.
I've not tested with ssl enabled environment. have tested without ssl.
ps: you can use ssh tunnel like ssh -L 8332:127.0.0.1:8332, so no more ssl needed XD
---
import pycurl, json, StringIO
amount = 1
address = '------'
postdata = json.dumps({"method":"sendtoaddress","params":[address,1.23456789],"id":1}).replace('1.23456789', '%.8f' % (amount))
rpcuri = 'http://user:pass@localhost:8332/'
b = StringIO.StringIO()
c = pycurl.Curl()
c.setopt(pycurl.URL, rpcuri)
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.NOSIGNAL, 1)
c.setopt(pycurl.POSTFIELDS, postdata)
#c.setopt(pycurl.SSLCERT, "/path/to/cert/file") # FIXME
c.setopt(pycurl.WRITEFUNCTION, b.write)
try:
c.perform()
print b.getvalue()
except pycurl.error, e:
print str(e)
c.close()
August 27, 2011, 01:19:07 PM
OK, techwtf's solution worked! Thank you. Would you please send your bitcoin address, techwtf?
I still wish I knew what was going wrong with the jsonrpc library in Python 2.7. People are going to start having more and more problems with this and Bitcoin payment solutions in Python.
Also, my concern is that I won't be able to use this script in a production environment unless it can be adapted to SSL. I can't be sending my json-rpc password unencrypted over the network, even if it is just within an AWS region and not the general internet. I'll add on another 2 if you can show me how to set it up with SSL authentication. The wiki has info on how to set this up in PHP. but nothing on Python:
https://en.bitcoin.it/wiki/Enabling_SSL_on_original_client_daemon
Thanks again for your help, techwtf and ruski.
I still wish I knew what was going wrong with the jsonrpc library in Python 2.7. People are going to start having more and more problems with this and Bitcoin payment solutions in Python.
Also, my concern is that I won't be able to use this script in a production environment unless it can be adapted to SSL. I can't be sending my json-rpc password unencrypted over the network, even if it is just within an AWS region and not the general internet. I'll add on another 2 if you can show me how to set it up with SSL authentication. The wiki has info on how to set this up in PHP. but nothing on Python:
https://en.bitcoin.it/wiki/Enabling_SSL_on_original_client_daemon
Thanks again for your help, techwtf and ruski.
August 27, 2011, 10:16:21 AM
import urllib2, base64, json
rpcuri = 'http://localhost:8332/'
user = '---'
passwd = '---'
amount = 1
address = '------'
postdata = json.dumps({"method":"sendtoaddress","params":[address,1.23456789],"id":1}).replace('1.23456789', '%.8f' % (amount))
print postdata
req = urllib2.Request(rpcuri, postdata, {'Authorization': "Basic " + base64.b64encode(user + ":" + passwd)})
try:
f = urllib2.urlopen(req)
print f.read()
except urllib2.HTTPError, error:
print error.read()
except Exception, e:
print str(e)
August 27, 2011, 09:59:04 AM
I mean, you say you are experiencing a serialization problem,
you can use tcpdump (-A is handy)/wireshare to read what was sent to bitcoind(mask the auth info first).
example, the post data I captured:
{"method":"sendtoaddress","params":["SOMEADDRESS",1.00000000],"id":1}
python code:
json.dumps({"method":"sendtoaddress","params":["SOMEADDRESS",1.00000000],"id":1})
but it returns '{"params": ["SOMEADDRESS", 1.0], "method": "sendtoaddress", "id": 1}'
so use a magic 1.23456789:
json.dumps({"method":"sendtoaddress","params":["SOMEADDRESS",1.23456789],"id":1}).replace('1.23456789', '%.8f' % (THEAMOUNT))
you now get the right json data like: '{"params": ["SOMEADDRESS", 1.00000000], "method": "sendtoaddress", "id": 1}'
You don't have to use any json-rpc library. Do basic auth in headers, and post the json data, that is enough.
you can use tcpdump (-A is handy)/wireshare to read what was sent to bitcoind(mask the auth info first).
example, the post data I captured:
{"method":"sendtoaddress","params":["SOMEADDRESS",1.00000000],"id":1}
python code:
json.dumps({"method":"sendtoaddress","params":["SOMEADDRESS",1.00000000],"id":1})
but it returns '{"params": ["SOMEADDRESS", 1.0], "method": "sendtoaddress", "id": 1}'
so use a magic 1.23456789:
json.dumps({"method":"sendtoaddress","params":["SOMEADDRESS",1.23456789],"id":1}).replace('1.23456789', '%.8f' % (THEAMOUNT))
you now get the right json data like: '{"params": ["SOMEADDRESS", 1.00000000], "method": "sendtoaddress", "id": 1}'
You don't have to use any json-rpc library. Do basic auth in headers, and post the json data, that is enough.
August 27, 2011, 09:40:54 AM
Ruski,
Thanks for your input. I tried assigning the float type and passing it to another variable instead of doing it all at once -- didn't work.
Your idea about transaction fees isn't a bad one. I would not have thought that this could be a problem. Unfortunately, my bitcoin.conf on the testnet was already set at transaction fees=0.00, so that didn't work either
I'm trying it out in a realnet env.-- no difference.
It's becoming clear to me that this problem has something to do with the new Python version I'm running: Python 2.7. I actually found an old version of this same kind of script for python at http://bitcointalk.org/?topic=1431.0 This script won't work on either of my systems running Python 2.7, either. It would appear that there were several changes made to the way Python handles floats in 2.7, which appear to have made the Bitcoin JSON-RPC library not work with certain calls.
Thank you in advance for your help!
PS -- techwtf: I'm not understanding what you're suggesting. Can you please write out the basic structure in Python?
Thanks for your input. I tried assigning the float type and passing it to another variable instead of doing it all at once -- didn't work.
Your idea about transaction fees isn't a bad one. I would not have thought that this could be a problem. Unfortunately, my bitcoin.conf on the testnet was already set at transaction fees=0.00, so that didn't work either
I'm trying it out in a realnet env.-- no difference.
It's becoming clear to me that this problem has something to do with the new Python version I'm running: Python 2.7. I actually found an old version of this same kind of script for python at http://bitcointalk.org/?topic=1431.0 This script won't work on either of my systems running Python 2.7, either. It would appear that there were several changes made to the way Python handles floats in 2.7, which appear to have made the Bitcoin JSON-RPC library not work with certain calls.
Thank you in advance for your help!
PS -- techwtf: I'm not understanding what you're suggesting. Can you please write out the basic structure in Python?
August 27, 2011, 03:47:15 AM
Convert the balance to a float in a seperate variable, and pass that to the sendtoaddress function. 
You'd be amazed the problems you can cause trying to pass a function result as a parameter of another function. Let me know if it works
PS - May be a stupid thought, but have you checked the transaction fee setting on your client? If you try to send the entire balance and it's trying to take out a fee, it's going to overdebit your balance and could cause an error.
PPS - Another stupid thought, it could simply be your test environment that's causing the error. Try it with real bitcoins if you haven't already.
Code:
#!/usr/bin/env python
# coding: utf8
import urllib
from gluon import *
from btcservice import access
OFFLINEWALLET_ADDRESS = 'mfo84Lum3gmk6v2cLhJMdtntS5DnTqkHYi' # CAUTION--DO NOT RUN THIS SCRIPT WITHOUT SETTING THIS FIRST.
balance = access.getbalance()
balance = float(balance)
if OFFLINEWALLET_ADDRESS != '':
access.sendtoaddress(OFFLINEWALLET_ADDRESS,balance)
You'd be amazed the problems you can cause trying to pass a function result as a parameter of another function. Let me know if it works
PS - May be a stupid thought, but have you checked the transaction fee setting on your client? If you try to send the entire balance and it's trying to take out a fee, it's going to overdebit your balance and could cause an error.
PPS - Another stupid thought, it could simply be your test environment that's causing the error. Try it with real bitcoins if you haven't already.
August 27, 2011, 03:23:39 AM
Oh...
so you may have to make the json request by hand to bypass the problem... ?
How access.sendtoaddress works? You may always send a magic number(known to have no rounding errors), then replace that number to ("%.8f" % (THE REAL AMOUNT)).
like this:
sendtoaddress(addr, amount):
#....make a fake request sending 12345 BTC ...
#....replace 12345 to the real amount you sent...
#...send the request...
so you may have to make the json request by hand to bypass the problem... ?
How access.sendtoaddress works? You may always send a magic number(known to have no rounding errors), then replace that number to ("%.8f" % (THE REAL AMOUNT)).
like this:
sendtoaddress(addr, amount):
#....make a fake request sending 12345 BTC ...
#....replace 12345 to the real amount you sent...
#...send the request...
August 26, 2011, 09:40:43 PM
Still haven't figured this out, but if I multiply the balance variable by 0.9, it lets me transfer half of the balance. If that helps anyone debug it. I'm hoping to transfer the entire balance.
August 26, 2011, 08:50:04 PM
To make it clear, the btcservice module maintains the serviceproxy. To repeat, the connection is not the issue, it's the code and something about the type.
For example, if I write this:
It subtracts a fraction of the balance, but only a fraction. If I multiple by 1e8 (which seems more logical to me), then it doesn't work at all (same Json exception as shown in my first post).
So the wallet is full, and the connection is just fine. The answer to this question will be something to do with types, floats, conversions, etc. Not JSON-RPC connections or the service proxy -- those are fine.
Thank you.
For example, if I write this:
Code:
OFFLINEWALLET_ADDRESS = 'mfo84Lum3gmk6v2cLhJMdtntS5DnTqkHYi' # CAUTION--DO NOT RUN THIS SCRIPT WITHOUT SETTING THIS FIRST.
balance = (float(access.getbalance() / 1e8))
if OFFLINEWALLET_ADDRESS != '':
access.sendtoaddress(OFFLINEWALLET_ADDRESS,float(balance))
It subtracts a fraction of the balance, but only a fraction. If I multiple by 1e8 (which seems more logical to me), then it doesn't work at all (same Json exception as shown in my first post).
So the wallet is full, and the connection is just fine. The answer to this question will be something to do with types, floats, conversions, etc. Not JSON-RPC connections or the service proxy -- those are fine.
Thank you.
August 26, 2011, 08:36:04 PM
OK, I'm running on testnet, and I have plenty of funds in the wallet. So that's not the problem.
Also, I'm quite sure the service proxy is running fine, I've been using access.getaddress() and access.getinfo() for a week now with no issues. Also, if I insert an actual decimal amount into my function instead of use the balance variable, it works fine. So the JSON-RPC connection is not in question.
I'm running Python 2.7, and I think I found the answer to my question -- apparently Python 2.7 makes all the difference. But I'll post the link that I'm looking at and see if someone beats me to posting an answer to my own question, if so you'll still get the 2 btc. See the entry from mikez302 on this page: http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
And yes, I should have posted on pastebin but I thought the code was so short it would still be readable.
Thanks again.
Also, I'm quite sure the service proxy is running fine, I've been using access.getaddress() and access.getinfo() for a week now with no issues. Also, if I insert an actual decimal amount into my function instead of use the balance variable, it works fine. So the JSON-RPC connection is not in question.
I'm running Python 2.7, and I think I found the answer to my question -- apparently Python 2.7 makes all the difference. But I'll post the link that I'm looking at and see if someone beats me to posting an answer to my own question, if so you'll still get the 2 btc. See the entry from mikez302 on this page: http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
And yes, I should have posted on pastebin but I thought the code was so short it would still be readable.
Thanks again.
August 26, 2011, 08:22:07 PM
I reproduced it until I had enough funds to send
So if balance=0, I think this is your problem
If not here is my code:
It works when balance>0 and prints your error when balance=0
So if balance=0, I think this is your problem
If not here is my code:
Code:
#!/usr/bin/env python
# coding: utf8
import urllib
#from gluon import *
#from btcservice import access
from jsonrpc import ServiceProxy
OFFLINEWALLET_ADDRESS = '19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3' # CAUTION--DO NOT RUN THIS SCRIPT WITHOUT SETTING THIS FIRST.
access = ServiceProxy("http://login:[email protected]:8332")
balance = access.getbalance()
if OFFLINEWALLET_ADDRESS != '':
access.sendtoaddress(OFFLINEWALLET_ADDRESS,float(balance))
August 26, 2011, 08:14:02 PM
You should post your code on pastebin or something, I don't know what happened in your code 
You want to transfer all you coins to an address that wallet is offline stored?
Make sure your RPC settings are right first, in ~/.bitcoin/bitcoin.conf:
rpcuser=username
rpcpassword=password
rpcallowip=xxx.xxx.xxx.xxx
rpcport=xxxx
first you need to do some basic auth in python, try to get getwork or getinfo command working, then getbalance and at last sendtoaddress.
You want to transfer all you coins to an address that wallet is offline stored?
Make sure your RPC settings are right first, in ~/.bitcoin/bitcoin.conf:
rpcuser=username
rpcpassword=password
rpcallowip=xxx.xxx.xxx.xxx
rpcport=xxxx
first you need to do some basic auth in python, try to get getwork or getinfo command working, then getbalance and at last sendtoaddress.
August 26, 2011, 08:00:20 PM
I've tried and tried to get this script to work. I want to run this as a cron job every hour or so to dump my online wallet's contents to an offline wallet). I'm trying to interact with the bitcoin json-rpc lib for python and I'm stuck in python type hell. Another individual (who is an actual programmer, unlike me --I'm just a web dev with some python skills) tried to get it done to no avail. I'll send 2 btc to the first person who posts a working solution to the boards. Currently, the script gets a "JSONRPCException(resp['error'])". Most of my alternate solutions tell me that my decimal types are serializable as json objects.
If it's not obvious, the access function is the serviceproxy to the bitcoin client running as bitcoind,
Thank you.
****
If it's not obvious, the access function is the serviceproxy to the bitcoin client running as bitcoind,
Thank you.
****
Code:
#!/usr/bin/env python
# coding: utf8
import urllib
from gluon import *
from btcservice import access
OFFLINEWALLET_ADDRESS = 'mfo84Lum3gmk6v2cLhJMdtntS5DnTqkHYi' # CAUTION--DO NOT RUN THIS SCRIPT WITHOUT SETTING THIS FIRST.
balance = access.getbalance()
if OFFLINEWALLET_ADDRESS != '':
access.sendtoaddress(OFFLINEWALLET_ADDRESS,float(balance))
Jump to: