btw: go google app engine!
I've managed to get https://github.com/jgarzik/python-bitcoinrpc to work on GAE
Manuel
Topic: Running on the Google App Engine (Read 14650 times)
Hi Gavin, I was reading this old topic and I wanted to ask: why did you add the secret parameter to the request? Isn't the connection secure already if you use SSL? Thanks!
btw: go google app engine!
I've managed to get https://github.com/jgarzik/python-bitcoinrpc to work on GAE
Manuel
btw: go google app engine!
I've managed to get https://github.com/jgarzik/python-bitcoinrpc to work on GAE
Manuel
You probably already found it out and this topic is extremely old, I know. But:
App Engine doesn't support file I/O. You could write it to the Blob store but that's probably too much overhead.
Why not have your thing read the secret from a file, so you can release all the code, but leave the secret key to your machines still hidden?
App Engine doesn't support file I/O. You could write it to the Blob store but that's probably too much overhead.
Why not have your thing read the secret from a file, so you can release all the code, but leave the secret key to your machines still hidden?
Is it possible to rewrite portions of bitcoin so it runs entirely in GAE? Particularly the feature that handles transactions, so it is fully scalable?
I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
We are in fact trying to build a small network module to interface the BitCoin Protocol in Python, if you'd like to help feel free to contribute: https://bitcointalksearch.org/topic/python-client-231 I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
Apparently it is considering at http://pastebin.com/10k74rSd (which is a summary of Bitcoin client hosts in #bitcoin on LFNet as of about 15 minutes ago) lines 432-529 are all GAE.
How did u generate that list?
Apparently it is considering at http://pastebin.com/10k74rSd (which is a summary of Bitcoin client hosts in #bitcoin on LFNet as of about 15 minutes ago) lines 432-529 are all GAE.
How did u generate that list?
https://bitcointalksearch.org/topic/m.13665
Is it possible to rewrite portions of bitcoin so it runs entirely in GAE? Particularly the feature that handles transactions, so it is fully scalable?
I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
So basically, you're running Bitcoin on a VPS and using an HTTPS connection from App Engine to signal it what to do? Why not just host the site on your server?
App Engine gives me:Free bandwidth and data storage (with the option to pay for what I use if I go over a gigabyte)
Built-in, already-configured-and-tweaked tools like memcache and BigTable
An architecture where scaling up is built-in (App Engine will automagically replicate my app and distribute it across Google's worldwide content distribution network if it gets a lot of traffic)
And all of that is administered by Google's sysadmins (who are WAY better at that than I am).
But mostly I did it as an experiment and because it was a technical challenge and because lately I'm an App Engine fanboy. I would've been done a day earlier if I'd run everything on the same server and wrote the front end in PHP+MySQL.
So basically, you're running Bitcoin on a VPS and using an HTTPS connection from App Engine to signal it what to do? Why not just host the site on your server?
Joozero asked if I'd make the source to freebitcoins.appspot.com available, and I promised to start a thread explaining how I created it-- and here it is.
freebitcoins is running on the Google App Engine. Well, the part you see is running on App Engine.
There's also a back-end bitcoind server running on a debian VPS that I've had for years. The App Engine code makes JSON-RPC calls via the App Engine url fetching API. Making that connection secure was a bit tricky; here's how I did it:
First, I connect to the VPS using https so the traffic is encrypted.
I need a secure connection because I add a secret pre-shared value to the JSON call. I'm not going to make the full source for freebitcoins fully open source because I don't want to bother with constantly working to keeping that secret value secret. I should hash the entire request with a secret key and an ever-increasing nonce, but I was lazy and I'm just not very worried about a man-in-the-middle attack between Google and my VPS.
I could have hacked my copy of the bitcoind C++ code to check for the secret value in requests and also modified it so it accepted connections from the Internet... but instead I wrote this little proxy server in Python that runs on the same box as bitcoind:
I'm happy with how it is working. My biggest worry is that the bitcoind process might unexpectedly exit, or fill up the disk with debug.log messages, or just generally be flaky. But so far so good...
freebitcoins is running on the Google App Engine. Well, the part you see is running on App Engine.
There's also a back-end bitcoind server running on a debian VPS that I've had for years. The App Engine code makes JSON-RPC calls via the App Engine url fetching API. Making that connection secure was a bit tricky; here's how I did it:
First, I connect to the VPS using https so the traffic is encrypted.
I need a secure connection because I add a secret pre-shared value to the JSON call. I'm not going to make the full source for freebitcoins fully open source because I don't want to bother with constantly working to keeping that secret value secret. I should hash the entire request with a secret key and an ever-increasing nonce, but I was lazy and I'm just not very worried about a man-in-the-middle attack between Google and my VPS.
I could have hacked my copy of the bitcoind C++ code to check for the secret value in requests and also modified it so it accepted connections from the Internet... but instead I wrote this little proxy server in Python that runs on the same box as bitcoind:
Code:
#
# WSGI Proxy into the bitcoind daemon (which only listens on 127.0.0.1 for security).
# Rejects non-HTTPs connections, or non-POST requests
# Rejects queries that don't include a pre-shared secret value
#
# Apache calls this because of a directive in /etc/apache2/sites-available/xyz.com
#
import hashlib
import urllib2
def application(environ, start_response):
serverURL = "http://127.0.0.1:8332/"
def error_response(message):
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(message)))]
start_response('500 Internal Server error', response_headers)
return message
if environ.get("HTTPS") != "1":
return error_response("Insecure connections not allowed.")
request = environ.get("wsgi.input").read()
secret = request[0:32]
json_request = request[32:]
if hashlib.md5(" pre shared secret goes here "+json_request).hexdigest() != secret:
return error_response("Authentication failed.")
req = urllib2.Request(serverURL, json_request)
response = urllib2.urlopen(req)
json_response = response.read()
status = '200 OK'
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(json_response)))]
start_response(status, response_headers)
return [json_response]
The other end of the connection is also django+Python code running on App Engine:# WSGI Proxy into the bitcoind daemon (which only listens on 127.0.0.1 for security).
# Rejects non-HTTPs connections, or non-POST requests
# Rejects queries that don't include a pre-shared secret value
#
# Apache calls this because of a directive in /etc/apache2/sites-available/xyz.com
#
import hashlib
import urllib2
def application(environ, start_response):
serverURL = "http://127.0.0.1:8332/"
def error_response(message):
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(message)))]
start_response('500 Internal Server error', response_headers)
return message
if environ.get("HTTPS") != "1":
return error_response("Insecure connections not allowed.")
request = environ.get("wsgi.input").read()
secret = request[0:32]
json_request = request[32:]
if hashlib.md5(" pre shared secret goes here "+json_request).hexdigest() != secret:
return error_response("Authentication failed.")
req = urllib2.Request(serverURL, json_request)
response = urllib2.urlopen(req)
json_response = response.read()
status = '200 OK'
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(json_response)))]
start_response(status, response_headers)
return [json_response]
Code:
import hashlib
import jsonrpc
from google.appengine.api import urlfetch
def make_bitcoin_request(server_url, server_secret, method, params):
json_request = {
"jsonrpc": "2.0",
"id": str(time.time()),
"method": method, "params": params
}
json_string = jsonrpc.dumps(json_request)
secret = hashlib.md5(server_secret+json_string).hexdigest()
try:
fetch_result = urlfetch.fetch(payment_server,
method="POST", payload=secret+json_string,
headers={ 'Content-Type' : 'text/plain' })
except urlfetch.Error, e:
logging.error('make_bitcoin_request failed: '+str(e))
logging.error('Request:'+json_string)
return None
if fetch_result.status_code != 200:
logging.error('make_bitcoin_request failed; urlfetch status code %d'%(fetch_result.status_code))
logging.error('Request:'+json_string)
return None
r = jsonrpc.loads(fetch_result.content)
if r['error'] is not None:
logging.error('make_bitcoin_request failed; JSON error returned')
logging.error('Request:'+json_string)
logging.error('Result:'+fetch_result.content)
return None
return r['result']
import jsonrpc
from google.appengine.api import urlfetch
def make_bitcoin_request(server_url, server_secret, method, params):
json_request = {
"jsonrpc": "2.0",
"id": str(time.time()),
"method": method, "params": params
}
json_string = jsonrpc.dumps(json_request)
secret = hashlib.md5(server_secret+json_string).hexdigest()
try:
fetch_result = urlfetch.fetch(payment_server,
method="POST", payload=secret+json_string,
headers={ 'Content-Type' : 'text/plain' })
except urlfetch.Error, e:
logging.error('make_bitcoin_request failed: '+str(e))
logging.error('Request:'+json_string)
return None
if fetch_result.status_code != 200:
logging.error('make_bitcoin_request failed; urlfetch status code %d'%(fetch_result.status_code))
logging.error('Request:'+json_string)
return None
r = jsonrpc.loads(fetch_result.content)
if r['error'] is not None:
logging.error('make_bitcoin_request failed; JSON error returned')
logging.error('Request:'+json_string)
logging.error('Result:'+fetch_result.content)
return None
return r['result']
I'm happy with how it is working. My biggest worry is that the bitcoind process might unexpectedly exit, or fill up the disk with debug.log messages, or just generally be flaky. But so far so good...
Jump to: