Author

Topic: Running on the Google App Engine (Read 14595 times)

newbie
Activity: 56
Merit: 0
September 24, 2013, 06:06:05 PM
#10
Hi Gavin, I was reading this old topic and I wanted to ask: why did you add the secret parameter to the request? Isn't the connection secure already if you use SSL? Thanks!

btw: go google app engine!
I've managed to get https://github.com/jgarzik/python-bitcoinrpc to work on GAE Cheesy

Manuel
full member
Activity: 530
Merit: 100
https://www.pax-coin.io/
April 22, 2013, 04:15:17 PM
#9
You probably already found it out and this topic is extremely old, I know. But:

Why not have your thing read the secret from a file, so you can release all the code, but leave the secret key to your machines still hidden?

App Engine doesn't support file I/O. You could write it to the Blob store but that's probably too much overhead.
hero member
Activity: 616
Merit: 500
Firstbits.com/1fg4i :)
March 11, 2011, 05:06:01 AM
#8
Why not have your thing read the secret from a file, so you can release all the code, but leave the secret key to your machines still hidden?
hero member
Activity: 489
Merit: 504
September 22, 2010, 08:48:43 AM
#7
Is it possible to rewrite portions of bitcoin so it runs entirely in GAE?  Particularly the feature that handles transactions, so it is fully scalable?

I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
We are in fact trying to build a small network module to interface the BitCoin Protocol in Python, if you'd like to help feel free to contribute: https://bitcointalksearch.org/topic/python-client-231
legendary
Activity: 1304
Merit: 1014
September 21, 2010, 09:33:19 PM
#6
Apparently it is considering at http://pastebin.com/10k74rSd (which is a summary of Bitcoin client hosts in #bitcoin on LFNet as of about 15 minutes ago) lines 432-529 are all GAE.


How did u generate that list?
hero member
Activity: 574
Merit: 504
September 21, 2010, 08:52:10 PM
#5
Apparently it is considering at http://pastebin.com/10k74rSd (which is a summary of Bitcoin client hosts in #bitcoin on LFNet as of about 15 minutes ago) lines 432-529 are all GAE.

How did u generate that list?
https://bitcointalksearch.org/topic/m.13665
legendary
Activity: 1304
Merit: 1014
July 13, 2010, 06:29:14 PM
#4
Is it possible to rewrite portions of bitcoin so it runs entirely in GAE?  Particularly the feature that handles transactions, so it is fully scalable?

I am thinking one can get the sourcecode from SourceForge and write the C code into GAE language (Python or Java).
legendary
Activity: 1652
Merit: 1186
Chief Scientist
June 12, 2010, 07:51:33 AM
#3
So basically, you're running Bitcoin on a VPS and using an HTTPS connection from App Engine to signal it what to do? Why not just host the site on your server?
App Engine gives me:
 Free bandwidth and data storage (with the option to pay for what I use if I go over a gigabyte)
 Built-in, already-configured-and-tweaked tools like memcache and BigTable
 An architecture where scaling up is built-in (App Engine will automagically replicate my app and distribute it across Google's worldwide content distribution network if it gets a lot of traffic)

And all of that is administered by Google's sysadmins (who are WAY better at that than I am).

But mostly I did it as an experiment and because it was a technical challenge and because lately I'm an App Engine fanboy.  I would've been done a day earlier if I'd run everything on the same server and wrote the front end in PHP+MySQL.
full member
Activity: 210
Merit: 100
June 12, 2010, 01:46:40 AM
#2
So basically, you're running Bitcoin on a VPS and using an HTTPS connection from App Engine to signal it what to do? Why not just host the site on your server?
legendary
Activity: 1652
Merit: 1186
Chief Scientist
June 12, 2010, 12:09:43 AM
#1
Joozero asked if I'd make the source to freebitcoins.appspot.com available, and I promised to start a thread explaining how I created it-- and here it is.

freebitcoins is running on the Google App Engine.  Well, the part you see is running on App Engine.

There's also a back-end bitcoind server running on a debian VPS that I've had for years.  The App Engine code makes JSON-RPC calls via the App Engine url fetching API.  Making that connection secure was a bit tricky; here's how I did it:

First, I connect to the VPS using https so the traffic is encrypted.

I need a secure connection because I add a secret pre-shared value to the JSON call.  I'm not going to make the full source for freebitcoins fully open source because I don't want to bother with constantly working to keeping that secret value secret.    I should hash the entire request with a secret key and an ever-increasing nonce, but I was lazy and I'm just not very worried about a man-in-the-middle attack between Google and my VPS.

I could have hacked my copy of the bitcoind C++ code to check for the secret value in requests and also modified it so it accepted connections from the Internet... but instead I wrote this little proxy server in Python that runs on the same box as bitcoind:
Code:
#
# WSGI Proxy into the bitcoind daemon (which only listens on 127.0.0.1 for security).
# Rejects non-HTTPs connections, or non-POST requests
# Rejects queries that don't include a pre-shared secret value
#
# Apache calls this because of a directive in /etc/apache2/sites-available/xyz.com
#

import hashlib
import urllib2

def application(environ, start_response):
  serverURL = "http://127.0.0.1:8332/"

  def error_response(message):
    response_headers = [('Content-type', 'text/plain'),
            ('Content-Length', str(len(message)))]
    start_response('500 Internal Server error', response_headers)
    return message

  if environ.get("HTTPS") != "1":
    return error_response("Insecure connections not allowed.")

  request = environ.get("wsgi.input").read()
  secret = request[0:32]
  json_request = request[32:]

  if hashlib.md5(" pre shared secret goes here "+json_request).hexdigest() != secret:
    return error_response("Authentication failed.")

  req = urllib2.Request(serverURL, json_request)
  response = urllib2.urlopen(req)
  json_response = response.read()

  status = '200 OK'

  response_headers = [('Content-type', 'text/plain'),
            ('Content-Length', str(len(json_response)))]
  start_response(status, response_headers)

  return [json_response]
The other end of the connection is also django+Python code running on App Engine:
Code:
import hashlib
import jsonrpc
from google.appengine.api import urlfetch

def make_bitcoin_request(server_url, server_secret, method, params):
  json_request = {
    "jsonrpc": "2.0",
    "id": str(time.time()),
    "method": method, "params": params
    }
 
  json_string = jsonrpc.dumps(json_request)
  secret = hashlib.md5(server_secret+json_string).hexdigest()
 
  try:
    fetch_result = urlfetch.fetch(payment_server,
                                  method="POST", payload=secret+json_string,
                                  headers={ 'Content-Type' : 'text/plain' })
  except urlfetch.Error, e:
    logging.error('make_bitcoin_request failed: '+str(e))
    logging.error('Request:'+json_string)
    return None
 
  if fetch_result.status_code != 200:
    logging.error('make_bitcoin_request failed; urlfetch status code %d'%(fetch_result.status_code))
    logging.error('Request:'+json_string)
    return None

  r = jsonrpc.loads(fetch_result.content)
  if r['error'] is not None:
    logging.error('make_bitcoin_request failed; JSON error returned')
    logging.error('Request:'+json_string)
    logging.error('Result:'+fetch_result.content)
    return None

  return r['result']

I'm happy with how it is working.  My biggest worry is that the bitcoind process might unexpectedly exit, or fill up the disk with debug.log messages, or just generally be flaky.  But so far so good...
Jump to: