bitHopper: Python Pool Hopper Proxy - page 201.

OCedHrt

member

Activity: 111

Merit: 10

mtred share information is actually incorrect. Numbers from both servers need to be combined.

Code:

def mtred_sharesResponse(response):
    global servers
    info = json.loads(response)
-   round_shares = int(info['server']['servers']['n0']['roundshares'])
+   round_shares = int(info['server']['roundshares'])
    servers['mtred']['shares'] = round_shares
    log_msg('mtred :' + str(round_shares))

Also just noticed mine.co is now PPLNS. Maybe should stop hopping to it or adjust its value.

enmaku

hero member

Activity: 742

Merit: 500

Here's my actual error, just crashed again:

Code:

---  ---
  File "D:\Python27\lib\site-packages\twisted\web\server.py", line 127, in proce
ss
    self.render(resrc)
  File "D:\Python27\lib\site-packages\twisted\web\server.py", line 147, in rende
r
    body = resrc.render(self)
  File "D:\Python27\lib\site-packages\twisted\web\resource.py", line 216, in ren
der
    return m(request)
  File "D:\bithopper\pool.py", line 342, in render_POST
    return bitHopper_Post(request)
  File "D:\bithopper\pool.py", line 281, in bitHopper_Post
    pool_server=servers[current_server]
exceptions.KeyError: None
Unhandled Error
Traceback (most recent call last):
  File "D:\Python27\lib\site-packages\twisted\web\http.py", line 1349, in dataRe
ceived
    finishCallback(data[contentLength:])
  File "D:\Python27\lib\site-packages\twisted\web\http.py", line 1563, in _finis
hRequestBody
    self.allContentReceived()
  File "D:\Python27\lib\site-packages\twisted\web\http.py", line 1619, in allCon
tentReceived
    req.requestReceived(command, path, version)
  File "D:\Python27\lib\site-packages\twisted\web\http.py", line 773, in request
Received

Edit: It's now crashing like this ~once per hour... Help!

koopa

member

Activity: 61

Merit: 10

Confirmed as working on Windows XP Grin

Was looking for a proxy + hopper and stumbled across bithopper... installed this morning and it seems to be chugging along just fine. Thanks to c00w for taking the time to put this up and continue to develop it. Cheers.

As a python virgin here's what I did to get this going on XP....

Files required for a 32bit Windows XP install:

Bithopper - Obviously.
https://github.com/c00w/bitHopper

Python 2.7.2 - Version 2.7.2 is the latest version compatible with Twisted on Windows XP
http://www.python.org/ftp/python/2.7.2/python-2.7.2.msi
Other Python versions available here:
http://www.python.org/getit/

Twisted 11.0.0 - XP compatible:
http://twistedmatrix.com/Releases/Twisted/11.0/Twisted-11.0.0.winxp32-py2.7.msi
Other Twisted versions available here:
http://twistedmatrix.com/trac/wiki/Downloads

pyOpenSSL 0.11.winxp32-py2.7.msi - XP compatible:
http://launchpad.net/pyopenssl/main/0.11/+download/pyOpenSSL-0.11.winxp32-py2.7.msi
Other pyOpenSSL versions available here
https://launchpad.net/pyopenssl

Zope.interface - XP compatible:
http://pypi.python.org/packages/2.7/z/zope.interface/zope.interface-3.6.4-py2.7-win32.egg#md5=887bffa4e7739c03f2dec72145a67b69
Other Zope.interface versions available here:
http://pypi.python.org/pypi/zope.interface#downloads

OK, so to get this up and running, install python, twisted and pyOpenSSL. Unzip the zope.interface file (it's a .egg file) and place the contents in C:\Python26\Lib\site-packages\

You should be able to run it following c00w's instructions from there.

*Edit 12-06-11 - Added link to pyOpenSSL and expanded.

anty

newbie

Activity: 40

Merit: 0

It just crashed for me too.
None the less, thanks for developing this proxy! I primarily use it to automatically switch to another pool when one goes down. The pool hopping is a nice bonus Wink

I want to +1 the request of putting the configuration into a separate file. I'd love to just do a "git update" without needing to merge my config into pool.py.

Edit: Here's another crash:

Code:

nishRequestBody
    self.allContentReceived()
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1641, in allContentReceived
    req.requestReceived(command, path, version)
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 807, in requestReceived
    self.process()
---  ---
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 125, in process
    self.render(resrc)
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 132, in render
    body = resrc.render(self)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 210, in render
    return m(request)
  File "/home/anty/bitHopper/pool.py", line 342, in render_POST
    return bitHopper_Post(request)
  File "/home/anty/bitHopper/pool.py", line 281, in bitHopper_Post
    pool_server=servers[current_server]
exceptions.KeyError: None

enmaku

hero member

Activity: 742

Merit: 500

Quote from: GoMaD on July 11, 2011, 09:30:42 AM

crashed during my breakfast!

last i saw was something like:

Code:

Traceback (most recent call last):
  File "pool.py", line 204, in server_update
    select_best_server()
  File "pool.py", line 179, in select_best_server
    new_server.callback(None)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
    self._runCallbacks()
---  ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "pool.py", line 337, in bitHopperLP
    json_request = request.content.read()
exceptions.ValueError: I/O operation on closed file

linenumbers didn't match cause i have some changes and comments

Got it to crash again, came back here to post about it and saw the exact error already there. Same error for myself as well

GoMaD

member

Activity: 74

Merit: 15

crashed during my breakfast!

last i saw was something like:

Code:

Traceback (most recent call last):
  File "pool.py", line 204, in server_update
    select_best_server()
  File "pool.py", line 179, in select_best_server
    new_server.callback(None)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
    self._runCallbacks()
---  ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "pool.py", line 337, in bitHopperLP
    json_request = request.content.read()
exceptions.ValueError: I/O operation on closed file

linenumbers didn't match cause i have some changes and comments

enmaku

hero member

Activity: 742

Merit: 500

Crashed during the night. I was running it directly so there was no helpful error message waiting for me, the window simply closed itself while I slept. I ran it from within a command prompt this time so that if it crashes again I'll have an error to post. Hopefully it's just a fluke Tongue

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: jkminkov on July 11, 2011, 05:13:42 AM

why default pool is not deepbit PPS but eligius Huh

Deepbit PPS costs 10%. Eligius SMPPS has a negligible fee.

jkminkov

hero member

Activity: 698

Merit: 500

why default pool is not deepbit PPS but eligius Huh

GoMaD

member

Activity: 74

Merit: 15

slush support (change USER and PASS):

Code:

mineco_pass = 'x
+slush_user = 'USER'
+slush_pass = 'PASS'   

            'api_address':'https://eclipsemc.com/api.php?key=YOURAPIKEY&action=poolstats'}
+,'slush':{'shares': 0, 'name': 'mining.bitcoin.cz',
+            'mine_address': 'api2.bitcoin.cz:8332', 'user': slush_user,
+            'pass': slush_pass, 'lag': False, 'LP': None,
+            'api_address':'https://mining.bitcoin.cz/stats/json/'}       
     
+def slush_sharesResponse(response):
+    global servers
+    info = json.loads(response)
+    round_shares = int(info['shares'])
+    servers['slush']['shares'] = round_shares
+    log_msg( 'mining.bitcoin.cz (slush)) :' + str(round_shares))
            
       'eclipsemc':eclipsemc_sharesResponse,
+     'slush':slush_sharesResponse)

GoMaD

member

Activity: 74

Merit: 15

Quote from: neptop on July 11, 2011, 03:43:10 AM

I think it would be a good idea to include the configuration from another file. Maybe just create a bithopper.conf.example, so you don't overwrite an existing one when checking out. Also with more and more pools being added it would be good if it'd ignore non existent ones. Maybe they should also be removed from the main file so one can simply add additional pools by creating a files inside a pools/ directory. Oh and of course a --quiet and a --daemon switch would be nice additions.

that would be great!

mining.mainframe.nl support (change USER, PASS):

Code:

mineco_pass = 'x
+miningmainframe_user = 'USER'
+miningmainframe_pass = 'PASS'  

            'api_address':'https://eclipsemc.com/api.php?key=YOURAPIKEY&action=poolstats'}
+'miningmainframe':{'shares': 0, 'name': 'mining.mainframe.nl',
+           'mine_address': 'mining.mainframe.nl:8343', 'user': miningmainframe_user,
+           'pass': miningmainframe_pass, 'lag': False, 'LP': None,
+            'api_address':'http://mining.mainframe.nl/api'}
         }

    log_msg( 'eclipsemc :' + str(round_shares))

+def mmf_sharesResponse(response):
+    global servers
+    info = json.loads(response)
+    round_shares = int(info['shares_this_round'])
+    servers['miningmainframe']['shares'] = round_shares
+    log_msg( 'mining.mainframe.nl :' + str(round_shares))

       'eclipsemc':eclipsemc_sharesResponse,
+      'miningmainframe':mmf_sharesResponse}

ups didn´t saw they use some kind of cheat-proof

neptop

sr. member

Activity: 314

Merit: 251

I think it would be a good idea to include the configuration from another file. Maybe just create a bithopper.conf.example, so you don't overwrite an existing one when checking out. Also with more and more pools being added it would be good if it'd ignore non existent ones. Maybe they should also be removed from the main file so one can simply add additional pools by creating a files inside a pools/ directory. Oh and of course a --quiet and a --daemon switch would be nice additions.

GoMaD

member

Activity: 74

Merit: 15

eclipsemc.com support (change USER, PASS and YOURAPIKEY)

Code:


mineco_pass = 'x
+eclipsemc_user = 'USER'
+eclipsemc_pass = 'PASS'

            'api_address':'https://bitclockers.com/api'},
+       'eclipsemc':{'shares': 0, 'name': 'eclipsemc.com',
+            'mine_address': 'pacrim.eclipsemc.com:8337', 'user': eclipsemc_user,
+            'pass': eclipsemc_pass, 'lag': False, 'LP': None,
+            'api_address':'https://eclipsemc.com/api.php?key=YOURAPIKEY&action=poolstats'}
         }


     log_msg( 'bitclockers :' + str(round_shares))

+def eclipsemc_sharesResponse(response):
+    global servers
+    info = json.loads(response)
+    round_shares = int(info['round_shares'])
+    servers['eclipsemc']['shares'] = round_shares
+    log_msg( 'eclipsemc :' + str(round_shares))
+

        'btcg':btcguild_sharesResponse,
+       'eclipsemc':eclipsemc_sharesResponse}

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: ?? on ??

for deepbit support i got a little idea Grin

deepbit dont offer live stats but if we can see when all the other pools got a block we can guess if deepbit did or did not get a block(if not other pool get a new block then there is a big % to be deepbit)

That's a great idea - I never figured out how multipool did it. Downside is there are a shit tonne of pools to check these days.

c00w

full member

Activity: 196

Merit: 100

1) Stats?
Yeah. I'm trying to make it rock solid and get LP to work. If I get that the reject rate may actually be 0% which is cool. A lot of miners just drop shares after a connection failure or are just written oddly/ ignore standards. Then stats. Then scraper for all the pools that aren't deepbit. Then deepBit.

*rant*
Miners I have found issues with:
poclbm: Drops connection way to quickly.
phoenix: Drops shares on a network failure. *I haven't confirmed this but it looks like it*
Diablominer + cgminer: Uses get instead of post for longpolling. Completely ignoring the standard. And it was an obscure bug to find.
Edit: I bet they all have the same issue as phoenix. But I don't use them enough to notice.
*end rant*

2)Deepbit support?
Yeah that sounds like the way to do it. I also want to look at how the multipool guy did it and build in a share estimator given scraped hash information to know when to switch off.

enmaku

hero member

Activity: 742

Merit: 500

This thing is FANTASTIC. If this is beta I can't wait to see a gold release Grin

I also can't wait for some stats.

c00w

full member

Activity: 196

Merit: 100

1) Json decoding error with eligius looping?
It looks like eligius doesn't like us hammering it with getwork requests in the case of one failure. I think its some sort of DDOS proctection. I'll throw in another backup pool for it to swap between.

2) IO Operation on a closed file
It looks like someone connected with LP and then died before it could read. Not a huge issue. I'll wrap it in a try except.

3) Why it probably died?
Same issue as the JSON decoding error with eligius looping. I need to add another backup pool so that if one dies/decides we are DDOSing it we can still do stuff.
Edit: I couldn't find another pool with pool hopper protection that works except btcmine. And well... registration is down. I did redo the networking code to hopefully be a little nicer to eligius and convince it to not DDOS flake out on us as much.

ahitman

sr. member

Activity: 302

Merit: 250

Code:

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "pool.py", line 195, in server_update
    select_best_server()
  File "pool.py", line 171, in select_best_server
    new_server.callback(None)
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 361, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 455, in _startRunCallbacks
    self._runCallbacks()
---  ---
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "pool.py", line 336, in bitHopperLP
    json_request = request.content.read()
exceptions.ValueError: I/O operation on closed file

I got this error, is there something wrong with my setup? It doesnt come up all the time, and it may have happened as bithopper decided to switch pools.

Also when I got home today my miners were sitting idle, but they didnt show a disconnected message, just that they had no work, ,looking back at the log from bithopper, looks like it was still getting the api data from pools, but was also showing the message:

Code:

 Error in pool api for btcg

multiple times, unfortunatley the RPC request log line I commented out to not show the work units so Im not sure if it was trying to connect or not. Do you know what could have happened? I know its not alot of info, so is there a way if something sismilar happens again that I can investigate what bithopper is trying to do? Thanks

c00w

full member

Activity: 196

Merit: 100

1) Burf good catch with the error call back. I'll fix it soon.

2) Defaulting to eligius at over difficulty instead of .40*difficulty?
Hmmm. Let me think about it. We are using eligius as our 'solo' mining pool because the payouts are roughly consistent mathematically with what solo mining should be. And the sweet spot for switching from a single pool is .435%

3) I figured out why diablo and cgminer are broken!
They completely ignore how long polling is supposed to work and use a post instead of a get. I'm fixing it now.

enmaku

hero member

Activity: 742

Merit: 500

Pointed a couple of my miners at a test setup last night, and so far they're performing about 20% over calculated estimates from hashrate alone. I think I'll be rolling out a full-scale setup very soon

Topic: bitHopper: Python Pool Hopper Proxy - page 201. (Read 355904 times)