Author

Topic: bitHopper: Python Pool Hopper Proxy - page 204. (Read 355689 times)

donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 08, 2011, 09:57:13 PM
#28
Quote
Well I almost have Long polling support enabled on the server side so it'll switch everytime a block is found.

looking forward to LP support, but how will you know where to switch from LP? Won't it just tell you that a block was found somewhere in the network? I suppose you could use that info to determine when bithopper checks to see which pool has the least shares to see if one of them was the one that found the block?  Or maybe I'm just misunderstanding LP.

Again, thanks for such a simple and easy to use/modify proxy.






full member
Activity: 196
Merit: 100
July 08, 2011, 09:10:50 PM
#27
Ah well, expected behaviour then! You could make it so that it only switches when the pool with lesser shares has say less than 50% of the pool you're on to avoid the jumping back and forth.

The earlier you come in on a round, the greater the value of a share on a prop pool. So you should really be hopping to whichever pool has started on a new block.

Well I almost have Long polling support enabled on the server side so it'll switch everytime a block is found.

Your idea of switching if its under 50% of the current pool sounds like a smart idea as well. I'll add that in.

Thanks
-C00w
donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 08, 2011, 08:58:12 PM
#26
Quote
2) Why doesn't it always hop to the lowest pool.
I currently have it set to only change pools if the shares of the current pool is over 40% of difficulty. However it would make sense to switch anytime a new block comes out. I'll add that once I get server side LP working. Its a pretty simple switch if I have server side LP.

The reason it only switches pools at 40% is I didn't want two pools with close amounts but desynced json api update rates to cause the proxy to hop back and forth between them wasting time and shares.

Ah well, expected behaviour then! You could make it so that it only switches when the pool with lesser shares has say less than 50% of the pool you're on to avoid the jumping back and forth.

The earlier you come in on a round, the greater the value of a share on a prop pool. So you should really be hopping to whichever pool has started on a new block.

I currently have bit hopper running with 6 pools and it never has to go to eligius, so I'd say bithopper's working well so far. No set up issues like I had (still having) with multipool.
full member
Activity: 196
Merit: 100
July 08, 2011, 08:13:14 PM
#25
Hi,

1) Does it only bind to localhost?
It uses the default twisted http server and should bind to all interfaces. So you can run it on one computer and have multiple workers, on other computers, pointed to it. This assumes your firewall doesn't get in the way.

2) Why doesn't it always hop to the lowest pool.
I currently have it set to only change pools if the shares of the current pool is over 40% of difficulty. However it would make sense to switch anytime a new block comes out. I'll add that once I get server side LP working. Its a pretty simple switch if I have server side LP.

The reason it only switches pools at 40% is I didn't want two pools with close amounts but desynced json api update rates to cause the proxy to hop back and forth between them wasting time and shares.

-c00w



donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 08, 2011, 07:55:42 PM
#24
I'm running this on one machine, 3 clients pointed at it. Seems to run well, only sometimes it will go for the one from the list of pools with the second lowest shares, or it wont change over. Eg last night it chose btcguild over eclipse (btc guild on over 300000, eclipse at 20000), but after switching off and restarting it changed to eclipse. This morning it stayed with eclipse at 280000 when it should have moved to mineco.in at 240000.

Otherwise stable enough for last 12 hours - no crashes.
hero member
Activity: 742
Merit: 500
July 08, 2011, 07:50:07 PM
#23
Does it only bind to localhost or is there anything stopping me from running this on one system and pointing all of my rigs at it?

(read: I don't want to deal with the dependencies eight times)
full member
Activity: 196
Merit: 100
July 08, 2011, 03:07:12 PM
#22
Hi,

I wrote this program so that each client could run their own pool hopper proxy. If you want a hopping pool multipool and multiclone already fill that gap.

However I am going to implement basic statistics and it should be pretty simple to add in the ability to record the user name and password people logged in with and store that with their shares.

So this should work pretty well as a pool. You are going to need to write your own custom payment functions however.

The license will be changed to something more pool friendly when this is in a decent state.

-c00w
member
Activity: 78
Merit: 10
July 08, 2011, 06:59:02 AM
#21
It might be interesting to reduce this to just the website scraping part of it. Then run in combination with Chris Howie's AMP based Proxy and just let it change the priorities in the MySQL database instead of doing its own proxying, simply reordering them in the order of number of shares (just set priority in the DB to negative number of shares or sth like that). That should relieve you of re-inventing complicated (and proven to work) proxying code.

I *might* make an attempt at that later today or tomorrow. Note I said *might*. No promises whatsoever.

As for license I strongly suggest the AGPL (GNU Affero General Public License). It has an additional rule that if you host it on a server that others can access, you also have to give them any modifications to the source that you made, even if they themselves don't run it. Makes lots of sense for server apps (or potential server apps, as this thing is).
donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 08, 2011, 05:11:21 AM
#20
thx burp Smiley
member
Activity: 98
Merit: 10
July 08, 2011, 03:51:44 AM
#19
I added support for mineco and bitclockers. I like how simple your program is, please don't overcomplicate things by unnecessary generalizations or such a regex-config. Writing a small sharesResponse function is much more flexible. One problem I can see is that shares get submitted to the wrong pool if the best pool changes while the miner has still data for the old pool. These few shares should be negligable though. If you add long-polling support you could make such a "new block" event when the best pool changes so that the miner flushes the work queue.

Code:
+        'mineco':{'time': time.time(), 'shares': 0, 'name': 'mineco.in',
+            'mine_address': 'mineco.in:3000', 'user': mineco_user,
+            'pass': mineco_pass, 'lag': False, 'LP': None},
+        'bitclockers':{'time': time.time(), 'shares': 0, 'name': 'bitclockers.com',
+            'mine_address': 'pool.bitclockers.com:8332', 'user': bitclockers_user,
+            'pass': bitclockers_pass, 'lag': False, 'LP': None}
         }
 current_server = None

@@ -86,16 +96,40 @@ def mtred_sharesResponse(response):
     print 'mtred :' + str(round_shares)
     server_update()

+def mineco_sharesResponse(response):
+    global servers
+    info = json.loads(response)
+    round_shares = int(info['shares_this_round'])
+    servers['mineco']['shares'] = round_shares
+    print 'mineco :' + str(round_shares)
+    server_update()
+
+def bitclockers_sharesResponse(response):
+    global servers
+    info = json.loads(response)
+    round_shares = int(info['roundshares'])
+    servers['bitclockers']['shares'] = round_shares
+    print 'bitclockers :' + str(round_shares)
+    server_update()
+
 def bclc_getshares():
     getPage('https://www.bitcoins.lc/stats.json').addCallback(bclc_sharesResponse)

 def mtred_getshares():
     getPage('https://mtred.com/api/user/key/d91c52cfe1609f161f28a1268a2915b8').addCallback( mtred_sharesResponse )

+def mineco_getshares():
+    getPage('https://mineco.in/stats.json').addCallback(mineco_sharesResponse)
+
+def bitclockers_getshares():
+    getPage('https://bitclockers.com/api').addCallback(bitclockers_sharesResponse)
+
 def update_servers():
     global servers
     bclc_getshares()
     mtred_getshares()
+    mineco_getshares()
+    bitclockers_getshares()

full member
Activity: 196
Merit: 100
July 08, 2011, 01:06:01 AM
#18
Well I could just have it dynamically import python modules. Should be pretty easy.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 08, 2011, 12:38:47 AM
#17
Thanks for the response! I understand this is still a work in progress, I wasn't hassling or anything.

re point 2, if you have the time and inclination to rustle up something at some point I think it would be a big point in the proxy's favour. People like to see how much ahead they are! But I just like chart porn. But you're right of course, LP is just a tad more important

I'd figured that the diff*.41 was something to do with when the proxy decides on which pool to look at next - delaying until all pools are more than 41% of expected shares for a block?

As for point 4 I followed along right up until my heart sunk at the evil word "regex". Sigh. Now I really will have to relearn regexpr. Stupid random symbols and letters  Angry I might just wait until you've had time to add it in.

Cheers!

(edited)
full member
Activity: 196
Merit: 100
July 07, 2011, 11:16:59 PM
#16

I think the *.pyc files are not really needed in the repository, hm?

You mean in jsonrpc? Yeah. I didn't notice them. I'll delete them once I get my code rewritten and LP support added.
full member
Activity: 196
Merit: 100
July 07, 2011, 11:13:08 PM
#15
Answers

1. I can't see how you got difficulty - is it entered manually or is it a wget from http://blockexplorer.com/q/getdifficulty ?

Right now its entered manually. Thats a good idea though

2. I can see you're getting user api data - are you providing an efficiency output? This is something I loved about Multipool - It's much harder to see that you're getting good efficiency otherwise. Easy enough: shares/difficulty*blockvalue.

Hmm. I was scraping the user data in order to get server data. I could probably whip up a crude API quickly. My priority right now is wrestling LP support into place.

3. Why 'shares':difficulty*.41 (line 33) for eligius?

Eligius has switched to a shared max pps. This makes it a bad pool for pool hopping but a good pool to mine when we can't find anything better. The difficulty*.41 is there so that it will check the other pools instead of assuming eligius is the best pool. Its a hack that will be removed when I clean up my code.

4. Can you give simple-ish instructions on adding other pools ourselves?

Sure. I'll add it to the readme at some point but its pretty simple: Copy the structure in the servers dictionary for your new pool.
Create a function to call a getPage from your info source.
Set its deferred to a function which parses your info source and sets server['server_name']['shares'] to the current shares.

Add the getpage calling function function to update_servers.

After writing that I realize its not that simple. I'm planning on spinning out some parsing functions with regex into some sort of file structure.

If you have a pool in mind and have the function written for extracting shares or api details, just post it and i can integrate it in if it is annoying.

Issues with the license:
I'll probably relicense it under a GPLV2 at some point. Its mainly so no one steals the script and resells it or trojans it without me being able to have a solid legal complaint.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 07, 2011, 10:57:25 PM
#14
I had a very quick look through and have a couple of questions:

1. I can't see how you got difficulty - is it entered manually or is it a wget from http://blockexplorer.com/q/getdifficulty ?
2. I can see you're getting user api data - are you providing an efficiency output? This is something I loved about Multipool - It's much harder to see that you're getting good efficiency otherwise. Easy enough: shares/difficulty*blockvalue.
3. Why 'shares':difficulty*.41 (line 33) for eligius?
4. Can you give simple-ish instructions on adding other pools ourselves?

Looks great - simple enough for me to (mostly) understand! Can't wait to give it a try tonight.

donator
Activity: 2058
Merit: 1007
Poor impulse control.
July 07, 2011, 10:38:15 PM
#13
i do see a problem
Quote
Creative Commons Attribution-NonCommercial-ShareAlike 3.0
  Sad

c00w prolly just doesn't want someone else to take some of the code and sell it as another proxy.
legendary
Activity: 2618
Merit: 1007
July 07, 2011, 10:30:03 PM
#12
i do see a problem
Quote
Creative Commons Attribution-NonCommercial-ShareAlike 3.0
 Sad
Where exactly is the problem there? Wanna use it commercially? (Btw. isn't generating "money" commercial per se?!)

Thanks a lot for the program btw.!

Edit:
I think the *.pyc files are not really needed in the repository, hm?
full member
Activity: 196
Merit: 100
July 07, 2011, 09:19:49 PM
#11
Its in both the readme and the initial instructions in the post. I modified it to make it more obvious though.
hero member
Activity: 504
Merit: 502
July 07, 2011, 09:01:49 PM
#10
You should mention that users should enter their own details in the py code, since its all filled with your own pool account details.

Just in case users dont know it, they will just end up mining for you.
full member
Activity: 168
Merit: 100
July 07, 2011, 06:38:59 PM
#9
Sweet. I found another error with submitting good work but I think I fixed it. I'm running my card with it but its only 180 MH/S so it might not catch all the errors.

subscribing.
Jump to: