bitHopper: Python Pool Hopper Proxy - page 202.

burp

member

Activity: 98

Merit: 10

Another fix:

Code:

@@ -241,7 +241,7 @@ def update_servers():
             info = servers[server]
             d = getPage(info['api_address'])
             d.addCallback(selectsharesResponse, (server))
-            d.addErrback(errsharesResponse, ('btcg'))
+            d.addErrback(errsharesResponse, (server))
             d.addErrback(log_msg)

And the recommended new btcguild server address is just us.btcguild.com.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Gotcha - you mean at from 0.435 * to proportional will pay better than eligius but after shares proportional should pay less than eligius? After re-reading Raulo's paper I realise Raulo was only considering one proportional and one 'connected' only pool, and after reading this: http://forum.bitcoin.org/index.php?topic=24966.0 I'd have to agree with you.

This is not based on math though, just what seems logical. Could be wrong.

Changing at would have the side benefit of disguising you from looking like the typical hopper who uses the 0.435* rule.

Edit: I suppose you could make the change and then run it for a week or so and see if there's a difference. Need an easy way to get stats though.

burp

member

Activity: 98

Merit: 10

Quote from: organofcorti on July 10, 2011, 06:29:28 AM

Quote

Yes, over 0.4*difficulty. But I mean this should probably be difficulty.

I'd say this relates to a paper written by a forum member, Raoulo, described and linked to here: http://forum.bitcoin.org/index.php?topic=3165.0

That is not what I mean. I know that it's optimal to switch after 0.435*difficulty. But only if you have a better pool then. Switching to eligius when you are over 0.4*difficulty but below difficulty is imo worse than staying at the proportional pools until difficulty is reached.

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote

Yes, over 0.4*difficulty. But I mean this should probably be difficulty.

I'd say this relates to a paper written by a forum member, Raoulo, described and linked to here: http://forum.bitcoin.org/index.php?topic=3165.0

burp

member

Activity: 98

Merit: 10

Quote from: organofcorti on July 10, 2011, 05:58:51 AM

Quote from: burp on July 10, 2011, 05:48:15 AM

What about using the backup pool (eligius) first when every other pool has reached number of shares? From that point on the maxpps system from eligius will be more beneficial than mining on the other pools with shares>difficulty.

doesn't it already? I thought I saw mention of it in the code, and bithopper does default to eligius if all pools have shares over six hundred and something thousand. Or I could be wrong.

Yes, over 0.4*difficulty. But I mean this should probably be difficulty.

An other issue is that you added api_addresses into the servers configuration now and use getPage for all of these now. I think this removes a lot of flexibility. Think for instance of a pool that publishes its statistics only in logged in state where some additional handling is required. I just wanted to add triplemining.com to my bitHopper and saw that now it's basically not possible anymore without reverting some stuff back to gain more flexibility.
For triplemining.com I have to login and parse a html page to get the statistics.
This is what I meant earlier with generalizations that complicate things Wink

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

Quote from: burp on July 10, 2011, 05:48:15 AM

What about using the backup pool (eligius) first when every other pool has reached number of shares? From that point on the maxpps system from eligius will be more beneficial than mining on the other pools with shares>difficulty.

doesn't it already? I thought I saw mention of it in the code, and bithopper does default to eligius if all pools have shares over six hundred and something thousand. Or I could be wrong.

burp

member

Activity: 98

Merit: 10

What about using the backup pool (eligius) first when every other pool has reached number of shares? From that point on the maxpps system from eligius will be more beneficial than mining on the other pools with shares>difficulty.

c00w

full member

Activity: 196

Merit: 100

Good catch. I'm fixing it.

toasty

member

Activity: 90

Merit: 12

Code:

def selectsharesResponse(response, args):
    log_msg('Calling sharesResponse for '+ args)
    func_map= {'bitclockers':bitclockers_sharesResponse,
        'mineco':mineco_sharesResponse,
        'mtred':mtred_sharesResponse,
        'bclc':bclc_sharesResponse,
        'bitclockers':bitclockers_sharesResponse,
        'btcg':btcguild_sharesResponse}

Bitclockers is defined twice here. Not hurting anything, but might cause a bug down the road if only one of them gets changed.

OCedHrt

member

Activity: 111

Merit: 10

Quote from: c00w on July 09, 2011, 06:44:49 PM

Its an LP issue. Diablominer doesn't actually read from the LP channel. Once it gets a packet it triggers and asks for more work from the normal channel. There is a way to rewrite the clientside LP and I'll add it to the list but for now just run with pool.py --noLP.

That's fine. I'm actually using poclbm with phatk because DiabloMiner is returning high rejects (but maybe this is also due to LP).

Code:

mhash: 279.6/280.2 | accept: 32 | reject: 5 | hw error: 0

I get 1-2% reject on poclbm.

c00w

full member

Activity: 196

Merit: 100

Its an LP issue. Diablominer doesn't actually read from the LP channel. Once it gets a packet it triggers and asks for more work from the normal channel. There is a way to rewrite the clientside LP and I'll add it to the list but for now just run with pool.py --noLP.

OCedHrt

member

Activity: 111

Merit: 10

Quote from: c00w on July 09, 2011, 06:20:23 PM

Quote from: OCedHrt on July 09, 2011, 05:47:54 PM

I actually encountered a new problem with DiabloMiner where it loops RPC Request in BitHopper continuously. This only happens IF DiabloMiner doesn't error out as described earlier.

Hmm. I ran diablo miner and didn't get the LP problem. I did see an abnormally high number of RPC requests. I am seeing an issue where my code is submitting 2-4 times as many RPC request as it should. I'm debugging.

EDIT: Turns out my logging function double printed certain things. Fixed now. It looks like Diablominer just pulls a lot of work. I'll keep playing with it and see if I can find the LP issue though.

If Diablo Miner errors on start, it's ok:

Code:

[7/9/11 4:29:32 PM] Started
[7/9/11 4:29:32 PM] Connecting to: http://localhost:8337/
[7/9/11 4:29:32 PM] Using AMD Accelerated Parallel Processing OpenCL 1.1 AMD-APP
-SDK-v2.5 (684.211)
[7/9/11 4:29:34 PM] BFI_INT patching enabled, disabling hardware checking
[7/9/11 4:29:34 PM] Added Barts (#1) (14 CU, local work size of 256)
Waiting...Exception in thread "DiabloMiner LongPollAsync" java.lang.NullPointerE
Waiting...
        at com.diablominer.DiabloMiner.DiabloMiner.forceUpdate(DiabloMiner.java:
504)
        at com.diablominer.DiabloMiner.DiabloMiner$LongPollAsync.run(DiabloMiner
.java:1363)
        at java.lang.Thread.run(Unknown Source)
[7/9/11 4:29:37 PM] Accepted block 1 found on Barts (#1)

If it doesn't error, then we have problem. Log @ paste bin below. There may be some logging differences since I've made some small changes, but it still has same problem with the pool.py I downloaded.

Code:

[7/9/11 4:29:47 PM] Started
[7/9/11 4:29:47 PM] Connecting to: http://localhost:8337/
[7/9/11 4:29:47 PM] Using AMD Accelerated Parallel Processing OpenCL 1.1 AMD-APP
-SDK-v2.5 (684.211)
[7/9/11 4:29:49 PM] BFI_INT patching enabled, disabling hardware checking
[7/9/11 4:29:49 PM] Added Barts (#1) (14 CU, local work size of 256)
[7/9/11 4:30:35 PM] Accepted block 1 found on Barts (#1)

http://pastebin.com/e4hUXD6R

c00w

full member

Activity: 196

Merit: 100

Quote from: OCedHrt on July 09, 2011, 05:47:54 PM

I actually encountered a new problem with DiabloMiner where it loops RPC Request in BitHopper continuously. This only happens IF DiabloMiner doesn't error out as described earlier.

Hmm. I ran diablo miner and didn't get the LP problem. I did see an abnormally high number of RPC requests. I am seeing an issue where my code is submitting 2-4 times as many RPC request as it should. I'm debugging.

EDIT: Turns out my logging function double printed certain things. Fixed now. It looks like Diablominer just pulls a lot of work. I'll keep playing with it and see if I can find the LP issue though.

EDIT AGAIN: Actually I remember causing that error in one of my commits. I fixed it once I tested it on my server and it didn't work roughly 2 minutes later. Git Pull and see if it works?

OCedHrt

member

Activity: 111

Merit: 10

Quote from: c00w on July 09, 2011, 04:53:17 PM

The issue is that when I use a twisted deffered mechanism to handle LP requests, which is the only way I can see to do it, twisted sends back a packet with no http data to say," We're alive but we are not done processing no http response"

All the python based miners are cool with this.

Diablominer takes this packet and tries to operate on it and breaks.

Cgminer accepts this packet, goes LP WAS FIRED and grabs a new block via getwork. Which isn't how LP is supposed to work either. They should try and read a getwork from the response like Diablominer and crash or they should check for a null packet and do something like oh not cycle endlessly through getworks.

I actually encountered a new problem with DiabloMiner where it loops RPC Request in BitHopper continuously. This only happens IF DiabloMiner doesn't error out as described earlier.

c00w

full member

Activity: 196

Merit: 100

Fixed.

burp

member

Activity: 98

Merit: 10

Btw, this is missing to make btcguild actually work:

Code:

 def get_difficulty():
@@ -171,6 +171,7 @@ def btcguild_sharesResponse(response):
     round_shares = int(info['round_shares'])
     servers['btcg']['shares'] = round_shares
     log.msg( 'btcguild :' + str(round_shares))
+    server_update()

 def bclc_sharesResponse(response):
     global servers
@@ -213,7 +214,7 @@ def errsharesResponse(error, args):

 def btcg_getshares():
     d = getPage('https://www.btcguild.com/pool_stats.php')
-    d.addCallback(bclc_sharesResponse)
+    d.addCallback(btcguild_sharesResponse)
     d.addErrback(errsharesResponse, ('btcg'))
     d.addErrback(log.err)

@@ -247,6 +248,7 @@ def update_servers():
     mtred_getshares()
     bitclockers_getshares()
     mineco_getshares()
+    btcg_getshares()

c00w

full member

Activity: 196

Merit: 100

The issue is that when I use a twisted deffered mechanism to handle LP requests, which is the only way I can see to do it, twisted sends back a packet with no http data to say," We're alive but we are not done processing no http response"

All the python based miners are cool with this.

Diablominer takes this packet and tries to operate on it and breaks.

Cgminer accepts this packet, goes LP WAS FIRED and grabs a new block via getwork. Which isn't how LP is supposed to work either. They should try and read a getwork from the response like Diablominer and crash or they should check for a null packet and do something like oh not cycle endlessly through getworks.

burp

member

Activity: 98

Merit: 10

Quote from: c00w on July 09, 2011, 04:31:24 PM

Quote from: burp on July 09, 2011, 04:12:25 PM

Here is how to make twisted less noisy:

http://stackoverflow.com/questions/5078980/twisted-starting-stopping-factory-protocol-less-noisy-log-messages

EDIT: I just noticed you fixed a couple of things, nice

Well the issue is that the noisy factories are embedded in getPage. I could rewrite it but its a lot simpler to not bother.

I'm going to add a option parser so I can have a flag to turn off client side LP for cgminer and diablominer users as well as a debug flag for me which turns on the twisted output.

You can just turn off LP support with --no-longpoll for cgminer, so it's no problem now. (A:4339 R:77, and accepted/rejected ratio is still good) But fixing the issue doesn't harm either Wink

c00w

full member

Activity: 196

Merit: 100

Quote from: burp on July 09, 2011, 04:12:25 PM

Here is how to make twisted less noisy:

http://stackoverflow.com/questions/5078980/twisted-starting-stopping-factory-protocol-less-noisy-log-messages

EDIT: I just noticed you fixed a couple of things, nice

Well the issue is that the noisy factories are embedded in getPage. I could rewrite it but its a lot simpler to not bother.

I'm going to add a option parser so I can have a flag to turn off client side LP for cgminer and diablominer users as well as a debug flag for me which turns on the twisted output.

c00w

full member

Activity: 196

Merit: 100

Quote from: ahitman on July 09, 2011, 09:36:20 AM

Here is some logic that popped in my head while looking through the code (I've never programmed in python but I have done some programming at school). Please let me know if this makes sense:

1. Check the api of each pool for share_count
2. If share_count[of the pool you are checking] < 1 then share_count[of the pool you are checking] = 10**10
3. Pick the pool with the lowest share count

Step 2 will catch any misbehaving api or a pool that is down and is returning a 0 for the share count.

Let me know if the code is doing this or something like this already, I have a hard time following some of the if statements, but you've inspired me to go and learn some python syntax!

I implemented something similar to this. Now if the API call fails it changes that server's difficulty to something along the line of 10^10. Your code is a good idea but wouldn't work because when the calls fail the server returns nothing and the error functions are called. Not a json api repsonse which can be parsed into a share count of 0.

Topic: bitHopper: Python Pool Hopper Proxy - page 202. (Read 355904 times)