No problem. I'll run eureqa on Monday and make you a utility function for slush that will be accurate to at least 0.5*diff. That ok? It won't be derived from the calculus but it will work.
As long as it's relatively accurate up until the crossover point (when it drops below 1.0) it should be fine for my purposes. 
Topic: [EXPERIMENTAL] poclbm-autohop: Yet Another Automatic Pool Hopper - page 2. (Read 10145 times)
September 02, 2011, 05:52:20 AM
September 02, 2011, 05:47:12 AM
No problem. I'll run eureqa on Monday and make you a utility function for slush that will be accurate to at least 0.5*diff. That ok? It won't be derived from the calculus but it will work.
September 02, 2011, 05:44:27 AM
Though, in order to add slush to this program, at least, I'd have to figure out how to calculate the utility of a given share. It should be possible once I've done the relevant analysis, but I don't exactly have the motivation to do such a thing at the moment.
How do you define utility? If it is the expected value of a share if a given number of shares in a round have already occurred, I can send you a score table, or you can generate one yourself. Otherwise, let me know if I can help.
September 01, 2011, 09:14:36 PM
Though, in order to add slush to this program, at least, I'd have to figure out how to calculate the utility of a given share. It should be possible once I've done the relevant analysis, but I don't exactly have the motivation to do such a thing at the moment.
How do you define utility? If it is the expected value of a share if a given number of shares in a round have already occurred, I can send you a score table, or you can generate one yourself. Otherwise, let me know if I can help.
September 01, 2011, 09:00:01 PM
New "how to hop" blog post:
How to hop part2: More on score
This week we discover Slush-type score systems fatal flaw, an update on the Slush hop point (previous estimate nearly 50% out), and what 'c' really means
Though, in order to add slush to this program, at least, I'd have to figure out how to calculate the utility of a given share. It should be possible once I've done the relevant analysis, but I don't exactly have the motivation to do such a thing at the moment.
As a second point, I'm worried that the list of pools I'm operating with is languishing a bit. It seems NoFeeMining changed to RSMPPS at some point, so it's been changed to a fair pool. Have there been any other proportional pools started? I hope we're past that point, given how easy it is to hop them. I'm unsure about adding any additional fair pools, as I already support several. In any case, they'd have to be added to pident first.
September 01, 2011, 06:09:37 AM
New "how to hop" blog post:
How to hop part2: More on score
This week we discover Slush-type score systems fatal flaw, an update on the Slush hop point (previous estimate nearly 50% out), and what 'c' really means
August 30, 2011, 06:34:47 PM
This doesnt look right, btcguild had a new block 20mins ago and several others have had blocks too.
30/08/2011 21:31:18, bitpit : 1.000
30/08/2011 21:31:18, eligius : 1.000
30/08/2011 21:31:18, mtred : 0.310
30/08/2011 21:31:18, ozco.in : 0.298
30/08/2011 21:31:18, bitcoins.lc : 0.211
30/08/2011 21:31:18, triplemining : 0.191
30/08/2011 21:31:18, bitminter : 0.190
30/08/2011 21:31:18, btcguild : 0.079
30/08/2011 21:31:18, nofeemining : 0.054
30/08/2011 21:31:18, bitpit : 1.000
30/08/2011 21:31:18, eligius : 1.000
30/08/2011 21:31:18, mtred : 0.310
30/08/2011 21:31:18, ozco.in : 0.298
30/08/2011 21:31:18, bitcoins.lc : 0.211
30/08/2011 21:31:18, triplemining : 0.191
30/08/2011 21:31:18, bitminter : 0.190
30/08/2011 21:31:18, btcguild : 0.079
30/08/2011 21:31:18, nofeemining : 0.054
Looks like the update process locked itself up ~18 hours ago. It should be catching up now, unless there's a bigger problem, which I'll find out shortly, I assume. Consider this a preview of how things will look when updates are suspended for ~8 days in late September.
August 30, 2011, 02:59:03 PM
This doesnt look right, btcguild had a new block 20mins ago and several others have had blocks too.
30/08/2011 21:31:18, bitpit : 1.000
30/08/2011 21:31:18, eligius : 1.000
30/08/2011 21:31:18, mtred : 0.310
30/08/2011 21:31:18, ozco.in : 0.298
30/08/2011 21:31:18, bitcoins.lc : 0.211
30/08/2011 21:31:18, triplemining : 0.191
30/08/2011 21:31:18, bitminter : 0.190
30/08/2011 21:31:18, btcguild : 0.079
30/08/2011 21:31:18, nofeemining : 0.054
30/08/2011 21:31:18, bitpit : 1.000
30/08/2011 21:31:18, eligius : 1.000
30/08/2011 21:31:18, mtred : 0.310
30/08/2011 21:31:18, ozco.in : 0.298
30/08/2011 21:31:18, bitcoins.lc : 0.211
30/08/2011 21:31:18, triplemining : 0.191
30/08/2011 21:31:18, bitminter : 0.190
30/08/2011 21:31:18, btcguild : 0.079
30/08/2011 21:31:18, nofeemining : 0.054
August 29, 2011, 08:17:40 PM
As it turns out, the donation percentage code never actually correctly read the donation percentage from the configuration file. This has been fixed.
In other news, there will be approximately a one week period in late September where the API will not be updated. As a result, anyone using this miner will most likely only mine at fair pools for that week, rather than hop.
The ultimate goal, however, should be to somehow improve the whole process so that it doesn't rely on a third-party API. However, the reliance on pident makes this quite difficult. I have little time for bitcoin at the moment, so I'm probably not going to come up with anything myself anytime soon.
In other news, there will be approximately a one week period in late September where the API will not be updated. As a result, anyone using this miner will most likely only mine at fair pools for that week, rather than hop.
The ultimate goal, however, should be to somehow improve the whole process so that it doesn't rely on a third-party API. However, the reliance on pident makes this quite difficult. I have little time for bitcoin at the moment, so I'm probably not going to come up with anything myself anytime soon.
August 15, 2011, 10:44:15 AM
I get the following when trying to start poclbm for the first time after adding the following pools to pools.conf
bitpit
bitcoins.lc
bitminter
btcguild
mtred
ozco.in
bitpit
bitcoins.lc
bitminter
btcguild
mtred
ozco.in
My copy of pident is currently having problems with its scoring system, causing it to not properly update the API. I'm attempting to fix it at the moment. It should work again in a few minutes, though.
EDIT: API is running again. Still not sure what caused pident to start having lots of division by zero errors. Since its base scoring code is opaque to me at the moment, I'm not in much of a position to debug it effectively. However, I have avoided the error as a stopgap measure.
August 15, 2011, 10:16:28 AM
I get the following when trying to start poclbm for the first time after adding the following pools to pools.conf
bitpit
bitcoins.lc
bitminter
btcguild
mtred
ozco.in
Traceback (most recent call last):
File "./poclbm.py", line 67, in
miner = BitcoinMiner(devices[options.device], options, VERSION, HttpTransport.HttpTransport)
File "/home/roos/src/aexoden-poclbm-2889219/BitcoinMiner.py", line 33, in __init__
self.transport = transport(self)
File "/home/roos/src/aexoden-poclbm-2889219/HttpTransport.py", line 21, in __init__
super(HttpTransport, self).__init__(miner)
File "/home/roos/src/aexoden-poclbm-2889219/Transport.py", line 32, in __init__
self.best_pools = self.pools.get_best_pools()
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 97, in get_best_pools
return sorted(self.pools.values(), key=lambda pool: (pool.utility, pool.priority), reverse=True)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 97, in
return sorted(self.pools.values(), key=lambda pool: (pool.utility, pool.priority), reverse=True)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 125, in utility
self.update_data()
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 116, in update_data
self.rate = get_pool_rate(self.pident_name)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 31, in get_pool_rate
return _pool_data[0]['rates'][pool] if pool in _pool_data[0]['rates'] else 0.0
KeyError: 'rates'
bitpit
bitcoins.lc
bitminter
btcguild
mtred
ozco.in
Traceback (most recent call last):
File "./poclbm.py", line 67, in
miner = BitcoinMiner(devices[options.device], options, VERSION, HttpTransport.HttpTransport)
File "/home/roos/src/aexoden-poclbm-2889219/BitcoinMiner.py", line 33, in __init__
self.transport = transport(self)
File "/home/roos/src/aexoden-poclbm-2889219/HttpTransport.py", line 21, in __init__
super(HttpTransport, self).__init__(miner)
File "/home/roos/src/aexoden-poclbm-2889219/Transport.py", line 32, in __init__
self.best_pools = self.pools.get_best_pools()
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 97, in get_best_pools
return sorted(self.pools.values(), key=lambda pool: (pool.utility, pool.priority), reverse=True)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 97, in
return sorted(self.pools.values(), key=lambda pool: (pool.utility, pool.priority), reverse=True)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 125, in utility
self.update_data()
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 116, in update_data
self.rate = get_pool_rate(self.pident_name)
File "/home/roos/src/aexoden-poclbm-2889219/pools.py", line 31, in get_pool_rate
return _pool_data[0]['rates'][pool] if pool in _pool_data[0]['rates'] else 0.0
KeyError: 'rates'
August 14, 2011, 05:09:23 PM
Recent updates:
Otherwise, not much is happening. BTC Guild results are still significantly better than for a full-time miner.
- Support for Bitminter was added.
- Since RFCPool has closed, support has been removed.
Otherwise, not much is happening. BTC Guild results are still significantly better than for a full-time miner.
August 08, 2011, 08:59:30 PM
Have you collaborated at all with c00w, bloodred, ryouiki or other hopping developers? Seems like some of them have different takes on various pools and are able to support some that you aren't. I also think their projects would benefit from your work with pident. Eventually I'd like to see all your projects roll together into one super hopping client.
My primary goal was to distrust the pools as much as possible. It'd be easy to hop a bunch of other pools if I were to just use their APIs. (In fact, older versions of poclbm-autohop did just that. Only pool I lost in the transition was bitclockers, though.) Of course, I'm trusting pools to report their blocks accurately. They might delay those stats, but I think most pools are unlikely to hide them completely. However, they might take a bitclockers type approach, where they make it almost impossible to associate the blocks with real blocks (by not providing the block number or hash).In other news, I'm ready to report the first results of efficiency hopping BTC Guild. At first glance, it won't look good. My efficiency has been a paltry 81%. However, it's not as bad as it sounds. BTC Guild is stuck in a period of bad luck. A full-time BTC Guild miner's efficiency over the same period is only 72%. (Both of these numbers are relative to what the miner would have earned at a 0% PPS pool.) In the end, it looks like we should still be able to effectively hop pools that are delaying their statistics, if at a reduced efficiency.
August 06, 2011, 07:41:27 PM
Have you collaborated at all with c00w, bloodred, ryouiki or other hopping developers? Seems like some of them have different takes on various pools and are able to support some that you aren't. I also think their projects would benefit from your work with pident. Eventually I'd like to see all your projects roll together into one super hopping client.
August 06, 2011, 06:13:11 PM
I've added support for MainframeMC to the miner. Note that since it carries a 1.5% fee, it won't be used unless you have no other fair pools configured.
I've also added the ability to set a donation percentage for each pool in pools.conf. This will allow the miner to more accurately calculate the pool's utility. The field is optional, so your old pools.conf will continue to work with the new miner. If you are, for instance, donating 1.0% to a pool, you should add 1.0 to the end of the line.
Finally, the reject reduction work seems to be working well. Long-term reject percentage on my miners seems to be somewhere between 0.5% and 1.5%.
There are still several pools that are not supported:
If a pool is not listed, it's not currently supported by pident. (Bitclockers is not supported by pident because it obfuscates its blocks, so no one knows exactly which blocks belong to it.) You can presumably talk to Artefact2 about getting pools added to pident.
I've also added the ability to set a donation percentage for each pool in pools.conf. This will allow the miner to more accurately calculate the pool's utility. The field is optional, so your old pools.conf will continue to work with the new miner. If you are, for instance, donating 1.0% to a pool, you should add 1.0 to the end of the line.
Finally, the reject reduction work seems to be working well. Long-term reject percentage on my miners seems to be somewhere between 0.5% and 1.5%.
There are still several pools that are not supported:
- Bitcoinpool: Uses a modified proportional system for which I need to derive an accurate utility function. Given the Draconian TOS they have, I'm unlikely to bother.
- BTCMine: Uses the exponential decay scoring system with an unknown magic number. It may be possible to estimate the utility function, but I haven't looked into it.
- BTCMP: May add at some point.
- DeepBit: I will not add this as long as it makes up more than 33% of the network.
- PolMine: As long as I can confirm and keep updated on its reward system, I can add it.
- Slush: Like BTCMine, I'd have to evaluate the scoring system.
If a pool is not listed, it's not currently supported by pident. (Bitclockers is not supported by pident because it obfuscates its blocks, so no one knows exactly which blocks belong to it.) You can presumably talk to Artefact2 about getting pools added to pident.
August 05, 2011, 06:28:24 PM
I've pushed a commit that may help to reduce rejected shares. Before this patch, my rejected shares ranged from 3-6%, depending on the frequency of hopping. My limited test suggests this has been drastically reduced, but the old bitcoins.lc URL wasn't working all day, so the test may be flawed. I'll post more results in about 12-18 hours.
August 05, 2011, 02:40:45 PM
The API on my server honors the If-Modified-Since HTTP header, and will send a 304 unless the data has been updated. It will also gzip compress the data if the user agent supports it.
Hmm, it doesn't appear to be the case in your repo. Also, unless you already use them, declareCache() and declareCacheExpiration() will do the job in 2 lines of code (Etags: and Expires: fields for client-side cache and server-side caching of the generated page). In fact, my cache is so awesome it's probably the piece of code in pident I'm the most proud of.
Meanwhile, I am working on reducing rejects, but it'll be a few hours before I can see if my changes have improved things.
August 05, 2011, 12:35:34 PM
In fact, my cache is so awesome it's probably the piece of code in pident I'm the most proud of.
Now you are throwing down some massive geek braggin'
"Oh yeah, well my cache is so awesome, apache tell me to keep the change. BOOM!"
August 05, 2011, 11:12:48 AM
The API on my server honors the If-Modified-Since HTTP header, and will send a 304 unless the data has been updated. It will also gzip compress the data if the user agent supports it.
Hmm, it doesn't appear to be the case in your repo. Also, unless you already use them, declareCache() and declareCacheExpiration() will do the job in 2 lines of code (Etags: and Expires: fields for client-side cache and server-side caching of the generated page). In fact, my cache is so awesome it's probably the piece of code in pident I'm the most proud of.
August 04, 2011, 09:43:19 PM
The code used to generate the API is now available on github: https://github.com/aexoden/pident
The code is located entirely in the new printpooldata file, which prints out the JSON object. I haven't included my custom update script, at this time, however. (It's badly organized and has some site-specific code in it.) The update script is executed by cron once a minute, and approximately does the following things:
The API on my server honors the If-Modified-Since HTTP header, and will send a 304 unless the data has been updated. It will also gzip compress the data if the user agent supports it.
By the way, I'm under no illusion that printpooldata couldn't use some optimization. However, doing so isn't really worth the effort for me. It might seem kind of haphazard because of changing my approach a couple of times in the middle, and not completely rewriting to match.
The code is located entirely in the new printpooldata file, which prints out the JSON object. I haven't included my custom update script, at this time, however. (It's badly organized and has some site-specific code in it.) The update script is executed by cron once a minute, and approximately does the following things:
- Executes updateblocks
- If a new block was detected, queues an updatepools to run in two minutes (who knows if this delay is long enough/too long--I'm just trying to minimize unnecessary network access).
- If there were new blocks, or if updatepools was executed, a new copy of the pool data is generated and copied to my server, which then serves it up.
The API on my server honors the If-Modified-Since HTTP header, and will send a 304 unless the data has been updated. It will also gzip compress the data if the user agent supports it.
By the way, I'm under no illusion that printpooldata couldn't use some optimization. However, doing so isn't really worth the effort for me. It might seem kind of haphazard because of changing my approach a couple of times in the middle, and not completely rewriting to match.
Jump to: