pool.yrral.net P2Pool backed mining pool ALPHA - page 2.

notme

legendary

Activity: 1904

Merit: 1002

Quote from: daemondazz on June 18, 2013, 11:29:40 PM

Previous patch didn't work, indenting on the exception handler was wrong. I've also updated it to log the network that p2pool is running on (I run BTC, LTC and TRC from one installation):

Code:

root@bitcoin:/var/lib/p2pool/p2pool/p2pool# git diff work.py
diff --git a/p2pool/work.py b/p2pool/work.py
index 7c5823b..d1bbec8 100644
--- a/p2pool/work.py
+++ b/p2pool/work.py
@@ -1,6 +1,7 @@
 from __future__ import division

 import base64
+import csv
 import random
 import sys
 import time
@@ -281,6 +282,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
         received_header_hashes = set()

         def got_response(header, user, coinbase_nonce):
+            got_block, got_share = False, False
             assert len(coinbase_nonce) == self.COINBASE_NONCE_LENGTH == 4
             new_packed_gentx = packed_gentx[:-4-4] + coinbase_nonce + packed_gentx[-4:] if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else packed_gentx
             new_gentx = bitcoin_data.tx_type.unpack(new_packed_gentx) if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else gentx
@@ -291,6 +293,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 if pow_hash <= header['bits'].target or p2pool.DEBUG:
                     helper.submit_block(dict(header=header, txs=[new_gentx] + other_transactions), False, self.node.factory, self.node.bitcoind, self.node.bitcoind_work, self.node.net)
                     if pow_hash <= header['bits'].target:
+                        got_block = True
                         print
                         print 'GOT BLOCK FROM MINER! Passing to bitcoind! %s%064x' % (self.node.net.PARENT.BLOCK_EXPLORER_URL_PREFIX, header_hash)
                         print
@@ -332,6 +335,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                     log.err(None, 'Error while processing merged mining POW:')

             if pow_hash <= share_info['bits'].target and header_hash not in received_header_hashes:
+                got_share = True
                 share = get_share(header, pack.IntType(32).unpack(coinbase_nonce))

                 print 'GOT SHARE! %s %s prev %s age %.2fs%s' % (
@@ -364,6 +368,15 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 print >>sys.stderr, 'Worker %s submitted share more than once!' % (user,)
             else:
                 received_header_hashes.add(header_hash)
+
+                # P2POOL SHARE LOGGING
+                try:
+                    with open('/tmp/p2pool-%s-shares.csv' % self.net.SYMBOL, 'ab') as csvfile:
+                        diff = bitcoin_data.target_to_difficulty(target) * 1e-8
+                        writer = csv.writer(csvfile)
+                        writer.writerow([time.time(), self.net.SYMBOL, user, diff, on_time, got_share, got_block])
+                except:
+                    log.err(None, 'Could not save share to pool database')

                 self.pseudoshare_received.happened(bitcoin_data.target_to_average_attempts(target), not on_time, user)
                 self.recent_shares_ts_work.append((time.time(), bitcoin_data.target_to_average_attempts(target)))

Good work. However, I don't see where you are changing the address that gets credit.

Anyway, thanks to all the testers! The first payout will go out tomorrow at noon EST. Currently, I see shares from these addresses:
178sstRXJ2aF4KHcrkbECzFopjDgHCigyr
1Q68UdKBYgBVy7h4gAAXcipaoBPeoJZrXQ
13WSYzi1h9S14oN7E8DGbGLk7vHoXC2x3W
1yrraLgmRZ1fbgtz2BSVXcWVrJnH785QZ

daemondazz

sr. member

Activity: 448

Merit: 250

Previous patch didn't work, indenting on the exception handler was wrong. I've also updated it to log the network that p2pool is running on (I run BTC, LTC and TRC from one installation):

Code:

root@bitcoin:/var/lib/p2pool/p2pool/p2pool# git diff work.py
diff --git a/p2pool/work.py b/p2pool/work.py
index 7c5823b..d1bbec8 100644
--- a/p2pool/work.py
+++ b/p2pool/work.py
@@ -1,6 +1,7 @@
 from __future__ import division

 import base64
+import csv
 import random
 import sys
 import time
@@ -281,6 +282,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
         received_header_hashes = set()

         def got_response(header, user, coinbase_nonce):
+            got_block, got_share = False, False
             assert len(coinbase_nonce) == self.COINBASE_NONCE_LENGTH == 4
             new_packed_gentx = packed_gentx[:-4-4] + coinbase_nonce + packed_gentx[-4:] if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else packed_gentx
             new_gentx = bitcoin_data.tx_type.unpack(new_packed_gentx) if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else gentx
@@ -291,6 +293,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 if pow_hash <= header['bits'].target or p2pool.DEBUG:
                     helper.submit_block(dict(header=header, txs=[new_gentx] + other_transactions), False, self.node.factory, self.node.bitcoind, self.node.bitcoind_work, self.node.net)
                     if pow_hash <= header['bits'].target:
+                        got_block = True
                         print
                         print 'GOT BLOCK FROM MINER! Passing to bitcoind! %s%064x' % (self.node.net.PARENT.BLOCK_EXPLORER_URL_PREFIX, header_hash)
                         print
@@ -332,6 +335,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                     log.err(None, 'Error while processing merged mining POW:')

             if pow_hash <= share_info['bits'].target and header_hash not in received_header_hashes:
+                got_share = True
                 share = get_share(header, pack.IntType(32).unpack(coinbase_nonce))

                 print 'GOT SHARE! %s %s prev %s age %.2fs%s' % (
@@ -364,6 +368,15 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 print >>sys.stderr, 'Worker %s submitted share more than once!' % (user,)
             else:
                 received_header_hashes.add(header_hash)
+
+                # P2POOL SHARE LOGGING
+                try:
+                    with open('/tmp/p2pool-%s-shares.csv' % self.net.SYMBOL, 'ab') as csvfile:
+                        diff = bitcoin_data.target_to_difficulty(target) * 1e-8
+                        writer = csv.writer(csvfile)
+                        writer.writerow([time.time(), self.net.SYMBOL, user, diff, on_time, got_share, got_block])
+                except:
+                    log.err(None, 'Could not save share to pool database')

                 self.pseudoshare_received.happened(bitcoin_data.target_to_average_attempts(target), not on_time, user)
                 self.recent_shares_ts_work.append((time.time(), bitcoin_data.target_to_average_attempts(target)))

daemondazz

sr. member

Activity: 448

Merit: 250

Ah, good ideas, here's up update patch for the part that writes the CSV:

Code:

@@ -364,6 +369,15 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 print >>sys.stderr, 'Worker %s submitted share more than once!' % (user,)
             else:
                 received_header_hashes.add(header_hash)
+
+                # P2POOL SHARE LOGGING
+                try:
+                    with open('/tmp/p2pool-shares.csv', 'ab') as csvfile:
+                        diff = bitcoin_data.target_to_difficulty(target) * 1000000000
+                        writer = csv.writer(csvfile)
+                        writer.writerow([time.time(), user, diff, on_time, got_share, got_block])
+                    except:
+                        log.err(None, 'Could not save share to pool database')

                 self.pseudoshare_received.happened(bitcoin_data.target_to_average_attempts(target), not on_time, user)
                 self.recent_shares_ts_work.append((time.time(), bitcoin_data.target_to_average_attempts(target)))

notme

legendary

Activity: 1904

Merit: 1002

Quote from: daemondazz on June 18, 2013, 07:58:57 AM

Here is a quick patch against work.py which will cause it to log a line to a CSV file every time an actual share is found. I haven't had time to fully work out how to get the hash rate inside that function yet. I'm not sure if you want to only pay out on shares that p2pool counts as valid, or a percentage of the hashrate.

Fields in the CSV are: time.time(), user, on_time, got_share, got_block

Code:

root@bitcoin:/var/lib/p2pool/p2pool/p2pool# git diff work.py
diff --git a/p2pool/work.py b/p2pool/work.py
index 7c5823b..a4c3f38 100644
--- a/p2pool/work.py
+++ b/p2pool/work.py
@@ -5,6 +5,8 @@ import random
 import sys
 import time

+import csv
+
 from twisted.internet import defer
 from twisted.python import log

@@ -281,6 +283,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
         received_header_hashes = set()

         def got_response(header, user, coinbase_nonce):
+            got_block, got_share = False, False
             assert len(coinbase_nonce) == self.COINBASE_NONCE_LENGTH == 4
             new_packed_gentx = packed_gentx[:-4-4] + coinbase_nonce + packed_gentx[-4:] if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else packed_gentx
             new_gentx = bitcoin_data.tx_type.unpack(new_packed_gentx) if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else gentx
@@ -291,6 +294,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 if pow_hash <= header['bits'].target or p2pool.DEBUG:
                     helper.submit_block(dict(header=header, txs=[new_gentx] + other_transactions), False, self.node.factory, self.node.bitcoind, self.node.bitcoind_work, self.node.net)
                     if pow_hash <= header['bits'].target:
+                        got_block = True
                         print
                         print 'GOT BLOCK FROM MINER! Passing to bitcoind! %s%064x' % (self.node.net.PARENT.BLOCK_EXPLORER_URL_PREFIX, header_hash)
                         print
@@ -332,6 +336,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                     log.err(None, 'Error while processing merged mining POW:')

             if pow_hash <= share_info['bits'].target and header_hash not in received_header_hashes:
+                got_share = True
                 share = get_share(header, pack.IntType(32).unpack(coinbase_nonce))

                 print 'GOT SHARE! %s %s prev %s age %.2fs%s' % (
@@ -364,6 +369,16 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 print >>sys.stderr, 'Worker %s submitted share more than once!' % (user,)
             else:
                 received_header_hashes.add(header_hash)
+
+                # P2POOL SHARE LOGGING
+                if got_share or got_block:
+                    try:
+                        with open('/tmp/p2pool-shares.csv', 'ab') as csvfile:
+                            writer = csv.writer(csvfile)
+                            writer.writerow([time.time(), user, on_time, got_share, got_block])
+                    except:
+                        log.err(None, 'Could not save share to pool database')
+

                 self.pseudoshare_received.happened(bitcoin_data.target_to_average_attempts(target), not on_time, user)
                 self.recent_shares_ts_work.append((time.time(), bitcoin_data.target_to_average_attempts(target)))

With the code I have, it logs every response from p2pool whether the "result" is true of false, but I only use the ones with true for payout calculations. I also log the current difficulty and time with each share. My current plan is to just do payouts proportionally based on successful shares weighted by their difficulty requirement. P2pool changes difficulty frequently, so that needs to be factored in if you are doing things share based. Using hashrate estimates should get you the same answer since the estimate would be derived from the rate of shares, so it seems like an unnecessary complication.

daemondazz

sr. member

Activity: 448

Merit: 250

Here is a quick patch against work.py which will cause it to log a line to a CSV file every time an actual share is found. I haven't had time to fully work out how to get the hash rate inside that function yet. I'm not sure if you want to only pay out on shares that p2pool counts as valid, or a percentage of the hashrate.

Fields in the CSV are: time.time(), user, on_time, got_share, got_block

Code:

root@bitcoin:/var/lib/p2pool/p2pool/p2pool# git diff work.py
diff --git a/p2pool/work.py b/p2pool/work.py
index 7c5823b..a4c3f38 100644
--- a/p2pool/work.py
+++ b/p2pool/work.py
@@ -5,6 +5,8 @@ import random
 import sys
 import time

+import csv
+
 from twisted.internet import defer
 from twisted.python import log

@@ -281,6 +283,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
         received_header_hashes = set()

         def got_response(header, user, coinbase_nonce):
+            got_block, got_share = False, False
             assert len(coinbase_nonce) == self.COINBASE_NONCE_LENGTH == 4
             new_packed_gentx = packed_gentx[:-4-4] + coinbase_nonce + packed_gentx[-4:] if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else packed_gentx
             new_gentx = bitcoin_data.tx_type.unpack(new_packed_gentx) if coinbase_nonce != '\0'*self.COINBASE_NONCE_LENGTH else gentx
@@ -291,6 +294,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 if pow_hash <= header['bits'].target or p2pool.DEBUG:
                     helper.submit_block(dict(header=header, txs=[new_gentx] + other_transactions), False, self.node.factory, self.node.bitcoind, self.node.bitcoind_work, self.node.net)
                     if pow_hash <= header['bits'].target:
+                        got_block = True
                         print
                         print 'GOT BLOCK FROM MINER! Passing to bitcoind! %s%064x' % (self.node.net.PARENT.BLOCK_EXPLORER_URL_PREFIX, header_hash)
                         print
@@ -332,6 +336,7 @@ class WorkerBridge(worker_interface.WorkerBridge):
                     log.err(None, 'Error while processing merged mining POW:')

             if pow_hash <= share_info['bits'].target and header_hash not in received_header_hashes:
+                got_share = True
                 share = get_share(header, pack.IntType(32).unpack(coinbase_nonce))

                 print 'GOT SHARE! %s %s prev %s age %.2fs%s' % (
@@ -364,6 +369,16 @@ class WorkerBridge(worker_interface.WorkerBridge):
                 print >>sys.stderr, 'Worker %s submitted share more than once!' % (user,)
             else:
                 received_header_hashes.add(header_hash)
+
+                # P2POOL SHARE LOGGING
+                if got_share or got_block:
+                    try:
+                        with open('/tmp/p2pool-shares.csv', 'ab') as csvfile:
+                            writer = csv.writer(csvfile)
+                            writer.writerow([time.time(), user, on_time, got_share, got_block])
+                    except:
+                        log.err(None, 'Could not save share to pool database')
+

                 self.pseudoshare_received.happened(bitcoin_data.target_to_average_attempts(target), not on_time, user)
                 self.recent_shares_ts_work.append((time.time(), bitcoin_data.target_to_average_attempts(target)))

notme

legendary

Activity: 1904

Merit: 1002

So far, in addition to my address, I see shares from these addresses:
1Q68UdKBYgBVy7h4gAAXcipaoBPeoJZrXQ
1yrraLgmRZ1fbgtz2BSVXcWVrJnH785QZ

If you have tried mining on my pool and you are not on the list, please let me know ASAP. Now that I have testers I will try to make some time tomorrow to work on some stat reporting.

notme

legendary

Activity: 1904

Merit: 1002

Quote from: daemondazz on June 17, 2013, 07:52:01 PM

Cool, I'll check it out.

I'm in Australia. I was thinking of a separate pool per continent, or multiple per continent maybe, rather than one worldwide pool. I guess that's what you mean by localised.

I've got no experience with Ruby, so I might rework in Python, then it can plug directly into p2pool work.py and there's no need for running it on a separate port. I've already started on a Django schema for the database side of things.

No need for AWS here, I run a hosting company

Cool. It's pretty simple so far, and if you make it possible to just patch p2pool that is awesome. I'm not particularly tied to my code, my objective is primarily to help make p2pool accessible to miners of any size. Feel free to play around with it and ask questions since the code is a bit dense in places. If you redo it in python, feel free to post that here. I'm not a ninja like with ruby, but I do have a bit of python experience.

daemondazz

sr. member

Activity: 448

Merit: 250

Cool, I'll check it out.

I'm in Australia. I was thinking of a separate pool per continent, or multiple per continent maybe, rather than one worldwide pool. I guess that's what you mean by localised.

I've got no experience with Ruby, so I might rework in Python, then it can plug directly into p2pool work.py and there's no need for running it on a separate port. I've already started on a Django schema for the database side of things.

No need for AWS here, I run a hosting company

notme

legendary

Activity: 1904

Merit: 1002

Quote from: daemondazz on June 17, 2013, 06:30:05 PM

This looks like a good idea, I was thinking about implementing something very similar on my p2pool node. What continent are you on? Might be worth pooling (pun intended!) resources.

Interesting.... If we could develop a network of trusted operators we could offer low latency from anywhere in the world. Actually pooling everything together brings up trust issues between the various operators unless we use a single modified p2pool instance that does the logging, but then we don't have low latency from everywhere. However, since each pool only needs to be big enough to find enough p2pool shares, I think smaller, localized pools are the way to go. More important is overall hashpower for p2pool. N. America, btw.

If you want to spin up your own proxy for your p2pool node the code is here: https://github.com/yrral86/proxy-p2pool
1. Spin up an aws instance near you
2. Install git via package manager
3. Install ruby via rvm (this will take some time)
4. gem install json em-proxy
5. git clone https://github.com/yrral86/proxy-p2pool
6. cd proxy-p2pool
7. In proxy.rb, modify pool address on line 8 and p2pool.address on line 38
8. ruby proxy.rb
9. Point your miners at aws_address:9339
10. Submitted shares will be logged to shares.log (just a csv for now, I'll likely back it with a real db eventually)
11. You're on your own for payouts. When I get some time I will put together some scripts for parsing the shares.log file.

Pull requests and bug reports are of course welcome.

daemondazz

sr. member

Activity: 448

Merit: 250

This looks like a good idea, I was thinking about implementing something very similar on my p2pool node. What continent are you on? Might be worth pooling (pun intended!) resources.

notme

legendary

Activity: 1904

Merit: 1002

Quote from: notme on June 15, 2013, 09:32:08 PM

Fundamentally, it comes down to a tradeoff between high stales and high variance. If you slow down the share rate, it is much harder to find a share and variance will go through the roof. In order to keep everyone happy from big miners to small, we need a tiered system. This is actually what I am working towards with my proxy project, but you seem to want to throw out everything already built and start from scratch rather than work incrementally towards a system that suits everyone. Yes, I want the lower tiers to eventually be decentralized, but I need to work out the kinks in just proxying to an unmodified p2pool for now. Ultimately, we need to allow multiple addresses to take credit for p2pool shares rather than a single address per share. Once this is in place, we can have second tier pools that submit p2pool shares that pay out based on their easier share chain. A centralized proxy is just a first step towards this vision and will help us understand a part of the target system that is achievable today. Then there is the issue of BFL ASICs. Once we have a tiered system, we will be able to lengthen the share time on p2pool to something acceptable for all hardware and the second tier pools can provide easier shares and lower variance.

notme

legendary

Activity: 1904

Merit: 1002

This is targeted at those who don't want to maintain a bitcoin daemon and p2pool instance. I can potentially spin up as many proxies as I want that point to as many different p2pool instances as I want. I just need to aggregate the share records to calculate the payment. This will allow for more ddos resistance than any individual p2pool instance or centralized pool.

PatMan

hero member

Activity: 924

Merit: 1000

Watch out for the "Neg-Rep-Dogie-Police".....

Hi,

A good idea in principal. However, this will lose the one great benefit of p2p mining - vulnerability to ddos attack. Many pools are suffering from this right now despite having many security measures, will your node/pool be able to resist such an attack? Surely the best option would be to create a more efficient p2p mining program that is compatible with all mining hardware & is fully stratum capable & keeps this anti ddos benefit? I wish you luck with this, as something definitely needs to be done, but this is not the answer I fear.
I'm trying to put together a team who is capable of writing a completely new enhanced p2p program that will put p2p mining back on the map, check out my thread:

https://bitcointalksearch.org/topic/m.2234739

I've been approached by a few programmers, but am still looking for somebody with more bitcoin experience if you're interested. I have also been contacted by many miners who are willing to help with testing of various fpga/asics setups, so the need is definitely there.

Again, I wish you luck with this endeavour, but if you decide you really want to help p2p mining as a whole I'd love to hear from you.

Regards & Peace.

notme

legendary

Activity: 1904

Merit: 1002

I am developing some mining proxy software that allows me to pool p2pool shares from multiple users together to the same address. This allows miners with less hashing power to use p2pool with less variance. This is alpha quality software and I will be doing manual payouts for now. Payouts will be proportional. Please do not use this pool without a fallback since I may restart it frequently. There are no public stats at the moment, but that is certainly on the TODO list. At the moment, the best you can do is monitor the pools genearation address (1B1kSM3KfcP9BvGDC1G3cxZAV9LbxovpQi), which will include everybody's hashpower:: http://pool.yrral.net:9339/static/graphs.html

The goal of this project is to bring more interest to p2pool and also keep it accessible to small miners.

In order to mine, connect to pool.yrral.net:9339 with your bitcoin address as your username. Password can be anything. If you are using cgminer, you will see "Accepted untracked stratum share" instead of the normal message, but rest assured your shares are being tracked. Also, if I restart the proxy you may see a message about shares being lost that you can safely ignore.

Fine print: My p2pool instance is currently set to collect a 2% fee. Payments will be made biweekly on Mondays and Thursdays at noon EST and will include any p2pool payout that has reached 120 confirmations. If your payout is below the dust limit for the bitcoin network, your payouts will be accumulated until they exceed the limit (if a transaction has an output that is too small it will not be relayed or included in blocks by most nodes).

Topic: pool.yrral.net P2Pool backed mining pool ALPHA - page 2. (Read 3185 times)