Pages:
Author

Topic: An enhanced alternative p2p “pool” – a Think Tank. (Read 3099 times)

hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Indeed it is. I've been really busy lately & have not been able to spend much time with it, but hopefully that will change within the next few days. I'm still looking for a programmer/coder with balls of steel to help me get this started - so if you are that person, or you know of someone - please drop me a line.....
hero member
Activity: 686
Merit: 500
WANTED: Active dev to fix & re-write p2pool in C
Ah! This sounds promising - is this project still in the mix?....
sr. member
Activity: 435
Merit: 250
How about turning this into a framework, for not only p2p mining, but also the coin network too.

Why do we need a separate bitcoind and litecoind daemons, when they could become plugin's.  Most coins need mining pools, so roll it into one app.

It would make creating new cryptocurrancies, much easier.

+1.
Add something like zeromq on top of that to reduce the network/block latency/propagation delays and we'd have a winner.
hero member
Activity: 546
Merit: 500
How about turning this into a framework, for not only p2p mining, but also the coin network too.

Why do we need a separate bitcoind and litecoind daemons, when they could become plugin's.  Most coins need mining pools, so roll it into one app.

It would make creating new cryptocurrancies, much easier.
legendary
Activity: 1904
Merit: 1002
Please can we have a plugin architecture to support different coins.  I have Bitcoin & p2pool running on a server, and today I just added Litecoin, which turned out to be less than easy.  Ports clashing, etc. I also intended to add one of the new experimental cryptocurrencies, but Iost patience, and will try another day.

Very confusing to have command line options & console output saying "bitcoin", when it's litecoin.

Ideally, I would like to see the new p2p pool software advertise to it's connected nodes, which coins it supports.  And then the nodes know which byte streams to share. No need for different ports set up for each coin.



Hi FlappySocks,

Funny you should say that, I had a long meeting/discussion yesterday with a potential coder and this subject was bought up. Although my first intention was to make the software Bitcoin only to start with, but with having provisions for adding other alternative coins when demand/time allowed, we toyed with the idea of maybe developing the software as a stripped down "bones" program so that the user can add which ever add-on they required to mine their preferred coin. This has never been done before and opens a whole new kettle of fish, but theoretically it makes great sense as it would help keep the program light, the user would only need to download the add-on he/she required for the coin they wanted to mine, as well as making it easier for the devs to implement updates to the add-ons as and when needed, which the user would pull when required. I like this idea as it is both ground breaking & serves a purpose, but more discussions are required before making a decision as it is a bold move. I will update the starter thread with this though.
Been going over some names for the software too lately and am hoping for some suggestions from you lot. A couple of the best so far are Multipeer & Multicoinp2p, but am totally open to this, so feel free to put a name forward - you might even win a prize....... Grin

This is one area for potential improvement over p2pool, but I believe it would have to be included in all clients.  Otherwise, you wind up with p2pool style merged mining where you only get the merged mining rewards if it is your node that solves the p2pool block.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Please can we have a plugin architecture to support different coins.  I have Bitcoin & p2pool running on a server, and today I just added Litecoin, which turned out to be less than easy.  Ports clashing, etc. I also intended to add one of the new experimental cryptocurrencies, but Iost patience, and will try another day.

Very confusing to have command line options & console output saying "bitcoin", when it's litecoin.

Ideally, I would like to see the new p2p pool software advertise to it's connected nodes, which coins it supports.  And then the nodes know which byte streams to share. No need for different ports set up for each coin.



Hi FlappySocks,

Funny you should say that, I had a long meeting/discussion yesterday with a potential coder and this subject was bought up. Although my first intention was to make the software Bitcoin only to start with, but with having provisions for adding other alternative coins when demand/time allowed, we toyed with the idea of maybe developing the software as a stripped down "bones" program so that the user can add which ever add-on they required to mine their preferred coin. This has never been done before and opens a whole new kettle of fish, but theoretically it makes great sense as it would help keep the program light, the user would only need to download the add-on he/she required for the coin they wanted to mine, as well as making it easier for the devs to implement updates to the add-ons as and when needed, which the user would pull when required. I like this idea as it is both ground breaking & serves a purpose, but more discussions are required before making a decision as it is a bold move. I will update the starter thread with this though.
Been going over some names for the software too lately and am hoping for some suggestions from you lot. A couple of the best so far are Multipeer & Multicoinp2p, but am totally open to this, so feel free to put a name forward - you might even win a prize....... Grin
legendary
Activity: 1904
Merit: 1002
FYI : BFL 480 Gh/s hashing daemon with cgminer up and running
https://bitcointalksearch.org/topic/m.2514804

edit : correction - p2pool is more than 22000 lines of code (and more than 8e5 characters)

That does seem a little more reasonable Tongue

Still, it takes a serious chuck of developer man hours to put something as small as 22k lines of code together.
.m.
sr. member
Activity: 280
Merit: 260
FYI : BFL 480 Gh/s hashing daemon with cgminer up and running
https://bitcointalksearch.org/topic/m.2514804

edit : correction - p2pool is more than 22000 lines of code (and more than 8e5 characters)
hero member
Activity: 546
Merit: 500
Please can we have a plugin architecture to support different coins.  I have Bitcoin & p2pool running on a server, and today I just added Litecoin, which turned out to be less than easy.  Ports clashing, etc. I also intended to add one of the new experimental cryptocurrencies, but Iost patience, and will try another day.

Very confusing to have command line options & console output saying "bitcoin", when it's litecoin.

Ideally, I would like to see the new p2p pool software advertise to it's connected nodes, which coins it supports.  And then the nodes know which byte streams to share. No need for different ports set up for each coin.

legendary
Activity: 1904
Merit: 1002
Hi all, I have found this thread googling p2pool + golang Smiley
I believe "notme" has a good understanding and I like his (proxy) proposal.
Is there any publicly available info about "Mhash/sec at miner's side" per "actual reward" for p2pool compared to other pools ?
If there is not clear advantage, I would not throw more than 8e5 lines of python code away, not mentioning I do not understand much of it.
But I would certainly enjoy going step by step and trying implement and possibly improve little pieces in google GO (asi it is well designed,compiled,clean,safe,concurrent .... language - try to play/edit/run/learn at tour.golang.org).
And that will improve lags/problems with "more miners" or "high Ghashes" connected to p2pool node as well.
I understand the urgency to allow connecting ASIC miners to p2pool. Avalon is mentioned not working,BFL not working(sufficiently) neither, but may improve soon : https://forums.butterflylabs.com/bfl-forum-miscellaneous/2164-demo-unit-forrestv-p2pool.html#post27922
Bitfury has no devices yet, KnC miner as well, ASICminer is mentioned to work well ("notme").
But - all vendors are (or will be) selling chips and there are many HW design threads and there is no reason not to support p2pool well.
I have no chips at my posession, but they can be simulated with many GPUs.
There is already an effort to replicate bitcoind and a client in golang.
So I will be happy to help with 'proxy' effort - but probably we shall create a new thread ?
Good luck to everybody !


You can join in the discussion here: https://bitcointalksearch.org/topic/poolyrralnet-p2pool-backed-mining-pool-alpha-234841

It started off just trying to get testers for my proxy which is written in Ruby.  Someone else came along and is looking at integrating it directly in p2pool using Python.  I really don't care if my thread devolves into a generic p2pool proxying discussion, so feel free to try and drum up support for a version in Go.  In the short term, I'm primarily interested in bringing down the block times for p2pool, and proxying is simple enough that having multiple approaches should help us all to understand the problem space better.

If anyone thinks I'm being hypocritical after my hard stance on not duplicating p2pool, as .m. points out, p2pool is 8e5 lines of code and my proxy and logging is providing all the functionality I need to operate a pool and is sitting at 185 lines.
.m.
sr. member
Activity: 280
Merit: 260
Hi all, I have found this thread googling p2pool + golang Smiley
I believe "notme" has a good understanding and I like his (proxy) proposal.
Is there any publicly available info about "Mhash/sec at miner's side" per "actual reward" for p2pool compared to other pools ?
If there is not clear advantage, I would not throw more than 8e5 lines of python code away, not mentioning I do not understand much of it.
But I would certainly enjoy going step by step and trying implement and possibly improve little pieces in google GO (asi it is well designed,compiled,clean,safe,concurrent .... language - try to play/edit/run/learn at tour.golang.org).
And that will improve lags/problems with "more miners" or "high Ghashes" connected to p2pool node as well.
I understand the urgency to allow connecting ASIC miners to p2pool. Avalon is mentioned not working,BFL not working(sufficiently) neither, but may improve soon : https://forums.butterflylabs.com/bfl-forum-miscellaneous/2164-demo-unit-forrestv-p2pool.html#post27922
Bitfury has no devices yet, KnC miner as well, ASICminer is mentioned to work well ("notme").
But - all vendors are (or will be) selling chips and there are many HW design threads and there is no reason not to support p2pool well.
I have no chips at my posession, but they can be simulated with many GPUs.
There is already an effort to replicate bitcoind and a client in golang.
So I will be happy to help with 'proxy' effort - but probably we shall create a new thread ?
Good luck to everybody !
donator
Activity: 2058
Merit: 1007
Poor impulse control.
One of the "problems" with bitcoin is block propagation.  Once a block is found, every other node needs to receive the block and mine on top of it.  If you don't mine on top of the longest chain, you will loose, and basicly, your block will become stale (sometimes called orphan)

To calculate the chance of this happening - anyone can jump in and correct my math: we have the following formulla: p/i

p would be the time it takes to propagate a block over the network, and i would be the interval time.  If we look at bitcoin, the value for i would be (on average) 600 seconds, as every 10 minutes a block is found.   Let's say it takes 5 seconds to propagate over the network: we would have 5/600, which means 0.8% of the time you are mining, that you are at risk of not being on the longest chain, and that you will create orphans, and if you find a block the same happens in reverse, someone else might build upon your block and causing your block to have a chance to be orphanned.

p2pool uses the same algorithm as bitcoin, only in a smaller version, their "blockchain" has an interval of 10 seconds.  Since the network is smaller, the propagation times are lower/better, but still, if it would take 1 second, you're down to 10% chance of finding a stale.

So unless my math is wrong, it is clear that shorter rounds will lead to higher stales.

So, anything "new" should find a way to use longer rounds inside the p2p network to make sure the stales are reduced.

Meni Rosenfeld derives something different here: http://bitcoin.stackexchange.com/questions/4953/how-will-a-massive-increase-in-hashpower-affect-orphan-rates/4958#4958

Quote
If the average time to find a block is T, and the typical time for a found block to propagate in the network is t, then the proportion of orphans among all blocks will be roughly 1/(1+T/t).


hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Your ideas are not bad ideas at all - they're just not applicable to what I want to do. Good luck.  Wink
legendary
Activity: 1904
Merit: 1002
I'm sorry, I'm not sure you get my drift. I'm doing this project for many reasons, some of them being: because I think it's a challenge, it will be fun, it will benefit mine & everyone involved knowledge & understanding of Bitcoin & p2p networking, because I want it to be better, because I believe it will benefit Bitcoin & mining, because I want to & because I can. This project is not about putting any pool out of business, it is about joining them in business with healthy competition which benefits Bitcoin & miners alike. It will be in competition with all pools, not just p2pool - all pools.
I started this thread to gather ideas & suggestions from the community, completely open & transparent to everyone who would like to get involved, it is not a "why" or "what for" thread - if anything it's a "why the hell not" thread, and the people who I'm interested in hearing from & possibly working with are the "why the hell not" type of crowd, plenty of them on this forum. I want to start from afresh & build it from the ground up because that is what I choose to do, in the knowledge that it has been 100% developed by the guys behind it - I don't believe I need a justification or reason to do this. On the contrary, it would take or need to be an exceptional reason for me NOT to do this.
Again, I wish you the best of luck with what you are trying to do, I hope it works for you. Of course, you are always welcome to give some positive input any time in the future if you wish. I'm open to everything - hence this thread.

Peace  Wink

I get that, I just don't see any ideas or suggestions here.... just talk about needing such ideas/suggestions.  And all of my ideas are bad in your opinion because I don't want to reinvent the wheel.

Anyway, I'm going to go back to being productive.  Best of luck to you two.  Hopefully you can find some other people to join in your conversation who are more enthusiastic about duplicating effort.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
I'm sorry, I'm not sure you get my drift. I'm doing this project for many reasons, some of them being: because I think it's a challenge, it will be fun, it will benefit mine & everyone involved knowledge & understanding of Bitcoin & p2p networking, because I want it to be better, because I believe it will benefit Bitcoin & mining, because I want to & because I can. This project is not about putting any pool out of business, it is about joining them in business with healthy competition which benefits Bitcoin & miners alike. It will be in competition with all pools, not just p2pool - all pools.
I started this thread to gather ideas & suggestions from the community, completely open & transparent to everyone who would like to get involved, it is not a "why" or "what for" thread - if anything it's a "why the hell not" thread, and the people who I'm interested in hearing from & possibly working with are the "why the hell not" type of crowd, plenty of them on this forum. I want to start from afresh & build it from the ground up because that is what I choose to do, in the knowledge that it has been 100% developed by the guys behind it - I don't believe I need a justification or reason to do this. On the contrary, it would take or need to be an exceptional reason for me NOT to do this.
Again, I wish you the best of luck with what you are trying to do, I hope it works for you. Of course, you are always welcome to give some positive input any time in the future if you wish. I'm open to everything - hence this thread.

Peace  Wink
legendary
Activity: 1904
Merit: 1002
Let me be clear.  I'm not here to replace p2pool.  I believe that most of the problems people have with p2pool are possible to overcome without changing the fundamental design.  My posts are to help you guys understand the technicalities of what you are attempting to do.

I can't see how you're going to do that if you are going to keep the mine-on-top-of-longest-chain system.   If you're going to do a new p2p pool, I think the fundamental design should be changed.  But feel free to convince me otherwise Smiley

In order to keep everyone happy from big miners to small, we need a tiered system.  This is actually what I am working towards with my proxy project, but you seem to want to throw out everything already built and start from scratch rather than work incrementally towards a system that suits everyone.  Yes, I want the lower tiers to eventually be decentralized, but I need to work out the kinks in just proxying to an unmodified p2pool for now.  Ultimately, we need to allow multiple addresses to take credit for p2pool shares rather than a single address per share.  Once this is in place, we can have second tier pools that submit p2pool shares that pay out based on their easier share chain.  A centralized proxy is just a first step towards this vision and will help us understand a part of the target system that is achievable today.  Then there is the issue of BFL ASICs.  Once we have a tiered system, we will be able to lengthen the share time on p2pool to something acceptable for all hardware and the second tier pools can provide easier shares and lower variance.

How do you intend to do it without a proof of work chain?  You have to have a way of reliably measuring the work of the various pool contributors.  All I've seen is complaints about p2pool and no new ideas other than the one I came up with, which seems to be a non-starter anyway.

Let me be clear.  I'm not here to replace p2pool.  I believe that most of the problems people have with p2pool are possible to overcome without changing the fundamental design.  My posts are to help you guys understand the technicalities of what you are attempting to do.

I can't see how you're going to do that if you are going to keep the mine-on-top-of-longest-chain system.   If you're going to do a new p2p pool, I think the fundamental design should be changed.  But feel free to convince me otherwise Smiley

I'm sure nobody wants to replace any pool, I just want to make a fully functional & efficient p2p pooling system. History shows that pools come and go - if a pool shuts down it is not because of other pools - it is because it was unable to provide it's miners with a good service that can compete with other pools. Ultimately, it is miners who decide weather a pool operates or not, not other pools. Sure, development is not going to be an easy task, I made that clear at the start, but it's also not as hard as many people believe it to be if it's done properly with the right programmer/coder. On a personal note, if I were a coder and was asked to develop such software, I would certainly want to develop it myself from the ground up rather than try and rectify someone else's programming with patches, workarounds and proxies. But that's just me.

I've grown up with the open source ethos.  It's about community and building on the work of others.  Reinventing the wheel seems pointless to me.  Perhaps you are right that a better system can be built from the ground up.  But I can't see it.  What I do see is a clear path from what is working today to what we want in the future.  So again, I ask you to lay out a path from 0 to your goal.  I've already laid out one from p2pool to your goal.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Let me be clear.  I'm not here to replace p2pool.  I believe that most of the problems people have with p2pool are possible to overcome without changing the fundamental design.  My posts are to help you guys understand the technicalities of what you are attempting to do.

I can't see how you're going to do that if you are going to keep the mine-on-top-of-longest-chain system.   If you're going to do a new p2p pool, I think the fundamental design should be changed.  But feel free to convince me otherwise Smiley

I'm sure nobody wants to replace any pool, I just want to make a fully functional & efficient p2p pooling system. History shows that pools come and go - if a pool shuts down it is not because of other pools - it is because it was unable to provide it's miners with a good service that can compete with other pools. Ultimately, it is miners who decide weather a pool operates or not, not other pools. Sure, development is not going to be an easy task, I made that clear at the start, but it's also not as hard as many people believe it to be if it's done properly with the right programmer/coder. On a personal note, if I were a coder and was asked to develop such software, I would certainly want to develop it myself from the ground up rather than try and rectify someone else's programming with patches, workarounds and proxies. But that's just me.
sr. member
Activity: 263
Merit: 250
Pool operator of Triplemining.com
Let me be clear.  I'm not here to replace p2pool.  I believe that most of the problems people have with p2pool are possible to overcome without changing the fundamental design.  My posts are to help you guys understand the technicalities of what you are attempting to do.

I can't see how you're going to do that if you are going to keep the mine-on-top-of-longest-chain system.   If you're going to do a new p2p pool, I think the fundamental design should be changed.  But feel free to convince me otherwise Smiley
legendary
Activity: 1904
Merit: 1002
Now that I've had some time to think about it I'm convinced my idea for equalizing variance won't work without punishing new address when they first show up on the network.  Without such a probationary period, people could mine to as many separate addresses as they want.  Even with a probationary period it might make sense to mine to several addresses if we don't get the incentives quite right.

If you are badly connected to any pool you will have many stales.  I still don't see how p2pool is any worse in this respect.  I'll concede it has problems with SOME ASICs.

I'm afraid this is not true.   Regular pools give you 30 seconds to submit a share.  p2pool uses the first share it finds of a certain difficulty, and you have to build on top of that.  So you need to race to be the first in p2pool in rounds that last 10 seconds.  You have 30 seconds on a regular pool and you don't have to race any peers. 

p2pool is greatly affected by your connectivity, regular pools are not

Those who get the shares first will always have an advantage.  I'm not sure it is possible to remove this advantage unless you somehow punish peers who have low latency.  Increasing the share time to 30 seconds would address the other concern.
But, yes, it is harder to maintain a sufficiently connected p2pool instance.  It is really easy to be sufficiently connected to a centralized pool or a particular p2pool instance.

The reason I've been making all my posts are simple: My message to you is think outside the box:  Do not make just another bitcoin clone.  p2pool did it, and it works fine but we see it has some drawbacks.  If you're going to create another implementation of the same basic idea, you're going to end up with the same problems.  I am not telling you I know the answer.  This is a difficult question to solve.  But if you want to start an alternative to p2pool, make sure you use a totally different way of thinking so you will have a product that doesn't work the same way so you have your own advantages (and perhaps disadvantages) over p2pool.



Let me be clear.  I'm not here to replace p2pool.  I believe that most of the problems people have with p2pool are possible to overcome without changing the fundamental design.  My posts are to help you guys understand the technicalities of what you are attempting to do.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Completely agree Kinlo, as I said earlier - a fresh start from the ground up is required. I appreciate your input Kinlo, it's always good to hear from someone who thinks the same way & realises what is needed - please do keep the ideas & suggestions coming - the more input from the bitcoin community the better chance there is of developing a successful p2p alternative.  Grin
Pages:
Jump to: