Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 301. (Read 2591971 times)

hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Just a heads up to those of us merge mining - the NMC problem has a temporary fix in place in git:

https://github.com/namecoin/namecoin/tree/relay_tmp_fix

Trying it now......
legendary
Activity: 1258
Merit: 1027
Fees are easy to see on any public node:

Node Fee: http://minefast.coincadence.com:9332/fee

Donation and Node Fee: http://minefast.coincadence.com:9332/local_stats

The API is built into every node. It would not be hard for a greedy node operator to spoof this though...

Edit: There is also the /patron_sendmany API, you give it an amount (100BTC in the example below), and it will give you a list of miners with active shares, dividing up the donation fairly by number of shares per miner:

http://minefast.coincadence.com:9332/patron_sendmany/100
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
It's funny, this debate comes round every few months when new arrivals realize the wasted potential p2pool has, but the conclusion is always undecided.....

Personally (again), there isn't really a lot wrong with how p2pool works, just what it (doesn't) works with due to no development - this is the problem. If there is to be any kind of hard coded donation feature, it should go to everyone depending on their hash rate, and the option to donate to the dev should remain as it is. It should be possible for the p2pool program to detect if the donation option is enabled or not on a particular node, then adjust the donation amount to that node accordingly. I still believe that if users are able to see that there is ongoing development on p2pool (which there must be in order for p2pool to be compatible with the ever increasing different hardware being released) - they will donate. This should also be displayed on their node for everyone to see - kind of name & shame so to speak.....

Just a rough suggestion...... Wink

I must admit it's a refreshing change to have a semi-intelligent debate about this, when I first suggested there was plenty of room for improvement with p2pool over a year ago - I was branded a blasphemous upstart witch!  How dare I question p2pool......Cheesy Cheesy
legendary
Activity: 1258
Merit: 1027
Not a single point of failure, a potential point of failure.....

The failure of the API (which is inevitable, eventually it will have some down time) will have 0 impact on mining.

The API will only provide data for the front-end, it will have no hooks into the node at all.

To be clear the stuff I'm talking about is basically the entire p2pool Global Dashboard here: http://minefast.coincadence.com/p2pool-stats.php

The platform data that is not available from p2pool and the block history on the Node Dashboard page here: http://minefast.coincadence.com/index.php

And the miner BTC reward and block luck here: http://minefast.coincadence.com/miner.php?id=19vXrwKGUhK4cCU8tA4kWZgbChcmh9a6qj

I'm not concerned with someone trying to hijack any ad space, the API will serve raw HTML that is embedded in the pages with Javascript and the ads will simply be there amongst the other data. Yes, someone could write some code to strip the ads and replace them with their own, but I don't see it as being a huge problem. If it became one it would be trivial to dynamically change the ad placement and code so its hard to strip...
member
Activity: 112
Merit: 10
Not a single point of failure, a potential point of failure.....
Ok, I'll give you that, if a peer node went down it could cause a "burp" for some folks, but I would still contend not a big one.  What happens if a peer you're connected to now goes down?  bitcoind blacklists it and uses the others in its list.  It's like a root nameserver going down... might cause an extra few hundred milliseconds to get a response on a domain query from a different one, but unless they all went down at the same time, you'd really not notice much of a difference.
member
Activity: 112
Merit: 10
So the technology is there to do something like that. But trouble is perhaps it's not really p2p anymore. You've just invented a semi-centralised pool that enforces a payment to the boss - even a benevolent one. So politically this could be hard to achieve in practice.

Exactly. So nobody will use it.....
And, currently there's nothing stopping anyone running a public node from charging a fee for miners to use it, and there's nothing stopping anyone from avoiding using that node (not intended to be snarky, just making the point since it's directly relevant).

And currently there are enough issues with p2pool where people are avoiding it and going with higher priced proprietary pools.  Arguably those issues could be fixed, but if only there was a way to get them fixed...........

We are at the point where you have to pool mine in order to make anything, are we all agreed on that?

That being the case, I would rather mine at a pool where I know any fee I pay is going to the betterment of the whole pool and all who mine on it, including myself (third point in my statement - distribute the rest back to all the miners), rather than to make someone or some company rich off my labor.  1% of a BTC is what, $5?  did anyone get a coffee on the way to work this morning?  How much did that cost you?  If forgoing a cup of joe for a couple days meant my Terraminers regained 20% of their hashrate (a piddly little 600GH/s, nothing big  Roll Eyes ), sign me up twice.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Not a single point of failure, a potential point of failure.....
member
Activity: 112
Merit: 10
That is creating a potential point of failure, the whole idea of a peer to peer network is to avoid that.
Please explain your conclusion, I don't see this creating a single point of failure, and it's quite the opposite.

A peering node would simply be another full instance of bitcoind running on a server that's well connected and very well resourced.  It would be able to service hundreds of connections from other, smaller, local nodes.  It's only purpose would be to ensure a well connected host closer to the smaller or home miners to hopefully increase their chances of getting their shares into the network faster.

In addition if it served the data that windpath is alluding to, and serve as a closer location to make the API calls to, it decentralizes the network further, and moves the processing of this data out closer to the individual miners.
newbie
Activity: 21
Merit: 0
To offset the costs of the API I was considering offering 2 options (miners choice):

Some great thoughts windpath.

What stops someone taking your code, running the complex node and replacing your advertisements with their own? Does this just create a way for others to monetise your work?

Or would the complex node stuff serving the API stay closed source?
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
So the technology is there to do something like that. But trouble is perhaps it's not really p2p anymore. You've just invented a semi-centralised pool that enforces a payment to the boss - even a benevolent one. So politically this could be hard to achieve in practice.

Exactly. So nobody will use it.....
newbie
Activity: 21
Merit: 0
I don't think there's anything stopping someone from re-coding it to remove the donation, however I'm not sure how the permissions are setup on the main git repo, whether those changes could be re-submitted back into the main branch without forrestv's approval.  That might require a fork.

Anyone is free to fork the code and make their own changes, but I'm sure also if someone did something like that with nefarious purposes it would be found out pretty quickly and that node could probably be blocked from interacting with the rest of the nodes.  Now I say that without knowing for sure, but that would be my guess.

They wouldn't need to resubmit to the main repo. They just fork it (like you did) include all the new development work done and paid for but leave out the little bit that makes the donation and then run the node. Presto - 0% fee. They could either release it publicly if they wanted to be an ass or better still just do it on the quiet and leach. It's just the existing system but without the convenient command line option that forrestv put in.

But the block chain concept already solved this problem. The same reason a miner can't just give themselves 50 BTC when they mine a block or double spend an input (analogous to removing a development charge). Although any miner could create a block awarding themselves extra BTC the rest of the bitcoin network agrees by consensus that this new block is not valid and simply ignores it - it doesn't enter the block chain. The rules of the system are encoded by the majority consensus of miners. That's the power of the 51% attack. With 51% you can change whatever rules you want and benefit yourself.

Taking this to the equivalent situation in the p2pool share chain. If the majority of miners on p2pool agree (or rather their nodes enforce) that a valid share must include a 1% donation to whomever then bang it's the law. Doesn't matter what an individual does with his nod. Nobody else will recognise it so he doesn't get that share. It's a hard fork though because you effectively create an entirely new p2pool which has an incompatible rule with the original p2pool.

So the technology is there to do something like that. But trouble is perhaps it's not really p2p anymore. You've just invented a semi-centralised pool that enforces a payment to the boss - even a benevolent one. So politically this could be hard to achieve in practice.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....

Would it not make sense to see if there are others who would be willing to put up very well connected peer nodes, which won't service miners, but will provide peering and maybe the data and API system so that it can be queried by the local nodes?  I've got resources to be able to contribute and am willing, Paul seems to, and I'm sure there are others who will pitch in as has been the case to date.


That is creating a potential point of failure, the whole idea of a peer to peer network is to avoid that.
member
Activity: 112
Merit: 10
It certainly looks like you're saying you want to move it away from being Open Source, tbh:

Open source and free will only take any project to a certain level, as long as people are willing to donate to foster a better cause.  I don't think this works in the BTC world, since we're all doing this for one reason - to mine BTC and make money......

Personally, I got into Bitcoin because I believe that it has the potential to change the world & dismantle the corrupt Banking systems, & I'm happy if I break even or even make a loss if it succeeds in doing so. I want BTC to replace money, & I also know I'm not alone in believing it can. I'm totally not interested in getting rich, and if I were, mining BTC is not the way I'd go about it  Cheesy Cheesy

There is far more worth in BTC than making a quick profit. The long term potentials are limitless. World changing even.
Well, since you point it out that way... It reads different than the way it sounded in my head.  Smiley

I probably should have left it at "free", and omitted the Open Source phrase from the sentence.

As anyone who has been involved with O.S. for any length of time knows, there is a point where people can no longer contribute because of other "real world" issues and as a project grows it needs more care and feeding and people to help - exactly the case of what happened here with forrestv.  At some point you run short of help and goals either become less achievable or frequently attained, or you need to seek alternatives to continue meeting them on time as needed.

We already have seen the challenges here with the main developer stepping away.  To date, no one has stepped up to fill the holes voluntarily, so it seems like there's a need now to explore the alternative means - i.e. money - to help fix what's broke and continue moving forward.

So, yeah, in summary, I didn't mean to insinuate it should be closed off... just that it might end up not being free (as in money) for long if things don't change.

PS - I agree wholeheartedly with your last sentence, I just didn't feel like separately blockquoting it...
member
Activity: 112
Merit: 10
Frankly, while my own desire for compensation is high, the benefits of being an open source and 0% fee pool are hard to ignore.
Well, technically it's still a .5% fee pool if folks don't know how to turn off the donations part... Wink

Quote
The primary Coin Cadence front-end that you see now on my site requires a lot of data that is simply not available to a regular p2pool node without some major changes that increase resources required by the server to the point where it would effect efficiency for most mining on their own nodes.

To solve this I will be providing an API that will allow nodes to pull the data they don't have access to from my servers.
I think this might tie into the idea behind my question yesterday about peering and "root peers".

Would it not make sense to see if there are others who would be willing to put up very well connected peer nodes, which won't service miners, but will provide peering and maybe the data and API system so that it can be queried by the local nodes?  I've got resources to be able to contribute and am willing, Paul seems to, and I'm sure there are others who will pitch in as has been the case to date.

It shouldn't be solely on you to be providing all of this, both because it's out of your pocket, and also a strain on your resources.  And, don't take this the wrong way, but we'd all be again relying on only one person for the fate of the project... Wink
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
It certainly looks like you're saying you want to move it away from being Open Source, tbh:

Open source and free will only take any project to a certain level, as long as people are willing to donate to foster a better cause.  I don't think this works in the BTC world, since we're all doing this for one reason - to mine BTC and make money......

Personally, I got into Bitcoin because I believe that it has the potential to change the world & dismantle the corrupt Banking systems, & I'm happy if I break even or even make a loss if it succeeds in doing so. I want BTC to replace money, & I also know I'm not alone in believing it can. I'm totally not interested in getting rich, and if I were, mining BTC is not the way I'd go about it  Cheesy Cheesy

There is far more worth in BTC than making a quick profit. The long term potentials are limitless. World changing even.

member
Activity: 112
Merit: 10
I agree with everything you said KGB. But given its open source what stops someone from stripping out the mandatory fee and gaining the benefits without paying.

The share chain solution is that if the majority of p2pool are willing to play by these rules and reject freeloading shares that don't include the development fee then it would be enforced. That would be a hard fork so would need managing.

Will the majority of miners play ball though?
I don't think there's anything stopping someone from re-coding it to remove the donation, however I'm not sure how the permissions are setup on the main git repo, whether those changes could be re-submitted back into the main branch without forrestv's approval.  That might require a fork.

Anyone is free to fork the code and make their own changes, but I'm sure also if someone did something like that with nefarious purposes it would be found out pretty quickly and that node could probably be blocked from interacting with the rest of the nodes.  Now I say that without knowing for sure, but that would be my guess.
member
Activity: 112
Merit: 10
Forget it. Won't work. It must & will stay Open Source, like Bitcoin itself.
Please point out anywhere in my post where I said it should be closed source?  That is nowhere in the intent of my thoughts.

Removing it from being controlled by forrestv, yes.  That would be "forking it".  That does not mean we take the fork and close it up - that would probably be impossible if forrestv setup the licensing properly in the first place anyways.
legendary
Activity: 1258
Merit: 1027
Wish I had a solution.
Personally, I don't think a solution is all that hard, it just requires a little press advocacy and some determination by a small handful of folks.  I also think that the ball has started rolling in that direction, thanks to windpath's efforts to date, and if momentum can be kept up and others pitch in to help out, it should be fine.  To that end, here's what my solution suggestion would be:

1) Get the project away from reliance on forrestv, which is already happening.  Again, nothing personal, but if he's not committed with or without donations, having it rest with one single person is detrimental, and as we have seen stunts growth horribly.
2) Re-work the p2pool project pages, sites, and get the important info updated - how to get it, how to set it up, where to go for answers, list of nodes and peers - need to be cleaned up and _maintained_ so that it's relevant.  This at least in part is happening, a-la windpath.
3) The hardest and probably most controversial - change the mindset of the pool from "open, 0%" to "proprietor-owned (note I didn't say proprietary, slight but important semantic difference) with minimal fee (1-1.5%)".

#3 I think is the answer to address the major problems facing the pool.  Open source and free will only take any project to a certain level, as long as people are willing to donate to foster a better cause.  I don't think this works in the BTC world, since we're all doing this for one reason - to mine BTC and make money, not to cure cancer or find alien life.

What's the biggest difference between proprietary pools and p2pool?  Proprietary pools have an entity (person, business, etc) behind them driving their pool to be better, because it attracts new miners and ultimately helps their bottom line.  That's the fundamental "problem" with p2pool - the ideology is great, but call a spade a spade in this case and admit the reality that without reward, you're not going to get a lot of people to do things to help beyond some forum posts.

I would suggest the only way to have this work is treat it like a true O.S. project - form a core group of members as caretakers of p2pool (you can easily see who could be a candidate by their actions).  Then change the code to remove the .5% donation to forrestv and replace with a 1% donation to an escrowed account overseen by the core group.  From this 1% amount, you could do 3 things that could ensure the ongoing health of p2pool - 1) form bounties for developers to tackle the coding of fixes big and small, 2) reward public pool node/peer operators for providing their time, services and resources maintaining high quality nodes that all miners can use and finally 3) re-distribute the leftover percentage directly to all the miners using the built-in reward/tip system already existing in p2pool.  This will provide a mechanism for properly funded development, and rewards to all who are doing their part to use p2pool and help it grow and flourish.


When we launch the front end on Github I have been considering forking the whole package.

I would love some feedback as to the suggestions made above.

I have been considering some other options to help raise funds for ongoing development.

Frankly, while my own desire for compensation is high, the benefits of being an open source and 0% fee pool are hard to ignore.

The primary Coin Cadence front-end that you see now on my site requires a lot of data that is simply not available to a regular p2pool node without some major changes that increase resources required by the server to the point where it would effect efficiency for most mining on their own nodes.

To solve this I will be providing an API that will allow nodes to pull the data they don't have access to from my servers.

To offset the costs of the API I was considering offering 2 options (miners choice):

1. Include paid ads in the data returned by the API. p2pool inherently attracts a highly targeted group of individuals that spend money on mining gear, it would be a manufacturers wet dream to "sponsor" the pool API with their message. If the front-end has a high adoption rate by you guys I believe this alone may raise enough to fund future development.

2. Pay a fee (annual?) to have an ad free version

None of this is set in stone, in fact I would love to hear feedback and other suggestions.

Once I've got p2pool.org re-launched and have released the beta of the front end on Github I plan to try and tackle the variance for small miners issue. It is not an easy problem to solve. I'm not sure if I will go the bounty route, or hire a full time python developer with some Bitcoin chops (if I can find one)... But for now focusing on the above.



newbie
Activity: 21
Merit: 0
I agree with everything you said KGB. But given its open source what stops someone from stripping out the mandatory fee and gaining the benefits without paying.

The share chain solution is that if the majority of p2pool are willing to play by these rules and reject freeloading shares that don't include the development fee then it would be enforced. That would be a hard fork so would need managing.

Will the majority of miners play ball though?
hero member
Activity: 686
Merit: 500
WANTED: Active dev to fix & re-write p2pool in C
Forget it. Won't work. It must & will stay Open Source, like Bitcoin itself.
Jump to: