[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 180.

Geremia

sr. member

Activity: 507

Merit: 253

Quote from: windpath on June 06, 2015, 02:51:00 PM

Code:

-- Host: localhost:3306
-- Generation Time: Jun 06, 2015 at 11:56 AM
-- Server version: 5.5.36
-- PHP Version: 5.4.28

SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";

--
-- Database: `p2pool`
--

-- --------------------------------------------------------

--
-- Table structure for table `block_payout`
--

CREATE TABLE IF NOT EXISTS `block_payout` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `block_id` int(11) NOT NULL,
  `address` varchar(34) NOT NULL,
  `ammount` decimal(10,8) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `address` (`address`),
  KEY `block_id` (`block_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=587605 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_blocks`
--

CREATE TABLE IF NOT EXISTS `found_blocks` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `share` varchar(15) NOT NULL,
  `time` datetime NOT NULL,
  `hash` varchar(64) NOT NULL,
  `height` int(8) DEFAULT NULL,
  `orphan` tinyint(1) NOT NULL DEFAULT '0',
  `luck` decimal(8,2) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `height` (`height`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2555 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_shares`
--

CREATE TABLE IF NOT EXISTS `found_shares` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `address` varchar(34) NOT NULL,
  `share` varchar(15) NOT NULL,
  `time` datetime NOT NULL,
  `doa` tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=17889 ;

-- --------------------------------------------------------

--
-- Table structure for table `pool_stats`
--

CREATE TABLE IF NOT EXISTS `pool_stats` (
  `id` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `global_rate` varchar(72) NOT NULL,
  `global_nonstale_rate` varchar(20) NOT NULL,
  `diff` varchar(20) NOT NULL,
  `miners` int(7) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Do you have all this and supporting code up on GitHub or somewhere else? If not you should put it all up.

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: windpath on June 06, 2015, 02:51:00 PM

This will be a high level explanation of how my node works, it's not an exhaustive tutorial...

So here goes...

I built this over a year ago (when p2pool.info died) in a bit if a rush, and there are certainly things I would change today to make it more efficient....

It runs an an AWS LAMP server, and pulls data from both bitcoind and p2pool. Everything is gathered and stored locally on the server, there are no external API's.

The first thing you will need to do (in addition to the existing p2pool setup) is add

Code:

txindex=1

to your bitcoin.conf, restart it, and force a re-scan. This will force a lengthy re-index, and makes the data of all transactions available from the bitcoin API. (note: if you do this on an operating node be prepared for some extended downtime during the re-index).

Next I set up the DB (MySQL), again it was quick and dirty, and this DB is specifically focused on p2pool:

Code:

-- Host: localhost:3306
-- Generation Time: Jun 06, 2015 at 11:56 AM
-- Server version: 5.5.36
-- PHP Version: 5.4.28

SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";

--
-- Database: `p2pool`
--

-- --------------------------------------------------------

--
-- Table structure for table `block_payout`
--

CREATE TABLE IF NOT EXISTS `block_payout` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`block_id` int(11) NOT NULL,
`address` varchar(34) NOT NULL,
`ammount` decimal(10,8) NOT NULL,
PRIMARY KEY (`id`),
KEY `address` (`address`),
KEY `block_id` (`block_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=587605 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_blocks`
--

CREATE TABLE IF NOT EXISTS `found_blocks` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`share` varchar(15) NOT NULL,
`time` datetime NOT NULL,
`hash` varchar(64) NOT NULL,
`height` int(8) DEFAULT NULL,
`orphan` tinyint(1) NOT NULL DEFAULT '0',
`luck` decimal(8,2) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `height` (`height`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2555 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_shares`
--

CREATE TABLE IF NOT EXISTS `found_shares` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`address` varchar(34) NOT NULL,
`share` varchar(15) NOT NULL,
`time` datetime NOT NULL,
`doa` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=17889 ;

-- --------------------------------------------------------

--
-- Table structure for table `pool_stats`
--

CREATE TABLE IF NOT EXISTS `pool_stats` (
`id` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_rate` varchar(72) NOT NULL,
`global_nonstale_rate` varchar(20) NOT NULL,
`diff` varchar(20) NOT NULL,
`miners` int(7) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

So some quick explanation of the DB structure:

3 things not to overlook: charset = UTF8, SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO", and SET time_zone = "+00:00".

The first 2 are self explanatory, the timezone must be set to UTC as that is what bitcoin uses globally in the block chain, was going to add a way for miners to set their local timezone in the views, but never got around to it.

Table block_payout contains the payout data by public address for every block P2Pool has found, it facilitates displaying total as well as per block payouts for miners (example). Originally I wrote a script to go over all the historical blocks and let it run, now it pulls them as we find blocks.

Table found_blocks has every block P2Pool has found, the p2pool share that found it, whether it was orphaned, and its luck. Again, added historical ones from a few sources provided by someone probably 100 pages back in this thread, new ones are added as they are found.

Table found_shares is specific to my node (I hated loosing all the shares on a re-start), so if you have ever found a share mining on my node, it's in there (example on miner shares tab).

Table pool_stats stores the bitcoin difficulty from bitcoind and pool hashrate from p2pool every minute, this is how luck is calculated - and why I could not calculate luck for historical blocks when I built it (no one had historical pool hashrate data).

The rest of the back end is basically 2 cron files, 1 that polls p2pool's log file and another that polls bitcoind. Both run once a minute, pull new data, check for orphans, calculate luck for new blocks, and store it in the DB.

The cron also renames the p2pool log file and saves it as a date stamped archive every 10MB, this greatly reduces memory usage when reading the log into memory.

Originally was adding blocks by searching the log for "GOT BLOCK", however this missed any stale shares that happened to find a block (and there are quite a few), so now I monitor a p2pool miner payout address for generation transactions as well (look out for donations!).

The front end is an ugly (code wise) mashup of p2pool's JS front ends and PHP running on the server. I used Bootstrap, a lot of cut and paste hacking, and some PHP to query the DB, calculate everything for the view, and display it.

In a nut shell, that's it.

All of this was done pretty openly in this thread with a lot of help from the community, a lot of the code logic is buried in this thread if you want to take a look at it.

If you give it a try and have any specific questions I'd be happy to try and help out.

prolly too long to reply here. u got pm & hopefully we can work from there.

whereistheblock ? i've bumped pool hashrate up earlier, i think it went over 3+ ph.

any other coins worth to merge ?

windpath

legendary

Activity: 1258

Merit: 1027

This will be a high level explanation of how my node works, it's not an exhaustive tutorial...

So here goes...

I built this over a year ago (when p2pool.info died) in a bit if a rush, and there are certainly things I would change today to make it more efficient....

It runs an an AWS LAMP server, and pulls data from both bitcoind and p2pool. Everything is gathered and stored locally on the server, there are no external API's.

The first thing you will need to do (in addition to the existing p2pool setup) is add

Code:

txindex=1

to your bitcoin.conf, restart it, and force a re-scan. This will force a lengthy re-index, and makes the data of all transactions available from the bitcoin API. (note: if you do this on an operating node be prepared for some extended downtime during the re-index).

Next I set up the DB (MySQL), again it was quick and dirty, and this DB is specifically focused on p2pool:

Code:

-- Host: localhost:3306
-- Generation Time: Jun 06, 2015 at 11:56 AM
-- Server version: 5.5.36
-- PHP Version: 5.4.28

SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";

--
-- Database: `p2pool`
--

-- --------------------------------------------------------

--
-- Table structure for table `block_payout`
--

CREATE TABLE IF NOT EXISTS `block_payout` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `block_id` int(11) NOT NULL,
  `address` varchar(34) NOT NULL,
  `ammount` decimal(10,8) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `address` (`address`),
  KEY `block_id` (`block_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=587605 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_blocks`
--

CREATE TABLE IF NOT EXISTS `found_blocks` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `share` varchar(15) NOT NULL,
  `time` datetime NOT NULL,
  `hash` varchar(64) NOT NULL,
  `height` int(8) DEFAULT NULL,
  `orphan` tinyint(1) NOT NULL DEFAULT '0',
  `luck` decimal(8,2) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `height` (`height`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=2555 ;

-- --------------------------------------------------------

--
-- Table structure for table `found_shares`
--

CREATE TABLE IF NOT EXISTS `found_shares` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `address` varchar(34) NOT NULL,
  `share` varchar(15) NOT NULL,
  `time` datetime NOT NULL,
  `doa` tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=17889 ;

-- --------------------------------------------------------

--
-- Table structure for table `pool_stats`
--

CREATE TABLE IF NOT EXISTS `pool_stats` (
  `id` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `global_rate` varchar(72) NOT NULL,
  `global_nonstale_rate` varchar(20) NOT NULL,
  `diff` varchar(20) NOT NULL,
  `miners` int(7) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

So some quick explanation of the DB structure:

3 things not to overlook: charset = UTF8, SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO", and SET time_zone = "+00:00".

The first 2 are self explanatory, the timezone must be set to UTC as that is what bitcoin uses globally in the block chain, was going to add a way for miners to set their local timezone in the views, but never got around to it.

Table block_payout contains the payout data by public address for every block P2Pool has found, it facilitates displaying total as well as per block payouts for miners (example). Originally I wrote a script to go over all the historical blocks and let it run, now it pulls them as we find blocks.

Table found_blocks has every block P2Pool has found, the p2pool share that found it, whether it was orphaned, and its luck. Again, added historical ones from a few sources provided by someone probably 100 pages back in this thread, new ones are added as they are found.

Table found_shares is specific to my node (I hated loosing all the shares on a re-start), so if you have ever found a share mining on my node, it's in there (example on miner shares tab).

Table pool_stats stores the bitcoin difficulty from bitcoind and pool hashrate from p2pool every minute, this is how luck is calculated - and why I could not calculate luck for historical blocks when I built it (no one had historical pool hashrate data).

The rest of the back end is basically 2 cron files, 1 that polls p2pool's log file and another that polls bitcoind. Both run once a minute, pull new data, check for orphans, calculate luck for new blocks, and store it in the DB.

The cron also renames the p2pool log file and saves it as a date stamped archive every 10MB, this greatly reduces memory usage when reading the log into memory.

Originally was adding blocks by searching the log for "GOT BLOCK", however this missed any stale shares that happened to find a block (and there are quite a few), so now I monitor a p2pool miner payout address for generation transactions as well (look out for donations!).

The front end is an ugly (code wise) mashup of p2pool's JS front ends and PHP running on the server. I used Bootstrap, a lot of cut and paste hacking, and some PHP to query the DB, calculate everything for the view, and display it.

In a nut shell, that's it.

All of this was done pretty openly in this thread with a lot of help from the community, a lot of the code logic is buried in this thread if you want to take a look at it.

If you give it a try and have any specific questions I'd be happy to try and help out.

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: p3yot33at3r on June 06, 2015, 08:14:29 AM

Yep, double checked all passwords & ports - although there's no need to forward the rpc ports as they are only used locally on 127.0.0.1 for merge mining. None of my rpc ports are forwarded on any other wallets & they work fine - it's only the UNO wallet that seems to be having issues. I'm compiling the QT wallet now.....

EDIT: QT compiled & running, I'll keep an eye on it for payments. In the meantime, if anyone else using the daemon hasn't received any payments - please post here, thanks.

Update: Well, I just received my first payment using the QT wallet, so now I know for sure there's no problem with my settings. I'm gonna switch back to using the daemon to see if the payments stop again.....

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: windpath on June 06, 2015, 10:21:11 AM

Quote from: yslyung on June 06, 2015, 10:11:48 AM

any guide ? want to add that to my node if possible.

and wut did i said few mins earlier a block ? yeah !!!

BITCOIN BLOCK FOUND by 1GEvGnXQbALcjvw6Ed4Xek6f3wyCgXQRfb! https://blockchain.info/block/000000000000000007eff6dfc2ed5478164b2627917b96d494f800175b53abcc

Unfortunately not, I'll spend some time this afternoon and describe the setup, it's not overly complex...

thx very much, looking forward to it !

windpath

legendary

Activity: 1258

Merit: 1027

Quote from: yslyung on June 06, 2015, 10:11:48 AM

any guide ? want to add that to my node if possible.

and wut did i said few mins earlier a block ? yeah !!!

BITCOIN BLOCK FOUND by 1GEvGnXQbALcjvw6Ed4Xek6f3wyCgXQRfb! https://blockchain.info/block/000000000000000007eff6dfc2ed5478164b2627917b96d494f800175b53abcc

Unfortunately not, I'll spend some time this afternoon and describe the setup, it's not overly complex...

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: windpath on June 06, 2015, 10:02:35 AM

Quote from: yslyung on June 06, 2015, 09:30:28 AM

btw windpath ... how do i add this to my node ?

thx ...

and also i can't seem to add or show node efficiency ...

feel a block is coming soon Cool

I poll bitcoin core and p2pool nodes, store it in a DB, and calculate it....

any guide ? want to add that to my node if possible.

and wut did i said few mins earlier a block ? yeah !!!

BITCOIN BLOCK FOUND by 1GEvGnXQbALcjvw6Ed4Xek6f3wyCgXQRfb! https://blockchain.info/block/000000000000000007eff6dfc2ed5478164b2627917b96d494f800175b53abcc

windpath

legendary

Activity: 1258

Merit: 1027

Quote from: yslyung on June 06, 2015, 09:30:28 AM

btw windpath ... how do i add this to my node ?

thx ...

and also i can't seem to add or show node efficiency ...

feel a block is coming soon Cool

I poll bitcoin core and p2pool nodes, store it in a DB, and calculate it....

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

btw windpath ... how do i add this to my node ?

thx ...

and also i can't seem to add or show node efficiency ...

feel a block is coming soon Cool

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: p3yot33at3r on June 06, 2015, 08:14:29 AM

Yep, double checked all passwords & ports - although there's no need to forward the rpc ports as they are only used locally on 127.0.0.1 for merge mining. None of my rpc ports are forwarded on any other wallets & they work fine - it's only the UNO wallet that seems to be having issues. I'm compiling the QT wallet now.....

EDIT: QT compiled & running, I'll keep an eye on it for payments. In the meantime, if anyone else using the daemon hasn't received any payments - please post here, thanks.

thx for headsup again ... i'm on windows as i'm a linux noob .... so far so good for me ... 1 block per hour sometimes more ...

keep us updated

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Yep, double checked all passwords & ports - although there's no need to forward the rpc ports as they are only used locally on 127.0.0.1 for merge mining. None of my rpc ports are forwarded on any other wallets & they work fine - it's only the UNO wallet that seems to be having issues. I'm compiling the QT wallet now.....

EDIT: QT compiled & running, I'll keep an eye on it for payments. In the meantime, if anyone else using the daemon hasn't received any payments - please post here, thanks.

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Hmm it should work though.

I port fowarded both 65534/65535. Use 65535 on the merged cmd line.

Your conf ?

Merged cmd line? Used the correct rpcuser, pwd & port?

Yeah with 10ths you should see something.

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: yslyung on June 06, 2015, 07:17:13 AM

Quote from: p3yot33at3r on June 06, 2015, 05:55:15 AM

Are those payments from merge mining? I've been using the daemon for a few days now but didn't receive any payments as of yet.....

Yes. You can tell from the pick logo & it says mined.

You sure you got the merge setup done correctly? Especially port fowarding & restarting the wallet once each step is completed. You might want to restart your p2p node too.

It's solo mining so a little bit of luck is needed.

How much hashpower does your node have?

Hmmm....everything seems in order:

Port 65534 is forwarded on my router, allowed on the ufw firewall, upnp is also activated on the router & daemon & is showing on my router logs. I've got no errors in my p2pool logs either - looks like there's a problem with the UNO daemon. I'll SSH into my rig, build the QT wallet & run with that for a while - that should give me a clue as to if the daemon is working properly or not. I'm hashing @ ~10TH

Has anyone else who is using the UNO daemon received any payments? If not, I'll contact FallingKnife & let him know there might be a bug.

EDIT: I've been running the daemon for about a week now, so should have received a few payments according to your wallet. I never installed QT as it's not needed for the daemons, so I'll have to do that first...... Sad

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: p3yot33at3r on June 06, 2015, 05:55:15 AM

Are those payments from merge mining? I've been using the daemon for a few days now but didn't receive any payments as of yet.....

Yes. You can tell from the pick logo & it says mined.

You sure you got the merge setup done correctly? Especially port fowarding & restarting the wallet once each step is completed. You might want to restart your p2p node too.

It's solo mining so a little bit of luck is needed.

How much hashpower does your node have?

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Are those payments from merge mining? I've been using the daemon for a few days now but didn't receive any payments as of yet.....

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

merged uno worked & i did the following:

1. goto uno official site & download official wallet http://unobtanium.uno/ *i'm using windows qt, just like nmc*

2. make a uno .conf file, guide at uno thread https://bitcointalksearch.org/topic/m.11422814

3. port forward ports, port to be used in merged cmd is 65535

4. let wallet sync - do remember to properly shutdown wallet & restart it if & when changes are made at .cof file or after port forward

5. add in merge cmd line for p2pool & restart pool

should be mining away ...

screenshot:

as usual, any questions, ask away.

btw, another block & keep 'em rolling in !

BITCOIN BLOCK FOUND by 1DHzUMUmiSrEzsKtpy3RMRTZhjEL475ZJp! https://blockchain.info/block/000000000000000013c43866e368a229f8e365c1db9927d18a43231c0b40dca8

yslyung

legendary

Activity: 1500

Merit: 1002

Mine Mine Mine

Quote from: p3yot33at3r on June 05, 2015, 11:33:44 AM

Quote from: FallingKnife on June 05, 2015, 11:09:12 AM

Thanks for mining Unobtanium. mm was released last month, and hash is still pretty low for Uno. Great time to get into it. The Uno community thanks you.

Hi FK,

Thank you for making UNO available to p2pool merge miners Wink

It's a good coin with a great community - always nice to see a dev who contributes to the community.....

I'm currently working on an updated p2pool set-up guide for noobs, as previous ones are now a little outdated - git repo's especially, and will include UNO & all other available merge mined coins in an effort to attract more miners to p2pool, as well as increase UNO hash rate

thx for the contribution ! looking fwd to the updated versions.

links & a quickie fo rUNO will be a good start for existing pool operators ?

i'm trying while waiting, hope i got it right ... using the same way as in other merge mining.

p3yot33at3r

sr. member

Activity: 266

Merit: 250

Quote from: FallingKnife on June 05, 2015, 11:09:12 AM

Thanks for mining Unobtanium. mm was released last month, and hash is still pretty low for Uno. Great time to get into it. The Uno community thanks you.

Hi FK,

Thank you for making UNO available to p2pool merge miners Wink

It's a good coin with a great community - always nice to see a dev who contributes to the community.....

I'm currently working on an updated p2pool set-up guide for noobs, as previous ones are now a little outdated - git repo's especially, and will include UNO & all other available merge mined coins in an effort to attract more miners to p2pool, as well as increase UNO hash rate

FallingKnife

legendary

Activity: 2450

Merit: 1076

keybase.io/fallingknife/

Quote from: p3yot33at3r on June 01, 2015, 06:14:50 AM

Last time the dev logged in here was Oct 2014, which might explain why it's so difficult finding answers....

Am I the only one here trying to run p2pool with pypy? Google has proven fruitless.... Cry

BTW, here's a list of ports I'm using for the merge mined coins:

coin / port / rpc
...
Unobtanium: 65534/65535
.,..

Thanks for mining Unobtanium. mm was released last month, and hash is still pretty low for Uno. Great time to get into it. The Uno community thanks you.

Geremia

sr. member

Activity: 507

Merit: 253

Quote from: jcumins on June 04, 2015, 06:40:22 PM

There great

They're.

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 180. (Read 2591920 times)