Pages:
Author

Topic: How to build your own Multipool - the Open Source Way - page 7. (Read 35477 times)

newbie
Activity: 6
Merit: 0
These are probably noob questions, but here goes anyways.

How much machine would this require (RAM, storage)?

What is the minimum hashrate you would recommend for before setting up your own Multipool?
legendary
Activity: 2002
Merit: 1051
ICO? Not even once.
lol you guy are dinks for stealting the first posts

There's an Edit button for a reason.
sr. member
Activity: 700
Merit: 250
Yep, I got lost at

Quote
Step #1 - start with a basic NOMP install. git clone from the NOMP repo: https://github.com/zone117x/node-open-mining-portal.git

Haha
sr. member
Activity: 378
Merit: 254
small fry
lol you guy are dinks for stealting the first posts
sr. member
Activity: 378
Merit: 254
small fry
How NOMP uses REDIS, and how we can use it to get what we want (and improve it)
All coin are given a redis key named - this breaks down into subkeys, :blocksCompleted, blocksPending, blocksOrphaned, blocksKicked
It also has a :balances key that keeps track of each workers current earnings per key, and a :payouts (which keeps track of the number of coins paid out to each user).

If we set up the coin configs right, we will never ever see a :payouts key in our install.
We are also going to create a new subkey, :blocksPaid that we are going to be moving every paid for round into, every time we run payouts.

Now the only stats that NOMP keeps are brutal.

We are going to implement a whole new level of logging into redis.

A master key, Pool_Stats contains everything.
We also have a kew named Coin_Names that contains (lowercase) a list of every coin we support.
We have Coin_Algos that lists every algo we support (lowercase)
We have Coin_Names_ that lists every coin of that algo.  These are all stored as hashes, by the way, with values of 1 (although the value doesn't matter, these are all accessed via the redis-command HVALS)

ie:
Coin_Names consists of two hashes:
feathercoin 1
terracoin 1

Coin_Algos consists of two hashes:
scrypt 1
sha256 1

Coin_Names_scrypt consists of 1 hash
feathercoin 1
Coin_Name_sha256 consists of 1 hash:
terracoin 1

Set these up like this:
 root@blackcoinpool:/: redis-cli hset Coin_Names feathercoin 1
 root@blackcoinpool:/: redis-cli hset Coin_Algos scrypt 1
 root@blackcoinpool:/: redis-cli hset Coin_Names_scrypt feathercoin 1
and so on for all of your coins/algos....

We are going to keep track of every users historical hashrate in a key named Pool_Stats:WorkerHRS:: as part of a redis SORTED SET.  The format for the sorted set will be to use epoch time as the score and the value will be set to: : -  the epoch time is requierd to allow redis to store duplicate hashrates (as it makes them all unique).
Pool_Stats will have seperate subkeys for every shift, but all of that will get created automatically through the next series of scripts I will provide.

We will keep CurrentShift datat in Pool_Stats:CurrentShift and current shift profitability data in Pool_Stats:CurrentShift:Profitability
Inside CurrentShift we will have:

Pool_Stats:CurrentShift:AlgosBTC - hash field listing each algos total value, in BTC, of each algo mined so far this shift.
Pool_Stats:CurrentShift:AlgosTgtCoin - hash field listing each algos total value, in target coin, of each algo mined so far this this shift.
Pool_Stats:CurrentShift:CoinsBTC - hash field listing each coins  total BTC that have been earned so far this shift.
Pool_Stats:CurrentShift:CoinsTgtCoin - hash field listing each coins total target coins that have been earned so far this shift.
Pool_Stats:CurrentShift:WorkersBTC - hash field listing each workers  total BTC they have earned so far this shift
Pool_Stats:CurrentShift:WorkersTgtCoin - hash field listing each workers  total target coin they have earned so far this shift.

We will keep track of historical stats in:
Pool_Stats:Profitability_ - set of profitabilities, using shift number as field and profitability a value.
Pool_Stats:WorkerHRs:: - sorted set list of hashrates, per worker, per algo.  Epoch time as field, value is hashrate:epochtime to ensure uniqueness.
Pool_Stats:Balances - outstanding balances in target coin
Pool_Stats:DetailedPayouts: - sorted set list of payouts, using epoch time as field and value as txn ids in target coin.
Pool_Stats:DetailedPayouts:: - Hash key named Date: with value as epoch time, hash key as Amount: and value as txn amount, hash key as URL and value set to full URL for txn in target coin's block explorer.
Pool_Stats:TotalPaid - hash key listing every worker ID as a key name and the total amount of target coins they have been paid in total as the value.

.

These values are all automatically calculated by the payment processor, which is coming up in a few posts - the reason for the duplicate storage (in both BTC and TgtCoin is it provides a level of sanity checking for the payment processor before it decides to pay out the shift automatically or not).


Pool_Stats:CurrentShift will always have a starttime set to the epoch time that shift started at.
We will have an incrementer named This_Shift in Pool_Stats (a hash value) that will be incremented by one whenver a shift ends.  
Whenever a shift end, we will rename all of the Pool_Stats:CurrentShift keys to be Pool_Stats: instead, and then start a fresh Pool_Stats:CurrentShift set of keys.


Let's start off with calculating and storing the users hashrates into the redis db.

hero member
Activity: 602
Merit: 500
that's great! Looking forward to your guide.
sr. member
Activity: 378
Merit: 254
small fry
As well as a miner_stat.html

Code:
                

                                               

{{? it.stats.balances }}

                               

                               

                                                       

                           

{{=it.stats.address}}


                           

                               

So far this shift you have earned: < (estimate).



                               

Your previous balance with the pool is currently:                              


                       

               

                               

                       




                                               

                               

                       
 

                       

                       

                               

                               




   
       


       
       


                   

                           


                       

                                                              {{ for(var balance in it.stats.balances) { }}
                                                                               

                                                                                 
{{=it.stats.balances[balance].coin}}

                                                                               
{{=it.stats.balances[balance].balance}}  

                                                                               

                                              {{ } }}
                                                                               

                       



               




       
       


{{?}}



next install Webdis from here https://github.com/nicolasff/webdis
and launch it with a .json config file like this:
Code:
root@blackcoinpool:/FORFREEDOM/# cat ~/webdis-home/webdis.json
{
        "redis_host":   "127.0.0.1",

        "redis_port":   6379,
        "redis_auth":   null,

        "http_host":    "0.0.0.0",
        "http_port":    7379,

        "threads":      5,
        "pool_size": 20,

        "daemonize":    true,
        "websockets":   true,
        "default_root": "/GET/index.html",
        "database":     0,

        "acl": [
                {
                        "disabled":     ["*"]
                },

                {
                        "enabled":              ["GET", "HGET", "ZRANGE", "ZCARD", "ZRANGEBYSCORE", "ZVAL", "LRANGE", "HGETALL", "HKEYS", "HVALS", "GETALL", "HGETALL", "ZRANGE", "SMEMBERS", "ZSCORE"]
                }
        ],

        "verbosity": 6,
        "logfile": "/root/webdis-home/webdis.log"
}

Next, you should be able to restart your nomp install and browse to /miner and input your worker address and see all the coin that worker has mined that shift.  the balance and outstanding amount will be blank till.  you can manually browe to /payout/ to see an estimated next payout.

Next up, let's do a deep dive into redis and how NOMP uses it (as well a re-write the entire payment processor)
sr. member
Activity: 378
Merit: 254
small fry
Next up, we are going to extend the stats.js file to extend the API to have some new functionality. This is going to be a long post.

In the /libs/ directory, rename stats.js entirely to stats.old
Open up a new stats.js and paste the following three snippets - modify the SECOND sections.  If your target coin is only on mintpal, uncomment the lines calling the mintpal API and fill in your ticker symbol.  If your coin is on cryptsy, fill in the appropriate market ID and the Cryptsy ticker symbol.  This file will extend your API in a few ways.

Most importantly, it will make /api/payout/ return the estimated number of coins a worker has earned during the current shift (in the payout coin of your choice).
It will also extend the 'my miner' page so that every worker can have a complete list of exactly how many coins of what type they have earned during that current shift.

Code:
var zlib = require('zlib');

var redis = require('redis');
var async = require('async');
var request = require('request');

var os = require('os');

var algos = require('stratum-pool/lib/algoProperties.js');


module.exports = function(logger, portalConfig, poolConfigs){

    var _this = this;

    var logSystem = 'Stats';

    var redisClients = [];
    var redisStats;

    this.statHistory = [];
    this.statPoolHistory = [];

    this.stats = {};
    this.statsString = '';

    setupStatsRedis();
    gatherStatHistory();

    var canDoStats = true;

    Object.keys(poolConfigs).forEach(function(coin){

        if (!canDoStats) return;

        var poolConfig = poolConfigs[coin];

        var redisConfig = poolConfig.redis;

        for (var i = 0; i < redisClients.length; i++){
            var client = redisClients[i];
            if (client.client.port === redisConfig.port && client.client.host === redisConfig.host){
                client.coins.push(coin);
                return;
            }
        }
        redisClients.push({
            coins: [coin],
            client: redis.createClient(redisConfig.port, redisConfig.host)
        });
    });


    function setupStatsRedis(){
        redisStats = redis.createClient(portalConfig.redis.port, portalConfig.redis.host);
        redisStats.on('error', function(err){
            logger.error(logSystem, 'Historics', 'Redis for stats had an error ' + JSON.stringify(err));
        });
    }

    function gatherStatHistory(){

        var retentionTime = (((Date.now() / 1000) - portalConfig.website.stats.historicalRetention) | 0).toString();

        redisStats.zrangebyscore(['statHistory', retentionTime, '+inf'], function(err, replies){
            if (err) {
                logger.error(logSystem, 'Historics', 'Error when trying to grab historical stats ' + JSON.stringify(err));
                return;
            }
            for (var i = 0; i < replies.length; i++){
                _this.statHistory.push(JSON.parse(replies[i]));
            }
            _this.statHistory = _this.statHistory.sort(function(a, b){
                return a.time - b.time;
            });
            _this.statHistory.forEach(function(stats){
                addStatPoolHistory(stats);
            });
        });
    }

    function addStatPoolHistory(stats){
        var data = {
            time: stats.time,
            pools: {}
        };
        for (var pool in stats.pools){
            data.pools[pool] = {
                hashrate: stats.pools[pool].hashrate,
                workerCount: stats.pools[pool].workerCount,
                blocks: stats.pools[pool].blocks
            }
        }
        _this.statPoolHistory.push(data);
    }


this.getCoins = function(cback){
        _this.stats.coins = redisClients[0].coins;
        cback();
    };

    this.getPayout = function(address, cback){

        async.waterfall([

            function(callback){

                _this.getBalanceByAddress(address, function(){

                    callback(null, 'test');
                });

            },

            function(msg, callback){

                var totaltargetcoin = 0;

                async.each(_this.stats.balances, function(balance, cb){

                    _this.getCoinTotals(balance.coin, balance.balance, function(targetcoin){

                        if(typeof(targetcoin) != "undefined"){
                            totaltargetcoin += targetcoin;
                        }

                        cb();
                    });

                }, function(err){
                    callback(null, totaltargetcoin);
                });
            }

        ], function(err, total){

            cback(total.toFixed());

        });
    };


    this.getBalanceByAddress = function(address, cback){

        var client = redisClients[0].client,
            coins = redisClients[0].coins,
            balances = [];
          payouts = [];



                    client.hgetall('Payouts:' + address, function(error, txns){
                                                                         //logger.error(logSystem, 'TEMP', 'txnid variable is:' + txnid);

                                                                        if (error) {
                                                                                callback ('There was no payouts found');
                                                                                return;
                                                                        }
                                                                        if(txns === null){
                                                                               var index = [];
                                                                               } else{
                                                                               payouts=txns;

                                                                               }

                                                                        });


        async.each(coins, function(coin, cb){

            client.hget(coin + ':balances', address, function(error, result){
                if (error){
                    callback('There was an error getting balances');
                    return;
                }

                if(result === null) {
                    result = 0;
                }else{
                    result = result;
                }

                balances.push({
                    coin:coin,
                    balance:result
                });

                cb();
            });

        }, function(err){
            _this.stats.balances = balances;
            _this.stats.address = address;

 
            cback();
        });
    };

this.getCoinTotals = function(coin, balance, cback){
        var client = redisClients[0].client,
        coinData = poolConfigs[coin].coin;
       logger.error(logSystem, 'TEMP', 'var is' + JSON.stringify(poolConfigs[coin].coin));
         //logger.error(logSystem, 'TEMP', 'coinData.ID variable is:' + coinData.ID);

        async.waterfall([

  // Get all balances from redis if no balance was provided already
            function(callback){

                if(balance) {
                    callback(null, balance);
                    return;
                }

                client.hgetall(coin + ':balances', function(error, results){
                    if (error){
                        callback('There was an error getting balances');
                        return;
                    }

                    callback(null, results);
                });
            },
THIS NEXT PART OF THE FILE YOU NEED TO MAKE SOME CHANGES TO - this is a continuation of the file above though
Code:
            // make a call to Mintpal to get targetcoin exchange rate
            function(balances_results, callback){
                var options = {
                  // url:'https://api.mintpal.com/market/stats//BTC',
                url:'http://pubapi.cryptsy.com/api.php?method=singlemarketdata&marketid=',
                json:true
                }

                request(options, function (error, response, body) {
                  if (!error && response.statusCode == 200) {
                   // var targetcoin_price = parseFloat(body[0].last_price);
                        var targetcoin_price =  body['return'].markets[''].lasttradeprice;
                    callback(null, targetcoin_price, balances_results);

                  } else {
                    callback('There was an error getting mintpal targetcoin exchange rate');
                  }
                });
            },

The rest of the stats.js is below - just paste all three of these into the same file, remembering in to fill in your info into the second part.
Code:

            // make call to get coin's exchange rate
            function(targetcoin_price, balances_results, callback){


               // logger.error(logSystem, 'TEMP', '#1 ---- coinData.ID variable is:' + coinData.ID);

                if(coinData.ID === 'mintpal') {

                        var optionsB = {
                   url:'https://api.mintpal.com/market/stats/' + coinData.symbol + '/BTC',
                json:true
                }

                request(optionsB, function (error, responseB, bodyB) {
            
                  if (!error && responseB.statusCode == 200) {
                   var coinB_price = parseFloat(bodyB[0].last_price);
                        logger.error(logSystem, 'TEMP', 'coinB_price variable is:' + coinB_price);

                    callback(null, targetcoin_price, coinB_price, balances_results);

                  } else {
                    callback('There was an error getting mintpal exchange rate');
                  }
                });

                } else if (coinData.ID) {

                    var options = {
                        url:'http://pubapi.cryptsy.com/api.php?method=singlemarketdata&marketid=' + coinData.ID,
                        json:true
                    }

                    request(options, function (error, response, body) {
                      if (!error && response.statusCode == 200) {
                        var coin_price = body['return'].markets[coinData.symbol].lasttradeprice;

                        /*
                        if(coin_price.toString().indexOf('-') === -1) {
                            // Good it doesn't have a dash.. no need to convert it to a fixed number
                        }
                        else {
                            var decimal_places = coin_price.toString().split('-')[1];
                            coin_price = coin_price.toFixed(parseInt(decimal_places));
                        }
                        */

                        callback(null, targetcoin_price, coin_price, balances_results);

                      } else {
                        callback('There was an error getting mintpal targetcoin exchange rate');
                      }
                    });
                }
                else {
                    callback(null, targetcoin_price, coinData.rate, balances_results);
                }
            },

            // Calculate the amount of targetcoin earned from the worker's balance
            function(targetcoin_price, coin_price, balances_results, callback){

                if(typeof balances_results !== 'object') {
                    var total_coins = balances_results
                    var bitcoins = parseFloat(total_coins) * coin_price;
                    var balance = (bitcoins / targetcoin_price);

                    callback(null, balance);
                    return;
                }

                var balances = [];

                for(var worker in balances_results){
                    var total_coins = parseFloat(balances_results[worker]) / 1;
                    var bitcoins = total_coins.toFixed() * coin_price;
                    var balance = (bitcoins / targetcoin_price);
                    balances.push({worker:worker, balance:balance.toFixed( 8 )});
                }

                callback(null, balances);
            }

        ], function(err, balances){

            if(balance) {
                cback(balances);
                return;
            }

            _this.stats.balances = balances;
         _this.stats.payout  = payouts;
        //logger.error(logSystem, 'TEMP', '_this.stats right before CB variable is:' + JSON.stringify(_this.stats));

    cback();
        });

    };





    this.getGlobalStats = function(callback){

        var statGatherTime = Date.now() / 1000 | 0;

        var allCoinStats = {};

        async.each(redisClients, function(client, callback){
            var windowTime = (((Date.now() / 1000) - portalConfig.website.stats.hashrateWindow) | 0).toString();
            var redisCommands = [];


            var redisCommandTemplates = [
                ['zremrangebyscore', ':hashrate', '-inf', '(' + windowTime],
                ['zrangebyscore', ':hashrate', windowTime, '+inf'],
                ['hgetall', ':stats'],
                ['scard', ':blocksPending'],
                ['scard', ':blocksConfirmed'],
                ['scard', ':blocksOrphaned']
            ];

            var commandsPerCoin = redisCommandTemplates.length;

            client.coins.map(function(coin){
                redisCommandTemplates.map(function(t){
                    var clonedTemplates = t.slice(0);
                    clonedTemplates[1] = coin + clonedTemplates[1];
                    redisCommands.push(clonedTemplates);
                });
            });


            client.client.multi(redisCommands).exec(function(err, replies){
                if (err){
                    logger.error(logSystem, 'Global', 'error with getting global stats ' + JSON.stringify(err));
                    callback(err);
                }
                else{
                    for(var i = 0; i < replies.length; i += commandsPerCoin){
                        var coinName = client.coins[i / commandsPerCoin | 0];
                        var coinStats = {
                            name: coinName,
                            symbol: poolConfigs[coinName].coin.symbol.toUpperCase(),
                            algorithm: poolConfigs[coinName].coin.algorithm,
                            hashrates: replies[i + 1],
                            poolStats: {
                                validShares: replies[i + 2] ? (replies[i + 2].validShares || 0) : 0,
                                validBlocks: replies[i + 2] ? (replies[i + 2].validBlocks || 0) : 0,
                                invalidShares: replies[i + 2] ? (replies[i + 2].invalidShares || 0) : 0,
                                totalPaid: replies[i + 2] ? (replies[i + 2].totalPaid || 0) : 0
                            },
                            blocks: {
                                pending: replies[i + 3],
                                confirmed: replies[i + 4],
                                orphaned: replies[i + 5]
                            }
                        };
                        allCoinStats[coinStats.name] = (coinStats);
                    }
                    callback();
                }
            });
        }, function(err){
            if (err){
                logger.error(logSystem, 'Global', 'error getting all stats' + JSON.stringify(err));
                callback();
                return;
            }

            var portalStats = {
                time: statGatherTime,
                global:{
                    workers: 0,
                    hashrate: 0
                },
                algos: {},
                pools: allCoinStats
            };

            Object.keys(allCoinStats).forEach(function(coin){
                var coinStats = allCoinStats[coin];
                coinStats.workers = {};
                coinStats.shares = 0;
                coinStats.hashrates.forEach(function(ins){
                    var parts = ins.split(':');
                    var workerShares = parseFloat(parts[0]);
                    var worker = parts[1];
                    if (workerShares > 0) {
                        coinStats.shares += workerShares;
                        if (worker in coinStats.workers)
                            coinStats.workers[worker].shares += workerShares;
                        else
                            coinStats.workers[worker] = {
                                shares: workerShares,
                                invalidshares: 0,
                                hashrateString: null
                            };
                    }
                    else {
                        if (worker in coinStats.workers)
                            coinStats.workers[worker].invalidshares -= workerShares; // workerShares is negative number!
                        else
                            coinStats.workers[worker] = {
                                shares: 0,
                                invalidshares: -workerShares,
                                hashrateString: null
                            };
                    }
                });

                var shareMultiplier = Math.pow(2, 32) / algos[coinStats.algorithm].multiplier;
                coinStats.hashrate = shareMultiplier * coinStats.shares / portalConfig.website.stats.hashrateWindow;

                coinStats.workerCount = Object.keys(coinStats.workers).length;
                portalStats.global.workers += coinStats.workerCount;

                /* algorithm specific global stats */
                var algo = coinStats.algorithm;
                if (!portalStats.algos.hasOwnProperty(algo)){
                    portalStats.algos[algo] = {
                        workers: 0,
                        hashrate: 0,
                        hashrateString: null
                    };
                }
                portalStats.algos[algo].hashrate += coinStats.hashrate;
                portalStats.algos[algo].workers += Object.keys(coinStats.workers).length;

                for (var worker in coinStats.workers) {
                    coinStats.workers[worker].hashrateString = _this.getReadableHashRateString(shareMultiplier * coinStats.workers[worker].shares / portalConfig.website.stats.hashrateWindow);
                }

                delete coinStats.hashrates;
                delete coinStats.shares;
                coinStats.hashrateString = _this.getReadableHashRateString(coinStats.hashrate);
            });

            Object.keys(portalStats.algos).forEach(function(algo){
                var algoStats = portalStats.algos[algo];
                algoStats.hashrateString = _this.getReadableHashRateString(algoStats.hashrate);
            });

            _this.stats = portalStats;
            _this.statsString = JSON.stringify(portalStats);



            _this.statHistory.push(portalStats);
            addStatPoolHistory(portalStats);

            var retentionTime = (((Date.now() / 1000) - portalConfig.website.stats.historicalRetention) | 0);

            for (var i = 0; i < _this.statHistory.length; i++){
                if (retentionTime < _this.statHistory[i].time){
                    if (i > 0) {
                        _this.statHistory = _this.statHistory.slice(i);
                        _this.statPoolHistory = _this.statPoolHistory.slice(i);
                    }
                    break;
                }
            }

            redisStats.multi([
                ['zadd', 'statHistory', statGatherTime, _this.statsString],
                ['zremrangebyscore', 'statHistory', '-inf', '(' + retentionTime]
            ]).exec(function(err, replies){
                if (err)
                    logger.error(logSystem, 'Historics', 'Error adding stats to historics ' + JSON.stringify(err));
            });
            callback();
        });

    };

    this.getReadableHashRateString = function(hashrate){
        var i = -1;
        var byteUnits = [ ' KH', ' MH', ' GH', ' TH', ' PH' ];
        do {
            hashrate = hashrate / 1024;
                        i++;
        } while (hashrate > 1024);
        return hashrate.toFixed(2) + byteUnits[i];
    };

};
[/code

Delete the stock website.js file as well, make a new one:
[code]

var fs = require('fs');
var path = require('path');

var async = require('async');
var watch = require('node-watch');
var redis = require('redis');

var dot = require('dot');
var express = require('express');
var bodyParser = require('body-parser');
var compress = require('compression');

var Stratum = require('stratum-pool');
var util = require('stratum-pool/lib/util.js');

var api = require('./api.js');


module.exports = function(logger){

    dot.templateSettings.strip = false;

    var portalConfig = JSON.parse(process.env.portalConfig);
    var poolConfigs = JSON.parse(process.env.pools);

    var websiteConfig = portalConfig.website;

    var portalApi = new api(logger, portalConfig, poolConfigs);
    var portalStats = portalApi.stats;

    var logSystem = 'Website';


var pageFiles = {
        'index.html': 'index',
        'home.html': '',
        'tbs.html': 'tbs',
        'workers.html': 'workers',
        'api.html': 'api',
        'admin.html': 'admin',
        'mining_key.html': 'mining_key',
        'miner.html': 'miner',
        'miner_stats.html': 'miner_stats',
        'user_shares.html': 'user_shares',
        'getting_started.html': 'getting_started'
    };

    var pageTemplates = {};

    var pageProcessed = {};
    var indexesProcessed = {};

    var keyScriptTemplate = '';
    var keyScriptProcessed = '';


    var processTemplates = function(){

        for (var pageName in pageTemplates){
            if (pageName === 'index') continue;
            pageProcessed[pageName] = pageTemplates[pageName]({
                poolsConfigs: poolConfigs,
                stats: portalStats.stats,
                portalConfig: portalConfig
            });
            indexesProcessed[pageName] = pageTemplates.index({
                page: pageProcessed[pageName],
                selected: pageName,
                stats: portalStats.stats,
                poolConfigs: poolConfigs,
                portalConfig: portalConfig
            });
        }

        //logger.debug(logSystem, 'Stats', 'Website updated to latest stats');
    };



    var readPageFiles = function(files){
        async.each(files, function(fileName, callback){
            var filePath = 'website/' + (fileName === 'index.html' ? '' : 'pages/') + fileName;
            fs.readFile(filePath, 'utf8', function(err, data){
                var pTemp = dot.template(data);
                pageTemplates[pageFiles[fileName]] = pTemp
                callback();
            });
        }, function(err){
            if (err){
                console.log('error reading files for creating dot templates: '+ JSON.stringify(err));
                return;
            }
            processTemplates();
        });
    };


    //If an html file was changed reload it
    watch('website', function(filename){
        var basename = path.basename(filename);
        if (basename in pageFiles){
            console.log(filename);
            readPageFiles([basename]);
            logger.debug(logSystem, 'Server', 'Reloaded file ' + basename);
        }
    });

    portalStats.getGlobalStats(function(){
        readPageFiles(Object.keys(pageFiles));
    });

    var buildUpdatedWebsite = function(){
        portalStats.getGlobalStats(function(){
            processTemplates();

            var statData = 'data: ' + JSON.stringify(portalStats.stats) + '\n\n';
            for (var uid in portalApi.liveStatConnections){
                var res = portalApi.liveStatConnections[uid];
                res.write(statData);
            }

        });
    };

    setInterval(buildUpdatedWebsite, websiteConfig.stats.updateInterval * 1000);


    var buildKeyScriptPage = function(){
        async.waterfall([
            function(callback){
                var client = redis.createClient(portalConfig.redis.port, portalConfig.redis.host);
                client.hgetall('coinVersionBytes', function(err, coinBytes){
                    if (err){
                        client.quit();
                        return callback('Failed grabbing coin version bytes from redis ' + JSON.stringify(err));
                    }
                    callback(null, client, coinBytes || {});
                });
            },
            function (client, coinBytes, callback){
                var enabledCoins = Object.keys(poolConfigs).map(function(c){return c.toLowerCase()});
                var missingCoins = [];
                enabledCoins.forEach(function(c){
                    if (!(c in coinBytes))
                        missingCoins.push(c);
                });
                callback(null, client, coinBytes, missingCoins);
            },
            function(client, coinBytes, missingCoins, callback){
                var coinsForRedis = {};
                async.each(missingCoins, function(c, cback){
                    var coinInfo = (function(){
                        for (var pName in poolConfigs){
                            if (pName.toLowerCase() === c)
                                return {
                                    daemon: poolConfigs[pName].paymentProcessing.daemon,
                                    address: poolConfigs[pName].address
                                }
                        }
                    })();
                    var daemon = new Stratum.daemon.interface([coinInfo.daemon], function(severity, message){
                        logger[severity](logSystem, c, message);
                    });
                    daemon.cmd('dumpprivkey', [coinInfo.address], function(result){
                        if (result[0].error){
                            logger.error(logSystem, c, 'Could not dumpprivkey for ' + c + ' ' + JSON.stringify(result[0].error));
                            cback();
                            return;
                        }

                        var vBytePub = util.getVersionByte(coinInfo.address)[0];
                        var vBytePriv = util.getVersionByte(result[0].response)[0];

                        coinBytes[c] = vBytePub.toString() + ',' + vBytePriv.toString();
                        coinsForRedis[c] = coinBytes[c];
                        cback();
                    });
                }, function(err){
                    callback(null, client, coinBytes, coinsForRedis);
                });
            },
            function(client, coinBytes, coinsForRedis, callback){
                if (Object.keys(coinsForRedis).length > 0){
                    client.hmset('coinVersionBytes', coinsForRedis, function(err){
                        if (err)
                            logger.error(logSystem, 'Init', 'Failed inserting coin byte version into redis ' + JSON.stringify(err));
                        client.quit();
                    });
                }
                else{
                    client.quit();
                }
                callback(null, coinBytes);
            }
        ], function(err, coinBytes){
            if (err){
                logger.error(logSystem, 'Init', err);
                return;
            }
            try{
                keyScriptTemplate = dot.template(fs.readFileSync('website/key.html', {encoding: 'utf8'}));
                keyScriptProcessed = keyScriptTemplate({coins: coinBytes});
            }
            catch(e){
                logger.error(logSystem, 'Init', 'Failed to read key.html file');
            }
        });

    };
    buildKeyScriptPage();

    var getPage = function(pageId){
        if (pageId in pageProcessed){
            var requestedPage = pageProcessed[pageId];
            return requestedPage;
        }
    };




    var minerpage = function(req, res, next){
        var address = req.params.address || null;

        if (address != null){
            portalStats.getBalanceByAddress(address, function(){
                processTemplates();

                res.end(indexesProcessed['miner_stats']);

            });
        }
        else
            next();
    };

    var payout = function(req, res, next){
        var address = req.params.address || null;

        if (address != null){
            portalStats.getPayout(address, function(data){
                res.write(data.toString());
                res.end();
            });
        }
        else
            next();
    };


    var shares = function(req, res, next){
        portalStats.getCoins(function(){
            processTemplates();

            res.end(indexesProcessed['user_shares']);

        });
    };

    var usershares = function(req, res, next){

        var coin = req.params.coin || null;

        if(coin != null){
            portalStats.getCoinTotals(coin, null, function(){
                processTemplates();

                res.end(indexesProcessed['user_shares']);

            });
        }
        else
            next();
    };


    var route = function(req, res, next){
        var pageId = req.params.page || '';
        if (pageId in indexesProcessed){
            res.header('Content-Type', 'text/html');
            res.end(indexesProcessed[pageId]);
        }
        else
            next();

    };



    var app = express();


    app.use(bodyParser.json());

    app.get('/get_page', function(req, res, next){
        var requestedPage = getPage(req.query.id);
        if (requestedPage){
            res.end(requestedPage);
            return;
        }
        next();
    });

    app.get('/key.html', function(req, res, next){
        res.end(keyScriptProcessed);
    });


        app.get('/stats/shares/:coin', usershares);
    app.get('/stats/shares', shares);
    app.get('/miner/:address', minerpage);
    app.get('/payout/:address', payout);



    app.get('/:page', route);
    app.get('/', route);

    app.get('/api/:method', function(req, res, next){
        portalApi.handleApiRequest(req, res, next);
    });

    app.post('/api/admin/:method', function(req, res, next){
        if (portalConfig.website
            && portalConfig.website.adminCenter
            && portalConfig.website.adminCenter.enabled){
            if (portalConfig.website.adminCenter.password === req.body.password)
                portalApi.handleAdminApiRequest(req, res, next);
            else
                res.send(401, JSON.stringify({error: 'Incorrect Password'}));

        }
        else
            next();

    });

    app.use(compress());
    app.use('/static', express.static('website/static'));

    app.use(function(err, req, res, next){
        console.error(err.stack);
        res.send(500, 'Something broke!');
    });

    try {
        app.listen(portalConfig.website.port, portalConfig.website.host, function () {
            logger.debug(logSystem, 'Server', 'Website started on ' + portalConfig.website.host + ':' + portalConfig.website.port);
        });
    }
    catch(e){
        logger.error(logSystem, 'Server', 'Could not start website on ' + portalConfig.website.host + ':' + portalConfig.website.port
            +  ' - its either in use or you do not have permission');
    }


};

Then inside the /website/pages folder create two new files.. Three actually.  First while in the /website/pages type 'touch user_shares.html'  just to create the file so NOMP won't puke for now.

Open a new file named miner.html:
Code:

        

        

                

Enter Your Wallet address


                

                        
                        
                                
                        

                

        






[/code]
sr. member
Activity: 378
Merit: 254
small fry
In this thread, I am going to go through the step required to build your very own POS multipool, complete with pretty front end stats and automatic exchange trading..  This will only work for bitcoin related coins.  I don't promise to share all the secrets, but enough to get a site up and running.

I am going to tell you all this completely free.
I am not going to polute this with donation solicitations - if you would like to send me some btc for writing this once you're done reading it, pm me and I'll send you my address.

 I will make some effort to support to help with the implementation of this if anyone has trouble.



It might not be the prettiest, but it works - and I won't blackmail coins out of any of you. Smiley  This is a big screw you to all the cunt MP operators who have forgotten where this entire scene came from. Satoshi releaed the bitcoin client opensource, quit charging people a bitcoin to set them up a multipool and quit trying to screw everyone over.
This implementation btw, does not fudge any of the stats like some of the other major pools, and this offers complete visibility into stats.  It also doesn't do any of that "everbody contributes shares to an algo specific bucket that pays out once per day" crap where the pool operator can steal 1/3 of the money off the top before anyone can tell.

This uses a combination of python, node.js and even basic cronjobs to get the job done.  


Step #1 - start with a basic NOMP install. git clone from the NOMP repo: https://github.com/zone117x/node-open-mining-portal.git
Step #2 - Set up your coin daemons as if this was a basic NOMP build.  There are a few exceptions.  In the /coins/ directory you are going to be limited to only using coins posted on either Cryptsy or Mintpal - I could eaily have added some other markets to this, so feel free.  For each coin, in the /coins directiory include an additional line: "ID": ""

IE:
Quote
root@blackcoinpool# cat feathercoin.json
{
"name": "Feathercoin",
"ID": "5",
"symbol": "FTC",
"algorithm": "scrypt"
}
The pool is going to use this to help it look up the values for the coin when it is calculating some of the stats later.

Also, when setting up the coin daemons - we aren't going to use the NOMP payment processor for anything more than calculating the balances tables in redis.  So make sure to set every coin's minimum payout to 999999999999999 and in the main config.json disable the worker name authentication.
Quote
main config.json...

    "defaultPoolConfigs": {
        "blockRefreshInterval": 1000,
        "jobRebroadcastTimeout": 55,
        "connectionTimeout": 600,
       "emitInvalidBlockHashes": false,
       "validateWorkerUsername": false,
        "tcpProxyProtocol": false,
        "banning": {
            "enabled": true,
            "time": 600,
            "invalidPercent": 30,
            "checkThreshold": 500,
            "purgeInterval": 300
        },
        "redis": {
            "host": "127.0.0.1",
            "port": 6379
        }
    },

    "website": {
        "enabled": true,
        "host": "0.0.0.0",
        "port": 80,
        "stratumHost": "everypool.com",
        "stats": {
           "updateInterval": 60,
            "historicalRetention": 86400,

            "hashrateWindow": 300
    },

Quote
this is from inside on the of the /pool_config/ coin configurations
    "paymentProcessing": {
        "enabled": true,
       "paymentInterval": 75,
        "minimumPayment": 99999999999,

        "daemon": {
            "host": "127.0.0.1",
            "port": ,
            "user": "birdsofafeather",
            "password": "flocktogether"
        }
    },
Pages:
Jump to:
© 2020, Bitcointalksearch.org