Pages:
Author

Topic: Smartcoin Linux mining administration. [MULTI-MACHINE SUPPORT NOW IN!] - page 9. (Read 105029 times)

full member
Activity: 238
Merit: 100
kennel -
Your settings look good to me.  If you get any false triggers of failover and/or lockup, you can always increase them 1 or 2 at a time until you find a good balance.

I'll experiment  with the /proc load average tonight (just got to decide whether to go with 1 minute or 5 minute average, though I'm leaning towards 1 minute) Additionally,  it would get rid of another dependency.

Regarding better descriptions of the settings, I'm probably going to leave it documentation in the future.  The settings system is very dynamic in that I just add an entry into the database and it automagically appears and just works - this makes development a little easier on me.  I may rethink it a bit once I can get out of beta, as I won't be adding entries so often then and it may be worth extending the system to be more verbose.
full member
Activity: 168
Merit: 100
I'll have a steak sandwich and a... steak sandwich
Note, that if you run an unusually high loop delay, you may want to reduce the Failover Threshold and Lockup Threshold settings as well, as they are measured in loop iterations.
I'll do that. My loop delay is 10 and I've set the failover threshold to 3 and the lockup threshold to 10. Would that make sense?

Perhaps you can add a sentence under "Edit Settings" describing what these settings actually mean?

I will look into other methods of determining CPU load, but an average works really well here, as the script its self has some areas of really high usage, and possibly even no usage at all (during the loop delay for example)  Can you post some comparisons of /proc/loadavg against iostat?
It's difficult to compare load averages with straight up CPU utilization. They're not measuring the same thing. I would say load averages are more interesting though as they reflect how the CPU (or CPUs) is coping with the work load. It's quite possible to run at 100% CPU without overloading the machine. It's also possible to overload the machine without running the CPU at 100%. More info here: http://en.wikipedia.org/wiki/Load_(computing)
newbie
Activity: 41
Merit: 0
Thanks -- I found it by searching through the thread.  I know you've already got tons on your plate, but it might be helpful to new users to include that info in the first post.
full member
Activity: 238
Merit: 100
What is the default miner output format string?  I tried editing the format string and ended up erasing it entirely.  Weird characters appeared every time I pressed one of the arrow keys or the delete key to move the cursor.  Now it's totally blank and I can't see a thing.

The default is:
Code:
[<#hashrate#> MHash/sec] [<#accepted#> Accepted] [<#rejected#> Rejected] [<#rejected_percent#>% Rejected]
newbie
Activity: 41
Merit: 0
What is the default miner output format string?  I tried editing the format string and ended up erasing it entirely.  Weird characters appeared every time I pressed one of the arrow keys or the delete key to move the cursor.  Now it's totally blank and I can't see a thing.
full member
Activity: 238
Merit: 100
Did poclbm change recently confusing smartcoin?  I just installed poclbm (git://github.com/m0mchil/poclbm) and smartcoin for the first time. The syntax smartcoin was using (--host, etc) was invalid.  It also does not appear to be able to scrape the output (I can see the miners running in screen, but smartcoin does not read it).   Additionally, it was (kind of) using invalid device ids.  Poclbm uses device 0 as the CPU.  Therefore the first miner ended up on the CPU.  

I was able to work around all of these except the scraping issue.  I was just wondering.

Dan

Can you post the smartcoin status screen output, along with the output of poclbm from the miner screen? Perhaps something has changed, I did test it and it worked just a couple of weeks ago

Also, to my knowlege poclbm doesn't always assume device 0 is cpu (unless something has changed in newer poclbm revisions). Here is the output of me running poclbm with no parameters:
Code:
jondecker76@miner1:~/poclbm$ ./poclbm.py                                       
No device specified or device not found, use -d to specify one of the following
                                                                               
[0]     Cypress                                                                 
[1]     Cypress                                                                 
[2]     Cypress                                                                 
[3]     Intel(R) Pentium(R) 4 CPU 2.80GHz                                       

full member
Activity: 238
Merit: 100
sr. member
Activity: 348
Merit: 251
Hi,
Just wondering if there is any update on the multi mining machine setup?

The thread is very long to read through.  Undecided

Thanks.
As soon as I make sure recent changes look good, and do some sqlite throughput testing, error logging and some other minor things, I will start digging into the multi machine support. Since most of it is already in place in the lower level of the code, it shouldn't be too awfully hard or time consuming to implement. I'm hoping for this next coming weekend to have things at least roughed out and working.

Excellent - great job.
newbie
Activity: 56
Merit: 0
Did poclbm change recently confusing smartcoin?  I just installed poclbm (git://github.com/m0mchil/poclbm) and smartcoin for the first time. The syntax smartcoin was using (--host, etc) was invalid.  It also does not appear to be able to scrape the output (I can see the miners running in screen, but smartcoin does not read it).   Additionally, it was (kind of) using invalid device ids.  Poclbm uses device 0 as the CPU.  Therefore the first miner ended up on the CPU. 

I was able to work around all of these except the scraping issue.  I was just wondering.

Dan

I think yet only supported is Phoenix (GPU) and Cgminer (CPU)
sr. member
Activity: 349
Merit: 250
BTCPak.com - Exchange your Bitcoins for MP!
Did poclbm change recently confusing smartcoin?  I just installed poclbm (git://github.com/m0mchil/poclbm) and smartcoin for the first time. The syntax smartcoin was using (--host, etc) was invalid.  It also does not appear to be able to scrape the output (I can see the miners running in screen, but smartcoin does not read it).   Additionally, it was (kind of) using invalid device ids.  Poclbm uses device 0 as the CPU.  Therefore the first miner ended up on the CPU. 

I was able to work around all of these except the scraping issue.  I was just wondering.

Dan
newbie
Activity: 40
Merit: 0
Jon,

I love the system and it works great for me.  I just thought I would share the scripts I wrote to help switch profiles via Web request incase anyone finds it useful.


poollookup.php
Code:

if (isset($_GET['SetPool']))
{
$pool = (int)$_GET['SetPool'];
// I use /tempdir on my server as a temp file directory, you can use whatever you choose
$res=fopen('/tempdir/pool.txt',"w");
fwrite($res,$pool);
fclose($res);
echo 'Pool is set to Profile ID '.$pool.'
Click here to return to main page.'
;

}
elseif (isset(
$_GET['GetPool']))
{
$pool readfile('/tempdir/pool.txt');
}
elseif (isset(
$_GET['KillMonitoring']))
{
$pool 'Stop';
$res=fopen('/tempdir/pool.txt',"w");
fwrite($res,$pool);
fclose($res);
echo 'Pool monitoring will be stopped';
}
else
{

//I have my actual list generated at runtime from my db file, below is just an example of what you can do.
 
?>



Current Pool Profile is ('/tempdir/pool.txt'); ?>:

Choose Which Profile you want to Run:

5)Profile A

2)Profile B

3)Profile C





}
?>

smartcoinmonitor.php
Code:
UpdateSmartcoin();
function 
UpdateSmartcoin(){

$ch curl_init();

curl_setopt ($chCURLOPT_SSL_VERIFYHOST0);
curl_setopt ($chCURLOPT_SSL_VERIFYPEER0); 
curl_setopt($chCURLOPT_URL,"http://{Webserver}/poollookup.php?GetPool");
curl_setopt($chCURLOPT_HEADER0);
ob_start();
curl_exec($ch);
curl_close($ch);
$res ob_get_clean();

if (
$res != 'Stop')
{

if (
$db = new SQLite3("/home/{USERNAME}/.smartcoin/smartcoin.db",SQLITE3_OPEN_READWRITE)) { 

$FindCurProfile "SELECT `fk_profile` FROM `current_profile` WHERE `fk_machine` = 1";
$currProfile $db->querySingle($FindCurProfile);


if (($currProfile != (int)$res) && ((int)$res != NULL))
{
$db->exec('UPDATE `current_profile` SET `fk_profile` = '.$res.' WHERE `fk_machine` = 1;');
echo 'Switching to Profile:'.$res."\n";
}
else
{

echo 'Sticking with Profile:'.$currProfile."\n";
}
$db->close();
} else {
    die(
"Sqlite failed to open\n");
}

//Have the system wait 30 seconds and check again.
sleep(30);
UpdateSmartcoin();
}
else
{
exit;
}
}

?>


smartcoinmonitor.php is run on your miner system(s), and poollookup.php is on your web server (which can also be on one of your miners).  Please remember to put the proper webser address in as well as the proper location of your smartcoin.db file.

Purpose:
I have 3 mining rigs running now using Smartcoin, but having 3 SSH's to change a profile on is a bit cumbersome sometimes, so I have all my miners (all with the same profile IDs) ask my webserver which profile they should run.

The code isn't pretty but it is useful to me and it may be useful to you.

Requirements:
a webserver with PHP5
PHP 5.3 or higher on your miner
LibCurl on your miner

Sample usage:
php smartcoinmonitor.php

or use Screen to be able to disconnect from it while it runs.

screen -S monitor php smartcoinmonitor.php

If this is helpful to you please consider a donation to 12MjprGQKxhSHUKaqT7d4QuZ54LCPqr1pd


full member
Activity: 238
Merit: 100
Hi,
Just wondering if there is any update on the multi mining machine setup?

The thread is very long to read through.  Undecided

Thanks.
As soon as I make sure recent changes look good, and do some sqlite throughput testing, error logging and some other minor things, I will start digging into the multi machine support. Since most of it is already in place in the lower level of the code, it shouldn't be too awfully hard or time consuming to implement. I'm hoping for this next coming weekend to have things at least roughed out and working.
full member
Activity: 238
Merit: 100
Update r501e now available
- SQL queries will retry for a lot longer if the database is locked. This greatly reduces the chances of a lost query!
- Major optimizations made in SQL queries.  There is still a little room for improvement, but SQL traffic is reduced by at least 70%!
Very nice to hear you managed to optimize the SQL traffic. However, even when running r501e, I'm still seeing about 26% CPU utilization (underclocked Sempron 140). In fact, it didn't decrease noticeable from the previous version. Any idea what might be causing the high CPU utilization? My miners aren't in my home, so I haven't been able to check whether the 26% CPU utilization actually translates into a higher power drain as compared to idle, but in theory, it should.

It should be noted that starting my miners manually in separate screens using the exact same startup string as in Smartcoin results in a CPU utilization around 1 or 2%.

Edit: Just noticed the update adding the loop delay. Will play with that and get back to you.
Yes, it will be up to you with the new loop delay setting to balance speed vs. CPU usage.  Smartcoin does a lot more than just open/close miner instances. It has to get a dump each miner screen window for each loop iteration and parse it for output, calculate hash rates, do the lockup detection, failover detection, etc - so of course there is a CPU penalty for all of this (I would guess that the dumping/parsing of the miner screen session windows is the most computationally expensive).

Regarding usage - the difference between 10% load and 30% load is under 20 watts on a 95W processor, so for me the difference extra load is a moot point.  Even so, my old p4 processor handles 3GPUs with 9 instances at under 13% load, which I don't think is bad at all.  For underclocked and low wattage processors, its even a smaller difference in wattage for extra load.  But again, just inject some delay with the loop delay setting and you can bring your CPU usage much lower if you want. Note, that if you run an unusually high loop delay, you may want to reduce the Failover Threshold and Lockup Threshold settings as well, as they are measured in loop iterations.

I will look into other methods of determining CPU load, but an average works really well here, as the script its self has some areas of really high usage, and possibly even no usage at all (during the loop delay for example)  Can you post some comparisons of /proc/loadavg against iostat?
sr. member
Activity: 348
Merit: 251
Hi,
Just wondering if there is any update on the multi mining machine setup?

The thread is very long to read through.  Undecided

Thanks.
full member
Activity: 168
Merit: 100
I'll have a steak sandwich and a... steak sandwich
I changed the loop delay to 10 and that dropped the CPU utilization drastically (from 26% to around 4%). It would be interesting to disable certain features in smartcoin_status to figure out what is actually causing the high load.

One thing to note though. The CPU utilization reported by iostat is the average since the last reboot so it's not super useful. I would prefer good old /proc/loadavg.
brand new
Activity: 0
Merit: 0
Has anyone experienced this?
When adding a new profile, sometimes there are 0 miners to select from. Running r496e
newbie
Activity: 41
Merit: 0
JD, man you're fast  Shocked

updated to r502e , used 8 as loop delay

cpu have much less stress, load from 0.76 to 0.10 after 10 minutes

abut that error flashing on one of rigs, still apears

it must be related somehow to database, because when i was changing delay loop value, def value was 11 (?) instead of 0 on other rigs

 
Ok, must have been an update error.  Does it clear up if you change the value then restart smartcoin?


do you mean delay loop value ?

if yes, changing delay doesn't fix anything

this error occurred after r452s to r496s update, in which delay loop wasn't existing  
full member
Activity: 168
Merit: 100
I'll have a steak sandwich and a... steak sandwich
Update r501e now available
- SQL queries will retry for a lot longer if the database is locked. This greatly reduces the chances of a lost query!
- Major optimizations made in SQL queries.  There is still a little room for improvement, but SQL traffic is reduced by at least 70%!
Very nice to hear you managed to optimize the SQL traffic. However, even when running r501e, I'm still seeing about 26% CPU utilization (underclocked Sempron 140). In fact, it didn't decrease noticeable from the previous version. Any idea what might be causing the high CPU utilization? My miners aren't in my home, so I haven't been able to check whether the 26% CPU utilization actually translates into a higher power drain as compared to idle, but in theory, it should.

It should be noted that starting my miners manually in separate screens using the exact same startup string as in Smartcoin results in a CPU utilization around 1 or 2%.

Edit: Just noticed the update adding the loop delay. Will play with that and get back to you.
full member
Activity: 238
Merit: 100
Update r506e now available

Thanks to plantucha, i fould and fixed a few bugs, and made a few changes.
- The auto profile now adheres to the auto_allow field for devices.
- The auto profile now only considers 'gpu' type devices. If you want to mix cpu and gpu types, do this via a manual profile!
- There was a bug in the new failover generation that is fixed. It only affected those using multiple miner types
- Lockup detection will now not happen on an instance showing Down (0 hashes per second). This way, in a profile with multiple instances, a single downed instance won't trip the failover.
- The miner parsers now differentiate between "Down" (0 hashes) and "Idle" (no miner output at all)

full member
Activity: 238
Merit: 100
JD, man you're fast  Shocked

updated to r502e , used 8 as loop delay

cpu have much less stress, load from 0.76 to 0.10 after 10 minutes

abut that error flashing on one of rigs, still apears

it must be related somehow to database, because when i was changing delay loop value, def value was 11 (?) instead of 0 on other rigs

 
Ok, must have been an update error.  Does it clear up if you change the value then restart smartcoin?
Pages:
Jump to: