Author

Topic: OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.0 - page 616. (Read 5805874 times)

legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
Hmm OK, so it's effectively 12 devices.
That's a lot of GPU devices Smiley

The RSS number is (obviously) what shows if you are using up more RAM and running out of it.
I guess what's needed now is to find out what is happening when that changes ... and does it jump up in steps, or is a gradual rise?

Edit: oh I just noticed another thing ... you have 2 pools and the main is GPUMax - does it swap pools a lot?
sr. member
Activity: 309
Merit: 250
OK 53 isn't a big number so it's not what I was considering may be the cause.

The other two numbers there are not big yet either
SZ=185303 RSS=388892
if you have a sizeable number of devices on your rig (your options suggest 6 GPU devices? Are any of them dual?)

How often does BAMT request status info from the API? And what API requests does it use? (I don't know exactly)
I guess if there was some accidental fast loop missing a delay accessing the API over and over again that might cause issues?

My monitoring in my current setup is a set of 3 API requests every 10 seconds, but in my previous configuration (same code and hardware a week ago) I also added on top of that another 1 API request every second - but that didn't cause extra RAM usage.

I guess I'll defer to the expert (ckolivas) Smiley
Going idle can be for two reasons, as you suggested if GPUMax isn't providing you with any work, but of course the normal reason is as ckolivas said, if your setting are too high.

Are those temp settings and clock settings in any way high for your GPU's?
(The numbers themselves don't seem high for many cards - but I guess that depends on what the cards are - and if they are old and no longer stable if OC'd)
Do you ever get any status change on any of the GPU's? (i.e. not Alive)


They are all 5970's (dual gpu) so those temps are okay. I was actually using Phoenix until I just recently started with cgminer and had those clocks stable on phoenix for several weeks.  Not sure how often BAMT requests via API.  I have BAMT running on 2 rigs with 3 7970's each with same version of cgminer and do not see the issue on there.  I'll try running stock for awhile and see what happens.
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
OK 53 isn't a big number so it's not what I was considering may be the cause.

The other two numbers there are not big yet either
SZ=185303 RSS=388892
if you have a sizeable number of devices on your rig (your options suggest 6 GPU devices? Are any of them dual?)

My rig has only 4 devices and it reads:
SZ=151618 RSS=127032
(and yes I use the code also from my GIT except I don't enable Bitforce being compiled in, I only add Icarus at the moment)

However, in my case it doesn't run rampant later on.

Definitely getting nowhere with this so far looking at the system process info.

How often does BAMT request status info from the API? And what API requests does it use? (I don't know exactly)
I guess if there was some accidental fast loop missing a delay accessing the API over and over again that might cause issues?
My monitoring in my current setup is a set of 3 API requests every 10 seconds, but in my previous configuration (same code and hardware a week ago) I also added on top of that another 1 API request every second - but that didn't cause extra RAM usage.

I guess I'll defer to the expert (ckolivas) Smiley
Going idle can be for two reasons, as you suggested if GPUMax isn't providing you with any work, but of course the normal reason is as ckolivas said, if your setting are too high.

Are those temp settings and clock settings in any way high for your GPU's?
(The numbers themselves don't seem high for many cards - but I guess that depends on what the cards are - and if they are old and no longer stable if OC'd)
Do you ever get any status change on any of the GPU's? (i.e. not Alive)

Oddly enough I am working on a notification Java app that uses the API that might help monitor this problem (that comment above about 3 API every 10 seconds - summary,devs,pools) if it is temp/clock related Tongue
(The code already done would be enough)
But I've been expending a lot of effort on threading the data handling part of it (and probably making it too complex) to ensure that blocking code is in threads unrelated to time sensitive code (and then there's the issue of RAM usage in Java that I need to resolve next ...)
So I seem to be talking way longer than I expected to get even a beta out Tongue
sr. member
Activity: 309
Merit: 250
Some questions regarding this Smiley
1) When you run "./cgminer --help" what does the 2nd line "Built with ..." say?

2) run "ps -eL | grep cgminer | grep -v grep | wc" and it will say how many threads there are

3) run "ps -eLF | grep cgminer | grep -v grep | tail -1" will show you the command running cgminer so you can be sure it matches exactly what you said Smiley

Here's the information you requested:

root@oconner:/opt/miners/cgminer# ./cgminer --help
cgminer 2.3.1f
Built with GPU bitforce icarus mining support.

root@oconner:/opt/miners/cgminer# ps -eL | grep cgminer | grep -v grep | wc
     53     265    2014

root@oconner:/opt/miners/cgminer# ps -eLF | grep cgminer | grep -v grep | tail -1
root     27782 27781 27188  0   53 185303 388892 0 18:25 pts/1    00:00:00 /opt/miners/cgminer cgminer -Q 2 --api-listen --gpu-engine 700-825 700-825 700-830 700-815 700-820 700-830 --gpu-memclock 240 --auto-gpu --auto-fan --temp-target 75 -I 8 -o http://gpumax.com:8332 -u x-p x -o http://x:80 -u x -p x
legendary
Activity: 1022
Merit: 1000
BitMinter
Hey Con. Will you release a version with kanos work that supports mining with Icarus and BFL (windows7, 2.3.1f) ? There is a 7.5 BTC bounty for that. Would be nice Tongue
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
...
The web server part is only for rig mining stats (BAMT).  It's cgminer that keeps growing in memory size.  I've been watching and I think every-time a thread is idle for 60 seconds and is restarted, it increases overall RAM usage (like the previous thread isn't getting removed from RAM).  I use gpumax, which can have high idle times occasionally....  cgminer starts at 174 meg of RAM for me and currently its using 800 meg, so I think adding another stick of RAM would help, but only by delaying the time it takes for the system to run out of RAM.  If I restart cgminer, then its back to normal mem usage and slowly increases again.

I've been reviewing the command switches and am not sure which one increases the time a thread can be idle before cgminer restarts it... can someone assist me with what switch does that?  I would like to set the thread idle limit to 120 or 180 seconds to see if that reduces my thread restarts (and therefore RAM usage).  I'm on cgminer v 2.3.1f.
Some questions regarding this Smiley
1) When you run "./cgminer --help" what does the 2nd line "Built with ..." say?

2) run "ps -eL | grep cgminer | grep -v grep | wc" and it will say how many threads there are

3) run "ps -eLF | grep cgminer | grep -v grep | tail -1" will show you the command running cgminer so you can be sure it matches exactly what you said Smiley
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
I've been watching and I think every-time a thread is idle for 60 seconds and is restarted, it increases overall RAM usage (like the previous thread isn't getting removed from RAM).  I use gpumax, which can have high idle times occasionally....  cgminer starts at 174 meg of RAM for me and currently its using 800 meg, so I think adding another stick of RAM would help, but only by delaying the time it takes for the system to run out of RAM.  If I restart cgminer, then its back to normal mem usage and slowly increases again.
Then you're onto something. It is dangerous to try and delete the ram used by the old threads because that just leads to a crash, so I specifically made the threads' ram not get cleaned up - so it is expected that ram usage will go up. However, you're doing something horribly wrong if your GPUs are going idle all the time. That's supposed to ONLY happen if your GPUs wedge because you're overclocking them too much and the GPU stops responding. I highly recommend reviewing your temperatures and clock speeds.
newbie
Activity: 50
Merit: 0

Sure you can.  Why wouldn't you be able to do that?

As long as you have your bitcoind/Bitcoin client started in server mode and the block chain up to date.

I don't know anything about GUIMiner and have no idea what that would have to do with anything.
Sam

Thank you very much! Wink

I mentioned guiminer becuase I used it in the past to do solo mining.
It sets my username/pass for solo mining and also launches btc client in server mode.
legendary
Activity: 3586
Merit: 1098
Think for yourself
Hi  Smiley

I have a doubt...
Sorry if this is already asked but 235 pages are too long to read  Shocked

Can I set "Solo Mode" mining in CGminer as follows?

Code:
cgminer -o http://localhost:3332 -u MyUserName -p MyPassWord -I 9 --auto-fan --auto-gpu --gpu-engine 800,800,775 --gpu-memclock 300

Username and pass for solo mode mining are previously set with guiminer.

Thanks in advance  Wink

Sure you can.  Why wouldn't you be able to do that?

As long as you have your bitcoind/Bitcoin client started in server mode and the block chain up to date.

I don't know anything about GUIMiner and have no idea what that would have to do with anything.
Sam
hero member
Activity: 642
Merit: 500
You can only set the 7970 memory 150 lower than the engine clock speed. That's precisely why the memdiff feature exists in cgminer, so add this to your commands:
--gpu-memdiff -150

I didn't know that command existed.  Nice.

A BIOS flash will take care of that clock/voltage issue nicely.  Smiley   (Assuming it's a dedicated rig, of course)
newbie
Activity: 50
Merit: 0
Hi  Smiley

I have a doubt...
Sorry if this is already asked but 235 pages are too long to read  Shocked

Can I set "Solo Mode" mining in CGminer as follows?

Code:
cgminer -o http://localhost:3332 -u MyUserName -p MyPassWord -I 9 --auto-fan --auto-gpu --gpu-engine 800,800,775 --gpu-memclock 300

Username and pass for solo mode mining are previously set with guiminer.

Thanks in advance  Wink
hero member
Activity: 630
Merit: 500
Uninstalled version "2.6", deleted the DLL's manually in System32 and SysWOW64 (necessary step going between versions!), then installed this new leaked version.  Deleted my kernel bins in cgminer dir as usual, then launched...

Can't really see any change in performance.  Hopefully this gives ckolivas and the kernel devs something to chew on and progress forward, though.

I'm very confused by the overall driver version thing. Is there a post (on this forum or somewhere else) that clarifies driver versions, OpenCL versions, which versions to use, how to change them and so on? I'm using Windows 7.
No and it's a shame because the good info is just spread out all over the place on these forums  Embarrassed
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
2.3.1f Smiley OK That's from my git that was added to BAMT after 2.3.1 was released
That was to add API support for FPGA's ... 29-Feb ... yep.

That problem description might suggest a possible problem ... not related to my git change ...
If no one else has said or done anything when I wake up I'll look at making a quick test binary for you then
- but it's 3am and I'm tired and likely to make mistakes now Tongue
sr. member
Activity: 309
Merit: 250

You mentioned those are not dedicated mining rigs running web servers as well.
Just add a $12 memory module. As a stop gap, create a swap file.
Quest complete, you gain 200 experience points.

The web server part is only for rig mining stats (BAMT).  It's cgminer that keeps growing in memory size.  I've been watching and I think every-time a thread is idle for 60 seconds and is restarted, it increases overall RAM usage (like the previous thread isn't getting removed from RAM).  I use gpumax, which can have high idle times occasionally....  cgminer starts at 174 meg of RAM for me and currently its using 800 meg, so I think adding another stick of RAM would help, but only by delaying the time it takes for the system to run out of RAM.  If I restart cgminer, then its back to normal mem usage and slowly increases again.

I've been reviewing the command switches and am not sure which one increases the time a thread can be idle before cgminer restarts it... can someone assist me with what switch does that?  I would like to set the thread idle limit to 120 or 180 seconds to see if that reduces my thread restarts (and therefore RAM usage).  I'm on cgminer v 2.3.1f.
full member
Activity: 210
Merit: 100
...
http://pastebin.com/8MDeRMkD
Anyone have any ideas to help me troubleshoot whats happening?

Your machines run out of memory.
It is the job of the linux 'oom killer' to sacrifice one or more processes in order to free up memory for the system when all else fails.

You mentioned those are not dedicated mining rigs running web servers as well.
Just add a $12 memory module. As a stop gap, create a swap file.
Quest complete, you gain 200 experience points.
hero member
Activity: 529
Merit: 500
Uninstalled version "2.6", deleted the DLL's manually in System32 and SysWOW64 (necessary step going between versions!), then installed this new leaked version.  Deleted my kernel bins in cgminer dir as usual, then launched...

Can't really see any change in performance.  Hopefully this gives ckolivas and the kernel devs something to chew on and progress forward, though.

I'm very confused by the overall driver version thing. Is there a post (on this forum or somewhere else) that clarifies driver versions, OpenCL versions, which versions to use, how to change them and so on? I'm using Windows 7.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
Thanks... I don't use the submit-stale option...  When i start it uses about 174 meg of mem, then a couple hours later, its using closer to 400 meg

this is my start script
Quote
/usr/bin/screen -dmS cgminer /opt/miners/cgminer/cgminer -Q 2 --api-listen --gpu-engine 700-780,700-780,700-810,700-805,700-820,700-815 --gpu-memclock 240 --auto-gpu --auto-fan --temp-target 75 -I 10 -o http://x:8332 -u x-p x -o http://x:80 -u x -p z


I'm not sure whats going on...

Neither am I. But I can recommend dropping intensity since 10 does not help anything less than a 7970 and consumes more CPU and makes hashrate more unstable. 7-9 depending on card is optimal.  There is a value, beyond which, the hashrate does not go up. 8 for most cards, 9 for 69x0s and 11 for 79x0s (only with export GPU_USE_SYNC_OBJECTS=1 on linux, otherwise again 9).
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
Since I've had a few cgminer updates waiting for quite a while, and I've also added another update to the miner.php that some might consider useful, I'll post about it here:

The latest version in my git waiting to be pulled:
https://github.com/kanoi/cgminer/blob/master/miner.h

This adds 3 changes to ckolivas git:
1) Split the devs output tables if there is more than 1 device type
2) A PHP deprecated update for 1)
3) A new option at line 10 "$readonly" that when set to true will force the script to not show any buttons to change cgminer
If it is set to false then it will use the cgminer API command 'privileged' to determine if it should show buttons

Change 3) is useful to force it to read only with true if you want to give public access to the web page but ensure it always is readonly no matter what API access level you set in cgminer
Will merge all this stuff whenever my mining rig is back online and I can actually test any code I commit.
Smiley I just noticed that link was wrong - it points to miner.h not miner.php Smiley
Oh well I'm sure anyone (other than Xiangfu who is also using it other than me) who did want it, they would have worked that out.
The correct link was:
https://github.com/kanoi/cgminer/blob/master/miner.php
sr. member
Activity: 309
Merit: 250

Yep you really have run out of ram. The only time I've seen cgminer itself waste lots of ram is when people have the submit-stale feature enabled and it caches lots of old work and keeps trying to submit them for extensive outages, creating more and more network threads since there is no upper limit to how long it will hold onto this work and try repeatedly to send them. I guess I should make them all time out and give up in a future version. I don't ever suggest using the submit-stale feature and I also recommend enabling the --net-delay feature. Why does the submit stale feature exist then? Because people asked for it. Nowadays it's redundant in my opinion.


Thanks... I don't use the submit-stale option...  When i start it uses about 174 meg of mem, then a couple hours later, its using closer to 400 meg

this is my start script
Quote
/usr/bin/screen -dmS cgminer /opt/miners/cgminer/cgminer -Q 2 --api-listen --gpu-engine 700-780,700-780,700-810,700-805,700-820,700-815 --gpu-memclock 240 --auto-gpu --auto-fan --temp-target 75 -I 10 -o http://x:8332 -u x-p x -o http://x:80 -u x -p z


I'm not sure whats going on...
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
I found a new version of OpenCL (AMD APP, Stream SDK, whatever they want to call it) in a leaked version of a beta driver.  This driver supposedly gives Windows 8 support.  Anyway, I don't give a bleep about the driver, just the OpenCL versions.
amdocl(64).dll       10.0.898.1
opencl.dll             1.2.1.0
898.1 is the sdk that's distributed from driver version 12.2 onwards, but it depends on which OS/combo etc. Having a different opencl.dll I'm not sure what it means.
Jump to: