Topic: Mining-Rig 3*HD7990 -- Problem -- Freeze after 10-30 minutes (Read 12645 times)

can you share how could you solve ıt
i have powercolor freezing too

Hello all

Thanks all of you guys for resoluting this problem. I have a same issue with AX7990 (I have only 1). My system (Dell Precision T7500): 1100W 80+silver PSU, Intel Xeon 5520 2.33GHz, 12GB 1066 RAM, 500GB HDD 7200rpm. I've tried a lot of things, including things that you dealt with in this topic. I do not know what to do, you have any idea how to setup cgminer config for this card? In full load, card temperatures are 66C, in idle 36C. PC Photo: http://oi42.tinypic.com/1r6jvl.jpg , official utility from Powercolor (default) - http://oi42.tinypic.com/21lsnsp.jpg

Thanks

edit: problem solved

CV 1060, CC 1000, MC 1500..   oh and TC 8192 , Intensity 13

thanks for the reply i did try i13 with someone elses settings did work as well for me im getting a stable 1300 1350 from the good cards @ i20

my current settings

color 03

bfgminer --scrypt --temp-cutoff 98 --url=stratum+tcp://litecoinpool.org:3333 --userpass= --url=stratum+tcp://us.litecoinpool.org:3333 --userpass= --failover-only --thread-concurrency=24000 --intensity=20



ive not dropped the freq of the card but am running at 1160mv stable for heat i cant keep my loop cool enough without undervolting and if your looking at my setup and seeing bfgminer ive also tried cgminer 3.1.0 and 3.3.0 but cgminer wont read temps at all and bfgminer shows me temps but they dont match the cards as in they are the right temps but in the wrong order

If you reduce the core frequency, it might help.

what i learned was that you should never use intensity 20 on a radeon 7990 ....
im running with these settings at 1400 kHs with one card:

setx GPU_MAX_ALLOC_PERCENT 100
setx GPU_USE_SYNC_OBJECTS 1
cgminer --scrypt -I 13 -w 256 -g 2 --thread-concurrency 8192 -o.....

if you look at the hardware comp. on the litecoin wiki you see that this is the way to go on these cards
maybe thats helps your problem...

well thought id add my progress in trying to solve this, i have now cleaned up the cores and repasted using arctic silver 4, and have replaced all the thermal pads,

its amazing the difference new paste and pads on a clean surface helps its now the coolest card in my water loop even after another card the same has used the water first, my temps have dropped by 10c easy under load.

unfortunatly it hasnt fixed my issue, BUT for stability i was playing around with just using one side of the card or the other to see witch it was, now i can run the one half full speed ahead and its stable but the other side will run for about 5 mins on i20 and upto 3 hours on i12.

one thing i might try is running the one core thats bad on its own in my water loop so it doesnt get hot and see if it is caused by heat, my way of thinking here is if ive damaged the VRM chip on my board it might work fine untill it hits this temp, in witch case i should be able to see about getting that chip replaced.

thoughts anyone does that sound right to you guys?

ive been having the same problem with one of my cards im running to psus

1 ax1200i that power mobo cpu 1hdd and 2 7990's the "i" part of the psu allows me to see how much they are drawing well under

i also have a cx750 running 1 7990 thats the one thats been causing me issues with this 511c temp issue has anyone found a fix ive had to unplug that card, im going to redo the thermal pads and paste tomorrow and see if that helps, but if a B1tch as im running these 3 cards watercooled tis a pain breaking the loop to diagnose this card.

are we thinking this is a cgminer issue, as its still running cgminer with the other 2 cards and they run perfectly?

Many Thanks

James

Edited but, just looked at that photo put up its exactly the same card thats failing in my system too number 4, if that helps, ive switched the power for the cards around and tried running the supposed bad card on the big psu no different

in my case it was def. a PSU problem .. i switched to 2x1050W gold PSU and i did the best i could in splitting the power 50/50 on the two PSUs... now it works great and i hash with this rig at 4,2 MHs, while undervolting to 1060 and temps between 69 and 78 °C

Hi, I have the same issue with my hd7990
I have this strange temperature (511 °C), then it freezes

Did you solve your problem ?

Does anyone know the cgminer configuration of this set-up?

7990, AX7990 6GBD5-M4DH    1363          N    490    cgminer 3.7.2       500GB HDD, RAMdrive, 80C, i7-4770K, 16GB RAM, 2x7990 @ 2721kH/s

https://litecoin.info/Mining_Hardware_Comparison

I have a Devil 13 AX 7990 for CAD work and mining   but can't get it stable above 1100 kH/s

EDIT

Forgot to say that I had some freezing problems and tried to add more RAM with a USB stick on readyboost to see if that would help and it did. Had 2x4gb corsair vengence DDR3 and added 2x4gb usb sticks (for now).

Ah, I see, I didn't know cgminer displayed each core as it's own gpu like that.

Also, your card has 4096 stream processors with GDDR5 memory meaning your thread concurrency should be 4096 x 5 (20,480)

Another thing, the Gigabyte HD 7990 minimum requirements call for 750w power supply for each one.

Last, I don't know if cgminer still has this limitation or not, but I know that it used to have problems with using large amounts of video ram. Users with high amounts of video ram who wanted to use their cards at full power were using Reaper, however this information may be outdated as when I look at the hardware comparison charts it seems that everyone else with your card is using cgminer with no issues.

I feel pretty confident that the issue is your system is starving for more power.

He has 3 7990s, that's 2 GPU cores each card. Essentially six 7970 cores total.

If you have 3 x 7970's why does cgminer report 5 gpu's in use...?
http://imageshack.com/i/14te5vj

*************************
RE:  -thread-concurrency 8192
*************************

 Wow, that's for like a much lesser card than even my tame 7950's. Optimum for mine with 3 gig memory is 24000


 I have difficulty in understanding that your cards must be "limited" by way of the  -thread-concurrency because reducing the thread-concurrency does exactly that. What it amounts to is reducing the stress on the card while being far from the optimum thread-concurrency setting for each and every different AMD model GPU, not to mention the onboard memory size.

 There are references, maybe at the link above, or absolutely elsewhere to guide one on the range of possible optimal thread-concurrency settings. But for the life of me 8192 is far too low.

Your giving up at least by my guess as much as 75% or more of those cards band width -thread-concurrency 8192 if I understand that. .
************************************************************************************************************

 Under-volting cards is fine, but it's a real possibly that your particular card(s) are not up to par enough for such a low undervolt as that, and can't be stressed that far. Lowering voltage increases card stress, and decreases stability. I should know cause I locked my rig up right on the start of it by going too low on volts with experimentations. Also the lower the volts the higher the amps. It's just the law of volts times amps equals watts as I recall. Higher amps equals two things: HEAT and Higher STRESS. In the same sense it's the very reason many delicate electronics today now instantly shut off as soon as voltage drops too low, say on a phone, tablet, or what not. So to protect the delicate circuits from being cooked by too much amperage surging as voltage drops too far for the design. Something I learned 32 years ago...

 All I am getting at here is to maybe increase the voltage by a few volts and see how the rig responds. And if that helps then hopefully you can dial up the bandwidth ( -thread-concurrency) too. Your results will improve unless those cards are fundamentally different than all previous AMD GPU's.


 Adjust only one thing at a time and then re-test. Never more than one thing till you get very wise as to likely results and all.


 And maybe further refine, review other's thread-concurrency settings for those highest end cards of yours at that link above, and elsewhere here at bitcointalk, and maybe elsewhere such as wiki ?


 I got my system so stable and reliable even when away for weeks all I do is reduce the overclock some and it's good to go, no worries ever. Even when overclocked to my personal max it's never had any issues ever but I take caution if gone for weeks. Never any problems at all. But I used top quality parts and basically found out what others did that worked best and copied their successes. I also have a decorative wire fencing around my rig to keep pets and kids from harm, and the rig too.


 Anyway best of luck. And yea, the other person is correct: use the best power supplies, absolutely. Read customer reviews at newegg.com . And be careful with the powered risers as twisting those ribbon cables around can damage the internal wires and cause serious problems too, or even cause fires due to shorts or broken, frail fine gauge wires. More than a few rigs burned down due to not being built up to par. IE: high quality parts and components. It's a must to be safe. Your running those cards wide open 24/7/365. Same for the entire rig. Gotta be safe doing such things! No matter the cost. btw: A ups battery backup, my rig takes about 650 watts, and my 850 watt ups backup lasts at most 45 seconds, lol, but at least it catches most flickers and very brief lights out for a few seconds too. The ups also cleans and conditions the voltage as well as smooths and regulates the voltage output too. Very wise on expensive rigs. But not cheap. But dirty power kills tons of components. I wont run any desktop without a ups, a larger one, a quality one. Read the customer reviews...

 AS for 1000 watts. Check that link, look at the row where it's watts, and scroll down to your model card. That's where I got the 470 watts to 490 watts from. At any rate even close to 1000watts consumption on a 1000watt PSU is far too dangerous for 24/7 use even for the best PSU. One needs at least 20% headroom from what Tacotime (hero member) stated a long while back. So keep that in mind too. But the other card and mobo could easily get by on a decent but not great 1000watt PSU that you already have!!! Just add up the watts of all the connections or estimate them. Save 20% for startup surging and for plenty of head room. Running a PSU hot is a fire hazard times ten!


Best Regards!

Do you have the Malta 7990 (AMD reference design)? In that case the 7990s do not take 470 - 490w each. That would include the system itself. Card itself draws ~ 350-370. It is the most efficient card out there, especially when undervolted.

2000w is more than enough. However wattage isn't everything... the PSU may be lacking in amps.

Personally, I'd go with a  Corsair 1200w for the system + 2 cards. Then put the third one on a 750w.

Btw I had no problem undervolting both cores of a Malta 7990 on Win 7 64-bit using MSI Afterburner.

hey thank you for your detailed answer...

hardware part:
- i have 3 * hd 7990 ...
- i looked it up and the system uses only 1,7 gb ram so there is enough left
- i have 3 powered risers and managed to put 50% of the whole system on each PSU with my cable configuration
- nothing gets too hot on my mainboard or on the risers.. i checked... everything feels cool except the gpus of course
- each graphics card has 2 pcie power-cables and 1 molex at the powered riser cable ...
- i have two 1000W bronze PSUs... and each has to do 50% of the work... but i think they are not so good ... maybe thats really my problem ... i already ordered new ones.. my friends rig is working pretty fine with 2 gold1000W PSUs... but i think you are right... it is really tight dimensioned ... 1200 W PSUs are way more expensive ... lets hope the 1000W gold ones do the job..

software part:
- i tried the 3 different settings for all my cards at once...sorry for the bad explanation but of course i have the same setting for every card... but i found the best and most stable thing is this
--scrypt -I 13 -w 256 -g 2 --gpu-fan 100 --thread-concurrency 8192  thats really the best setting for this card (dont knwo why -g 2 but it is rocking i can tell)--> my friend runs the same rig really stable with this config and 3 * radeon 7990 at 4190KHs .. i think thats really nice... and with undervolting to 1055 nothing gets to hot (nothing is over 79°C and some gpus are at 62°C)!
- i think it is no driver problem because i reinstalled and tried several versions of the catalyst and appsdk drivers... and i did it carefully and correctly


thank you very much much for getting turned on by my problem
i report back when the new PSUs are here... you confirmed my assumption with your really nice list... thank you again

edit...
I gotta run:
--------------
NOT EDITED: I will try to do so later!
------------------------------------------

------------------
*not edited yet*
------------------


I will take a stab at this. Let's "work the issue".


Maybe this will help, I hope...see the PSU issue first. That is likely the issue. Or lack of Powered Risers, and or lack of secondary Video Molex outlet on mobo not having any juice (voltage) to it. Also is the motherboard designed to handle 3 such beasts as those 7990's


 1st - 2*4gb RAM = 8 gig correct? If so then maybe with (3) 7990's Cards your under memory requirements.
                                         
 Open up Task Manager on the Task Bar at the bottom of your desktop. Then click the PROFORMACE tab in Task Manager.
That will bring up both the CPU and Memory usage data and charts. Note how much memory is being taken by the entire system the first moments after the rig is mining. This may be your problem - not quite enough memory. But maybe not?

 I don't know how much memory is on the 7990's - let's look:

http://www.amd.com/us/products/desktop/graphics/7000/7990/pages/radeon-7990.aspx


OKAY:


Questions:

ARE YOU USING "powered risers" and are they getting voltage (plugged in to the PSU)

 And is the motherboard PCIexpress secondary voltage OUTLET  PLUG  (Molex likely)  for extra Video Cards have a Molex cable or whatever feeding the Mobo extra juice??? Your system is running Hot Rod style cards btw, those such energy like crazy!! (a PCIexpress secondary voltage OUTLET  PLUG  (Molex likely)  for extra Video Cards may or may not be on your board). If none is there and no powered risers either it's obvous your running a very dangerous fire hazard to say the least, and maybe why it's crashing quickly through voltage drops which triggers sky high amperage as a result (massive heat issues). And fire hazzard galore in your case, IF no powered risers plus NO secondary  PCIexpress secondary voltage OUTLET  PLUG on board your mobo In your case you need BOTH I would hazzard to merely guess. Others may chime in to assist but running those beasts will result in massive current flows.

 Is your power supply big enough? High Quality (Corsair Gold?, and highly rated model ? ), Is it a Single Rail, or more than one rail? Are you drawing too much from one rail on a dual rail or multi-rail PSU?

 Again,  Is your power supply big enough? - That's very critical. I would hazzard to even guess.




 I just reviewed the link below again. (while working this issue):

 Your cards require 470 watts to 490 watts EACH, according to those that reported testing them on their rigs. WOW!!!
Toss in your motherboard, hard drive, and other attached drives and what not and well one needs a 20% reserve capacity too at a minimum for safety (don't run a PSU at max output 24/7) so a 2000 watt PSU is required here!!! Or damn close to 2000watts. Or say 1 850 watt PSU for the mobo and 1st card. Then another 1200 watt for the 2nd two 7990's. That may result in solving this issue if your underpowered.

 LOL, okay, well AFTER THE FACT of writing the above I see you have two 1000 watt power supplies. So your likely running two cards off one of those PSUs correct ? If so your up against the wall so to speak with it's peak output. If not a damn fine PSU it wont likely be stable, much less long term reliable from demanding it's peak output be met 24/7.

 Try at least a 1200 Watt power supply (PSU) on those 2 cards that one is attached to and powering. IF THAT DOESN'T solve it then move on below. Better off though getting bigger than 1200 watt PSU since really those cards are far and away big drains. Gold Rated Corsair that is very well reviewed by users!!! Or whatever that size that is best reviewed whatever brand(s) are. But no way use any cheap PSU's that are not at least Gold Rated here.

 Trouble Shootings sometimes a bitch. I worked here for you cause electrical trouble shooting is one thing that turns me on, sadly...



Do you have 3  7990's or four ? Not so clear in that above? I take it just 3 cards from further review! I base all this on 3 cards. And wow that's a beast of a rig in terms of power requirements and energy costs. But hey it's all about hash #'s too now that every coin is tough to mine and growing in value so there ya go!



NEXT:

7990 = 6GB of GDDR5

So 6gig x 3 = 18gig.


 That's way more than say my 7950's (the most energy efficient cards to use). 7950's have 3gig as I recall each, or 9gig total for the 3 card rig I have.


 My memory consumption with my VPN Application & Service running for encypted, secure internet, plus Kaspersky Internet 2013 and Malware Antimalware Lifetime, along with C3 GUIMiner's running all together while mining
all together in the Performance tab inside Task Manager equals = 3.21GB

 Hum. I thought it would be a bit more memory consumption.

 I run (2) 8 gig sticks = 16gig total since I got them free with matching motherboards and wanted spare motherboards which already came to be needed since one board ended up bad quickly.

 So at any rate I don't know about this.


 At first I thought maybe it's not enough memory but now not so sure.


 Might want to re-install your AMD Video Drivers. Did you get them straight from AMD? That's the ticket. Same for the Uninstaller from AMD too.

 Any Screen Freezes and Hard Shutdowns can cause the Video drivers damage, and until re-installed, or simply clicking the uninstaller inside control panel and then clicking repair instead of uninstall will likely fix any driver issues. If not use the AMD uninstaller app, and then do a fresh clean install again. Maybe run Ccleaners Registry cleaner though first to clear out the old left over AMD crud. But if you  don't know what your doing then registry cleaning can be very dangerous.
Same for regedit and manual registry cleaning.

 At any rate maybe Drivers, or not enough memory even though it is seeming possibly okay in the Performance tab. Or bad memory.


 I never ran cgminer in command line by itself. But instead in GUIMiner-Scrypt which is well EASY. But I did wonder why your first card is set at 8192 but the 3rd one is at 24000, and the 2nd card doesn't display any data on your post??? :

--scrypt -I 13 -w 256 -g 2 --gpu-fan 100 --thread-concurrency 8192
--scrypt -I 18
--scrypt -I 18 -g 1 -w 256 --thread-concurrency 24000


 I have read and reviewed many posts on cgminer and it's real picky/different than using it inside GUIMiner-scrypt when it comes to I (intensity settings) and also sometimes even Thread Concurrency settings.

 Your I (Intensity setting) that seems to get a lot of folks in mesery with cgminer if they set it too high. Results in hardware errors. Evidently it must be lowered to result in no hardware errors for best mining results, plus of course even to resolve cgminer freezing up/screen freezes, etc.

 -1 13 sounds correct for my 7950's as about as far as one is said to safely go with cgminer in the command line by itself, as I recall but am not too sure Where as with cgminer inside GUIMiner-Scrypt I can usually run all the way dialed up to I 20, or at least I 19 without too many Stales resulting (<~3%). Much higher then I reduce the Intensity, but inside the GUI of GUIMiner for cgminer we have no reporting of hardware errors itself. I still wont switch to the bare cgminer since GUIMiner-Scrypt is so nice, easy, stable and well I can quickly switch from pool to different coins on the fly or even mine three different coins and three different pools quite easily. I don't know if cgminer alone can do that trick near as easy, I doubt it. Oddly some settings such as I for cgminer vary drastically if used with GUIMiner-scrypt compared to using it alone in the command line setting.

 The I 18 setting of yours may be far too high, or not Verify that if your not certain.
 And why??? do you have the 1st card at I 13 but the 3rd at I 18, makes no sense if their identical cards or even the same series (7990's)

 The best thread-concurrency for all my many scrypt mining has been 24000 btw, but that is for 7950's and whats best for them. See below at the link to find your ideal settings or at least what's been effective. That's ideal. But 8191 is far too low for say my 7950's, let alone your fire breathing hot rod 7990's! Just remember that thread-concurrency must always divide by 64 resulting in a whole number (no fractions) for smooth mining results, for whatever reason(s).

 See the Link below and other threads (do a search) to find the best cgminer settings for your 7990's. That also may be where the issue is unless your certain your setting are correct. But for the life of me I know 8192 is not correct ?!!?

 The -g 2 what is that??? GPU threads maybe? For the correct settings there see the link below as well but even though some mining software sets that at 2 GPU threads for my 7950's they work ever so slightly better at 1 GPU thread. Not sure why or whatever but yours may well be very different for that settings. See below.



 You got some trouble shooting to do. Work the problem. One step at a time. Verify everything down this list if need be.

 The first place I would zoom to is test the rig again and check in the Performace tab for total memory usage and what's available still. That likely will check good? Maybe, or NOT!!!

 Then adjust the -I down to 13 or even 12 for ALL Cards, just to see if that helps, if so then find the correct I settings for your card. In fact find the correct settings (VERIFY below) to make certain that is correct before doing any more risky settings adjustments!



https://litecoin.info/Mining_Hardware_Comparison



...

7990, CGAX-7999    1125    1075    1700          cgminer v 2.11.4    -I 13 --thread-concurrency 14336    84C, Catalyst 13.1, https://imageshack.us/scaled/large/607/proof7990.png

7990, CGAX-7999    1150    1090    1725          cgminer v 3.1.0    -I 13 -g 1 -W 256 --thread-concurrency 8192    87C, Catalyst 13.1, http://img24.imageshack.us/img24/5343/imagetfc.jpg

7990, GV-R799D5-6GD-B    1250    980    1350    1.050 N       cgminer 3.4.0    -I 13 -g 2 -w 256 --thread-concurrency 14336 --gpu-powertune 20    Windows 8, 70-80C, i5-2500k, ASUS P8P67

7990, 102-C47601-00-AT    1325    1025    1500    1.075 N       cgminer 3.4.0    -I 20 -g 1 --thread-concurrency 24000 --gpu-powertune 20    Windows 7 2061, 66-79C, 6Z77-V DELUXE, 3770K, 16GB RAM
(M379B5273DH0-YK0), 2 x 7990, CoolerMaster Sniper case, AX1200 PSU, 660-670kH/s per GPU, 113-C47601M0-101 firmware

7990, R7990-6GD5    1344    965    1360    1.050 U    470    cgminer 3.7.2    -I 13 -g 2 -w 256 --thread-concurrency 8192    Windows 7, 87C, Celeron G530

7990, AX7990 6GBD5-M4DH    1363          N    490    cgminer 3.7.2       500GB HDD, RAMdrive, 80C, i7-4770K, 16GB RAM, 2x7990 @ 2721kH/s

7990, HD7990-6GD5    1518    1100    1480          cgminer 3.2.2    -I 13 -g 2 -w 256 --thread-concurrency 8192 --gpu-powertune 10    60C, Catalyst 13.9, FX-8320, 3x7990, watercooled




Caveat emptor - let the buyer beware!

ok cool i didn't know that there is a 64bit version.. juhu ... thanks a lot!! my friend is really happy too because he didnt knew as well... the temps drop around 8-10° thats insane ^^

after disabling the catalyst centre & co... it still crashes after a few minutes.. im pretty sure now that my PSUs are doing some crazy shit... new ones are on the way...