Topic: 7/8 GPUs per Motherboard (Read 9614 times)

I just came back from the basement myself where I was resetting one of the circuit breakers because it couldn't not handle 1750w of power. I do have multiple circuits in the house too (it was built 2007, IL) ranging from 15 to 30amps, but all the circuits I can plug in my rigs, are 15 amps. I guess I could just replace them with 20 amps breakers but then I would have to check if wires are up to it.

Need to nudge you there - US circuits vary from state to state and what year the house was built in.  This is further broken down as per what is the wiring in the house used for (kitchen or washer/dryer units tend to have higher limits).  It is not uncommon for houses in the US to have a mix of 15 amp and 20 amp breakers.  My house is almost entirely wired up with 20 amp breakers (and I still overload them).

My 5 x 7950 rig fluctuates between 1575 and 1600 watts.  I have them load balanced between a 900w and a 850w.  

Because 7950 went down in price recently (I get them for $199), so they cost 3x as much as 6770 and at the same time they are 3.5 times more powerful than 6770 (hashing power). In addition I don't have to buy as many motherboards/cpus/memory/OS/hdd to run the same hashes. For 3250Khs (1 rig of 5 cards) I'm only getting one set of motherboard/cpu/memory/os/hdd.

8x6770 would produce about 1600Khs in one rig, right? So you would need 2 sets of hardware to run 3200Khs.


PSU would be the same because $/W of power would be about the same for 6770 and 7950 GPUs.

I only use 6770 since they cost the equivalent of 66.5$ (new with shipping included) where I live.
I undervolt them to 0.8v (stock is 1.125v) so having 8 on a PSU is not a big deal.

But yeah, idk for 7950...

I wish there was a PSU with enough power to feed two motherboards with 8 GPU each. Usually the reverse is true: you would need 2 PSUs just to provide enough power to run 8 GPUs in one motherboard.

7950 card would take about 270-300W (mining scrypt), even if you have 5, it is already 1500W for GPUs only.
5970 card would take about 330w to run, so 4 of them connected to the one motherboard is 1320 for 4x5970 only.

Even if there was a PSU rated at 3000W, where would you plug it? You would need 30amp circuit to plug it in, and regular US circuits are only rated at 15amp, which is 1650W.

Just for your information : It is possible to run 2 motherboard with one PSU. If you can easily find cheap motherboard/gpu/ram that accept 4+ GPU (using pcie1x) then run 8+ GPU with one PSU.

I know that's not exactly what you aim for but it can be an alternative.

I have been told that this is a limitation of the OS.

I have heard that 8 GPU limit is imposed by the PCI express design or operating system design. I never seen anybody who run more than 8 GPUs on the same motherboard. Please post a link if you have seen otherwise.

I have one of these boards in my setup.  The board has no problem recognizing 6 physical cards.  There seems to be a quirk with this board I have found that prevents it from recognizing card #7.  Seems that the board disables one of the 1x slots when the the adjacent 1x slot is occupied.  Found a couple of YouTube vids confirming this issue.

Under that premise, I see no reason why you couldn't load this board up with 12 GPUs.  I do not have any way of testing this though since all of my cards are single GPU.  I'd love to see pictures of something like that.

I settled with 5 x 7950 cards on mine.

Edited for clarification.

Red = 2 GPU Cores on on card

I don't get it. Which GPU count double ?

I was finally able to run 8 GPU under Windows 7 Home 32bit:

7 GPUs are mining Scrypt and 1 Card 7970 is mining SHA256 (not enough memory to run it without hardware errors)

So, 3x5970
1x5830
1x7970

@ssateneth, several questions about your 8 GPU MSI Z77A-G45 setup.

1.  You mention that you have to use a processor with an Ivy Bridge core due to IRQ conflicts that disable the 8th GPU, there are number of Ivy Bridge cores of widely varying capability, and I assume you're using these for Scrypt mining, so does it matter which processor you get as long as it's Ivy Bridge?  Would Scrypt mining benefit from the increased processor cache from the i3 and i5 CPUs?

Ivy Bridge CPUs:
Celeron various G440 (1.6 GHz) through G1620 (2.7 GHz)
Core i3-3240, 3245, 3250
Core i5-3570, 3570K, 3770, 3770K
Full List at the bottom of this page: http://www.cpu-world.com/CPUs/Celeron_Dual-Core/Intel-Celeron%20G1620.html

2.  The MSI Z77A-G45 only has 7 PCI-e slots, so I'm assuming that you are using at least one 5970 to get up to 8 GPUs with 7 slots, does it matter which slot the dual GPU goes in and/or have you noticed problems using the dual GPU in any specific slots?  I've heard from other forums that sometimes using a dual GPU will disable adjacent PCI-e slots depending on the mobo config (with or without jumpers).

3.  Do you have to use the PCI-e 1x jumper procedure on the MSI Z77A-G45 mobo?

4.  If you do scrypt mine, how much system memory is required for your 8 GPU setup on the MSI mobo?  I've heard sometimes 1GB per GPU and other times as much GB as the video cards have, but everyone seems to agree that memory is important for scrypt mining.

Much appreciated on your informative posts here, packing as many GPUs per mobo as possible really helps to cut overall costs.

I don't know how to solve your hangs.
Several points:
- Disable Sleep option on your computer and disable "monitor switch off" after some period of time.
- Make sure you monitor is always connected to the same card and to the same port - if you change it, your cards will be mapped differently on reboot.
- Set intensity lower on the card/slot where the monitor is connected.
- Set resolution to a lower level of 1024x768 and only enable one monitor.
- if you are only using 5970 cards, you can just replace couple of files inside c:\windows\system32 (OpenCL.dll and amdocl.dll
) to get to downgrade to 2.5 SDK which is the best for 5970 cards.

Here is how to correctly map GPUs:
- Launch cgminer with one GPU enabled (0 for example), press g inside cgminer and see which GPU has activity inside it, if it is GPU 2, then your first mapping would be 0:2
- Disable GPU 0 and enable GPU 1, see the activity, if you see activity inside GPU 3, then your second mapping is 1:3
- Disable GPU 1 and enable GPU 2, see the activity, if you see activity inside GPU 0, then your third mapping 1:4
and so on

once you done for all the GPUs, put the following into cgminer.conf:
"gpu-map": "0:2,1:3,2:4....",

To combat heat, you should map GPUs correctly first, so auto-fan and auto-gpu would actually regulate the fan speed and the GPU speed of a right GPU. For the overheating card, enable auto-gpu  so it would auto adjust engine speed based on temperature and set fan range to 30-90, then set max GPU speed to 825 and memory speed to 1150, it would allow you to lower the voltage to 1050-1074 for this card only and would still hash at about 383MH.

Please consider my tool to always mine the best coin: https://bitcointalksearch.org/topic/ann-jsonkarasoftcom-scryptsha256-profitability-managerswitcher-260940, it will switch your cgminer to the most profitable coin based on current difficulty and exchange rate. (if you like it, you may click donate button to get even better results   )

Bless you man...got my rig working on scrypt. Now only 2 MH but thats because the damn XFX gets overheated and i have to run it on stock. But really appreciate it man, i had more or less given up on scrypt. And with difficulty rising, i think it is the way to go. Once again thanks 

will donate to you once i am able to mine some coins...

One question through? on my MSI Z77A-G45, rig gets hanged sometime, and than the 5970 dont work/get recognize on the same slot (whether PCI 1x or 16x)...after i change the card to some other slot they start work, have you ever faced anything like it ?Also you said regarding mapping 5970, how does that work?

I have found a crude of maping by changing cards in different slots etc. would be interested to know your way/thoughts?

Once again thx  .....

thx man. As always valuable insight from a seasoned miner. Appreciate it

CGMiner can only set to voltages that are programmed into the programmable VRM. On a stock 5970, these voltages are 0.95, 1.0, 1.0375, and 1.05. Trixx and afterburner uses other methods to get the voltage set to different steps.

I use program called TRIXX (sapphire tweak utility), works great for 5970, Somehow cgminer always fails in setting the voltage for 5970, it works for other cards, just not for 5970.

1600 PSU should be enough to run 3x5970.

Thx for the detailed guide.

One question ? how to set voltage ? i thought cgminer does that when we state the voltage in cgminer.config?

I have a LEPA 1600 gold, do you think it will be enough?

I'm getting 394Khs per GPU core, so 394x6=2364Khs total

do you see all your GPUs inside Device Manager without yellow icons. if so, you will be ready to run your 5970 to mine scrypt.

Here is what I have inside cgminer.conf:
CGMiner id 3.3.1

...
"intensity" : "18,18,18,18,18,18",
"vectors" : "1,1,1,1,1,1",
"worksize" : "256,256,256,256,256,256",
"kernel" : "scrypt,scrypt,scrypt,scrypt,scrypt,scrypt",
"lookup-gap" : "0,0,0,0,0,0",
"thread-concurrency" : "6400,6400,6400,6400,6400,6400",
"gpu-engine" : "840,840-840,840,840,840,840",
"gpu-fan" : "30-85,50-85,50-85,50-90,50-90,50-92",
"gpu-memclock" : "1180,1180,1180,1180,1180,1180"
...

All GPU cores are set to 1099 voltage (do it before you run cgminer)

When you start cgminer, do it for 1 card first and then enable other GPUs one by one to make sure you have enough power to run it:

cgminer -d 0

Make sure that your GPUs are correctly mapped (if not, you will have to remap them manually, it is always the case for me. You would see that they are not mapped correctly if you have 1 gpu enabled but activity would show up for another gpu, let me know if it is the case for you, I will tell you how to map them correctly)

If your machine doesn't have enough power in PSU, the computer will just reboot or hang when you enable enough power draw. You need at least 1250W PSU to run 3x5970.

I recomend enabling 1 GPUs inside each card initially, wait for couple of minutes for the temperature and fan to reach about 80 degree and 4000 rpm, only then start enabling second GPU inside each card.

Let me know how it goes