Author

Topic: Maximum Cards Per Rig (Win 10) (Read 2957 times)

member
Activity: 75
Merit: 10
December 30, 2017, 02:18:37 PM
#18
Also note that the Win10 "Fall Creator's Update" from October(?) did specifically increase the limit, though I don't know what the maximum is now.
full member
Activity: 322
Merit: 105
December 30, 2017, 09:13:56 AM
#17
I have 9 nvdia on windows without flaws, only problem is afterburner not working with more than 8 cards well.

I had to play a little with what card was on each place to be  detected

Intel g4560 (support 16 pcie lanes)
Msi z170 gaming 7 ( 7 slots)
Two cards on mainboard
One card with m2 to pcie adapter and riser
Three cards with risers
Trhee cards with risers on same pcie with 1-4 multiplier


Im trying now to build a 12 nvidia cards rig


ummm , u are the first that I see running 9 NVidia gpu on windows  everyone else was saying that the limit is 8 gpu
that is interesting indeed
anyway first i will have to try 7 cards rig then i can consider adding the 8th one with the M.2 slot


I dont know if it is possible due to differents models, i have 3x1060,3x1070 an 3 1080ti

sr. member
Activity: 672
Merit: 252
Until the end
December 30, 2017, 08:19:04 AM
#16
Server PSU's can be used to just run the 12v supply for GPU's, so you can have a standard ATX PSU plus a server PSU to run everything 12v

The HP 1200 power supplies are generally a lot more cost-effective too than regular name-brand power supplies.  They were also built with reliability in mind.
newbie
Activity: 32
Merit: 0
December 30, 2017, 06:55:46 AM
#15
I have 9 nvdia on windows without flaws, only problem is afterburner not working with more than 8 cards well.

I had to play a little with what card was on each place to be  detected

Intel g4560 (support 16 pcie lanes)
Msi z170 gaming 7 ( 7 slots)
Two cards on mainboard
One card with m2 to pcie adapter and riser
Three cards with risers
Trhee cards with risers on same pcie with 1-4 multiplier


Im trying now to build a 12 nvidia cards rig


ummm , u are the first that I see running 9 NVidia gpu on windows  everyone else was saying that the limit is 8 gpu
that is interesting indeed
anyway first i will have to try 7 cards rig then i can consider adding the 8th one with the M.2 slot
full member
Activity: 322
Merit: 105
December 30, 2017, 05:33:08 AM
#14
I have 9 nvdia on windows without flaws, only problem is afterburner not working with more than 8 cards well.

I had to play a little with what card was on each place to be  detected

Intel g4560 (support 16 pcie lanes)
Msi z170 gaming 7 ( 7 slots)
Two cards on mainboard
One card with m2 to pcie adapter and riser
Three cards with risers
Trhee cards with risers on same pcie with 1-4 multiplier


Im trying now to build a 12 nvidia cards rig
hero member
Activity: 812
Merit: 500
December 30, 2017, 03:19:34 AM
#13
You can burn your motherboard if you start playing how many gpu can connect.
newbie
Activity: 32
Merit: 0
December 30, 2017, 02:57:09 AM
#12
i am very interested in this topic as well and was hoping that someone with good experience can make things more clear here
right now I am running a 6 gpu mining rig with the asus prime Z370A  motherboard which have 7 pcie slots and two slots of m.2 which can be used
as a pcie slots as well , I have seen people run 9 gpu on that motherboard but I have some doubts that it would be an easy setup
right now my question is while I am using the built in vga port as my monitor primary port what would be the maximum number
of cards that I can run with it  7 gpu + the built in  or 8 gpu + the built in  also if I am to use the two ports of m.2 slots would it be possible
to run it on windows 10 ?
sr. member
Activity: 305
Merit: 250
December 04, 2016, 10:47:31 AM
#11
Is there a way to virtualize these fixed IRQs?
Shut down USB system clock and other things to free IRQ

How do servers handle multiple boards at once?

I think that has to do with sharing pcie lanes using alternate timings of sharing lanes. (My puny understanding)

Other thing is that consumer boards and processors have restricted number of lanes free. Server boards and processors such as 2011 sockets have 40 lanes free. So you can theoretically add 40 GPUs to one system.


I think it does not have much to do with the limit of available lines
https://en.wikipedia.org/wiki/PCI_Express
newbie
Activity: 19
Merit: 0
December 04, 2016, 10:39:43 AM
#10
Server PSU's can be used to just run the 12v supply for GPU's, so you can have a standard ATX PSU plus a server PSU to run everything 12v
sr. member
Activity: 546
Merit: 250
It takes a lot to build but not much to lose
December 04, 2016, 10:35:35 AM
#9
Is there a way to virtualize these fixed IRQs?
Shut down USB system clock and other things to free IRQ

How do servers handle multiple boards at once?

I think that has to do with sharing pcie lanes using alternate timings of sharing lanes. (My puny understanding)

Other thing is that consumer boards and processors have restricted number of lanes free. Server boards and processors such as 2011 sockets have 40 lanes free. So you can theoretically add 40 GPUs to one system.
sr. member
Activity: 305
Merit: 250
December 04, 2016, 10:27:31 AM
#8
Is there a way to virtualize these fixed IRQs?
Shut down USB system clock and other things to free IRQ

How do servers handle multiple boards at once?
member
Activity: 98
Merit: 10
December 04, 2016, 09:57:54 AM
#7
Lets say we have a board which easily supports 7 PCI-E connectors thus 7 cards. But now the question is, what happens in case we take 7 PCI-E multipliers from 1 to 3 and turn our 7 motherboard PCI-E slots into 21 slots. Will Windows 10 and the board detect all of them? If not then what is the maximum achievable config?

First, you are limited by the PCIe lanes that the motherboard/cpu can offer, and skylake sucks there.
Then you are limited by the number of GPU's a consumer grade OS can support, which, for Windows 10 is 8.
Then you are limited by the IRQ addressing of the hardware in the OS which is what causes issues for people using PCIe 1 to 3 splitters and over 7 cards.

Also, even IF you had 21 slots, you would need a 3000W powers supply atleast and that alone would cost more than 3 sets of motherboards, CPU's and 1000W power supplies.
I have personally found 5 cards to be the sweet spot in terms of VFM. Or if you want to maximize resale and use current gen hardware, go for 4 card setups.

1) PCI-E Lanes not a problem.
2) Windows 10 - 8 GPU limit, that's a real limiting factor.
3) IRQ addressing in the OS, I guess this is linked to (2)

4) Power supply not a problem, can easily use multiple 700W to 1000W PSUs.
 
So currently there is no OS option to support over 8 cards?
full member
Activity: 168
Merit: 104
December 04, 2016, 09:16:48 AM
#6
can you give me links to mobos good for rigs which can handle 7 gpu's?

msi z97 gaming 5... you should sub to my YT channel, i have a how to build video using that board up

now using a 1-3 multiplier on that board does not yield more GPUs being used - the board will post and boot up , but it still only uses 7 GPUs.

Now i know people have gotten it to work with 8, but only seen it in ubuntu based distros.. and with a said "modified drivers"

so I would also like to know - how do we get 8 working on that board in windows 10?

the 1-3 multiplier of which I also have a video of online, will get 7 working on PRO BTC or Asrock Anniversary, but trying to use 8 will yield a black screen and no post/boot.

however the z97 gaming 5 msi does boot with many more attached through multipliers - it just doesnt want to use them.

So i like where this thread is heading and I want to learn more

I just met a guy who runs a huge farm in china, so I am curious if he can teach me how to get 8 working.
sr. member
Activity: 466
Merit: 500
December 04, 2016, 09:12:28 AM
#5
Lets say we have a board which easily supports 7 PCI-E connectors thus 7 cards. But now the question is, what happens in case we take 7 PCI-E multipliers from 1 to 3 and turn our 7 motherboard PCI-E slots into 21 slots. Will Windows 10 and the board detect all of them? If not then what is the maximum achievable config?

First, you are limited by the PCIe lanes that the motherboard/cpu can offer, and skylake sucks there.
Then you are limited by the number of GPU's a consumer grade OS can support, which, for Windows 10 is 8.
Then you are limited by the IRQ addressing of the hardware in the OS which is what causes issues for people using PCIe 1 to 3 splitters and over 7 cards.

Also, even IF you had 21 slots, you would need a 3000W powers supply atleast and that alone would cost more than 3 sets of motherboards, CPU's and 1000W power supplies.
I have personally found 5 cards to be the sweet spot in terms of VFM. Or if you want to maximize resale and use current gen hardware, go for 4 card setups.
I used 7 cards on win10, right now I'm using 6 cards on win 7 ( and other 6 cards on win 10) I'm just testing... But your point of view is valid.
hero member
Activity: 751
Merit: 517
Fail to plan, and you plan to fail.
December 04, 2016, 08:21:16 AM
#4
Lets say we have a board which easily supports 7 PCI-E connectors thus 7 cards. But now the question is, what happens in case we take 7 PCI-E multipliers from 1 to 3 and turn our 7 motherboard PCI-E slots into 21 slots. Will Windows 10 and the board detect all of them? If not then what is the maximum achievable config?

First, you are limited by the PCIe lanes that the motherboard/cpu can offer, and skylake sucks there.
Then you are limited by the number of GPU's a consumer grade OS can support, which, for Windows 10 is 8.
Then you are limited by the IRQ addressing of the hardware in the OS which is what causes issues for people using PCIe 1 to 3 splitters and over 7 cards.

Also, even IF you had 21 slots, you would need a 3000W powers supply atleast and that alone would cost more than 3 sets of motherboards, CPU's and 1000W power supplies.
I have personally found 5 cards to be the sweet spot in terms of VFM. Or if you want to maximize resale and use current gen hardware, go for 4 card setups.
newbie
Activity: 18
Merit: 0
December 04, 2016, 08:14:42 AM
#3
can you give me links to mobos good for rigs which can handle 7 gpu's?
full member
Activity: 251
Merit: 100
December 04, 2016, 06:42:57 AM
#2
Maximum count of cards for win 10 - 8 cards per rig.
For me 8 won't run, 7 runs ok
member
Activity: 98
Merit: 10
December 04, 2016, 06:30:53 AM
#1
Lets say we have a board which easily supports 7 PCI-E connectors thus 7 cards. But now the question is, what happens in case we take 7 PCI-E multipliers from 1 to 3 and turn our 7 motherboard PCI-E slots into 21 slots. Will Windows 10 and the board detect all of them? If not then what is the maximum achievable config?
Jump to: