Pages:
Author

Topic: 8 - 11 GPU Rig, which PSUs to choose? (Read 1218 times)

kjs
full member
Activity: 188
Merit: 105
June 28, 2017, 11:13:59 PM
#25
I've had multiple systems with 11 x NVIDIA 1080 Ti's and 11 x AMD RX 580's running simultaneously without any issues.
Could you please tell me, which components you used at the Nvidia system?
Why don't you run that systems any longer?

Who says I don't run the systems any longer?

SMC X9DRX motherboards
EVGA 1200W PSUs
PCIe Version 007S risers
NVIDIA 1080 Ti FE GPUs
Sapphire AMD RX 580 GPUs
Ubuntu 16.04 LTS
Add2PSU adapters

Nothing particularly special or out of the ordinary there.
full member
Activity: 154
Merit: 100
June 28, 2017, 07:08:30 PM
#24
I've had multiple systems with 11 x NVIDIA 1080 Ti's and 11 x AMD RX 580's running simultaneously without any issues.

Great then point to a guide with the specifics of how it is done.  As I said most of the posts here complain about not being able to run Nvidia cards on PCI-E splitters or getting them working above 7 cards.  IF you were able to find a combination of parts that allows it please share the info.
full member
Activity: 184
Merit: 101
June 28, 2017, 06:33:14 AM
#23
I've had multiple systems with 11 x NVIDIA 1080 Ti's and 11 x AMD RX 580's running simultaneously without any issues.
Could you please tell me, which components you used at the Nvidia system?
Why don't you run that systems any longer?
kjs
full member
Activity: 188
Merit: 105
June 27, 2017, 10:06:12 PM
#22
Nvidia cards do not work with PCIe splitters as far as I've seen.  If you get a ASUS Prime z170-A or Pro you can probably get 7 cards working, but you'd need to likely have 2 - 850W+ PSU's in order to not fully max them.  A single 1500W might work as well, but you'd likely be pushing it.

Make sure you have a 20A breaker on the circuit you use or split between two circuits.
Thanks for that information.
Wyh do you think only 7 cards will work on Asus Z170-A? It has 7 PCIe ports + 2 M.2 slots. The M.2 to PCIe adapter is an "adapter" not a splitter, so at least 8 cards should be possible (i thought). Or am I wrong?
Why the 20A breaker? You mean the thing in the fuse box?

People have struggled getting that many Nvidia cards working simultaneously.  You might be able to get 8 working, but it will likely take a lot of fiddling and may not work.   Nvidia's draw more power and require more PCI-E resource channels than AMD cards so at least on Windows it tends to complain and although will see the card may not work properly.  Look around there are plenty of people who have tried to do what you are doing, but as I've said I haven't seen a lot of luck with more than 6-7 Nvidia cards at once on the same system.

As for the 20A breaker it's because you will get roughly 1800W of total power at 100% load out of a 15Amp breaker.  

Assuming you're in the US with standard 120v home power.
Ohm's law = 15 amps x 120 volts = 1,800 watts

You don't want to run at full power otherwise you run the risk of popping the breaker when you have power spikes and such.  Circuits can also get hot and cause fires if they are run too close to maximum capacity 24/7.  It's just better to be safe than sorry IMO.  There have been plenty of stories of people having meltdowns/fires because they didn't pay attention to how much power they were trying to draw, especially on older wiring.  Also I presume you will have things like a monitor for setup and potentially fans running for cooling that will likely be on the same circuit.  All of those things take more power and even though you may plug them into different outlets most home power circuits will have all the outlets in a room on the same circuit.  Most home breakers are only 15A unless you are looking at a utility, laundry, or kitchen.

I've had multiple systems with 11 x NVIDIA 1080 Ti's and 11 x AMD RX 580's running simultaneously without any issues.
full member
Activity: 184
Merit: 101
June 27, 2017, 02:19:42 PM
#21
Also this may not be a profitable rig, profits are rapidly dropping, and you may not breakeven before profits go to under a $1 a day.
Thank you for that hint. Wyh do you think it won't be profitable and which components would you suggest instead of my planned ones?
Is ETH the most profitable currently?
full member
Activity: 223
Merit: 100
June 27, 2017, 11:58:30 AM
#20
Yep in Europe should be plenty of power, but still something to keep in mind.   As I mentioned I was speaking from a US perspective.

Indeed you did  Grin

Why with the 120V USA I do not get it  Huh

Well the US was the first to get a power transmission system originally, but it was also during a period when Thomas Edison was arguing that AC power was dangerous compared to DC power.  Even though AC power was ultimately selected as the winner the influence of Edison's DC +110V - 0 - -110V 3 wire system influenced the ultimate design.  110V DC was the maximum voltage that could be used and not burn out the carbon filament of the lights at the time.  Europe later started with 110V, but switched to 220V because it allowed the power company to increase capacity without changing out the wiring at the time.  From there is was just whichever standard was adopted and never changed because the equipment switch was always far too expensive.

Very interesting, thanks  Smiley
full member
Activity: 154
Merit: 100
June 27, 2017, 11:43:27 AM
#19
Yep in Europe should be plenty of power, but still something to keep in mind.   As I mentioned I was speaking from a US perspective.

Indeed you did  Grin

Why with the 120V USA I do not get it  Huh

Well the US was the first to get a power transmission system originally, but it was also during a period when Thomas Edison was arguing that AC power was dangerous compared to DC power.  Even though AC power was ultimately selected as the winner the influence of Edison's DC +110V - 0 - -110V 3 wire system influenced the ultimate design.  110V DC was the maximum voltage that could be used and not burn out the carbon filament of the lights at the time.  Europe later started with 110V, but switched to 220V because it allowed the power company to increase capacity without changing out the wiring at the time.  From there is was just whichever standard was adopted and never changed because the equipment switch was always far too expensive.
full member
Activity: 223
Merit: 100
June 27, 2017, 11:28:35 AM
#18
Yep in Europe should be plenty of power, but still something to keep in mind.   As I mentioned I was speaking from a US perspective.

Indeed you did  Grin

Why with the 120V USA I do not get it  Huh
full member
Activity: 154
Merit: 100
June 27, 2017, 11:26:32 AM
#17
As for the 20A breaker it's because you will get roughly 1800W of total power at 100% load out of a 15Amp breaker.  

Assuming you're in the US with standard 120v home power.
Ohm's law = 15 amps x 120 volts = 1,800 watts

You don't want to run at full power otherwise you run the risk of popping the breaker when you have power spikes and such.  Circuits can also get hot and cause fires if they are run too close to maximum capacity 24/7.  It's just better to be safe than sorry IMO.  There have been plenty of stories of people having meltdowns/fires because they didn't pay attention to how much power they were trying to draw, especially on older wiring.  Also I presume you will have things like a monitor for setup and potentially fans running for cooling that will likely be on the same circuit.  All of those things take more power and even though you may plug them into different outlets most home power circuits will have all the outlets in a room on the same circuit.  Most home breakers are only 15A unless you are looking at a utility, laundry, or kitchen.

He is located in Austria so 230V is standard in Europe.

15A * 230 = 3450W

Will probably have at least 2 x 15A circuits for the normal plug sockets around the house.

Normally a further 18A or 20A for the kitchen/utility room.

You could run easy double the amount of GPU you are going for spread around the house.

Yep in Europe should be plenty of power, but still something to keep in mind.   As I mentioned I was speaking from a US perspective.
full member
Activity: 223
Merit: 100
June 27, 2017, 11:20:04 AM
#16
As for the 20A breaker it's because you will get roughly 1800W of total power at 100% load out of a 15Amp breaker.  

Assuming you're in the US with standard 120v home power.
Ohm's law = 15 amps x 120 volts = 1,800 watts

You don't want to run at full power otherwise you run the risk of popping the breaker when you have power spikes and such.  Circuits can also get hot and cause fires if they are run too close to maximum capacity 24/7.  It's just better to be safe than sorry IMO.  There have been plenty of stories of people having meltdowns/fires because they didn't pay attention to how much power they were trying to draw, especially on older wiring.  Also I presume you will have things like a monitor for setup and potentially fans running for cooling that will likely be on the same circuit.  All of those things take more power and even though you may plug them into different outlets most home power circuits will have all the outlets in a room on the same circuit.  Most home breakers are only 15A unless you are looking at a utility, laundry, or kitchen.

He is located in Austria so 230V is standard in Europe.

15A * 230 = 3450W

Will probably have at least 2 x 15A circuits for the normal plug sockets around the house.

Normally a further 18A or 20A for the kitchen/utility room.

You could run easy double the amount of GPU you are going for spread around the house.
full member
Activity: 154
Merit: 100
June 27, 2017, 11:18:00 AM
#15

Also this may not be a profitable rig, profits are rapidly dropping, and you may not breakeven before profits go to under a $1 a day.

Depends on what you are mining.   I have a 5x1080 rig that is still pulling in $40+ per day so it has a long way to go before it is below $1 per day.
newbie
Activity: 28
Merit: 0
June 27, 2017, 11:13:00 AM
#14
Hi,

I am going to build an 8 - 11 GPU Rig (depends on how many cards I can get to run on an Asus Z270-A mainboard (using 1 M.2 to PCIe adapter and also 1 PCIe to 4 Port PCIe adapter, see pic).
I will use only NVIDIA 1060 (some 3 GB / some 6 GB) cards, cause others are extremely overpriced.

I would prefer to use 2 seperate PSUs, cause it's much cheaper than 1 big PSU.
Would 2 700 Watt PSUs (be quiet! Pure Power 10-CM 700W) be enough to power the whole rig?
I am planning to undervolt (65%) and overclock the cards (core +120, mem +850) and think 1 card will user 90 - 110 Watt.
If I am right, the maximum for 11 GPUs would be around 1210 Watt and then there would be some power for the rest of the system (CPU, RAM, etc.).
Or is this too less power?

If I use 2 PSUs, will I have to use a special link cable, between the PSUs or will it also work if I just connect the additional cards to PSU2?

Do you think, that these 1 PCIe to 4 port PCIe adapter will work on the Asus Z270-A mainboard? Or will even more than 1 of that adapters will work on that mainboard?
https://i.imgur.com/PmsZeyY.jpg

Last question:
The Asus Z270-A mainboard has 2 x M.2 slots. I will use one slot for M.2 to PCIe adapter. So can I run my system using a M.2 SSD in the second M.2 slot or is it blocked cause of the use of slot 1? I am asking, cause I did read, that only one M.2 to PCIe adapter will work on that mainboard, so I am unsure if a SSD will work in the other M.2 slot.

Also this may not be a profitable rig, profits are rapidly dropping, and you may not breakeven before profits go to under a $1 a day.
full member
Activity: 223
Merit: 100
June 27, 2017, 11:11:06 AM
#13
Where are you located?

There are some IMB 2880w on eBay in the UK at least.

The breakout board and wires I'm not sure just ask the sellers on this forum
I'm in Austria. I found that 2880w IBM PSU on eBay now, but I don't know which cables + adapters I need for a server PSU.

https://bitcointalksearch.org/topic/bladecenter-h-2880w-breakout-boards-selling-from-stock-limited-time-discount-893159 - J4bberwock i think is located in France.

He will have the breakout board and the necessary 6pin cables.

You will need a bunch of 18awg pci 6pin to 6+2pin splitter cables.

Also a c19 power cord

And 6pin risers

Any small & cheap PSU for the CPU and MOBO.
full member
Activity: 154
Merit: 100
June 27, 2017, 10:52:39 AM
#12
Nvidia cards do not work with PCIe splitters as far as I've seen.  If you get a ASUS Prime z170-A or Pro you can probably get 7 cards working, but you'd need to likely have 2 - 850W+ PSU's in order to not fully max them.  A single 1500W might work as well, but you'd likely be pushing it.

Make sure you have a 20A breaker on the circuit you use or split between two circuits.
Thanks for that information.
Wyh do you think only 7 cards will work on Asus Z170-A? It has 7 PCIe ports + 2 M.2 slots. The M.2 to PCIe adapter is an "adapter" not a splitter, so at least 8 cards should be possible (i thought). Or am I wrong?
Why the 20A breaker? You mean the thing in the fuse box?

People have struggled getting that many Nvidia cards working simultaneously.  You might be able to get 8 working, but it will likely take a lot of fiddling and may not work.   Nvidia's draw more power and require more PCI-E resource channels than AMD cards so at least on Windows it tends to complain and although will see the card may not work properly.  Look around there are plenty of people who have tried to do what you are doing, but as I've said I haven't seen a lot of luck with more than 6-7 Nvidia cards at once on the same system.

As for the 20A breaker it's because you will get roughly 1800W of total power at 100% load out of a 15Amp breaker.  

Assuming you're in the US with standard 120v home power.
Ohm's law = 15 amps x 120 volts = 1,800 watts

You don't want to run at full power otherwise you run the risk of popping the breaker when you have power spikes and such.  Circuits can also get hot and cause fires if they are run too close to maximum capacity 24/7.  It's just better to be safe than sorry IMO.  There have been plenty of stories of people having meltdowns/fires because they didn't pay attention to how much power they were trying to draw, especially on older wiring.  Also I presume you will have things like a monitor for setup and potentially fans running for cooling that will likely be on the same circuit.  All of those things take more power and even though you may plug them into different outlets most home power circuits will have all the outlets in a room on the same circuit.  Most home breakers are only 15A unless you are looking at a utility, laundry, or kitchen.
full member
Activity: 184
Merit: 101
June 27, 2017, 10:45:59 AM
#11
Where are you located?

There are some IMB 2880w on eBay in the UK at least.

The breakout board and wires I'm not sure just ask the sellers on this forum
I'm in Austria. I found that 2880w IBM PSU on eBay now, but I don't know which cables + adapters I need for a server PSU.
full member
Activity: 223
Merit: 100
June 27, 2017, 10:41:08 AM
#10
https://bitcointalksearch.org/topic/ibm-2880w-psubreakout-boards-new-package-deals-for-t9s9-a7-966135

Cheapest and most efficient option - also you have room to expand.

Server PSU's are pretty much all 80+ platinum and are designed to run 24/7 365 for years unlike most domestic units.

Bear in mind you should have a 15AMP circuit

Might be a bit more power than you need though and remember these do not have the resell value or demand of a domestic unit.

Ok, understand. But here I can't find any PSUs in stock with more than 1200 Watt, so I would have to buy 2 PSUs anyway.

Where are you located?

There are some IMB 2880w on eBay in the UK at least.

The breakout board and wires I'm not sure just ask the sellers on this forum
full member
Activity: 184
Merit: 101
June 27, 2017, 10:37:12 AM
#9
https://bitcointalksearch.org/topic/ibm-2880w-psubreakout-boards-new-package-deals-for-t9s9-a7-966135

Cheapest and most efficient option - also you have room to expand.

Server PSU's are pretty much all 80+ platinum and are designed to run 24/7 365 for years unlike most domestic units.

Bear in mind you should have a 15AMP circuit

Might be a bit more power than you need though and remember these do not have the resell value or demand of a domestic unit.

Ok, understand. But here I can't find any PSUs in stock with more than 1200 Watt, so I would have to buy 2 PSUs anyway.
full member
Activity: 168
Merit: 100
June 27, 2017, 10:37:00 AM
#8
Get a 2400w server power supply. They are way cheaper, Platinum grade, and meant to be on 24/7. Then a ~400w psu for board and SSD.

Check out parallelminer.com
newbie
Activity: 62
Merit: 0
June 27, 2017, 10:34:01 AM
#7
I think better with one pay, 10 x 1060 can condone maximum 1200watt without any power limitation so a 1500 watt is more than enough or even a 1200watt
Thanks for your fast reply. What's the advantage of only 1 powerful PSU instead of just using 2?
2 x 700 Watt = 1400 Watt what should also be enough, or not?



I think it is cheaper to run 2x PSU instead of one.
full member
Activity: 184
Merit: 101
June 27, 2017, 10:12:45 AM
#6
Nvidia cards do not work with PCIe splitters as far as I've seen.  If you get a ASUS Prime z170-A or Pro you can probably get 7 cards working, but you'd need to likely have 2 - 850W+ PSU's in order to not fully max them.  A single 1500W might work as well, but you'd likely be pushing it.

Make sure you have a 20A breaker on the circuit you use or split between two circuits.
Thanks for that information.
Wyh do you think only 7 cards will work on Asus Z170-A? It has 7 PCIe ports + 2 M.2 slots. The M.2 to PCIe adapter is an "adapter" not a splitter, so at least 8 cards should be possible (i thought). Or am I wrong?
Why the 20A breaker? You mean the thing in the fuse box?
Pages:
Jump to: