Topic: Dual PSU Woes - 3GH/s w/ 3x5970 + 2x5870 (Read 7783 times)

Roger that.
The startup peaks are insignificant - the real load begins once the OS boots up and the miners launch.

This.  Peak rating for ATX supplies is marketing nonsense to make the supply look better to uninformed consumers.  While there will be startup peaks they are relatively small (as a % of overall continual load).

Honestly don't even look at peak power to avoid it possibly influencing you subconsciously.  I wouldn't pay $0.01 more for a unit w/ "higher" peak power.  Computers are continual loads, mining even more so.


jake is also right on the "history".  In the dark ages of ATX spec the 5V rail was initially used to power CPUs.  As CPU became faster they had to operate at lower voltage to avoid a thermal loads that would melt steel.  So the 3.3V rail became more important.  As CPU became even faster, even 3.3V was too high of an operating voltage.  Rather than have a new voltage rail with each new CPU (2.2V, 2.0V, 1.8V, 1.5V, etc) the load on 12V rail was increased and the motherboard converted the voltage to whatever the CPU/northbridge/memory needed.  As power increased the ATX spec was changed 4 times.  First the ATX 20 pin connector was expanded to 24 pins, next a 4pin 12V MB connector was added and then later doubled to 8pin, and finally direct GPU power connectors 6pin and 8pin were added.  Overtime the % of load that was 12V in average PSU went from 20%, to 30% to 50%, to 70% and now is 80%+ even on NON-MINING RIGS.

Due to that designing PSU that did direct conversion from 120VAC/240VAC to 5VDC (and 3.3VDC) became very inefficient.  Also any time spent making 3.3V & 5V efficient was largely wasted since 70% to 90% of the load was on 12V rail.  This caused PSU designs to move to "rail to rail" conversion.  Since most of the load was 12V convert everything to 12VDC FIRST and do it was efficiently as possible.  Then convert some of the 12VDC as needed to 3.3V & 5V but honestly efficiency isn't important because at most a modern computer only has a token amount on 3.3V/5.0V rails.

Really the ATX spec should be overhauled.  Today there is no reason that a PSU couldn't be 100% ultra high efficiency 12VDC. Technically 18VDC or 24VDC would be even better but you would lose all backwards compatibility. So 12VDC is a reasonable compromise.

Kindly allow me to elaborate on that:
Way back when we were all younger, life was easier: any PSU had discrete 12V, 5V, and 3.3V conversion circuitry.
A 300W PSU was considered a powerful, high-end unit. 5V was enough to feed a CPU. 5V was the primary rail of a PSU and had an evil twin: the -5V rail.
Any PSU could deliver up to X watts at the 12V rail, Y watts at 5V and Z watts at 3.3V rail, e.g. a 400 Watt unit being capable of 80W at 3.3V, 120W at 5V and 200W at 12V.
The load at any specific voltage rail was pre-determined by how powerful curcuitry had been installed.

Let's skip forward to 2012 - the dominant approach today is called DC-to-DC conversion.
For efficiency's sake the circuitry for transforming 220V⁽¹⁾ into lowly 3.3V or 5V has been ditched.
Instead those non-primary rails are being fed by the 12V circuitry and need only to regulate down from 12V.
Almost 100% (let's ignore the -12V and standby 5V rails) of the PSU's output is converted to 12V internally.
Success! - this gives you flexibilty previously unheard of: if you have a 650W DC-DC PSU, it doesn't matter whether your load pattern is 100W at 3.3, 130W at 5V and 400W at 12V or 630W⁽²⁾ at the 12V rail as long as you don't trip the internal overcurrent protection.

Mind you, that it's still the end-user's responsibilty to double check the topology and rating of any PSU before integrating it into such an edge case as a mining rig sporting 4 double-GPU cards.

The "50% load" rule-of-thumb advocates sound as if they spent a couple of years in a deep, dark dungeon... at least I hope they got some serious experience points out of it

Oh, and let me reiterate once more: disregard the peak rating⁽³⁾.

Notes:
(1) I'm not being Euro-centric. The 110V input voltage is being raised to 220V by the active PFC circuit or by the voltage doubler.
     The reason why PSUs achieve a couple per cent higher efficiency when plugged into 220V is that one circuit less needs to be used.
(2) I didn't use 650W as max load to accommodate for the discrete -12V and 5V standby rails often included in the max rating by those damned bean counters.
     Guys, if you're marketing a 650W unit, let me draw that 650W at the USEFUL voltages without overloading the PSU, ok? 650W should mean 650W, not 636.36251W
(3) Historically, the peak rating has been horribly abused to the dismay of many a user ending up with a dead PSU or even the whole machine.
     Huntkey PSUs were a blatant example: units with the overcurrent protection set at eg. 500W were being marketed as 500W units.
     Had the damned marketroids marketed them as 400W units which they clearly were many disasters could be averted.

Forget peak rating.   Peak rating is meant as a safety valve against high transient loads.

You know why the 80% load is reasonable? To leave a buffer in case the cpu ever gets pegegd at 100% (CPUs consume power as well), the cards are ran at stock memory speed due to a misconfig, Windows must be installed and booted⁽¹⁾, or some other serious incident affecting the power usage. It also makes the life of your PSU a tad easier.


(1) just kidding, a bit of Windows-bashing is always in order

Are you running them at 80% of peak or 80% of the continuous wattage rating?

I purchased an OCZ ZT 750w, was going to purchase an AZZA 850w, reading around here changed my choice (DAT, Inaba, jimi).
The OCZ ZT has a continuous rating of 750w, can I draw 750w (12v*62A=744w) without worrying about potential issues?

If it is designed to run at 750w continuously how is that bad to run it at 744w?

Gozintas are cool.

With a single 1200w continuous PSU:
1 @ 1500w@w = 12.5A @120v
1 @ 1500w@w =  6.81A @ 220v

Compare to 750w continuous PSU:
1 @ 930w@w = 7.75A @ 120v
1 @ 930w@w = 4.22A @ 220v

[offtopic]
I have to find my Tesla gozintas, you like those.
[/offtopic]

From the link you provided there is a 2% difference in efficiency between 50% load and 100% load.  2%.

On higher quality 80Plus-Gold units from good brands like Seasonic the difference is efficiency between 50% load and 85% load is <1%. 

The "50% rule" is seriously outdated by about 4 or 5 years.    Skip the Silverstone and get a Seasonic.  I have 7 running at 80% load 24/7 for 9 months now.  They are cool to the touch.

The 80Plus-imposed 20-50-100% efficiency curve is well known.

Please note that you're crying wolf over an efficiency decrease of 3 to 4% whilst at the same time you need to spend a great deal more money on powerful PSUs trying to keep the load at ±50%.
Try doing that with a huge mining farm, like the one Gigavps has... good luck.

Note that the scale Silverstone are using begins at 79% efficiency and ends at 91% to make the curve look more dramatic than it actually is.


Also, don't forget that the internal temperature increase of any PSU is inversely proportional to its efficiency.
A high-efficiency PSU will generate less heat, thus can be run at high load levels without incurring the temperature-caused MTBF degradation.
Start with a gold-certified PSU and you really can disregard that 2% efficiency loss when running it a 80%.

At 100% load, a gold-certified PSU may waste no more than 12% energy. For a 1000W unit that translates to up to 120W of heat - this really isn't beyond a 140mm fan's ability to dissipate.
Silverstone emphasize low fan speeds (and hence low noise) - this is completely orthogonal to mining as you won't hear the PSU fan with all the howling GPU turbines anyway.


I also have strong reservations about the temperature chart, I don't like it starting at 40°C. Any PSU drawing air from outside of the case will draw much colder air than that.
During this load test of a Silverstone PSU, delta between the intake and exhaust temperatures is just 16°C at 100% load. You can rest assured the innards aren't boiling at 70°C as Silverstone's article would suggest.
I can't agree with the claim that "components are rated at 85°C", this day and age 105°C rated components are being used in high-quality PSUs, including the said 1000W Silverstone one.


At best, the article you linked to suggests that Silverstone PSUs aren't necessarily the best choice for mining due to the emphasis on low-noise, low-speed fans and use of only 85°C rated components.
At worst, it can be regarded purely as propaganda: "Why use a 1500W power supply?" Of course Silverstone prefer to sell you an oversized and overpriced 1500W unit where a much cheaper 1250 watter would suffice.

No, its for all of them. They all have efficiency charts, go look at them. Peak is not continuous. Even the nice ones should not be run at full capacity. Here http://www.silverstonetek.com/techtalk_cont.php?area=en&tid=wh10_005

the 60 number is an for older, cheap PSUs


if your going to mine. buy a very good PSU, it will pay for itself.

the seasonics advertise they're continuous wattage.

60% of the peak wattage rating or 60% of the continuous wattage rating?

NOT tru. maybe you shouldn't buy rosewells.

 the seasonics are rated at full capacity.   ( a 1250 will peak at about 1400)

It's best have 50-60% load on your psu. Even if you get it running your power supplys will burn out. I would run a 1200 or 1500 with a 1000w. In the end you will pay more if you don't. In extra power costs or replacing them.  You could always go with third power supply. I run all my systems with 3x 5970's 1500w psu. I tried a 1000w rosewell originally with just 2 5970's, after 2 weeks it caught on fire.

Agreed - might do that but like the bells and whistles of BAMT so it stays as is for now...

I was using Xubuntu on HDD for the last year so know how to do it 

Sorry.
I recommend you base your rig on any standard Linux distro. Follow the installer, then just toss in the driver, miner, and sdk and you're ready to go.
It's not that hard and there are tutorials you could follow.
If that secondary PSU shuts down with lower load than before, it is clear that it took some damage when you overloaded it. Judging from how the X-750 is built, I'm guessing one of the 031N06L power transistors responsible for +12V rail died. You should have that PSU looked at.

Thanks for taking this way OT 

Once again, ATI drivers limit you to 8 AMD GPUs.
Bulanula, check_status never said it was an AMD-based machine.

NVidia-based machines are totally orthogonal to mining unless Kepler kicks some serious ass when it finally launches (April?).

Although the machine you linked to is a beauty, it's not the way to go due to hardware cost of the non-standard HPC platform.
While a VDACTr8-based rig would further decrease the system power usage overhead, any decent rig pulling 1100W has only 10% overhead at 110W for the fans, drives, board, and cpu.

http://www.secmaniac.com/blog/2011/02/06/building-the-ultimate-bad-arse-cuda-cracking-server/

18 gtx580 cards, 3200 watts, 220v.

Update:
Renderstream VDACTr8 Motherboard
http://www.youtube.com/watch?v=QFUpPe3o1rE
$15,500 (probably with 8 GTX580's)

Please show us some links or proof because drivers restrict you to 8 GPUs per rig AFAIK.

Thanks !

Don't forget there is also a driver-introduced limit of 8 GPUs... you can thank ATI engineers for that.

I wonder if the GPU restriction is a northbridge or southbridge issue?
There are manufacturers making custom 10 pcie boards for the brute force community. I believe, might be wrong, that the creator of the Social Engineering Toolkit had one built for hash cracking with CUDA.

Edit: I was correct.
 David Kennedy (ReL1K)
http://www.social-engineer.org/framework/Computer_Based_Social_Engineering_Tools:_Social_Engineer_Toolkit_(SET)

It is an issue with BAMT...

Nope on the powered extenders...

Have gone to 2x5970 and 3x5870 for now = 7 GPUs.  Working fine at 2.8 GH/s 

Are you using any PCIe extender cables?

I am running the x64 version - should have mentioned that...


Didn't realize the X64 didn't fix the 7 GPU max limit though...

BAMT only takes 7 GPUs, lodcrappo thought on moving BAMT to x64 and so support 8, but it's showing as problematic for some beta testers, and he is giving up on the idea.

UPDATE...

Have moved the main PSU to a Corsair 950 and the secondary as Seasonic Gold 750.

Good news - all 5 cards show up - with a minor caveat...

The 5970 on the secondary PSU only has 1 core showing active -- so 7 GPUs total.

Any bright ideas?

Thanks guys - 2 things I should have made clear earlier (hastily posting earlier).

1.  Total watts should be 1250 at the wall.  Assuming 88% efficiency the PSU is providing ~1100 watts or roughly 73.33%.

2.  I am struggling to get it to properly load all 3 5970's even without the last 5870 thrown in.  It seems that the 2 x 5970 on one 750 is causing problems but believe I have seen others using this combination of PSUs w/ 4x5970.

From a strictly engineering perspective, this would usually be true, but according to the PCIe spec it should work fine. ArtForz has posted on this issue several times with links to the technical documents explaining why.

However, I do think you could be underpowered still, OP. I never run my PSUs beyond 50-60% load, and with that hardware combo you might be pushing it over 80-90% load which might have stability issues.

The only thing that comes to mind here is DON'T split  a card between two PSU. I've never tried that but I'm pretty sure that would cause problems. Each PSU has potentially slightly different voltage outputs and this would cause some cross currents between the supplies as they try to "merge". Depending on how the board power connectors actually hook together and what components run off each one there could be various other effects.

I'm not a 5970 user so I don't know what they draw. I would think that 2x5970+MB on one 950W PSU should work and 1x5970+2x5870 on the other one. Definitely don't split a GPU between 2 PSU.

Been trying to achieve a 3 GH/s box.  Specs below:

AsRock 970 Extreme 4MB
1 x 2GB ram
Sempron 145
2 x 750 Seasonic Gold (usign Add2PSU adapter)
Running BAMT (USB Key)

I have had it running 2x5970 and 2x5870 and 1x5770 for a day or two.  Added a 3rd 5970 with just one 5870 and it booted and mined OK.  Then went for the final 5870 and it all fell apart.  Now I am unable to get even a single 5970 working off the secondary PSU.

My plan was:

Main PSU
 - MB
 - 1 x 5970
 - 1 x 5870

Second PSU
 - 2 x 5970
 - 2x10W Delta screamer fans (someone mentioned putting some 5v load on a secondary PSU).

The last 5870 was being powered via dual 4pin molex to 6pin PCI-e with each PSU pushing one of the 6pin plugs (2x4pin molex).

Any words of wisdom here?  Maybe use a 950 as the secondary PSU (gives it more headroom with 2x5970).  Put the 2x5970's on the main PSU?

Yes, I understand that a 1500W PSU would be simpler  

So all you folks running 1200W+ rigs what have you learned with respect to what cards where on multi-psu setups.

I am planning to mess with it some more this evening but would love some pointers if I am missing something simple.

Thanks in Advance!

EDIT -- Yes, the 5970's are in the x16 slots and the 5870's are in x1