Topic: Rig Master Build Tips (Read 1442 times)

Yes, ive been build Beowulf clusters and GPU render farms for close to 10 years now.

The question about why someone would use a board with onboard?  Foxcon makes a great AM3 board with (2) 16x and (2) 1x slots. Total cost of board at time of buy $28. No jumping required for detection and runs like a dream. 

yes, running a motherboard with an onboard gpu means it will consume the same power if you use it for a display: There are, however, two problems with this: a) why would you have bought a board with that to start with, and b) why would you be displaying something on it? I have six rigs with multiple cards, with uptimes measured in months, and see no appreciable difference in stats on primary/secondary/etc cards.

OTOH, 80+ efficiency is ignored too often, power supply efficiency is huge!

If you must use KVM, and display you're a noob.

I cant believe you're even giving other advices.

Ever build a cluster of computational farm?

Hey BAMT has tenth a percent precision.  Hmm I might want to look into that.  Or maybe I should offer cgminer author a bounty because likely it is possible in cgminer too.

I use BAMT and there is no GUI/display involved.

I agree,

There is this thing that show the number of share
on my win7 machine It is 99.9% the same on the card without a display.


Onboard video is a good sign the board is a POS.

And IIRC all board I encountered would disable onboard when a GPU is detected.

The claim of 70% to 90% GPU load if you use it for desktop is nonsense.

Unless your are running HD movies or trying to game with it you should get 97%+ GPU load on the GPU which is primary display and that is when it is in use.  The other (headless) GPU should be 99%+.  If you aren't getting that there is something wrong with your rig.

On my 3x5970 display according to cgminer my load is
GPU 0 - 99%
GPU 1 - 98%
GPU 2 - 99%
GPU 3 - 99%
GPU 4 - 99% (don't ask me why the first GPU on first card is GPU 4 according to cgminer)
GPU 5 - 98%

So my GPU used for desktop actually you performs two of the other GPU although the difference is less than 1% and due to rounding could be even smaller so it likely is just statistical noise.

I also have quite a few machines/cards mining (see sig).  


I don't see this behavior on my rigs...  I'll get into that below.


Er...  Sort of.  That assumes you're using a board with an onboard GPU in the first place.  I'm not.  I assure you that you'd see slightly less power used if the board *didn't* have that onboard GPU.  If you already have it, then yes -- using what's already there wont significantly affect wattage.  If your onboard GPU doesn't use any power or generate any heat, it's supernatural and magic.  


15 of my rigs have 4 GPUs and they don't exhibit this behavior.  Maybe it's something specific to your setup?  All of these rigs except for 1 (my main PC) are headless.  Most of my rigs currently have about a 2 week uptime and I don't see the disparity between cards on any of them.  In fact, on some rigs, I have slightly *more* accepted shares on GPU0.  Even then, the highest share difference I can find between GPUs on *any* of my rigs is 0.269 percent.


Yeah, sorry about that.  I realize that you wanted to lock the topic and have PMs, but I wanted to post this publicly to reduce what is [possibly] misinformation.  It wouldn't be a public forum without public discussion.  


Here's what I'm using for my 4 GPU rigs:

ASRock 870 Extreme3
Athlon II X3 450 with 2 cores disabled (overkill, but the resale will be higher if I sell assembled machines)
4x 5870's, 4x6970's, 4x6870s, and a couple machines with 2x 5970s.
OCZ ZX 1250W PSUs, OCZ ZX 1000W PSUs, OCZ ZX 850W PSUs, and a few various Seasonic PSUs (all 80PLUS Gold)
No hard drives (just 8gb and 16gb flash drives).  Hard drives add a little extra power/heat to the mix (though you save $8-10 per machine if you happen to have a bunch laying around).

All dedicated rigs are running 32 bit Ubuntu Natty.


Again, I just don't see this behavior on any of my miners...  unless I'm actually using the machine to browse, etc (on my Windows box).  I saw this to some degree back when I was using Linuxcoin and Phoenix.  Maybe that's the issue?  If you would like some statistics from my miners, I'd be happy to send them to you.

Learn about your hardware: One of the biggest mistakes I almost made was in a choice of 6pin adapter. I had come across an ebay listing for sata->6pin adapters and they were cheap in bulk. So I went ahead bought myself a good bunch for about $50 and days later they were in my hands. Now even before I hooked these up, I noticed something. Normally on a 6pin, you have 3 grounds and 3 12v+ connections running parallel

[gnd][gnd][gnd]
[12v][12v][12v]

Well these had all the grounds right, but the middle 12v was not present. So I took my pos test box out and fired it up not expecting the card to run, and no it did not.
For people out there with a Sapphire brand card, your middle 6pin on the adapter is a loopback to one of the outer 12v pins. So seeing that, I cut some wire and made it like the Sapphire adapter. Worked.

Take the time and know what your hardware is, and it can save you time on simple issues.

KISS: Keep It Simple Stupid: My first few machines were based on either linux coin or Windows XP x64 with Trixx or Afterburner for an Oc option. Ran ok, but like crap compared to when I started running CGminer. The options in that program allowed me to eliminate my Oc program, and control cooling and oc at once. My machines INSTANTLY ran smoother and colder but more efficient. Thinking that could use a different miner with hard programmed settings for my Oc and such, I tried multiple builds on multiple os's, in the end, one miner for all got me my best results.

10amps for your house: NEVER, EVER! Use more than 10 amps on one outlet. Yes, they are rated for 15amps. Just dont. There is not a reason. If you are that pressed for amperage, you need to look into other options. In most homes, you shouldnt run more than 6 circuts with 10amps on them nonstop without large fears. I once walked into a home out here in SoCal where the guy had about 12 20amp breakers running at 19amps each and him telling me that "They will go as high as 24 without fliping at times." WTF?! I thought I was going to explode due to that current in that room. DO not do what that guy did. He got lucky and had equipment fault and short at the psu level and not trip the line (quality homes of southern california) I was told there was fire and a desperate grab at a extension cable which is similar to something I had happen.  Well, everything apparently tripped when he became the ground. Non-fatal but severe harm.
This was actually a data backup center in a home and not bitcoin, but same thing, just more hardware, less heat.

You know, I used to think that, but I have spent a long time looking at 4 screens rotating between 16 machine each on kvms. While you may have momentary high burst of hash power, you really are only getting 70% at the min and maybe 90% at the max. Using onboard gpus will not affect wattage (or every mobo I have with one lies on my Kill-a-watts). Now your still going to say this will not be a big loss, but over a month, its huge. Look at your shares for your main gpu and look at the rest, 70-80% diff. If you are a true miner, you should not have more than 4 cards on your board anyway for a bunch of reasons. But having a display only card will not kill you, especially if you are not using it for heavy acceleration.
I dont do anything without testing. For my test I used the following systems in the following builds.
Each build has a non-mining gpu either pcie or onboard, and one with all cards mining and/or no onboard.

Foxcon (cant remember, will edit with model number) AM3 (2)16x (2)1x
Semp 130 Cpu
4x Sapphire 5830s run stirctly off a Thermaltake Rx2 750 with some brand I cant recal 80+ bronze on the mobo.
120gb hdd

Msi890FXA-GD70
Semp 130 Cpu
4x Sapphire 5830s stirctly off a Thermaltake Rx2 750 with some brand I cant recal 80+ bronze on the mobo.
120gb hdd

On the other two machines, same build but the Foxcon has an onboard VGA/DVI gpu and fo the Msi we used a 5450 via the DVI port (GPu disabled in this setup).

28 days into runtime, here is what we had. Both the Foxcon with the onboard and the Msi with the Hd5450 had around 20% more shares and while visible but unexplainable, less rejected shares by 4.5% (calculations became finite for rejected shares as this made no sense)

This theory was first observed with Hd 4550 but uptime was not high enough to monitor. Has been re-verified with 100% functional 5770 and a 5830 that cant mine anymore.

I forgot to lock the topic, and will do so in the morning. Until then, ask some questions, then I will answer and lock to keep this as a reference topic.

Good start so far.  I would heavily refute the first bit about the onboard GPU though.  It's accurate if you're going for nothing but maximum hash rate.  If you're going for maximum efficiency however, adding an extra card is counter-productive.  Adding an extra 15-20 watts (or more) for a few extra Mhash/sec from GPU0 doesn't add up.  You're simply not getting the 15-20 watts of mining power back out of the primary video card.  With those numbers, NVIDIA based mining would actually be cost effective.

...unless you can find a really old PCI card that can output a 2D display and idle at a few watts.  Most lower end 3D cards will suck up 20-50 watts while idling though (for example - the NVIDIA G92 chips are brutal.  My roommate's 200 series GTX drains 87 watts just sitting there).  

This is going to be a locked thread with all the useful things I have discovered and questions I answer often kept in one place. Feel free to PM me if you see an error.

Use a the onboard GPU as much as possible!: This one sounds odd, but here is how and why. Your main GPU will not perform to its top if you are using it for display. Even on a headless system you will not get a full hash score out of it. On alot of my rigs, I may sacrifice a slot for a cheap 5450 or old GeForce just to act for the display. Onboard works well to depending on the mobo and chipset. If you run 5770, I would suggest this as the card can run as low as 90mhs while displaying a desktop.

Count your riser, not your cards: The hardest part of running more than 4 gpus on single cards is balancing the GPUs over the rails or multiple psus.
In the case of 2 Psus in tandem, just running half the cards on one and half on the other wont work. The psu powering the board will be giving around 75w to the card over the pcie connection in the case of a 5770. So you would have a Mobo, HDD, 6 x 75w for pcie connections and 6 x 12v 6pin connections on one power supply while the second psu is only powering 6 x 12v 6pin connections.
The ideal setup would be to have a lower power psu powering the mobo, pcie connections and one card, while a 900w powers the other 5 cards alone.

80+ 4 Life!: Running the same setup on a non 80+ psu and one that is on a 80+ psu will not only cost you more in electrical cost, but also limit your mining capacity. As much as 4 amps can be lost here alone, and thats enough to eliminate a full rig for ever 2 your running per circuit.

Make a Load Light!: Some psus when running as a secondary will require some amperage to be drawn over the 5.5v (the red wires on your psu) in order to stay operational under the large 12v draw. One way to fix this is to take a standard automotive 5.5v dome light and connect the red line off your molexed adapter to the tip of the light, and the black ground to the metal side of the light. Solder away and plug in to your secondary psu ONLY. Watch your baby pur away. This is helpful if you use a setup like 2 Thermaltake RX2 750 modular psus. The second one will drop without the light.


More to come