Topic: Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] - page 5. (Read 169362 times)

I don't know when I'll be able to test it beyond just booting up (it does boot fine, BTW), might have to wait til Saturday.

I decided to communicate with someone via email, and they googled my address and reported me.
What's funny is it was my first purchase of some GPUs to get into mining about a year ago.

I was really lucky a I had just traded a good sum of cash for pp with a friend less than a month before that, and pp only allowed me to refund money, so I only have 20someting in there now.

I ran into some problems with the controller last night when trying to specify channels, but I've almost sorted that out, so you can still expect it to ship Monday.

How did they find out? I set up my PP account several years ago when I was also under 18, had it connected to my bank account and card (visa electron) and never had any problems buying or selling worldwide, before and after turning 18.

Will do. Let me know how much it is worth to you.

Is Dwolla good? That and BTC are the only good ways for me to receive money right now, as my PayPal account is frozen because I'm not 18. A check would work too.

Sweet. How do you want to be paid? If you want BTC, I'll need to move some funds around.

I should be able to do the pot fallback, I'll give you an update tonight.

I'll also make it so you can just send a percentage without a group, which will set all of the fans to that speed.
I'm also looking into reading the tach. I'm going to plan on shipping it Monday.

Yeah that seems to be the easiest.  Say you have 3 VM

VM1 - GPU 1-6
VM2 - GPU 7-12
VM3 - GPU 13-18

So you have the controller on VM1 (or maybe a dedicated VM4 to it remains functional even if VM1 crashes).

Wherever it is you have
a) the physical controller
b) the serial controller software
c) a custom program which pulls temp stats (over API) from all the GPU on all the VM and sends instructions to the fan controller

Since the API is simply JSON and works over the network inter-VM communication is trivial.  Much how ANUBIS pulls temp stats from multiple physical rigs over the network your controller would pull temp stats the same way.

As far as who?  Kano has experience with both serial communication and he wrote the cgminer API.  conman wrote the "auto-gpu" fan controller code in cgminer.  If they aren't interested it isn't that tough of a task.  Luckily people like me offering a bounty to add API to cgminer makes it easy to extend it by 3rd party programs.

How much? No idea but my guesstimate is it shouldn't take more than an hour or two to code.  I would say 10 BTC maybe more depending on functionality.  I would ask for it to be controlled by a config file.  The config file would need:

list of hosts to monitor and which fan group they belong to
desired temp
hysteresis
fan % increase increments
polling interval

It would be easiest if all the GPU in one host (VM) are part of one fan group.  If not you need to add a parameter which will map host/GPU to fan group.

So in high level psuedo code it would work something like.

Technically "b" above may not be needed (still good to have for manual control).  As long as the fan controller just accepts commands over serial the same program which monitors the GPU temps could send send the command over serial link.

Yeah I got some fan grilles on ebay. So who might be able to write up a cgminer API interface, and what would it cost me? Oh and I just remembered - If I have several VMs with their own instances of cgminer, then what happens? I guess I could run some software in another VM to gather the stats or something.

Also Garr255, if it isn't too hard, the only feature I need you to build in is an input for a potentiometer that controls them all in sync. This is so I don't go deaf when the OS isn't running. Either that, or when it doesn't have a serial signal, all fans go to 10 or 20%.

Yeah to avoid derailing the thread let me provide some context.

1) Pressure is a lot.
2) System is dissipating ~ 4KW of heat.

The bitspower crystal tubes worked fine for years in my water cooled workstation both before and after I started mining.  Also I have never had a problem with any of their other nozzles, compression fittings, clamps, etc).  When the pump "failed" it actually was still pumping but at a very low flow (guestimating <0.2 gpm)  this caused the temp in GPU block to rise.  The plastic "clear" part deformed at ~85C.   All of the links deformed but only one deformed enough to leak.  Luckily it was a "slow" leak and it only killed the motherboard.  If your rig never overheats then you likely will never have a problem.

I think Gomeler may be right on the phobya links.  I am using a high end aquarium pump (20ft+ of head @ 15gpm+) so that may have contributed to the leak.  I was connecting a GPU power connector and pressed down too "hard" (not that hard IMHO) on the card and it must have shifted the link just enough.  I got a spray yes a fraking spray of water.  When I let go of the card it stoppped.  There are not many more things scarier than seeing a spray of water shoot out inside your server.  Amazingly I suffered no damage.  When I am doing leak testing I take one sheet of a shop towel (paper) fold it into quarters and then roll it up.  Fits perfectly between two cards suspended below the link.  I usually leave that shop towel in for 3 days or so to check for any small leaks.  I was lucky and the towel absorbed the blast.  One of my rigs still uses the phobya links.  I am just super careful/paranoid around them.

After that I bought a set of the danger den links and koolance links to test.  Both worked and are still in use but by that time I found the Swiftech ones.  They are solid.  I ended up buying 3 sets for the last 3 rigs.  

So 6 rigs
0 set of Bitspower crystal links (trashed them all)
1 set of Phobya SLI links
1 set of Danger Den SLI links
1 set of Koolance SLI links
3 sets of new Switftech SLI links

The Danger Den links are very similar to the Phobyas but looking inside the o-rings are much larger providing a tighter seal.
The Koolance links are very good (likely the best SLI links) but at $15 a pop they are just highway robbery.

I like the swiftech ones even better because they are almost the same as the Koolance ones but they are designed to be tightened with a wrench, have a more industrial feel (function over form) and they are only $8 ea.

As long as it is controllable via some software mechanism having someone else right up a program which checks temps over cgminer API and issues changes to fan controller over the serial port.


LOLZ.  Good safety tip.  I would imagine (hope) he is using safety grills.  Back in my hardware days I remember changing out the hot swap fans on IBM servers.  For some reason they had no safety grills not even cheap plastic ones.  I never lost a finger but it always scared the crap out of me.  The same server supported hot swap CPU and memory too but we never used those features.  No we shut down the server to change those out but who cares if a the tech loses a fan right?

Right now its manual control over serial. I think a probe would be a bad idea, and cgminer interfacing is something I haven't a clue how to do.

I have an arduino uno on hand so that's what you'll be getting if you have no preference.

You'll need to install the drivers for the emulated serial port over USB: http://arduino.cc/en/Guide/HomePage

If you have any requests for commands, let me know.
Right now I'm thinking something like , which would look like 1,77

I could also make some handy features such as the allowance to spin the fan at light speed, or even ludicrous speed, so the command would look like 1,lightspeed

Just keep in mind your connections need so be very solid when turning off the fans if you need to get in and tweak some hardware or something, because pwm 0 = 100%!

D&T had a pump failure which caused block temperatures to spike and deform the plastic. I had a similar temperature case where I had a leak empty the loop and cause cgminer to idle the GPUs when they hit 65C. This was with just 4 5970s in serial. Luckily I was using the Phobya links and didn't suffer any plastic deformation. I do agree, if you are in your rig, wiggling cards around you could cause a leak. I've had no leak issues though as I don't touch my watercooled rigs when they're running. That's just asking for a dead mobo.

The also have a copper version:
http://koolance.com/index.php?route=product/product&path=28_42&product_id=1172

and as for the crystal link tubing and fittings, I have personally never had any problems with them, but of course YMMV.

That's weird D&T I never had any issues with the crystal link tubes from Bitspower.  The rig I have it on is my main one with crossfire 5870s oced 920 and no leaks thus far.

How much heat were you dissipating may I ask? Also was this under higher pressure than a mcp655?  I really don't want to have to break my system down to replace the xfire and northbridge connections.

Well, if I were to assume 300 watts per overclocked 5970 (not sure yet on 7990), x 17 slots == 5.1 KW of heat. I was looking at this rad from Grainger:



It's good for a little over 7KW, (10 horsepower) with forced cooling (8 HP with no forced air, they say). But it takes up 8U or so.

Beware that Koolance contraption has an aluminum radiator.
 
You could mount 3 120mm wide radiators across the width of a rack.  Imagine a variation of one of my radiator stands (a three rad wide version) mounted in between the rails.




What's the total wattage that needs to be dissipated?

I agree with DT. Those Swiftechs are solid. The smallest clearance model sports 11mm spacing between mount points.

I have those Phobya connectors, but I don't see an issue with them unless you pull them too far apart. One failure point more though.
I think DT's setup has more pressure than average Joe's so he'd be better off with the sturdiest he can find. That silicone sealant wouldn't be bad idea either.  

Oh and that koolance thingy is rubbish. The price is ridiculous and it doesn't have even nearly enough cooling power for rjk's needs.

For the record those Bitspower SLI connectors are worthless.  The plastic deforms at about 85C.  I had a pump failure and the loop overheated.  System shutdown but every single of those SLI connectors deformed and leaked. Utterly utterly worthless.

For the record the Phobya SLI connectors are also junk.  Thin o-rings and loose tolerances means it is possible to shift them and cause a leak.

Koolance SLI connectors are solid but expensive.  
http://www.frozencpu.com/products/9342/koo-177/Koolance_Dual_VID_Connector_-_Adjustable_1_Slot_Spacing_CNT-VDA2_.html?tl=g30c101s873

Danger den makes some good ones (don't have a link though) but they have a floating middle connector which means 2 potential places to leak.

The best ones I have found are these:
http://www.frozencpu.com/products/15732/ex-tub-1252/Swiftech_G14_Male-Male_Lok-Seal_SLI_and_CrossFireX_Connector_Fitting_-_20_to_33mm_-_Chrome_G14-MMEXT-MEDIUM-CHR.html?tl=g30c101s873

I provided no context for these claims.  In the interest of keeping this thread on topic please see this post before providing a rebuttal.
https://bitcointalksearch.org/topic/m.875386

Hey sweet, thanks, I'll look into that stuff.