If I did the math right using a heat exchanger before the cold water intake of the hot water heater, at 2gpm I could raise the water temp from 60F to 105F before it ever gets to the hot water heater.
Topic: A journey of extreme watercooling: Cooling a rack of GPU servers without AC. - page 6. (Read 27312 times)
February 29, 2012, 10:10:00 PM
Isepick that is a nice method to force the waste heat outside I am looking for quiet and I think that kind of setup exceeds the WAF boundary at the D&T hashing farm. Plus I would like to see how much waste heat I can capture and use to heat hot water for the house.
If I did the math right using a heat exchanger before the cold water intake of the hot water heater, at 2gpm I could raise the water temp from 60F to 105F before it ever gets to the hot water heater.
If I did the math right using a heat exchanger before the cold water intake of the hot water heater, at 2gpm I could raise the water temp from 60F to 105F before it ever gets to the hot water heater.
February 29, 2012, 10:07:16 PM
When did you build this? It looks unbelievably similar to mine.
3 or 4 weeks ago, and you were the direct inspiration for it. I meant to do a build thread about it, but got lazy. It works pretty good for me. I had wanted to relay my positive experience with this design to DAT in case he wanted to try a much cheaper route. As I said, the temperature the room this is in used to be unbearable (hot enough to melt candle wax!), now it is tolerable with even more rigs in it. WAF greatly increased since I pulled 2 rigs off the kitchen table and dropped the ambient temp in the living/dining room area by several degrees.
February 29, 2012, 03:07:16 PM
I had considered going under water for my 10 Ghash/s, but since I have none of the blocks or rads, it was going to be way too expensive. Instead I went the ghetto route:
That's a 12", 1000 cfm vortex fan (pulling 350w). It is far cheaper and easier to suck the hot air outside. The whole thing (rack, fans, foamboard, tape, etc) took $500 and a weekend to build. I live in S. Florida and this is my solution for the summer. It used to get over 100F in that room with only 4 rigs, now it is maybe 5 degrees warmer than the rest of the house on a hot day with 7 rigs. This may or may not help you, and watercooling is a hell of a lot neater, but from a maximum profit point of view it is cheaper to move a large quantity of air over those cards than it is to water cool them. And yes, that fan is louder than anything any video card
. Anyhow, just throwing this out there for ideas. Good luck.
That's a 12", 1000 cfm vortex fan (pulling 350w). It is far cheaper and easier to suck the hot air outside. The whole thing (rack, fans, foamboard, tape, etc) took $500 and a weekend to build. I live in S. Florida and this is my solution for the summer. It used to get over 100F in that room with only 4 rigs, now it is maybe 5 degrees warmer than the rest of the house on a hot day with 7 rigs. This may or may not help you, and watercooling is a hell of a lot neater, but from a maximum profit point of view it is cheaper to move a large quantity of air over those cards than it is to water cool them. And yes, that fan is louder than anything any video card
. Anyhow, just throwing this out there for ideas. Good luck. When did you build this? It looks unbelievably similar to mine.
February 29, 2012, 06:07:22 AM
If you had chosen BitFORCE, you could've built the same system (3328 MH/s, about 328 MH/s higher than the actual solution) with only 4 units,
costing total of 2400USD (+ shipping) and consuming only 330 Watts (less if a high-efficiency power-supply was used to power all 4 units).
Virtually silent as compared to GPUs (26dB each unit) and a fraction ( A third so to speak ) on electricity costs
Regards,
costing total of 2400USD (+ shipping) and consuming only 330 Watts (less if a high-efficiency power-supply was used to power all 4 units).
Virtually silent as compared to GPUs (26dB each unit) and a fraction ( A third so to speak ) on electricity costs
Regards,
That's what I figured when I ordered 6 units from you a few days ago. Assuming they get here within the "4 to 6 week" period, I will have a very nice high-hash, low-power and low noise upgrade to my mining rigs within a few weeks.
BTW, it's nice to see you on the forum, BFL-Engineer (Sonny, I presume).
Keep shippin' em' and I'll keep orderin' em as long as I get them within reasonable delays.
Sonny is in sales, I'm in engineering. We'll try our best to satisfy customers. Efforts are
under way to reduce shipping delay.
Regards,
February 28, 2012, 10:41:16 PM
I had considered going under water for my 10 Ghash/s, but since I have none of the blocks or rads, it was going to be way too expensive. Instead I went the ghetto route:
That's a 12", 1000 cfm vortex fan (pulling 350w). It is far cheaper and easier to suck the hot air outside. The whole thing (rack, fans, foamboard, tape, etc) took $500 and a weekend to build. I live in S. Florida and this is my solution for the summer. It used to get over 100F in that room with only 4 rigs, now it is maybe 5 degrees warmer than the rest of the house on a hot day with 7 rigs. This may or may not help you, and watercooling is a hell of a lot neater, but from a maximum profit point of view it is cheaper to move a large quantity of air over those cards than it is to water cool them. And yes, that fan is louder than anything any video card. Anyhow, just throwing this out there for ideas. Good luck.
That's a 12", 1000 cfm vortex fan (pulling 350w). It is far cheaper and easier to suck the hot air outside. The whole thing (rack, fans, foamboard, tape, etc) took $500 and a weekend to build. I live in S. Florida and this is my solution for the summer. It used to get over 100F in that room with only 4 rigs, now it is maybe 5 degrees warmer than the rest of the house on a hot day with 7 rigs. This may or may not help you, and watercooling is a hell of a lot neater, but from a maximum profit point of view it is cheaper to move a large quantity of air over those cards than it is to water cool them. And yes, that fan is louder than anything any video card
. Anyhow, just throwing this out there for ideas. Good luck. 
February 28, 2012, 10:14:26 PM
If you had chosen BitFORCE, you could've built the same system (3328 MH/s, about 328 MH/s higher than the actual solution) with only 4 units,
costing total of 2400USD (+ shipping) and consuming only 330 Watts (less if a high-efficiency power-supply was used to power all 4 units).
Virtually silent as compared to GPUs (26dB each unit) and a fraction ( A third so to speak ) on electricity costs
Regards,
costing total of 2400USD (+ shipping) and consuming only 330 Watts (less if a high-efficiency power-supply was used to power all 4 units).
Virtually silent as compared to GPUs (26dB each unit) and a fraction ( A third so to speak ) on electricity costs
Regards,
That's what I figured when I ordered 6 units from you a few days ago. Assuming they get here within the "4 to 6 week" period, I will have a very nice high-hash, low-power and low noise upgrade to my mining rigs within a few weeks.
BTW, it's nice to see you on the forum, BFL-Engineer (Sonny, I presume).
Keep shippin' em' and I'll keep orderin' em as long as I get them within reasonable delays.
February 28, 2012, 07:38:50 PM
Good work. Looks clean and has this professional touch.
February 28, 2012, 07:00:08 PM
/drool
February 28, 2012, 06:54:44 PM
Although with right MB and if AMD ever fixed their fracked drivers you could use more than one to get 10 or 12 GPU in one case. 6x7990 in one 4U chassis. Hmm... See now you distracted me.
Stay out, that's my territory 
February 28, 2012, 05:55:46 PM
Some updated pics.
...
Thoughts, comments, suggestions?
She's a beauty, DAT....
Thoughts, comments, suggestions?
Well done - and without a single blue led or cold cathode. Looking at the average watercooled rig I wouldn't deem omitting some sort of l337 light source possible
February 28, 2012, 01:50:56 PM
Yes, it isn't cheap. I assume you would only need one per case (last card using it) and you could prop it against the side of the case. How well do those water fittings fit together?
Yeah it would be one per rig but I will need 6 all together.
Although with right MB and if AMD ever fixed their fracked drivers you could use more than one to get 10 or 12 GPU in one case. 6x7990 in one 4U chassis. Hmm... See now you distracted me.
Quote
That bracket was just from a quick search, I am sure DD and others make cheaper ones. You save because you don't have to spend as much on the rack case
I looked and I couldn't find a store in US that carries it. I called frozen cpu and they don't stock it because it is explictly not designed for water cooling. The manufacturer puts in small print not designed for heatsinks over 1 pound. WTF? What else would you use it for? Cramming hot aircooled 5970s even closer together?
Quote
How well do those water fittings fit together?
The fittings work well as long as you don't move the cards.
The SLI fitting is adjustable (necessary to get multiple cards installed) but there is a max limit. If you pull the "end caps" too far apart the tube in the inside can slide past the double o-rings and leak. You really need to stupidly pull them "far" apart (say 1" more than they are now). The risk is likely small inside a crammed case but I am a coward.
There are other "single piece" methods but they add a lot of cost.
February 28, 2012, 01:47:19 PM
Thanks! Chenbro's site shows that as a tower. i ended up just going with a 3 card setup for my cluster b/c i could not find any decently priced 4 gpu rackmount cases. Maybe i should replace my setup with these RM41300-FS81?
It is designed to be both. It has rack dimensions and rack handles on the front but it also comes with stand you can attach to use it as a tower (likely adding to the cost).
Other than the fact that it is stupid that noboody else has a 8 slot case under $400 and I felt kinda forced into it getting this chassis it has been solid so far. Fits my needs exactly. I wouldn't use it for air-cooling though which is supposedly what it is designed for. I would worry about not getting enough air in.
I think with alot of 133+ cfm fans you can get the temps down. like this guy
http://www.ebay.com/itm/New-nVidia-Tesla-S870-6GB-Memory-GPU-Processing-Computing-Server-System-BitCoin-/220938944468?pt=LH_DefaultDomain_0&hash=item3370fcbfd4#ht_4992wt_1163
But of course those heat sinks are aligned with the air flow, but if you design some simple copper heat sinks and mount them to the 5970s this might work. It would be loud as hell with those 40mm fans but it would only take up 1U
February 28, 2012, 01:41:50 PM
It will be in my own rack no datacenter. The largest advantage comes from dumping heat directly outside so datacenter doesn't really make sense.
>directly outside
How do you plan to do that?
Is it a BARN, with a overhanging roof and a huge open window?
Half a year ago I had eight tower cases perched on the windowsill of an office, four where the right windowpane opened and four where the left windowpane opened. I added external ducting and 12cm "exhaust fans" at the mosquito screen of the windowpanes.
Problem was, they were blue LED fans - the building owner drove by at night, went WTF? - and eventually asked me what the f*** I was doing. I gave an evasive answer ("research") and was evicted shortly thereafter with a 3-day notice.
Now you know why I'm interested in BFL Singles and FLGA boards...
February 28, 2012, 01:39:25 PM
Thanks! Chenbro's site shows that as a tower. i ended up just going with a 3 card setup for my cluster b/c i could not find any decently priced 4 gpu rackmount cases. Maybe i should replace my setup with these RM41300-FS81?
It is designed to be both. It has rack dimensions and rack handles on the front but it also comes with stand you can attach to use it as a tower (likely adding to the cost).
Other than the fact that it is stupid that noboody else has a 8 slot case under $400 and I felt kinda forced into it getting this chassis it has been solid so far. Fits my needs exactly. I wouldn't use it for air-cooling though which is supposedly what it is designed for. I would worry about not getting enough air in.
February 28, 2012, 01:38:05 PM
Couldn't you use a 7-slot case that has a little more room where the 8th slot would go, and replace the bracket with a single slot bracket on the 5970?
http://www.plinkusa.net/4u5u.htm <-- lots of cheap 4U 7 slot cases
EDIT: Bracket: http://shop.aquacomputer.de/product_info.php?products_id=2460
http://www.plinkusa.net/4u5u.htm <-- lots of cheap 4U 7 slot cases
EDIT: Bracket: http://shop.aquacomputer.de/product_info.php?products_id=2460
I thought about it but the waterblocks are heavy and I worry about all that weight supported by a single slot screw. Then you combine the catastrophic damage of slot bending and water pouring over $3000 in components.
The RM41300-FS81 is $120 so getting cheaper case would save $70 or so per rig. The bracket you linked to is very hard to find and Aqua wants $35 to ship to US so it ends up being almost $15 ea. I could dremmel the slot bracket to make it a single slot but that makes it kinda hard to resell and still got the risk of "lots of weight + tiny screw".
In the end I decided to pay the extra $60 or so and play it safe. Likely being over cautious but I likely will buy the same expensive case for the next 5 rigs too.
That bracket was just from a quick search, I am sure DD and others make cheaper ones. You save because you don't have to spend as much on the rack case
February 28, 2012, 01:35:27 PM
Couldn't you use a 7-slot case that has a little more room where the 8th slot would go, and replace the bracket with a single slot bracket on the 5970?
http://www.plinkusa.net/4u5u.htm <-- lots of cheap 4U 7 slot cases
EDIT: Bracket: http://shop.aquacomputer.de/product_info.php?products_id=2460
http://www.plinkusa.net/4u5u.htm <-- lots of cheap 4U 7 slot cases
EDIT: Bracket: http://shop.aquacomputer.de/product_info.php?products_id=2460
I thought about it but the waterblocks are heavy (even heavier filled with water) and I worry about all that weight supported by a single slot screw. Then you combine the catastrophic damage of slot bending and coolant pouring over $3000 in components.
The RM41300-FS81 is $120 so getting cheaper case would save some money. The bracket you linked to is very hard to find and Aqua wants $35 to ship to US so it ends up being almost $15 ea. LOLZ. I could dremmel the slot bracket to make it a single slot but that makes it kinda hard to resell and still got the (lots of weight) + (water) + (single tiny screw) = risk issue.
In the end I decided to pay the extra $60 or so and play it safe (probably oversafe).
February 28, 2012, 01:34:58 PM
What chassis is that? i have been looking for a 8 exp. slot rack mount for months can can only find pricey tesla supermicros and the RM41300-FS81.
That is it. As far as I know it is literally the only sub $400 8 expansion slot 4U chassis on the planet. Kinda bizzare given punching an eighth slot has essentially no real cost. Its not like someone wanting 7 slots would say "forget this case it has 1 too many slots".
Provantage is out of stock but looks like they have the cheapest price: $128
http://www.provantage.com/chenbro-micom-rm41300-fs81~7CHEN0RM.htm
February 28, 2012, 01:26:17 PM
What chassis is that? i have been looking for a 8 exp. slot rack mount for months can can only find pricey tesla supermicros and the RM41300-FS81.
That is it. As far as I know it is literally the only sub $400 8 expansion slot 4U chassis on the planet. Kinda bizzare given punching an eighth slot has essentially no real cost. Its not like someone wanting 7 slots would say "forget this case it has 1 too many slots".
http://www.provantage.com/chenbro-micom-rm41300-fs81~7CHEN0RM.htm
http://www.plinkusa.net/4u5u.htm <-- lots of cheap 4U 7 slot cases
EDIT: Bracket: http://shop.aquacomputer.de/product_info.php?products_id=2460
February 28, 2012, 01:23:40 PM
What chassis is that? i have been looking for a 8 exp. slot rack mount for months can can only find pricey tesla supermicros and the RM41300-FS81.
That is it. As far as I know it is literally the only sub $400 8 expansion slot 4U chassis on the planet. Kinda bizzare given punching an eighth slot has essentially no real cost. Its not like someone wanting 7 slots would say "forget this case it has 1 too many slots".
Provantage is out of stock but looks like they have the cheapest price: $128
http://www.provantage.com/chenbro-micom-rm41300-fs81~7CHEN0RM.htm
February 28, 2012, 01:19:14 PM
What chassis is that? i have been looking for a 8 exp. slot rack mount for months can can only find pricey tesla supermicros and the RM41300-FS81.
Jump to: