So the original thread is here (but the plan changed a lot and the thread got sidetracked at points w/ flames & trolls)
https://bitcointalksearch.org/topic/a-journey-of-extreme-watercooling-cooling-a-rack-of-gpu-servers-without-ac-66606The Purpose (also know as "are you fraking crazy FPGA are going to take over the world ... ")
I ALREADY have a farm of six 3x5970 rigs plus another 6x 5970s cards looking for a home. The hardware has long since been paid off and I have some measure of pride knowing the rigs I built are just about as efficient as is possible with GPU tech (although some optimizes 7970 rigs might edge them out). The goal is to extend that investment as long as possible. Kinda a GPU "last stand" against the encroaching FPGA hoards
My goal was to find a way to:
a) survive the summer without the 1/3rd increase in electrical cost
b) avoid paying ~$3K to the upgrade AC system (farm was expanded in the fall so current AC won't be able to handle it)
c) have fun building a big insane system.
d) see if it would be possible to capture some of that heat for preheating hot water heater and/or heating the house.
So I had this crazy idea. Can you put 24+ HD 5970s in a standard server rack and using a really powerful pump move all that heat (18,000 BTUs) outside?
Well the farm is 50% converted.
The Rack From top to bottom is APC PDU, management server, gigabit switch, second APC PDU, and the first three water cooled rigs.
Closeup of the rigsNo I am not cooling it with milk. I guess the flash made it look like that. Just using plain distilled water.
Connectivity & PowerI need some better cable management but power on left, connectivity on the right is a good start. I hope to eventually replace all the flash drives with PXE.
ManifoldFirst three rigs are hooked to the three left most ports. Isolation valves on top, flow control valves on bottom.
The connected loop on the right ensures even if I absent mindedly disconnect all 3 rigs the pumps will still get about 1gpm flow (important to keep the pump cool). The photo is kinda hard to tell but this is mounted on the right hand side of the rack when looking from the POV of the first photo.
http://i.minus.com/iBLM12pskhT2g.pngThe pumpA real monster but quiet and has a nearly silent fan in the back which reduces the amount of heat which ends up in the loop. A pair of unions simplify hooking it up to the loop. The pump and fan are the only component which can't run on 240 which necessitates running 120V extension cord.
The radiatorHas about 16x the pipe surface area as your "normal" 3x120mm radiator and about 6x the fin surface area.
I am still using the poly tubing but will be replacing it with PEX when I get the time. Fan is just a "normal" floor fan. Probably 1800 cfm. I may replace it with a car radiator fan or maybe just a outdoor rated workspace fan.
The rack contains:
3x quad 5970s rigs
2x APC 9571 PDU (240V, 30A, 80% derate) for 11.5 KW usable power
1x Cisco 24 port gigabit switch
1x 1U watchdog server (runs bitcoind, p2pool, remote power relay card, and monitoring software)
Current config:
12 5970s (24 GPUs) ~ 9.2 GH/s
When complete:
24 5970s (48 GPUs) ~ 18-20 GH/s
Theoretical max:
40 5970s (80 GPUs) ~ 30-32 GH/s