Topic: GPU rig design and cooling at scale (Read 71 times)

Our farm has outgrown our initial space and we are soon moving to a new, much larger, location.  With all the pains this brings, it also gives me a chance to rethink my rig design and cooling strategy.

So, currently, I've been building 8 card rigs in their own angled aluminum cages and those cages sit on wire rack shelves.  2 cages fit on a shelf and each shelving unit has 6 shelves - 12 rigs per shelving unit.  These are your typical design with MB and PSu on the bottom and a row of cards on top of them.  We have a big 36" exhaust fan and a fresh air intake that keep them relatively cool - +/-70c.  Other than that, we have not gotten into hot aisles and cool isles.

For the next location, I am wondering if there are any advantages to scrapping the cages and directly mouting the cards to the wire rack shelving.  And if I do that, are there any advantages to hanging them (DVI/HDMI inputs up, power plugs down) from the wire racks (vs long rows of cards all with their HDMI end facing out). Hanging them would put the HDMI side of the card directly under the MB of the rig on the shelf above it.

The warehouse itself will still have exhaust fans (+/-15 CFM per rig) and fresh air intakes, but no other mechanical cooling (ie air-conditioning).  We'll have multiple rows of shelving units and I need to be sure each rig on each shelving unit is getting fresh air across its fans. 

Does anyone have any thoughts or best practices on how to set this up in an efficient way?  This will be in a wide open space of about 4,000 SF (50'w x 80' l).

Thanks in advance!