Topic: Submersing a rig - page 3. (Read 5768 times)

http://www.petrochemcarless.com/white-oils.htm
I don't know where you live but you may want to try this place for the oil?


If I was going to build something like this here is what I would do assuming of course I had a pond

I would use a 240v recirculating pump (have it) and a car radiator (have it) and make the exchanger out of 10mm plate (have it). Only thing I would have to buy is the water hoses, steel pipe and the oil.
 

or how about.... no container at all? I just saw this:

http://www.youtube.com/watch?v=okVPiYD7ny8

Thats nuts. He is not even using a heatsink on his Pentium4, just spraying the oil on it


Hmmm... food for thought.  Though it would also require  a second pump doubling the potential points of failure and further increasing costs.
But if I go ahead with this, Ill almost certainly go with vegetable oil, and that makes the cost a non issue. Local supermarket sells frying oil for 1 euro per liter and its very liquid, almost water like. It might go bad in months, but I read people running their rigs in aquaria (so with light oxidizing the oil)  for 18 months without changing their vegetable oil, so who knows.

BTW, Im bidding on a Formula 1 transmission cooler that used to be in Michael Schumacher's Benetton LOL:
http://www.benl.ebay.be/itm/350525626493?ssPageName=STRK:MEWAX:IT&_trksid=p3984.m1423.l2649

3 days to go, Im sure it will go too high and Ill buy something more sensible but it would be fun

I wish you luck P4man.  Nobody has yet produced a oil cooled setup which is more cost effective than traditional water cooling but you access to a massive heat sink (pond) means you might be able to.

Your concept is the first that at least on paper makes me say "hmm" that might work.  The largest barrier is the high cost of mineral oil.  I almost wonder if some single GPU container would make sense.

To visualize.  Imagine a container (no top) which is roughly the size of a single graphics card.  If you have 4 GPU on  a board you set 4 of the containers side by side, turn MB upside down so the GPU are hanging into the 4 smaller containers.  Now you get a 4 way manifold and run 4 tubing lines to the 4 containers, opposite side of container has an outlet where 4 lines connect to a manifold again. 

A parallel cooling setup.  While more complicated if it saves you a gallons of mineral oil on each rig that savings could be worth it.  The large problem is graphics cards are "tall" (say 5").  If you have a large single pan (12" by 8") and you need to have 7" of depth you are talking 3 or 4 gallons. 

Some more "volume efficient design" (say 7" by 3" by 8") may allow you to use only half a gallon per card (plus amount in the loop).

You may still want to consider a heat exchanger if you need to pipe the cooling fluid a long way to the pond.  Having a short "oil loop" for each rig and then all rigs connected in parallel and then dumping that heat into a heat exchanger for a long range water loop to the pond would reduce the amount of oil you need. 

If cost of oil in long distance loop > cost of heat exchanger well you are golden.  A more expensive setup would be 1 heat exchanger for each rig connected to a single long distance water loop to the radiator in the pond.  The nice thing about using an exchanger is it would require 2 leaks to pollute your pond.

No. Though I see no reason it wouldnt work, Id still be dumping the heat in the shed where I dont want it, and not in the water where at least it has some uses. Also I dont have a spare fridge and its very bulky and probably beyond my skills.


I would pump the oil with a ... pump . You could use most water pumps, if it can pump water, it can pump light oil. A large aquarium pump might do the job, but overheating is something to pay attention to. I dont know how hot the oil will end up being, but assuming 50+C, thats not something all pumps will like. So , Im looking at this pump:

http://www.ebay.co.uk/itm/High-temperature-100C-Water-Oil-Pump-w-speed-Control-/160526138814

As for the number of GPU's; I only have 5 atm, but I plan on expanding that in the coming months. Im waiting for HD 7xxx0 cards to go mainstream and hoping 5xxx cards will become cheap secondhand.  But for now, I started work on a small bath that will hold 4 or 5 gpus max. Depending on temps I get, I might scale that up, possible by "daisy chaining" several oil baths using the same single pump and radiator or add more if need be.

But I actually suspect the above pump combined with a car or motorcycle oil radiator will be overkill for 10 cards, it might handle 100+ cards. After all, the oil radiator on my bike is no bigger, its only air cooled and my bike produces 80+ KW of power. Granted not all of that heat is dumped in the oil, but much of it is.

But Ill only know for sure when I have tried.


Its a concern yes, certainly if I wanted to use pure mineral oil. All I have found here is hideously expensive. If you live in the US, this would seem good and cheap:

http://www.amazon.com/Durvet-Mineral-Oil-1-Gal/dp/B000HHLUE6

Ive not found anything remotely that cheap here.

For vegetable oil, its not much of a concern, its really cheap, though it will also depends how long it will last before it needs replacement. If it ever needs replacement, who knows? I dont .

 Since the amount of oil would have no real impact on the temperature theoretically, I plan on minimizing the amount. Currently working on a small 25L prototype. When my extender cables and a spare motherboard+cpu combo arrive I will plunge in a single old nvidia card with no oil cooling just to see what happens. If it doesnt overheat with just a single ~100W, I may even keep that running for a few months  to see how the videocard, fans and the oil cope.


Pumping dirty pond water with leaves, needles, fish poop, gunk, algae and what have you through a radiator is asking for trouble. Its gonna clog in a matter of days, at most weeks. I see what happens in my filters, its not pretty.  

I was under the impression you had the pump and water flow already (pond filter) needing only a 3/4 inch (garden hose) bypass setup 20-30 psi. As for the rest I guess I was looking at my shed, the drum and scrap is sitting there....although I would have made a purpose built heat exchanger out of the same junk pile.

Have you considered having a fridge evaporator submerged into the oil? More power used but better cooling/control. I have the vac and reclaim gear so maybe not an option if you don't have access.

in your diagram you have not shown how you planned to pump the oil to the oil cooler. Also missing is the number of gpu you need to drown i.e. min dimensions of oil container need to house the gpu.

As i see it your issue currently is the amount of oil required? What if you submerged the oil cooler in the oil and pumped the water though it. This would mean that the water side is an open loop and would need cleaning every so often....

 

Lol. I might try that, if there were any side panels. Or top or bottom

ever think of just keeping the side panel off when you run it?

I just did a small test, mixing vegetable oil with light paraffin oil; the parafin oil does float on top, but if you stir the fluids thoroughly, it seems to mix. An hour later I cant really say I see a layer of parafin oil floating on top. Bummer .

I see no reason why that wouldnt work, but boy, sure sounds a lot more complicated than buying a $40 oil cooler. In either scenario you still need tubing, a pump, oil container etc. It would also take up even more space in my shed for no apparent advantage. Perhaps if you have a water source and no filter or pumps to use, then your approach might make sense, though Id probably still prefer to just put the radiator straight in the pond or swimming pool or whatever.

Also dont underestimate how difficult it is to make a pressurized vessel of any kind. Ive had a smallish commercial pressure filter once, with a 19mm in and outlet.
when I accidentally hooked up the wrong (too powerful) pump, it exploded. Water pressure is nasty.

Sorry I had this picture of a PC hanging from a tree (little birds nesting on it) GPUs hanging down from it into the pond......


Ok so how about this. Take a 44 gallon drum and flow the water in/out, pressurize it. Weld a metal well into the drum to seat the GPUs in an oil bath. The welding would be a little more fiddly and you would want to give it a good hydrostatic test but there is no reason you couldn't do it length ways with the drum on its side. You could also weld fins onto the bottom of the well to maximize surface area.

Its no different than with a submerged radiator. Assuming your ammo box has equal thermal transfer characteristics as an oil cooler, which I doubt considering the thickness, coating etc, all that matter is surface area and temperature difference between the liquids.  A radiator has a huge surface area for its volume, while with your approach, surface area is directly linked to the amount of oil and scales rather poorly with it (cubic root) .

Mind you, Im not saying it wouldnt be enough but I fail to see the advantage, particularly since I wouldnt be able to flow nearly as much water through the "containing container", so Id have higher water temps (and evaporation).  The rigs will be above pond water level, so I cant use gravity feed, I need pump fed. Since you cant pressurize the container, you need gravity return. Unless I plumb at the very least 50mm water return pipes to the shed, thats going to be a serious limitation on flow rate or cause headaches with the water level. To give you an idea, I use 130mm return pipes in my filter, and with my pump capacity even that creates a 10+cm increase in water levels in the filters. Now I dont think youd need anything like the kind of waterflow I have through the filters, but it sure doesnt hurt, and I already have it, so why not use it? The higher the flow, the lower the water temps.


Not sure what you mean by protect. Protect from what? Take a motherboard with a few gpu's (probably using extender cables), turn it upside down. Drop the cards in an oil bath. Even if oil would splash on the MB or PSUs or, or the oil creeps up, it doesnt matter. Ill probably prevent this by creating a lid over the oil box though, mostly to prevent dirt from getting in.  I will wire the cables through the lid and seal it (and may use PCIe power extender cables that are fixed in the lid, so the psu cables plug in to the lid), but thats not even necessary.


Here is sketch made by the 5 year old in me to explain :

I think you might be surprised how much heat a metal surface area can transfer with liquids on both sides. It would be easy enough to test if you have a submersible heater or jug element.

I am trying to picture what you are going to do. Are you only putting the cards into the pond if so how will you protect the MB and PSU etc?

While this may work, you effectively reduce the surface area for the thermal transfer to the size of the box. I suspect an oil cooler would have a larger surface area, and for my setup at least, its much easier to drop a cooler in my existent filters where I already have waterflow, than create a box and somehow circulate large amounts of water through there without overflowing it due to pump head. With gravity return that means HUGE pipes or low flow.  You would also risk a fair amount of water evaporating in a place where I dont want humidity, ie, in the shed where my rigs are.

The idea of an ammo box is excellent though.


Oh, and no zinc in the water. My Koi dont like it


Yes, the problem is thermal grease can be conductive. You dont want it dissolved in the oil. But indeed, for temperatures it should not be a big concern as it would be replaced by oil.  To make sure no air gets trapped, you will want to smear some oil on the die before re-mounting the heatsink.

What about if you used a metal box like an ammo box that you can just fit the cards into. Leave the fans on the cards to circulate the oil. Because the cards fit snugly into the box not as much oil is needed to submerge them, this way you can use the mineral oil rather than run the risk of vegetable oil.

Place the metal box into a larger plastic container of water. Place it the top of the metal box is sitting above the top of the plastic container so any accidental water overflow goes onto the floor not into the box. Pump water into the plastic container and use gravity to take it back to the pond via an overflow. The oil/metal/water interface should be a good heat conductor.

Make sure you put a zinc in the water to stop galvanic series eating the box away.

I cannot see the issue about thermal grease being eaten away, if it was wouldn't just be replaced by the oil that ate it??

Its not so far out of the box, since I got the idea while reading about mineral oil often being used to seal other liquids. So why not vegetable oil
Anyway, Ill go shopping this weekend and Im buying some extra parts to build a rig around an old 8800GT to experiment with.

Some other points I came across:
- thermal grease will dissolve in oil. Before plunging in the cards, youlll have to remove all thermal grease. I hope the vrm pads will manage.
- I came across one report of overheating with a S775 CPUs, despite the oil being cool. apparently caused by air trapped in the socket; though Im not quite sure I understand how that would work, Im assuming this is not a problem with GPUs as there are no sockets.

I like this idea.  Way to think outside the box!

Ive been reading and searching further on this subject to decide on the oil. I can not find a cheap local source of mineral oil, so Im gonna go for vegetable oil. Im aware this is less stable and can oxidize (go rancid). It seems the biggest factors contributing to oxidation are sunlight and humidity. The first is easy enough to tackle, I have no need for a transparent aquarium, by setup is going to be butt ugly anyway.

To counter humidity, Im going to try to add a layer of mineral oil on top of the vegetable oil as sealant. Mineral oil is lighter and should therefore float on top, but Ill have to see if it mixes or not.

If anyone has other suggestions, Im all ears. Is there something lighter than mineral oil that is stable and not conductive to use as sealant?

Oh, there are some prettier species than that but they are all pretty fugly in the grand scheme of things.

I only suggested tilapia cause it can tolerate the highest density. You could grow out some walleye pike or something awesome like that, but like all the other delicious apex predator fish, they are super cannibalistic and have like, 90% mortality on an ok run.

I know a couple people who culture Koi in California and make a good amount of money on it, but for every nice koi, there are a thousand flushers.

So you are suggesting I replace my precious Koi with these hellish creatures:



No thanks. Its so ugly I wouldnt even want to eat it .

If I were in it for the money, I even think growing Koi is the better investment. Though I would need warmer water... (or indoor winter tanks).

No, I meant you should be farming food fish and vegetables in there. 10,000g with some upgrades to the biofilters should be able to handle a good five or six thousand pounds of tilapia, easy...just cut out some raceways and start growing greens and herbs for your local restaurants.