Topic: 3M Novec 7100 and antMiner S7/S9 (Read 4049 times)

this is just an ideea, but i think it's deserving:

one does not need ONE liquid to do the cooling. one can MIX 2 liquids. maybe 3, even.
oils are soluble in oils. the thicker will act as a condenser, in its whole mass, and the lighter, lower boiling point oil, will fractionally distill. then it will condense in the mass of the thicker, heavier oil.

one only needs to pay attention to combustibility and dielectric constant.

other than that, i'm sure we can hack a dyi-in-your-kitchen solution, once all the chemists get their focus on this.

brainstorm away ?

LE: maybe best moved on a topic of its own ? immersion fluid diy cooking ? req for help from fellow chemistry forums ?

Sure, Arizona isn't bad, but it isn't the best either - the blended yearly rate is basically right around $0.07...

But i don't understand how many dollars per kilowatt do you Pay...
Can i know??

Just because you can only see one rack in the picture, doesn't mean that's all there is...    It was actually only 24 S5's in the horizontal build, the picture of 32 machines is actually Avalon A6's, so each rack was a bit under 40Kw in operation, and you can see the 16 IBM 2880W PSU's that ran up and down both sides.  I wish I took a picture of one of the newer versions of the racks, as I really improved the way the A6's mounted on them - which made maintenance a breeze.  With the old one, if one had to be removed from the middle, you had to remove the ones to the left of it to slide it out.

That's one nice thing about my GPU miner design, it's dead simple to work on, and it only takes 8 screws and the fans are easily removed to work on the back side (if needed).  I actually don't use the support tray in the middle either now that sidehack makes custom cables for me, so that's gone also in the new ones.

wow!!!
thats look really great!!!
it looks very clean and all in to an accurate structure!!!

32 S5 mean about 18,8Kw

 

wich kind of electricity contract do you have???
here 1 Kwatt/Hour cost 0,25euro cents...

what about your contract?

In general I tend not to take any pictures until I'm somewhat satisfied with the build - and like I said, I really didn't get too much time with it until it all evaporated, and that kind of ended that endeavor.

But just so you know that I'm not full of crap, here's some of the other builds I've done:

My individual S5 enclosure:
https://bitcointalksearch.org/topic/antminer-s5-laser-cutter-mods-1086882


My plenum S5 enclosure:
https://bitcointalksearch.org/topic/the-ultimate-s5-mega-enclosure-maybe-1184272


My updated plenum Avalon A6 enclosure:


My GPU mining enclosure:


As you can see from it all, I make everything out of aluminum extrusion and acrylic - which are basically the two building blocks when working with Novec - so I was excited to use it...

can you share some photos??
or maybe do you have a thread in bitcointalk forum?

No, I didn't use a condenser coil - since this was more experimentation and I was trying to use more easily accessible things I was using a small radiator instead.  While they perform very different functions, the general principle is the same (tube with fins, one to radiate heat the other to collect heat).  I suspect there was some performance loss from the radiator choice but I also figured it was small enough not to matter - as at least in my little test rig it didn't seem to affect it.

Really, very little is different than a normal water-cooling system in that regard - you'll basically pass water through the radiator, then you need to cool it back down in some way.  In my case it was just another radiator with fans I had outside of the Novec enclosure.  Just a closed loop with two radiators, a pump and a reservoir - and it was all PC cooling stuff I have from other cooling systems I've done.

Hi Mark!!!
i'm really interested to your experience!
did you buy condenser coil or did you do by your self?
If you can share...

thank you for now!

1000$ (about 900eur) for 1 gallon or for EU 245eur for 1 liter is too much.
specially in EU i will never see ROI
I thought novec cost a lot less!!!!

While I haven't done oil immersion myself, I can tell you that you'll learn very little that will translate between the two - the way Novec works is completely different than traditional submersion.  Keep in mind that traditional submersion still requires you to move around the liquid just like you move around air - if you don't move the liquid, it will ultimately burn up.  Novec boils, so you don't need anything circulating it around - on top of that, because of the state change you're now dealing with extracting heat from a gas, instead of a liquid.  The whole process is totally different and far more complicated than what I've read about traditional immersion cooling...

Not saying that you shouldn't play around with immersion if it interests you - just saying you're more like likely not going to learn anything that is relevant for Novec.

For me, playing with Novec would be fun but it's cost kills that idea. I really don't care for the issues with any other type of fluid submersion cooling so that brings me back to using cooling blocks.

Now just need to think of how to couple a cold plate to the short topside heat sinks... Using a thermal gap filler putty like from Bergquist would be dandy. Just put a blob on each chipsink to fill most of the open area between the fins. But once again, it is not cheap although far more affordable than Novec or most other engineered thermal transfer fluids.

For those that want to try this, but the cost of the Novec is way too much.  You can test this out with a much cheaper alternative and there's been threads on Mineral Oil Submersion systems here before.  Why not test it with that first and then decide if you want to go to Novec?

As far as the plastic\brittle aspect of mineral oil goes, it definitely has a negative effect on the cables that come in contact with the oil. Back in highschool we put PC in a fishtank submerged in mineral oil. The power cable was hard as a rock after sitting in it for a couple months.

Must have been - I don't recall there being one that had a higher temp, but there are so many caveats to it, plus so many variations.  It actually has the same issue as the dialectric oil you mentioned - although not nearly as bad - you have to check each material that it comes in contact with and ensure that it won't have a chemical reaction with it.  I also know there can be issues with contaminants getting into the Novec and causing issues - but alas I didn't have enough time with it to encounter anything like that.

I played around quite a bit with water cooling - sold two of my quad S5 water cooled builds on here, and dismantled my custom stuff (once again, acrylic is great with water cooling).  I've never done anything with oil immersion just because of the mess involved, plus you have terrible viscosity in most cases to try to move it around in any meaningful way.


It's actually probably even more expensive than you think!    Don't quote me on this, but I seem to recall it was in the $500/gallon range.  The $1000 we spent was enough to do a couple S5's - you learn quickly with it that you're best off putting them inside of a kind of acrylic sandwich, so there's as little Novec actually used as possible, and what is used can be used effectively.

Let me also say that I still think this stuff is bad a**, but it just isn't cost effective for mining...

damn!!! this liquid it is very expensive!!!
I never thought to this possibility!!!

Bet you used Novec 7000 which boils at 34C (93.2F)?
I was thinking about the 7100 which boils @ 61C (141.8F). Means it's stable even in very very uncomfortable if not deadly ambient temps. Also means that you can have the higher delta-t between its boiling/condensing temp and the water cooling loop.

Ja some slight pressurization to leak detection purposes would be a good idea. BitFury's data tanks are sealed and also show a rather large pipe coming out. An over pressure relief pipe perhaps if the water loop fails and China Syndrome sets in?

With oil immersion what I see is 2 points:
1st being what is the long term effect of the oil or other fluid on the miners? Will it seep into metal can caps? Harden/embrittle plastics or make them swell? Attack whatever thermal glue used to bond the top sinks to the chips? etc. Anyone have a miner that has been in oil for a few years to examine then fire up and see it it still works?

2nd deals with the heat-capacity and viscosity of most oils. Rather poor on both counts and far from ideal but can be dealt with if needed. There are other other heat transfer fluids like ones from Paratherm that are designed just for this: Thin viscosity so pumps and flows very well with much smaller eddies around parts along with much higher capacity than mineral oils. Also decent lists and support on what materials the fluids are compatible with. But of course- like Novec, damn pricey.

edit: Damn you... We have a 55gal drum of Shell Diala-AX high dialectic transformer and other electrical component insulator/cooling oil. Stuff is not much thicker than water. I do know that it rapidly sucks the plasticizers out of vinyl and hardens it within a couple months. Bigger possible downside is that it is designed as a penetrating oil (for getting into the winding's of transformers). O-ring seals do not stop it so it getting inside metal can caps is a concern as well. But it is also compatible with most other materials used in electronics. Think I'll dunk an old s1 board in some for a few months...

I looked into this quite a bit back in the S5 days - really, you can pick whatever miner you want, I think everyone who has seen the videos of Novec immediately goes "I WANT THAT!".  I actually went so far as to buy some of it to experiment with a bit, and I had three big takeaways from it... The first is that the actual system in production is much more complicated than it appears.  You aren't just tossing the boards into the Novec, you have to manage how the cables come into the enclosure, and this is surprisingly challenging (and I love acrylic, look at any of the enclosures I've designed in other threads).  The other big enclosure issue is that you still have to have some sort of cold water loop to bring the gas Novec back into it's liquid state - and this leads to my second big takeaway, and that is that you're still left having to deal with the heat.  The best way I found to think of Novec is that it's a REALLY efficient way of transferring heat from one place to another - in this case it's transferring heat from the ASIC to your cold water loop... But the net result is that the water loop now contains the heat.  Sure, dealing with a radiator and being able to convey the heat trapped in water is much more convenient than air.

And then that lets to my final takeaway - part of the system failed at some point in my testing, and the Novec turned to gas and ran away.  All of it.  $1000 basically vaporized overnight, oh, and they took the ASIC with them, since it promptly burned itself up once the Novec took off.  The system itself worked fine for a few weeks, so I wasn't some catastrophic design flaw, but in the end something failed and it was able to escape.  Now keep in mind that I'm in Arizona, so the ambient temps were 100f+, which certainly helped Novec in it's jailbreak - but the real bitch is that I never was able to figure out what failed.  I poured over the enclosure trying to find the leak, but never was successful - and I suspect this is also why in some of the 3M units they actually do some really light pressurization, so they can detect a leak before it's visibly failing.

So for me, the completed system (sealed enclosures, pressurization, water loops, etc) cost so much more than just fans blowing air - and the potential downside of failure being the complete and utter destruction of the system (and Novec, which ironically was the most expensive single component), made this a something not worth pursing - for me at least.  

So does all this brainstorming mean your wheels are spinning on some S7/S9 Water block mods 

Liquid cooling of any sort 101:
For now, putting aside how heat is getting into the primary fluid to-air cooling loop be it direct contact using cooling blocks, oil-immersion fluid to air, or a 2-phase liquid/vapor 'Hot side' to fluid to air setup:

#1) All heat flow calcs are based on using either BTU/HR or for heat loads over several kW, Tons
2) Conversion factors:
  1kW = 3412.142 BTU/hr
  10kW = 2.84345 Tons
  A handy site is http://www.rapidtables.com/convert/power/kW_to_BTU.htm and http://www.unitconversion.org/power/kilowatts-to-tons-refrigeration-conversion.html

That covered, we come to basic laws of heat transfer between two objects or mediums. Again, for simplicity media heat capacities, thermal resistance of the heat exchangers used, Heat exchanges size and even flow rates is left out.

That leads us to:
3) Based purely on what temp we want to maintain at the load (miners) vs the temp the sink or where the heat is going  (outside air, a large pond, a swimming pool, etc) the general rule is:

The amount of heat energy (power, BTU's whatever) that can be moved is directly related to Temp_load - Temp_sink or more commonly referred to as Δt.

Now to apply these basics to miners, 1st remember that:
a) miners are quite happy at say, 61C (141.8F). That's pretty toasty and more than uncomfortable to the touch.
b) Even using an extreme sink temp - in our case, eventually ambient air - of 50C (122F) we still have a pretty decent Δt With a more reasonable ambient air temp even better.
c) That in turn means we only have to dissipate enough heat into the sink (air) by using a honking big outside fluid-to-air heat exchanger to keep the desired miner tank temp where we want it. At a minimum you will need desired temp and Δt between that and the sink (air) along with BTU's or Tons to be dissipated to start sizing a radiator or cooling tower.

In summary, given that it is MUCH easier to transport massive amounts of heat energy in a fluid moving through pipes vs just moving a massive amount of air it gives a huge advantage where feasible or appropriate. Liquid is of course much more compact on the load end (miners packed in tanks) and pipes are a helluva lot smaller/easier to route than air ducts. Only really big part is the sink end but since that is outside, not a huge issue to deal with.

To be continued...