Topic: Cairnsmore1 - Quad XC6SLX150 Board - page 119. (Read 286370 times)

I'm seriously considering watercooling these. It depends on if a faster bitstream is released that could be thermally limited by the small heatsinks we're using on these FPGAs. If I can get another 15-30% by watercooling and keeping the core temps low then I'm all for it.

That's not passive then if you're relying on a fan to move air. Convection should be enough.

I meant a case fan in the enclosure, but not a fan on the heatsink, a bit like we get passive PC video card, but still need a casefan to have the air moving.

Yes, they are serious.  But not as serious as me.  Four Peltiers, then the heatsinks, then the watercoolers, all submerged in mineral oil, sealed into a 55 gallon metal drum, sunk down into Challenger Deep.  I figure I can get about a half a TH/s from each core.

If a fan is pushing air across it, how is it passive?

lol are you guys serious water cool 15watts ? and the chip runs at 1.2v ? A passive heatsink with 120mm fan pushing air across will be enough.

Watercool that shit.

Thats up from 200 mhash, right? I wonder how many of these boards should have been equipped with big copper heatsinks instead.

Yes, definitely. I wouldn't want to get nearly that close to the limit. That being said, if someone like eldentyrell releases a bitstream that does 250MH/s on an LX150 but pushes power consumption up to 15W, there are options out there that might be worthwhile to deal with that.

Ok so time for a little more info.

We have enough heatsinks arriving next week to do the first 100 boards so that is good and solves the problem for May and early June shipments. We might have a little gap when we run out before our main batch arrives. The initial heatsink will be 10mm higher than the final solution. They are both from the smae family of parts. The higher one is about 1.5 Deg/W and the smaller 1.9 Deg/W at 200 LFM. at higher air flow rates they are very close in performance and both better again. That's why we choose to run with the smaller part as standard as it did not make much difference on thermal performance with our preferred fan option. These thermal resistance numbers are about half the thermal resistance of the Spartan-6 itself so they are not much of a limiting factor in the scheme of things. We can't do anything much about the Spartan-6 thermal performance that's with the Xilinx team that designed it so we just have to work with what we have and make the best use of it.

The second part of the cooling solution is an Arctic F12 fan which is a good balance of low noise (22.5dbA), long lifetime and high air blow. There are higher flow options but much noisier and we might offer those as an option somewhere down the line. Similarly copper versions of the standard heatsink make also be offered but only offer a little more performance for a lot of weight, direct cost, and shipping cost. 4 of the best are 0.5 Kg in weight and that is expensive to ship.

We won't ship a PSU as standard as many people will want to use options other than the 2.5mm jack. The weigh of these PSU is such the carriage will cost more than the PSU. However the 2.5mm jack ties up with 60W power supplies commonly available for LCD monitors at low cost. We may offer a PSU as a cost option for those that want it to arrive ready to go but there will be the extra cost of carriage. We will attempt to test a range of models and list our results but that won't be immediately.

On the schedule we are probably 2-3 days behind our original schedule plan but we still expect the first units to ship in May. All of the array FPGAs are programming under JTAG and the local SPI flashes are working and do configure the FPGAs now. The controller we have more work to but it is running a basic build now and is generally ok so far.

The Issue 1.1 design is now frozen and that starts it's path though manufacture on Monday. We have added a fan monitoring hardware now to all 4 fan headers and that connects to the controller.

JTAG driving capability is in the controller and it would probably use a micro engine of the FT4232 to drive it but there are other options. Update of the SPI flash is also viable if support is built into the array FPGA design.

LEDs - each FPGA has 4 LEDs.

We have yet to look at Bitcoin bitstreams and the compatibility and performance of them with Cainsmore1 but I expect we will get some of that done next week.

Generally I am happy with our progress so far. We do still have some way to go yet but that is the norm for this sort of project. On the manufacturing side our component supplies are shaping up well and we now have very few problems now in area.

I think what we have achieved in the 15 days since we started is pretty good even though the team here do things like this on a regular basis. This time we didn't beat our record delivery for designing a major new board and sending some to a customer fully tested but you can't win them all. Maybe we should have run a night shift like we do on really critical project time lines and we could have shaved a couple of days off but that's all academic now.

More to come over the next week or so.

Yohan

But no one wants to do that The whole idea about FPGAs is that they are smaller, cooler, less noiser than GPUs. Even if you achieve 2.7C/W top power draw shouldn't be more than 17W, commercial grade chips, 25W industrial grade chips at 25C ambient temp. Compared to 40W of one chip in BFL Single is damn small value. And we have to remember that propagation delays thru silicon raises when temperature raise. More power draw, faster MHz limit = less MH/s. There is somwhere a sweet spot and it's defenitevely below 17W thanks to high Rthjc... Are we agreed?

Of course, you eventually trend towards the point where the surface of the heatsink is at ambient and the heat dissipation is just a product of the ability of the heatsink to transfer heat to the surface. The jump from 200 to 800 LPM isn't that large though, and is well before most designs stop reacting to increased airflow. Digikey's also quite expensive, for that price you could get something like this that would probably have no trouble with that kind of spec.
http://ultimatepccooling.com/xiptponobrch.html

Again, I just said you can get a 0.5C/W or lower heatsink if you go large or loud enough. I still think 1.8C/W (4-2.2) isn't really a good heatsink/fan combination, and is pretty pessimistic.

Price is quiet huge. Pushing more air, as you said doesn't mean that you will be able to go low as you want. It depends on how a heatsink is made (shape), which materials was used etc, etc.  There always will be some point where it will impossible to achieve lower C/W. And no one wants to have another fan that generates 50dB of noise spinning at 10000RPMs.
Spartan6 are designed as low power devices. Things will be much more better with Atrixes7. Spec says that they will have Rthjc lower than 0.2C/W. So, even with not so big heatsink 30W from that chip will be no problem to dissipate (with total Rthja of 1.5C/W).

With 40x40 heatsink, 200LFM is only 3.4CFM. That's not exactly a lot of airflow, you can get a 40mm fan rated for 6 times that or more if you're willing to pay the penalty. You still have to derate the airflow for the obstruction, but putting the equivalent of 800LFM over a heatsink isn't an issue. You also are not limited to a 40x40 heatsink, anything with a reasonably thick baseplate can have a larger are than it's contact area to increase surface area. This one is 45x45 and rated at 1.1C/W with 200LPM, and it was just something I found in a minute or two on digikey.
http://media.digikey.com/PDF/Data%20Sheets/Advanced%20Thermal%20Solutions%20PDFs/ATS-61450W-C1-R0.pdf
The scaling isn't linear, but cooling drops with the square root of airflow, so increasing airflow by a factor of 4 tends to drop thermal resistance by a factor of 2. Put 800LPM through that heatsink and it should be capable of close to 0.5C/W. If you actually looked for something bigger than that, or one of those crazy heatpipe northbridge coolers it would be even better.

Difference between paste one or another don't play major role. Rthjc is like 10 times bigger, so more important is how long paste can hold their parameteres. Also I wouldn't count to find a heatsink with (of that dimensions) Rthja lower than 1C/W, with decent airflow.

So how would this change if I used a solid copper high performance heatsink plus expensive thermal paste like AS5?

Ok, so find me a heatsink with 0.5C/W with dimensions like 40x40 mm, size that is standard among those FPGAs. Best what i was able to find was 1.7C/W @200LFM.

That's without a heatsink, just blowing air at the chip. With a heatsink to get the thermal resistance from junction to ambient you add Rthjc and Rthca, which encompasses the interface between the case and the heatsink as well as the heatsink itself. When I said 2.7C/W Rthja, that was the system resistance with a heatsink with good airflow over it.

I'm not talking about Rthja but Rthjc. Thats a big difference. Rthja for FGG484 package is 9.3C/W @750 LFM. So, without heatsink you can dissipate maks 5W with ALOT of air blowing at chip. At still air Rthja is 15.8C/W. Thats 3W and you are boiling a core.

I don't think this info is known (by them) yet. They are likely doing testing on the prototype boards now. Yohan will likely come let us know "All the Juicy Bits" (as he put it) once they have completed testing and have actually hashed on a physical board.

I suspect right now they are focusing on getting it hashing. If they then have time/resources to complete/test additional features and still hit their shipping timelines, they likely will.

My understanding right now is the hardware is fully capable of it, in that the USB chip itself can directly drive the jtag chain, requiring nothing fancy on the control FPGA at all. But I don't know exactly how they have it all wired. depending on how this is implemented it MAY require something in the control bitstream to drive the JTAG (in which case it's a feature they need to deliberately add). If it's wired up in a way that allows the USB chip to natively drive the JTAG, that means the only real limitation is software for your PC that can talk to that USB chip in the appropriate way, to load the bitstream to the board. (so in otherwords the board will support it fine, but you might not be able to use the feature until someone writes said software for the PC).

Also from what I see, there are 2 JTAG headers. It appears the right-angle header next to the USB is the main JTAG chain for the 4 "Worker" chips in the matrix. The other JTAG header appears to be for flashing the Spartan3 "Control" chip. So chances are because there are 2 different JTAG chains, the USB chip can likely only drive the JTAG chain for the 4 worker chips. And the JTAG for the Control chip is probably "hands off" (unless you have a JTAG cable). So I suspect that bitstream will be fairly "fixed" and considered not user-servicable, but the bitstream for the 4 worker chips will be USB flashable by the user fairly easily.

Also of note, Yohan has mentioned that there is the possibility (but nothing confirmed, ie: hardware can do it, but Enterpoint may not have time to roll the feature out before shipping, who knows if it can be added in future). To have multiple bitstreams stored in the flash. So that you can "dual boot" the boards, or have options like a "failsafe" bitstream. So you can install new ones, and it can fail back to the "factory default" bitstream on it's own for example.

Again, same disclaimer as before. All just my own speculation based on available info. Don't assume anything until it's officially stated by Yohan or other Enterpoint "official" channels.