Modular FPGA Miner Hardware Design Development - page 5.

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: fizzisist on August 10, 2011, 10:15:08 AM

[...] Did Olaf change the feedback resistors to the suggested ones in the power supply datasheet? [...]

Not yet. I also wanted to incorporate the suggestion by TheSeven to increase the 1.2V rail to 1.26V, and I haven't thought about how to find the closest matching resistor values for a given ratio.

fizzisist

hero member

Activity: 720

Merit: 528

O_Shovah, good idea to summarize the list of tasks remaining. I would add to it:

1) Decide if we should switch from the LMZ12010 to the smaller and cheaper LMZ12002 (or 3) for the 2.5V rail.

2) Did Olaf change the feedback resistors to the suggested ones in the power supply datasheet?

3) Decide on cooling method. These could mean leaving mounting holes (or clearance for the Radian EZ-Clip) and fan headers.

O_Shovah

sr. member

Activity: 410

Merit: 252

Watercooling the world of mining

Maybe just to have a little roundup,

wich task are still to be fullfilled for finalising our prototype on the hardware stage ?

- completing MSP 430 integration ?

- I asume the power section is complete so far

- clarify if the 0402's are nessecary ( i personally would use them to stick to the specification as close as possible)

- intergration of debugging devices ( pin headers, leds ,peepers) ?

- completing routing on the DIMM

Please correct me and complete this list.

DiamondPlus

member

Activity: 68

Merit: 10

I find it interesting because of the FX2 chip, but apart from that it has too much: extra DDR SDRAM, extra EEPROM, extra CPLD, microSD card slot. The pricetag is only a factor of 2 above what we hope to achieve (note that they are missing the power supply section and need 3 different voltage rails).

I don't know about the others, but I am in it for the fun of developing a board. The board you linked may be a very good start to get FPGA developers to write code, but once our board is go, it won't be cost effective (unless you figure their support and warranty are worth the price). I am not arguing against the board: it is good for early adopters, but eventually we should beat it.

li_gangyi

full member

Activity: 157

Merit: 100

Hi, yeah I've been busy lately. I've no problems with 0402s, but it'll probably slow things down abit. Do we really need the 0402s at this stage?

Regarding the heatsinks, I don't see a problem with the clipons as long as it isn't bumped around. Alot of times I see them thermal epoxied or thermal taped to the package. I don't think we'd be able to find a heatsink that'll cover both the FPGA and Vregs, since they have vastly different heights. The 100uF decoupling capacitors around the perimeter of the FPGA will also probably prevent a long heatsink from being used to cover both chips.

O_Shovah

sr. member

Activity: 410

Merit: 252

Watercooling the world of mining

Hello i'm back

Im happy to see , you made good progress.

I only have read through the last pages partialy yet so just some thougts on the last points.

- Regarding the parts size: We will need to have a comment from li_gangyi how much hassel he would have soldeirng smaler components but i havent seen him around lately.
Maybe im gonna pm him later.

- Regarding the Power consumption: The Calculations i ran were a total first try for a absolutly theoretical maximum. @ TheSeven maybe you could run the estemination tool again as you are experienced with Xilinx software.
But in the end nothing will replace test cycles with the protoype.

- Regarding the heatsink: the solution http://www.radianheatsinks.com/Products/INM23001-33W/2.6_155.aspx you postet is certainly good choice for the prototype,
but for series we should consider a solution using mounting holes. I consider the way of snapping it below the BGA a bit risky.
Im on search for a plate fin heatsink wich covers the whole DIMM in order to act as heatsink for both the the FPGA's and the voltage regulators.But different heiths and themeratures might be an obstacle to that.

TheSeven

hero member

Activity: 504

Merit: 500

FPGA Mining LLC

Quote from: fizzisist on August 07, 2011, 04:59:28 PM

Quote from: O_Shovah on July 15, 2011, 12:31:57 PM

I ran the xilinx power estimator tool using a utilisation of 105% for all parts avaidable (Logic,DRAM,DSP,DCM,....).
It seems 5.5 W is the maximum the FPGA allows before getting cooked.
This splits to:

- 1.2 V : 4,2 A
- 2,5 V : 0,3A

This numbers heavily scale with temperature. So cooling is a major influence factor as power consumption ranged from 7.7W (no cooling, no heatsink, -T_case 125 °C)
to 4.5 W (500 LFM,huge heatsink,T_case 40°C) using the same setup.

This estmination should cover for the absolut possible maximum, but as i am totaly new to this tool i would like somebody more used to the FPGA design to confirm this results.

Here's O_Shovah's estimate from a few weeks ago, to remind us. 300 mA x 2 + 100 mA for the MCU gives 700 mA total. I think the LMZ12002 should be sufficient then, and it's half the price of the LMZ12010. Does anyone object to switching to this for the 2.5V switcher?

By the way, it would probably be a good idea to include mounting holes for a heatsink around each FPGA. Also nice would be some headers for fans. These could be directly connected to the 12V unregulated rail. Maybe something like this fansink?. It's a little bit large, but I think would do a great job at cooling these FPGAs. It's probably not required that we use a heatsink like this, but it would be nice to include the holes for it so that we have the option. A more compact option would be something like this, but it's too large for our FPGAs. And finally, a passive heatsink that fits the FG(G)484 package perfectly: HS2135DB.

Does anyone else know of other good options? This is the result of 5 minutes searching on my part, I'm sure there are thousands of other heatsinks out there.

Use the maximum possible values from the datasheets and add some safety margin when designing the PSU and cooling. If the chips stay cool, they consume less power and yield higher hashrates. I'd calculate 8-10W per FPGA. 45°C ambient temperature, and I'd like the FPGA to stay below 65-70°C if possible. So if you don't want to use a fan, you would need at least something like this.
If no other components are in the way, I'd consider using one huge passive heatsink that covers both FPGAs. This would also be favorable for a backplane setup in a server case, where there room between the DIMMs is almost filled by the heatsinks, so you can just add some big fans that push some air through the whole thing from one end of the DIMMs to the other one. I'm thinking of something along the lines of this

fizzisist

hero member

Activity: 720

Merit: 528

Quote from: O_Shovah on July 15, 2011, 12:31:57 PM

I ran the xilinx power estimator tool using a utilisation of 105% for all parts avaidable (Logic,DRAM,DSP,DCM,....).
It seems 5.5 W is the maximum the FPGA allows before getting cooked.
This splits to:

- 1.2 V : 4,2 A
- 2,5 V : 0,3A

This numbers heavily scale with temperature. So cooling is a major influence factor as power consumption ranged from 7.7W (no cooling, no heatsink, -T_case 125 °C)
to 4.5 W (500 LFM,huge heatsink,T_case 40°C) using the same setup.

This estmination should cover for the absolut possible maximum, but as i am totaly new to this tool i would like somebody more used to the FPGA design to confirm this results.

Here's O_Shovah's estimate from a few weeks ago, to remind us. 300 mA x 2 + 100 mA for the MCU gives 700 mA total. I think the LMZ12002 should be sufficient then, and it's half the price of the LMZ12010. Does anyone object to switching to this for the 2.5V switcher?

By the way, it would probably be a good idea to include mounting holes for a heatsink around each FPGA. Also nice would be some headers for fans. These could be directly connected to the 12V unregulated rail. Maybe something like this fansink?. It's a little bit large, but I think would do a great job at cooling these FPGAs. It's probably not required that we use a heatsink like this, but it would be nice to include the holes for it so that we have the option. A more compact option would be something like this, but it's too large for our FPGAs. And finally, a passive heatsink that fits the FG(G)484 package perfectly: HS2135DB.

Does anyone else know of other good options? This is the result of 5 minutes searching on my part, I'm sure there are thousands of other heatsinks out there.

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: fizzisist on August 06, 2011, 09:37:39 PM

[...]
I have a question about the PSU. What is the load on the 2.5 V rail? Aren't we going a little overkill with the 10A supply? Maybe we could save a little money and space by switching to the LMZ12002 or 12003 (2A and 3A, respectively). The package is slightly smaller, too.

Good question, the exact current can probably be given by TheSeven, as he compiled (some version of) the HDL code that runs on the FPGA and the tools can give a power estimate. He insists these numbers are highly unreliable, but they are still the best we have at the moment. That said, we can also look at the datasheet: the maximum current for VCCAUX should be 600mA per FPGA, but there is no power consumption specified for VCCIO. As we don't connect many io pins, my guess is that this is low (100mA, maybe). So a 2A switcher may be sufficient. But a guess is all it is! We have to build the prototype and measure the currents in each path.

fizzisist

hero member

Activity: 720

Merit: 528

Thanks for the input, old_engineer.

@Olaf
Thanks for putting together these totals. It's good to see how the price is shaping up.

I have a question about the PSU. What is the load on the 2.5 V rail? Aren't we going a little overkill with the 10A supply? Maybe we could save a little money and space by switching to the LMZ12002 or 12003 (2A and 3A, respectively). The package is slightly smaller, too.

old_engineer

sr. member

Activity: 387

Merit: 250

Quote from: fizzisist on August 04, 2011, 02:51:48 PM

In general, a smaller capacitor will be able to get closer to the pin and therefore do a better job decoupling. I think it would make sense to just use as few 0402s as possible: the probability of making a mistake soldering is just going to be increased for each additional capacitor. I don't think we have to think about it like reducing trace width or such (therefore raising the cost of the whole board). When these boards are eventually loaded by a professional shop, they won't even blink when we ask them to load 0402s.

Seconded that pro shops won't blink at 0402s. If there's going to be a hand-soldered prototype, avoid them unless you absolutely need them. Later on, switch to 0402s to reduce board space and cost (assuming large quantities of boards, of course).

I rework 0402s all the time. My suggestion: when laying out a prototype that has to include 0402s, like in the bypass capacitor example above where using a 0805 might cause a difference in behavior, make sure to leave room around the 0402 so you have room to get a soldering iron in there. Just because it's a small component doesn't mean that the board needs to be packed densely, and densely packed 0402s make it tough to only remove the desired resistor without accidentally removing nearby resistor(s). Those little suckers come on in like two seconds if your soldering iron strays. A quality pair of tweezers and a magnifying glass for inspection are also must-haves.

FWIW, I use one of these cheapo $90 USB "microscopes", which is a handy gadget for documenting issues:
http://www.amazon.com/ViTiny-Handheld-Microscope-measurement-functions/dp/tags-on-product/B003NGIBQI

and a pic of a 0402 cap that is accessible enough, with enough space to the left & right for soldering iron:

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: TheSeven on August 05, 2011, 05:24:49 PM

[...]
Are there any components on the back of the board as well? If yes, would you mind posting a picture of the back side of the board as well? I'm a bit worried by the small number of capacitors that I'm seeing on the front of the board...

There are caps on the back (the smallest ones, only). The board is only 1.2mm thick, in which case the Xilinx PCB design guide assigns them to the back. Instead of an image, how about a PDF with the schematics, views of the board with all layers, from the front and back and a BOM? This PDF is available on GIThub if you don't have Dropbox access. The only problem: it's too large for direct download from the webpage, you need to clone the repository.

TheSeven

hero member

Activity: 504

Merit: 500

FPGA Mining LLC

Quote from: Olaf.Mandel on August 05, 2011, 04:45:36 PM

Quote from: TheSeven on August 05, 2011, 04:10:12 PM

[...]
Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

I think moving the resistors below the FPGAs is a given by now. But if we increase the core voltage to achieve higher clock rates, then I would ask how close we want to cut it: is there a risk of having any EMF mess with the trace from the resistors to the switcher, so that the switcher runs at a too high voltage? What about required or suggested resistor tolerance: is 1% sufficient?

And our of curiosity: what is the gain in terms of clock rate for a voltage increase of 0.05V? I am willing to take ISE estimates after whichever stage you feel is remotely reliable.

IIUC the resistors shouldn't be below the FPGA, but instead as close to the switcher as possible, but fed from a trace that's coming back from one of the FPGA VCCINT pins. Also remember that this will only compensate for the losses on the VCCINT path, not the GND return, so the effective voltage will still be a bit below the programmed value. Ask one of our more experienced electrical engineers regarding how close to cut it, I'm just a computer engineering student. I just wanted to suggest to stay on the positive side of tolerance, as the suggested values were on the negative side, which doesn't seem like a good idea.

I don't think ISE even takes these values into account before the PAR stage, so you'll probably have to do a full synthesis run without timing constraints enabled or even step constraints to check the maximum achievable clock for both voltages. I've done a quick search on the web and didn't find anything even remotely related, so we'll probably have to analyze this ourselves.

Are there any components on the back of the board as well? If yes, would you mind posting a picture of the back side of the board as well? I'm a bit worried by the small number of capacitors that I'm seeing on the front of the board...

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: TheSeven on August 05, 2011, 04:10:12 PM

[...]
Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

I think moving the resistors below the FPGAs is a given by now. But if we increase the core voltage to achieve higher clock rates, then I would ask how close we want to cut it: is there a risk of having any EMF mess with the trace from the resistors to the switcher, so that the switcher runs at a too high voltage? What about required or suggested resistor tolerance: is 1% sufficient?

And our of curiosity: what is the gain in terms of clock rate for a voltage increase of 0.05V? I am willing to take ISE estimates after whichever stage you feel is remotely reliable.

Olaf.Mandel

member

Activity: 70

Merit: 10

I went ahead and merged the different parts temporarily. I still feel that at least the MCU is unfinished enough to warrant not doing the final merge, but this temporary merge lets us see where we are going:

The BOM total for one board is about 340EUR (a few components are still missing, but those should fit in that price limit). The board price at pcbcart is 27EUR per board plus 135EUR one-time costs with the following settings:

Material: FR4
Layers: 4 layer
Material Details: Standard Tg 140C
Board type: single unit
Board Size (width): 135mm
Board Size (height): 95mm
Quantity: 5pcs
Thickness (Finished Board): 1.2mm
Layer Stack: As pcbcart default
Layer Stack Details: -
Impedance Control: No
Surface Finish: Lead Free HASL - RoHS
Outer Layer Copper Weight (Finished): 35um
Inner Layer Copper Weight: 35um
Min. Tracing/Spacing: 0.15mm
Min. Annular Ring: 0.10mm
Smallest Holes: 0.30mm
Holes Number: Over 600
Buried/Blind Vias: No
Times of Buried/Blind Via: --
Surface Mount: 2 sides
Soldermask: Both Sides
Peelable Soldermask: None
Soldermask Color: Green
Matt Color (only add to Green or Black): None
Silkscreen Legend: 2 sides
Silkscreen Legend Color: White
Gold Fingers: Yes
Gold Fingers Number: 240
Gold Fingers Chamfer: 60°
Slots in Board: No Slot in Board
Slots quantity in board: -
Testing: Yes
UL Marking: Yes - as pcbcart default
Date Code Marking: Yes - as pcbcart default
Lead Time: in 18 days
Special Requirement Note: -

So a full set of 5 prototypes is about 5x400EUR = 2000EUR.

PS: When doing the merge, I also found two small bugs in the PSU (not replaced the resistors, yet), so reuploaded that, too.

TheSeven

hero member

Activity: 504

Merit: 500

FPGA Mining LLC

Quote from: Olaf.Mandel on August 05, 2011, 10:45:30 AM

Quote from: fizzisist on August 04, 2011, 10:19:58 AM

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: MonkeyThink on August 05, 2011, 11:34:50 AM

[...]
What other people's idea's on this. Is everyone thinking that they will just hook up a hardware miner to their desktop machines or laptops and leave them running 24x7? Because I was thinking it would be nice to have a totally stand alone solution.
[...]
If people are interested then perhaps we could start a discussion along these lines with some suggested solutions. Obviously it would be possible to build the solution into the FPGA but it seems like it would be a waste of valuable space and effort and might be better (and more cheaply) implemented in other ways.

I was thinking along the lines of something like one of this tiny Gumstix boards: http://www.gumstix.com/ running Ubuntu or perhaps a more mainstream, small motherboard. Mini-itx perhaps.

Thoughts?

This has already been discussed and is on the roadmap. Basically, the plan is as follows: First we walk, then we run.

Meaning that first we get something working that requires a computer and then we change the backplane to include an embedded, ethernet capable computer. This will leave the FPGA design completely untouched (it being on a different PCB and all). We haven't discussed which CPU to use later, because we first have to get this one working...

fizzisist

hero member

Activity: 720

Merit: 528

Quote from: Olaf.Mandel on August 05, 2011, 10:45:30 AM

Quote from: fizzisist on August 04, 2011, 10:19:58 AM

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

Good eye, I didn't see that equation before, only the table. So, li_gangyi's choices would work because the ratio is right. On the other hand, the table values are the recommended values. Also, there's this note in the datasheet:

These resistors should generally be chosen from values in the range of 1.0 kΩ to 10.0 kΩ.

R_FBT is only 620 ohm on the 1.2V supplies right now, so we should increase those. I'd suggest just going with the recommended values as long as we're changing things...

MonkeyThink

newbie

Activity: 33

Merit: 0

Hi,

Not sure if this should go in a separate thread or if it's OK here, but while the guys have been doing such fine work on the mining hardware design I've been wondering about the 'front end' computer that will be feeding the miner hardware and submitting proof of work back to the network.

What other people's idea's on this. Is everyone thinking that they will just hook up a hardware miner to their desktop machines or laptops and leave them running 24x7? Because I was thinking it would be nice to have a totally stand alone solution.

I remember that someone did post about having an FPGA development board mining totally stand alone, but I can't seem to find that thread at the moment.

If people are interested then perhaps we could start a discussion along these lines with some suggested solutions. Obviously it would be possible to build the solution into the FPGA but it seems like it would be a waste of valuable space and effort and might be better (and more cheaply) implemented in other ways.

I was thinking along the lines of something like one of this tiny Gumstix boards: http://www.gumstix.com/ running Ubuntu or perhaps a more mainstream, small motherboard. Mini-itx perhaps.

Thoughts?

Olaf.Mandel

member

Activity: 70

Merit: 10

Quote from: fizzisist on August 04, 2011, 10:19:58 AM

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

Topic: Modular FPGA Miner Hardware Design Development - page 5. (Read 119320 times)