Topic: Modular FPGA Miner Hardware Design Development - page 8. (Read 119320 times)

Ok, what else needs to be done to the MSP section?
I'll have a look to change the package tomorrow.
EDIT
Did i miss something ? In the version i uploaded the MSP 430 f 5507 has a RGZ package i thought we agreed to use this one. ?

It seems that FPGA and PSU are ready or nearly ready, but the MCU part needs a lot of work, still. For example, the package is wrong (PT instead of RGZ).

Has anybody started merging the parts of MSP 430, FPGA and Power supply yet ?

You guys are doing some ambitious stuff here, has been a great thread to follow.
 The first one I click on in new replies section. 
Am very interested to see how the testing phase goes ... keep it up.

I uploaded a new version of the FPGA design to github and dropbox. Commit logs:

Added second 100MHz oscillator: The oscillators are now located close to their respective FPGAs. Added SUSPEND signal to FPGAs.

Edit: For everyone who cannot see it in dropbox, these are pictures of the current designs for the FPGAs and li_gangyi's PSU units:

I had a look at the MCU schematics: I prefer the MSP430F550x, because it has the smaller package. For the actual implementation, I suggest the MSP430F5507 because it is in the MSP430F550[4-7] range of the the simplest chips that have a temperature sensor and in that range it has the most memory.

The USB connection seems to be too simplistic. Please look at figure B-33 in the MSP430 Hardware Tools User's Guide (SLAU278F): There is a diode in the +5V line that is also mentioned in the MSP430x5xx/MSP430x6xx Family User's Guide (SLAU208H). Then there are the ESD diodes, and RC filters plus caps on the power supply.

The JTAG connection is done in such a way that it debugs the MSP430, but the required connection to NMI is missing. Please look at figure 2-1 of SLAU278F for a suitable connection. If the idea is to connect these JTAG signals to the FPGAs, then that may also work (these pins work fine as GPIO), but please rename TDI and TDO to TDI0 and TDO1. But then you cannot use the pins to debug the MSP430. If you want to have a combined thing, where a JTAG connector is used to debug the MSP430 and the MSP430 can debug the FPGAs, then you need to add JTAG signals on different pins.

Have you decided on which pins to connect the two SPI busses: one bus for the FPGAs, one going to the DIMM connector?

If three people agree then there is not much point pretending I know how bad a bussed signal would get. I will add the second oscillator today.


I think the OHL is ok. The suggestion of packaging everything into an archive is useful for distribution, but not feasible on dropbox or git, because we don't want the developers to always have to download the full archive if only a single file changes. We are required to use the license document in PDF form, changing it is explicitly forbidden. So we cannot concatenate the GPL and OHL licenses into a single text document "COPYING". We will need three different files: "COPYING" containing just the copyright notices, warranty disclaimers and explanations which license applies to which directory, "GPLv3.txt" that contains the GPL license and "OHLv1.1.pdf" that contains the OHL license.

Regarding the clock sources:
I agree with TheSeven and li_gangyi. One individual clk soure for each FPGA no matter how many they might become.This would provide the best quality signal for each FPGA with the least noise.
As i also consider the bandwith of the BUS a minor problem(see fpgaminer's post on data rates).

Regarding the copyright:
As i already stated  the GNU GPL v3 and the CERN OHL in combination seem the most reasonable way at the moment.I certainly may not guarantee there isn't another better approach out there but it will take me time to find it. So for the time being this would be my proposal.
Please give me a short comment what you think of it. We may change this later in case we see the need to do so, as we still remain the inventors.


And in own matters:

Yesterday i've been on the second bitcoin meeting here in munich.
Very interesting dicussions and people. We will try to hold those regular.
I got a lot of positive feedback regarding our project and many people reported interest
in the final design or wanted to support us in our development.
So i see again there is a lot of potential in this technology on the market already.     

While this might be feasible for JTAG/SPI where we can just choose whatever frequency works consistently at a later point, and aren't really constrained on bandwidth, a non-ideal clock signal for the FPGA directly impacts the maximum achievable hashrate. It's a cost/performance tradeoff, and I don't think the oscillator cost will make up for the performance risks that you'd take otherwise.

I don't see it as a problem, if it is, we could always tie a suitable capacitor onto it's softstart pin to ram up the voltages slower.

li_gangyi: do you think there will be an inrush current issue with the voltage regulators you suggested?

The board I work on at work used to have issues with the regulators locking up if they could not grab a lot of amperage on power up.  At running mode, the system only sucks out like under 1 A @ 24 V, but when you power it on, if you limit the current at the power supply to somewhere around 2-3 A, it will lock up the regulators, and they never get to their full voltage.  Our system has a bit more stuff on it that's sucking out power on boot, like a DSP, FPGA, a bunch of 24-bit A2Ds, and probably a handful of other stuff I'm forgetting at the moment - and mebe that's part of the problem, all the chips turning on at the same time.  But I'm unsure if this is typical for voltage regulators, and I want to be sure that once this board gets fabbed/assembled, that it will actually turn on without some deadbugging.

for the clocks - I don't think phase synchronization between the two FPGAs in an issue - they shouldn't be communicating with each other, and thus it doesn't matter if the phase relationship between the two FPGAs is synchronous - everything will be talking to the MCU.  Personally I would say go with 2 clock crystals as you'll get a better clock signal this way - and we don't gain much from having the 2 clocks in phase - other than PCB area, routing, and crystal cost.  Having a pure clock signal with low jitter and an even duty cycle will prevent the FPGA from fucking up because to get a fast hash engine out of the FPGA we are going to have to push it to it's limit.  Just for this reason alone I would say go for 2 clock crystals.  But then again, if you guys are sure you won't get a polluted clock signal this way, then it is viable.

Also, if you guys add any more IO connections to the FPGA - keep them all in the same bank.  I believe the Spartan6 has restrictions on which signals can be used in which clock domains based on quadrant.  You can get by the ISE by forcing it, but it's typically not good practice because the tools will have to route that one signal a long distance, which will impact max core speed.  But the FPGA should have more than enough IO on that one Bank 2 you guys are already using.

When you're at 100MHz and especially for a clock signal you have to do this coz you want the steeper edges to have lowest jitter.
JTAG and the SPI are less critical and you can slow those edges down.  At say 25MHz you have one driver resistor and just use 10R resistors at the junctions splitoff ( and/or connector crossover points) and omit them for short T split lengths. At say 2MHz you just need one "slow down the edge rise/fall" resistor at the driver end.

For a bigger board you can/must use several  buffers for the clock and then you can make other (distributed) topologies rather than the usual star if you will.
Even with a star topology you don't have to route in a star, nor do they have to be the same length as each trace is individual after the resistor at the driver.

15pF is just a measurement point. It will be more "happy" with lower pF but still work with higher within reason.

The drive is no problem according to the data sheet (hopefully the load capacitance is close to 15pF). The problem is the star configuration: doing this for one signal is not so much of problem. But having to do that three or even five times makes the bus considerably wider. I know we first want to build "just" a two-FPGA board, but I still want to know how to build a scalable bus for later boards with more FPGAs. An option that uses a star configuration just seems wasteful to me.

Can you give us some info on exactly how many Watts of power this device uses?

Just use one osc but make sure it has proper cmos drive output rail to rail (not clipped sine).
Then put two resistors close to the osc with individual trace from each resistor to each FPGA and with a continous GND plane under the traces from end to end.

The reason why I am harping so much on keeping it down to one oscillator and using a suitable termination: we have the same problem with the SPI bus (clockrate there is 25MHz instead of 100MHz) and to a lesser extend also with JTAG. So I want to learn how to make the bus the correct way, and if at all possible I want to not use a star-shaped bus with the MCU at the centre.

You'd need seperate resistors to each FPGA and yes, 2 seperate oscillators is better then anything. 

I uploaded a new version of the FPGA and PSU designs to github and dropbox. Commit logs:

Added 100MHz oscillator to FPGAs and added vias below switchers in PSU section.

So you think we need more oscillators. Would a better termination also suffice (reintroduction of the Thevenin termination) or is that still much worse than two separate oscillators?

That's what I planned on doing, populate 1 board, test it out and if it works, populate the rest. I have no idea how to test the functionality yet except for the PSUs and stuff, unless some1 can come up with some smallish test firmware to be loaded on the FPGA + MCU.

We could just get 1 cheaper FPGA to populate on the 1st board, if that works (meaning the PCB is adequate), I'll then populate the rest of the boards with LX150s.

@Olaf: I'd think a 50Ohm resistor should suffice, however you'd need to route the oscillator traces to only have stray capacitances between it and ground (so it doesn't pickup stuff from power or other IOs). The oscillator you've in mind is pretty good (10ppm), however you'd be pissing away that performance if we do this, I'd prefer to have the oscillator dedicated to each FPGA and keep everything nice and tight, it doesn't take up much board space anyways.