Topic: Modular FPGA Miner Hardware Design Development - page 11. (Read 119320 times)

I just added my mail to the authors and copying file.

I found a recent development in hardware licencing.The "CERN OHL" so the Center European de Research Nuclear  Open Hardware License http://www.ohwr.org/projects/ohr-meta/wiki/CERNOHL.
Its meant explictly and exlusiv for hardware and is similar like the GPL for software.
This would need for a spliting of the software part of the project to the GPL (all firmware,FPGA binaries,additional software) and the Hardware Part (all technical drawings, Documentation, Layouts, Prototypes).
But it would supply us with a reliable Licencing for both software and hardware, as the GPL has proven to be solid and reliable even in court and the CERN OHL can be seen simmilar as it was created by a International Authority and meant for Universitys and other societys wanting to publicate their hardware projects open to everyone without losing the initial rights to it.

In addition i just added my mail to the authors and copying file.


So thats my current status and proposal.I will dig further into this matter from monday onwards (last test ).

I see the Layout party being quite busy.Thank you everybody for making my little startthread such a great project

Ok, will remove them in the next iteration of the board. That will leave just 5 resistors for the FPGA board. As for the splitting at the source: does this apply for JTAG as well (should be slow enough, right)? And do you think one can run 25MHz through a T for SPI? The reason I ask is that this would make the bus even wider.


"x via with an y pad" means an annulus of (y-x)/2=175um, right? That leaves a clearance of ~157um on either side.

The default trace size and clearance for pcbcart are 200um (though 150um and 100um are also available) and min drill size is can be selected between 400um and 100um. The annulus sizes are either 250um or 100um. Their "standard PCB" product range is limited to 200um for the trace and drill size and the 100um annulus size is also not available there. The price difference seems to be small, though...


Don't get it: how is the (power, I assume) connector related to this? And I haven't tried your settings, yet, but with the design rules I used I cannot access any of the inner pins without routing this to a different layer.

I uploaded a new version of the FPGA schematic, board and PDF+PNG to github and dropbox. Commit log:

Converted the supply layer for VCCINT back to polygons, which allow the use of different nets. Also, moved some decoupling caps closer to their respective via.

This takes care of pusles first comment. I didn't change the vias, yet.

Also uploaded an AUTHORS and COPYING file, specifying GPLv3+. Please add or edit your info into these when you upload. The GPLv3+ license is tentative at the current time, while O_Shovah looks into the legal aspects of hardware vs. GPL a bit further. Anyone has a strong feeling about GPLv2 vs. v3? Tell us and edit the COPYING file if there is a consensus.

We can swap up to the current sharing capable LMZ modules, and make the Vccint 1 single supply, that'll need an external clock to make sure both modules run at the same frequency though. Can be a simple 555 or driven off the MSP430. With the MSP430, u can even generated a 180deh phase shifted clock, to reduce ripples in the output.

Good note on the Vccint plane, it should decrease routing complexity some.

Of course the pullups are needed and the conf done should be routed to the MCP.
I was talking about the thevening 100R+100R termination.
This is only needed for really high speed and even then it's kind of a tossup.
At work I'm using series termination at the driver end for a DDR bus running at 160MHZ ( 320Mbit pr wire) single end.

Just avoid T-junctions and split the signals at the source perhaps with a series resistor for each wire.
If you really want rx end termination I would suggest using a cap+resistor in series. Then there is no DC current drawn + driver issues but
it's still terminated for the higher frequencies.


For the fpga ball & escape routing,  I  use 0.50mm pads for the fpga, and 0.25mm vias with 0.60mm pads. ( 1.0mm pitch BGA)


You don't have to work your ass off for every decoupling cap, but to me it looks like it should not take much time to get a large improvement here.

I guess the connector will be on one side of the fpga. Then you have 2-4 rows you can route "straight" out and make routing
of the special purpose pins + vcc quite easy.

Ok, so you are saying polygons instead of a supply plane.


I was afraid someone would comment on the slightly longer trace length...


There is two types of resistors there: Thevenin termination for the SCLK (CCLK), MOSI, CLK, TCK and TMS signals. This is actually strongly suggested by Xilinx UG380 for the SCLK signal (they don't mention it for the others, though). This is a bit weird: the argument is that in master mode, the FPGA uses only a small drive strength on the CCLK output. But even for the slave mode (figure 2-3), they use the termination resistors. Unless you are very sure there, I would leave them in at least in the prototype: they can be desoldered for a test then and we can see. There is no such suggestion for the other terminations, I just thought they couldn't hurt. Same as above: remove them from the prototype to be sure.

The other three resistors are not termination at all: they are pullups for DONE, !IRQ and MISO. !IRQ must have a pullup during configuration (unless the MSP430 can do a weak drive?). Done can be left completely unconnected, in which case this pullup as well as R1 and R2 can go. But we may want to know if uploading the bitstream worked or not. And MISO is there to prevent an undefined logic level on that net before and during configuration.


It's a bit of a problem: the drill of 0.2mm and the annulus of 0.25mm that I was shooting for don't fit inside the BGA. I erroneously configured 0.2mm annulus, so everything seemed to be fine. When I found my mistake, I chose to reduce the annulus to the (very slightly) more expensive 0.1mm annulus. The price difference is marginal (<2EUR per board for one example). Different suggestions?

Since there are two regulators for VCCINT it's better if they are separate even if they are happily connected together. ( most dc-dc don't like this).
With separate planes you can have individual  feedback from each FPGA.

You  don't need the VCCINT planes to cover the whole board, just under the FPGA extending out the the caps surrounding it and a super fat trace to the DCDC switcher.

The GND layer can/should cover the entire PCB. If you have to split it up, make sure the unbroken areas covers the FPGA's, the decoupling  caps,  the powersupply and the bus lines to the connector. There should be no "obstacles" or detours in the path of the return current.

The decoupling caps under det FPGA should be moved so one pad is as close to the VCCINT via as possible, then add a new GND via
that fits close to the other pad of the cap. ( if there isn't a GND via with suitable distance there already )

The termination resistors at the receiving end are not needed and just causes driving issues and draws lots of current.
Use series resistor at the output pin/buffer instead. 100R should be fine for anything up to 25MHz.
The FPGA has selectable impedance/drive strength on all pins so no need for  series drive resistors for signals  driven by the FPGA.

Is it me or does the via pads look very small?   But if the pcb manufacturer doesn't complain, neither shall I

I like the GPL. The problem with using a software license for hardware is the concept of "source code" becomes ambiguous. If it is made out of molded plastic, am I required to supply a copy of the mold upon request?

For images, are the original vector files used for rendering needed? Do all of the rendering parameters need to be spelled out?

I think for the FPGA, releasing the verilog files is enough, even if the "compiler" is very proprietary.

For the board, releasing the eagle files is probably enough as well, despite being another proprietary program.

"Public Domain"  may cause legal problems in some countries. That is what the WTFPL was written for.


Edit: The GPL licenses have some anti-patent provisions. This may be a problem if some patented functionality is included in the cost of one of the components.

@O_Shovah: I asked three days ago about clarifying the terms under which what is created here becomes available. These are the suggestions that were made:

li_gangyi : distribution under the GNU GPL license
MountainMan : distribution under the MIT license, don't use GPL (Troll?: one post, no reply)
Olaf.Mandel : distribution under the GNU GPLv3+ license or possibly a different OSI approved license
O_Shovah : founding a company and distributing stake holdership or distribution under the GPL license
TheSeven : distribution under a FOSS license (MIT, BSD, GPL, LGPL, ...) or (reluctantly?) release into the public domain

Since then, the discussion has ceased. I guess people where agreeing by staying silent... Seems like distribution under one of the GPL variants or possibly a different OSI approved license (=FOSS license) are the most preferred solution. There were open questions with licensing under the GNU GPLv3+ (or a different variant):

• Can it be applied to hardware designs without problem? -> There seem to be some open hardware licenses (Wikipedia: TAPR, CERN-OHL and more. In contrast, OpenCores page suggests use of the GNU LGPL. The home-fabrication crowd (3D printers) are discussing this also: I haven't found an authoritative answer.
• What about contributions in this forum, in the form of ideas and management resources that do not lead to authorship of copyrighted documents? -> In my opinion, the person implementing the ideas into a file (design, whatever) has to decide if this is sufficient input to award the supplier of ideas shared authorship or not. This works well for normal publications, so why not here?

I myself don't see why a software license should not be usable for our purposes: the design files (Eagle, tables with BOM, ...) are the source-code files and the actual hardware built from these designs are compiled binaries. In that sense, we would be fine:

• there is no restriction whatsoever on what you can do with the hardware
• everyone who gets the hardware is entitled to the designs it is based on
• if someone uses our design, and modifies it (and gives out the hardware) they have keep giving our names as (co-)authors
• the stuff can still be sold (at a low margin of profit)

Please either discuss this question more energetically to get a conclusion or make a decision if you think there is a consensus. We cannot keep going on without these things clarified.

I uploaded a new version of the FPGA schematic, board and PDF+PNG to github and dropbox. Commit log:

Defined layers 2 and 15 as supply layers, removing them from the routing. This required merging VCCINT0 and VCCINT1 again. Additionally, it now was cheap to add a net NC for all unconnected pins of the FPGA.
    
All big components are on the mil 25 grid, just the small caps on the back of the FPGAs are on a local 0.5mm grid around their FPGA.
    
Unfolded the busses. Now they need more space but should be "safe". Moved around the termination resistors.

Unfortunately, after I finished I realised that I had an anulus that was slightly too small (0.2mm instead of 0.25mm). And that was the straw that broke the camels back... So I reduced the anulus to 0.1mm (the next pcbcart size).

Do people like this layout with the supply layers or should we go back to using polygons in layers 2 and 15?

I just rewired most of the MSP430 in two versions now.

I uploaded a MSP430f550x and a MSP430F552x version into the layout MSP430 folder so we may choose the one wich seems best fit.

Currently i would prefere using the 5528 version. It certainly is an overkill but wouldn't make for a big price difference ang gives us plenty of overhead recources.

I also added a PNG for thos without Eagle.

Im not here one evening and i get a whole new page to read

Maybe we should decide on a MSP we will use first. I case we take something bigger than the 550x series (wich seems very likely now) i will have to reroute it anyway.
As far as i know there is a free version of eagle avaidable. I doesnt have many features but should be sufficient for inspecting the files.
I case it doesn't work for our files i will upload a png of the new schematic. 

@ magik:
Also for you, just send me  pm with you email adress so i may invite you to the dropboxfolder .

It's for cases of having a network outage or your computer crashing, ...


A prototype board needs at least the XC6SLX75, because of the housing and because of the way you need to roll the miner up to fit inside. And the price difference to the 150 is small enough compared to the constant board costs (power supply, ...) that I would suggest going full hog on the first try. Maybe not both FPGAs, but at least the correct size.


The MCU is the USB chip, so there is no overhead there. The main ideas for not putting interface logic into the FPGA: the debugging is easier for non-HDL specialists, the FPGA can be brought up without needing a programmer cable (which hobbyists may not have!), the MCU can run on a different power rail than the FPGA and power up and down the FPGA, it can act as a brownout detector and temperature guard, there is no need for a dedicated USB chip when using a suitable MCU.


Hybrid design: reduces entry level investment because you don't need the backplane for your first DIMM and the cost for the extra MCU, Mini-USB and Molex connectors together is small compared to the DIMM cost.

This is not quite right: the bandwidth needed for mining is negligible. The bandwidth available through either of the two interfaces still in the designs (JTAG and SPI) is much higher. Assuming 2MHz clockrate on SPI (the default value in master mode), we need 17s to bootload the FPGA. As the FPGAs have the same bitstream, they can be loaded in parallel. For a higher clock speed we are faster, e.g. for 33MHz we need only 1s.

I suppose to quickly answer questions like yours, the first post of the thread should be kept up to date.

The back-plane is supposed to be optional. There will be a USB port on the modules that is inaccessible while plugged into the back-plane. This is to allow stand-alone operation.

AFIAK, the point of the back plane is to make managing several boards easier. I envision powering down the chips being useful in off-grid applications where power may be at a premium, depending on weather conditions. It may also be useful if you want "standby" hashing power to protect the network, but you would probably use (cheaper, more power hungry) general-purpose machines for that.

I was going to say my post was the first mention of a temperature sensor, but it actually wasn't. I don't think any temperature-sensing scheme is finalized yet. I just noticed the MCU has that feature when reading the datasheet (+- 20C).

- Post 184

- Post 235

- Post 351
(I don't think the preceding has been formally decided yet.)

- Post 374 Update: Post 416

I don't think anybody has volunteered to program the MCU yet. In theory, I could do it if I get my computer situation straightened out in the near future.

And that's a spartan 3E i'm talking about - I imagine the 6 series has a lot more cells that need to be programmed.

It's fairly easy to get an evaluation license for Xilinx - and I think it's even pretty easy to just re-generate more if you have more e-mail addresses....  Nobody in the industry really pays for Xilinx licenses - they just give them away when you buy chips typically - but the bad news is that it's unlikely they will even give you a license if you aren't purchasing semi-bulk.  I would guess it'd be kind of hard for hobbyists to nab a non-eval license.

So what exactly is the purpose of powering down FPGAs?  If this was my mining rig, I'd be running it at 100% at all times.  I see you mentioned temperature sensor - so maybe in places without adequate cooling or if a cooling system breaks you might want to turn things off... again I think I need to see a bit more of what you guys have designed so far.

The amount of FPGA space you will use for simple things like comm protocols is nearly negligible - especially when these modules are running at much reduced clock rates compared to the important stuff ( the hashing engine ).  Another thing to think about - see what other FPGAs in the family have the same foot-print.  A prototype board doesn't need to have the most expensive FPGA if a cheaper smaller FPGA can handle 1 hash engine, and this may help a little in the first round of prototyping.

I guess if the MCU is very cheap it's not that much of a hit to have one... but you are adding one more layer of complexity to the design. PC <==> USB chip <==> MCU <==> FPGA.  And the MCU itself will need to have code written for it...  I guess it matters what you really want the MCU to be able to accomplish.  Or what you think the MCU may be able to accomplish in the long run.

I think I need a bit more info on what the high level design architecture is going to be.  How does the backplane/motherboard fit into this if each DIMM module has it's own MCU and USB controller?  I would imagine ideally you would put these type of comm things onto the backplane/mobo itself - but I understand in terms of prototyping and development it'd be easier to have it on the module itself so you wouldn't need a backplane to debug/develop with it.

Also, I'd like to mention that I'd be willing to throw up a bit of cash ( couple hundred dollars ) when prototyping time comes.  And I am interested in helping with the project.  I have access to Xilinx ISE tools, and all kinds of hardware equipment ( logic analyzers, power supplies, o-scopes, sig gens, etc... ).  I am also a hobby programmer with experience in C/C++, Java, python, perl, PHP, etc... Trained as an EE in school, but focused more on computer engineering, hence why I'm now a firmware engineer.  But yeah I'd love to help develop this with you guys and I have some money I could pool together with you guys to get the first round running.  I don't do much of the hardware layout - but I can always ask my coworkers to take a look at things.

It's worse than that: we want to be able to bring down the FPGAs deliberately if they are drawing too much power. How much data would need to be moved across the bus every time we do this? So far the assumption is that bandwidth is negligable. Edit: 1.6MB can take minutes at standard serial speeds.

Another reason for using the MCU is that it saves space on the FPGA for hashing. The MCU can translate between USB and SPI for example.

Edit2: MCUs can also be reprogrammed without software costing $3000 to produce the bitstream. This makes things like minor protocol changes or calibrating the built-in temperature sensor easy.

At my work we use a lot of Spartan 3E 1600s and the bit files are somewhere around 1.6 megabytes....  No idea how large the bit files are for the 6 series....  USB could definitely handle it - but then you have to also think about the uC being able to handle it in a reasonable amount of time too.

From what I've seen typically for every FPGA you would have a SPI flash memory for holding the programming - and on boot, the FPGA's bootloader can be told to talk to the SPI flash chip and unload it's programming from there.

I was dabbling around with some code to field-upgrade the firmware in the flash and the Spartan 3E itself doesn't even have enough BRAM to hold the entire flash image on the FPGA itself - I'm going to have to go down the road of possibly having two flash chips, one with some "bootloader" code and the other with the actual running program - and to field upgrade the firmware, I would have code that would be able to access the 2nd flash to write directly to it.

But I guess if you program the chips through the USB comm interface that's 1 less part you'll need ( flash memory ), but it also means much more complicated uC code to be able deal with programming the FPGAs.  And it also means that if the device loses power, that it would need to be reprogrammed again - and the software communicating through USB would need to be able to catch this, or be restarted.  I guess it could work, but in my mind it just sounds kind of hacky.

I'd like to try to help with this project, as I'm a firmware engineer ( FPGA programmer - Xilinx ) by trade, and I'd love to dabble in the FPGA bitcoin mining world.  If you guys want I could run some of your schematic designs by our mechanical engineer that does our layouts, see if you guys are missing anything or anything else that jumps out.  It seems a bit early for the FPGA work, and that stuff already seems a bit worked out by others ( Open source FPGA project, etc... ).  But I'd love to help in any way I can - this project just sounds magical, and I don't think I've ever really seen an open source FPGA/hardware project of this magnitude before ever.

I find it a bit hard to figure out where you guys are right now without access to this shared dropbox folder and without reading this entire thread.  But what's the current status of everything right now?  Where do you guys need help/work right now?  I am also a bit of a software/hardware engineer as well and might be able to help out in some of those areas as well?

In regards to the uC talk above - if all the uC is going to do is route work from the USB comm path to the FPGAs, and the reverse - I'm not sure why a powerful uC is even needed.  You could probably get away with doing away with the uC completely and doing the USB chip interface on the FPGA?  Maybe something like the "master" platter controls the USB comms and then communicates to the other FPGAs via bus.

I probably need to brush up a bit on what exactly you guys have planned for the design, how you plan on implementing things from a higher level perspective.

But I'd love to help in any way I can.