Topic: Modular FPGA Miner Hardware Design Development - page 22. (Read 119276 times)

AVRs are definitely the strong version though; they have hardware write-protect for the bootloader that as I understand it can't be overridden without connecting up a programming cable, and neither can anything else that would prevent you from booting from said bootloader. At least that's the theory...

Ah, sorry. Well, Cyclone IV and earlier can, as can Spartan-6 obviously. Kintex-7 and Cyclone V will apparently also support 3.3V IO on at least some pins, though I'm not sure if they do for the configuration pins...

In a weaker version of that sense, the MSP430 is also "unbrickable": as long as you don't touch the default BSL and as long as your code leaves the write-protect alone, everything is fine. The problem is that an inexperienced user (or someone who types command-lines on autopilot) may easily reflash the BSL.


Actually, you shifted what I wanted to say where a bit: I meant that while the current FPGA can do 3.3V, I am not sure how many others can do so.

And having to use level shifters is very annoying. I just looked, though: the MSP430F5528 (and comparable chips) has several banks of GPIOs, but there is only a single supply voltage for IOs. And if you use 2.5V for the FPGA connections, you cannot use 3.3V for the connections to the DIMM bus (except for the USB, of course).

So long as you stick to only reprogramming these USB AVRs via the factory-preprogrammed USB bootloader, I don't think you can brick them. Which is fortunate, because recovering from a disabled RESET pin is apparently a pain; it requires a high-voltage parallel programmer that has to connect to pretty much all the pins and apply 12V to the RESET pin, which makes doing it in-circuit just a tad tricky.

The shipped bootloader is a bit annoying in that you have to manually hold a particular pin low during reset to enter it, but that's no big deal unless remote firmware updates are a requirement.

Yeah, it just makes doing I/O interfacing a tad more tricky. As Olaf said, most of the FPGAs we're likely to use can run at 3.3V for I/O anyway, and if necessary level shifters are an option. (Looking at the Kintex-7 data sheets those support 3.3V on a subset of the I/O pins too.)

Bricked would mean either completely unrecoverable or not recoverable without desoldering from the PCB. My understanding was that, like you said, the fuses can accidentally be set to lock out the RESET pin. But I thought this is completely unrecoverable, even by a high-voltage programmer. Or isn't it? The curious thing is that the different Atmel chips have different programming strategies, and I only read about the one for the ATtiny. For a USB capable MCU: no clue.


Important are the IOs that go to the FPGA. While the Spartan 6 can be operated at VCCO = 3.3V, this may not be the preferred voltage. And other FPGAs for a second generation of DIMMs may not support 3.3V at all. Basically, it's not a show stopper, not even a very huge demerit at the moment, but it is something to keep in mind when comparing otherwise nearly identical MCUs.

In my mind, if you have an MCU on the board, you could reflash the MCU to adapt to a future motherboard design. This gives maximum flexibility. We should route as many IOs to the DIMM connector as possible, with at least a basic idea of what each one will serve as, but they can be reassigned (or disabled) by downloading new firmware for the MCU on the daughterboard. If we do this, we would need to make sure that reflashing can be easily done without extra hardware by including a USB bootloader or something similar.


What exactly would you define as bricked? If you had a USB bootloader and accidentally overwrote the firmware, you would need to reload the bootloader with a programmer (or have the motherboard do this). The only "bricking" I can think of with the AVRs is if you accidentally lock the flash memory or disable the RESET pin (needed to load new firmware). These can still be recovered from by using a special programmer.


Is 2.5V operation a requirement? Can USB run at 2.5V? I don't think so.

I am still not convinced that putting extra protocols on the DIMM bus cannot hurt: in the best case, we are locking future DIMMs and backplanes into a more complex interface than it needs to be. How about this: we define several interfaces, e.g. the ones we were talking about all the time (JTAG, I²C, SPI, USB) and we make all of them optional. The DIMM can inform the backplane via a couple of low impedance connections which protocols it supports and which are not connected. That makes the backplane a (very little) more complicated because you need to read out the signals from these key-pins. But it keeps the DIMM design completely open: a later revision could really employ "head-less" DIMMs that don't work stand alone. Or use an FT2232H design that is incapable of tri-stating the JTAG and SPI pins for use by the backplane.

Of course, this way all DIMMs will work on a backplane that supports all protocols. But if the backplane designer starts cutting corners, then some DIMMs won't work in that backplane. There are three solutions:

• Make all protocols mandatory (current situation). Safest option, but precludes making the boards simpler at a later stage.
• Define one of the protocols as compulsory (my preference: USB) and leave the others optional. All DIMMs and all backplanes must support the compulsory protocol and can fall back to that if no other protocol is available.
• If you have one compulsory protocol that must be supported, why do you need the others? So only support that one protocol. Very lean bus design, but forces all future DIMMs to have adapter logic to translate between the bus protocol and what is used on the DIMM.

It's basically a philosophy question: many choices but (mostly unneeded) overhead to provide them on one hand and lean design but less flexibility on the other hand ("less", not "no"! You just cannot remove the adapter logic). A bit of an IBM/PC clone vs. Mac question   


Haven't found one, yet. I was suggested the Analog Devices ADUC series as unbrickable, but that one does not support USB.


My understanding was that it is possible to brick the Atmels. Can you confirm (or better: rebut) this?

The only real problem I can see with the AT90USB82 is that it doesn't support 2.5V operation.

I agree with TheSeven, too. As for MCU, it would be wise to use one that includes USB functionality already. The MSP430F5528IRGCT suggested by Olaf.Mandel seems good, but I don't know the MSP chips well. I have more experience with Atmel AVRs and I would prefer the AT90USB82. That's only a personal preference, so don't give too much weight to that vote, especially if someone else will be writing the firmware.

Sorry, that's not really my area. But given the vast number of MCUs out there, there just must be some perfectly fitting one.
Anyone who knows that market better will hopefully be able to dig that one up

I second TheSeven's statement

@ TheSeven : Do you have a more detailed idea of wich MCU's to use and wich effective setup and wiring ?

Yes, we should keep the early tasks as simple as possible. But we shouldn't sacrifice too much flexibility for that.
That's why we want to build hybrid cards, so that we can first focus on the cards and run them standalone, but at a later point are able to make a nice backplane for them.
The difference between standalone or hybrid cards is really just a DIMM connector and a couple PCB traces. Possibly a better USB bridge or MCU. Yes, it's a <$10 difference on a >$200 board. Do you really want to sacrifice the possibility of building a backplane for it later for these few bucks?

And regarding that backplane-only card idea, with a trivial one-slot FTDI backplane, I have two concerns:

• While this reduces cost for the board by a bit, it adds cost for a second board and a DIMM connector.
• It makes things less flexible. USB on the DIMM would allow future cards to change internal protocols as neccessary.

Nevertheless I would still route the SPI and JTAG interfaces to the DIMM as well. It just can't hurt.
If we're going for an MCU instead of an FTDI anyway, tristating won't be an issue. Just provide a dedicated tristating pin on the DIMM, which has a pulldown on the card, and gets pulled high on the backplane. If the MCU has the neccessary resources, adding an additional SPI link between the backplane and the MCU can't hurt as well. Oh, and having the MCU's JTAG pins on the DIMM can't hurt either. That way you can do MCU firmware upgrades from the backplane. And yes, we should go for an unbrickable MCU.

True, but if you put an FTDI USB chip on every daughterboard (is this what you propose?), how do you connect it to the FPGA? Here you still need I2C, SPI or JTAG. Then you can also put it directly on the backplane and connect the daughterboards only using i.e. I2C. USB on the daughterboard will probably also require more careful routing and special impedance matched connectors because it is a high speed bus.

I was sort of hoping that if the motherboard is compelling enough, people would build ASIC boards that are pin-compatible

Edit: the FPGA prototypes would be a "proof of concept".

I couldn't agree more. This is why I don't see the advantage of doing all the work to implement a bus protocol, when USB functionality costs nothing more than an FTDI chip ($3-4). If we're discussing overall strategy here, I think the priority should be on developing the FPGA daughter board. Right now, we're spending this time trying to design the perfect backplane, that will have sorts of cool features and allow for all different types of daughterboards. That's a very difficult task. It's very easy to get bogged down in this design and delay the overall development.

On the other hand, as O_Shovah said, we really shouldn't go back now and rediscuss the issues that have already been decided. It would be better to just try to move ahead with the design.

Is anyone working on the daughterboard schematic and or layout? We could probably get started now and add the bus protocol to it later.

I think you all should try to keep it simple. That increases chances to produce something that is working in time and without multiple iterations for PCB production and assembly (money!). The later this thing is ready probably the higher is mining difficulty, this in the end also costs money.

As far as I can tell USB is a much more complex protocol than SPI or I2C. Thats why FTDI can sell chips. Implementing USB inside the FPGA costs LE cells that could otherwise be used for mining. In addition there is a host needed to connect multiple slaves. For mining no bandwidth is needed. SPI, I2C, JTAG all are capable of pushing enough bits for mining.

My proposal:

• make the motherboard simple, put only the FPGA + decoupling + one of (SPI, I2C, JTAG), put an direct JTAG connector on the board for debugging the FPGA.
  
• let the motherboard supply one of (SPI, I2C, JTAG) + programmable power supply + reference clock. Power needs cooling and you normally don't want to have this on the daughterboard. Connect the motherboard to a standard PC power supply.
  
• The simplest motherboard would then just put an FTDI USB connector on the motherboard and you are ready to go. Easy debugging. Anybody who feels up to the task can then replace it in a second step with a microcontroller and ethernet.
  

We build a similar board (not for mining) at work and you will find out that it is a pretty complex task. You will succeed much faster if you try to keep complexity as low as possible.

And just as a reminder: to my knowledge there is at least one startup working on an ASIC, announced to be ready in October. If this is true, an FPGA board only suitable for mining will be worthless.

Yeah, that could be... fun.

O_Shovah, I'm sorry to derail the thread. I think Salzgitter had a great idea:


If we wanted to add ethernet to the motherboard later, this would mean adding a MCU that could serve as a USB host.

USB has my vote.

No offence to anyone,but to make this clear:

I don't tihnk we should change or rediscuss decicions allready made or we will end up running in circels.


The Daughterboards are intended to be hybrid capable wich means they will be abled to operate standalone via USB or as slave in the motherboard.

So there are some points

- Minimising the design of the daughterboard would require a complete redesign of the daughterboard if we add a cpu and ethernet on a later motherboard version later.

- As the daughterboard is stand alone capable you dont need to buy the motherboard if you don't want it.

- According to my rough calculation each daughterboard should end up at a price of 300 Euro,so a additional price of maximum 140 Euro should be bearable if you buy more than one daughterboard.


I Would really appreciate it if we could return to ther design discussion regarding the BUS system to be used.

You hit the nail on the head here: the cost of this project will be inversely proportional to the number of orders we can get. The cost of PCB production and assembly, as well as the components, is very strongly affected by economies of scale. Anything we can do to increase the number of orders will have a huge effect on reducing the price. I believe that reducing the cost to get started in FPGA mining will encourage many more people to get into it. If the daughterboards can't run without the motherboards, the motherboard cost will deter many people, therefore bringing up the cost.

Of course, the real question is how much interest is there? How many people will pledge to buy? How many will provide the money up front? I for one am willing to invest the amount of money for a 10 FPGA system, possibly even more so that the cost per FPGA is lower. Maybe something like Kickstarter will help to gauge this?

Going from N hybrid DIMMs plus one backplane to N standalone cards, the savings are:

• Nx DIMM connector: < N x 3EUR
• 1x Extra connectors and chips: <20EUR
• 1x Board cost: ~30EUR
• 1x Population cost: ~50EUR

So the board solution would cost approx 120EUR more, compared to a total cost of 2xNx(114EUR+50EUR???), e.g. 1640EUR. Actually, it's a bit better because I neglected the cost of the USB hub and the many cables for a stand-alone solution. So approx 7% of the total is due to the board. That is probably acceptable for more flexibility.

Going from N single-purpose DIMMs plus one backplane to N standalone cards: Same as above, but you save a bit more for the DIMM+backplane solution, because you don't need the USB interface on each board. That may even add up to cover maybe half the cost of the backplane.

For everything I wrote above: it's difficult to make precise calls on the money aspects without having designed the system and costed the manufacturing costs (the numbers I gave above may even be on the low side, if we don't reach small series count: prototypes cost more).