Topic: Use Beagleboard/Pandaboard to run FGPA mining rig - page 2. (Read 14418 times)

Exactly. All these fools thinking they will go for ARM or MIPS or another BS architecture like that will hit a brick wall.

Fact of the matter is that x86 is KING and most supported. No x86, no dice.

Stick to i686 or amd64 if you don't want to be dreaming about cross compiling in your sleep

Now I run 5 via USB, but its only 1 java process that controls the cluster. CPU is 0% for that process... It would be close to 0% for the 128 I could run with the limitations of USB.

Just try to use your ARM linux as a workstation with java, eclipse, chrome WITH flash support and spotify, skype, wine etc. If you wan't to do anything else in your life you won't be able to.

How many FPGAs do you run?
How much is theCPU load?

I'm going to have to get an FPGA and see if I can plug it into my chumby.

That's wrong. If you buy a Smarttop from Genesi there is a software support community at http://www.powerdeveloper.org/. Of course, if you're not interested in playing around with a new architecture, which will have issues that are not present on firmly established archs like x86, it's probably not for you.

And come on dude, do you really know all vendors of Linux ARM hardware well enough to say that none of them offer software support? It's a bold statement to make at least, and pretty much impossible to verify.

No, the processor draws 13W for the one I have and 10W for the next generation. If you compare that to the ztex FPGA 8W it's really ok.

The peripherals will draw on any other card too.

And the linux you get on any non x86 architecture has NO software support.

The choice is easy!

Here's a few more images to check out.

Schematic of MasterPIC.
Schematic of Flash support for MasterPIC.
PCB Top Layer MasterPIC.
PCB Bottom Layer MasterPIC.
(I may still replace Xtal with Oscillator pkg on Net2Blox. I like the small size but it costs about 50 cents more.)

I've created them as little I2C/SPI modules allowing them to be used for other things. I tentatively call these modules "2Blox" as I may market them later.

Net2Blox and Flash2Blox. The Flash2Blox allows using either W25Q64 64Mbit flash devices or alternately a microSD card. I haven't decided my choice yet but designed board to work with either. If the microSD can't output at 16Mbps then I'd choose the chip instead. I don't want it to take too long to load FPGA CFG. At 16Mbps it's approx 2 seconds to do the whole cluster.

The Flash2Blox has a gated oscillator so the Net2Blox (MasterPIC) can set it up, enable the clock, and let the CFG fly out to the slaves from the Flash - very fast.

These little 2Blox boards stack on top of each other. So the Flash sits just on top of the Net one. And the Net one plugs into the first FPGA slave.

This is very much a work-in-progress.

watching

The program for the Ethernet PIC (MasterPIC below) would have a web interface like your router. You can login and edit the pool values (no reason it can't have a few for failover). Also, later should be able to view some stats/graphics, though haven't thought much about that yet. It would have an upload button for updating firmware for both itself and each FPGA, and PIC too. Both these model PICs support self-writing to their EEPROM allowing online updates. Only the initial programming needs to be done via programmer to install bootloader. That's why little space is given to ISP connectors.

Each FPGA-PIC has simple code to inspect I2C data, queue work data for FPGA, or relay on to slaves. During power up it does a process of device enumeration to set addresses for whatever devices are connected. Then it disconnects itself and passes thru CFG data until "Done" on each FPGA tells it to reconnect as I2C relay. This code has to fit in 2K so it's assembler and minimal. Since Spartan 6s are happy with hotplugging I don't see a reason why devices couldn't be added/removed while running, with minimal disruption, as long as my addressing code adapts. It would have to rescan every few seconds maybe.

Here is a PNG of the overall idea of how you would connect a cluster of these units. Note that each FPGA board has a Molex MicroFit connector (small!) for power. The I2C connectors don't provide power to slaves (each has it's own cable to PSU). But the master connector (top one, red) has power on it to provide power to the EthernetPIC.

BkkCoins, epically awesome. So does the program that gets flashed to the PIC also contain stuff like what pool to mine on, and IP address, etc? Or am I misunderstanding how it will be used?

I did consider POE but it was too much hassle. And since there is power from the FPGA board I chose the economy approach. The first FPGA board provides 3.3V power to the Ethernet PIC via it's I2C connector. Simple.

I did the design in Eagle on Linux. I don't mind sharing the design files but since they use a custom library I'm not sure it's the easiest for a quick view. So I've exported a rather large PNG of the schematic, and then converted a couple Gerber views to PNG for easy viewing.

Some Notes:

This is based on an Icarus compatible FPGA design. I tried pretty hard to synthesize and place my own version of the Verilog so I could optimize everything into the FPGA (no on board MPU) but kept on having failures (maybe due to some Ubuntu issues or whatnot). Finally I decided to just adapt to the serial input on the Icarus design. The Ztex uses the CSBGA and I can't see myself working with 0.65 pitch layout on a 2 sided board. That's even more crazy than just attempting a 2 sided board in the first place.

Power is from a good quality PSU. It uses 12V, 5V, and 3.3V input. 5V is used as bias for FAN2108 DC converter, 12V -> 1.2V, and 3.3V is used directly for 1 FPGA bank and dropped to 2.5V for VCCAUX and other banks (as required by Icarus design).

Each (5cm x 5cm) FPGA board can act as master for up to 3 slaves. They tile together like square dominoes, up to 127 boards (limited by I2C address, seems like enough to me). There is a PIC16LF1503 on board that queues work/results and acts as multiplexer for CFG init and work data. It co-ordinates data from the Ethernet PIC and passes it off to slaves. It talks via bitbanged serial to FPGA. This chip was chosen as it has a CLC (configurable logic block) inside allowing routing high speed CFG data via itself without external parts. It talks to master and slaves using I2C.

There is fan control, temperature monitoring on each board. It uses the PWM of the PIC and internal temp sensor. Not the most accurate but likely good enough for this.

The Ethernet PIC is master of the whole cluster. It gets work, feeds it to the first FPGA slave, and requests work back. It has a 64Mb Flash chip on board to store FPGA CFG. I'm building these parts as modular units so that I can use them for other purposes.

So, here is PNG version of schematic and PNG version of PCB layouts with screenprint overlay.
This is just the FPGA board now. I can post the EthernetPIC/Flash board soon.

Miner Schematic
Top Layer
Bottom Layer

I'll draw up a quick overview diagram so it makes more sense how these connect together in a cluster. I'm going to send the Gerbers off for making 10 boards once I'm more confident this is a final layout. As you can see this is my 4th revision (miner4)!

^ I must admit that I'm satisfied with the 5W that the Smarttop consumes. Personally, I'd be worried about the upgradability of custom hardware. What if I want to do p2pool mining on BkkCoins' hardware, where I have to verify transactions and store the block chain?
Also, Smarttops are ready to ship now, with working software. It seems BkkCoins hasn't even had hardware built yet. But if OP is not in a hurry, and he wants to save a couple of watts, it might be worth waiting for, cause we won't get lower power than a custom design like that.

Well, BkkCoins' post up at the top of this page makes x86 (and ARM) look like power hogs by comparison. I would like to know how well his design can scale, and how much power it uses.

^ Installing the OS is as simple as downloading an image and booting it. The Smarttop is $129 (€129 in Europe with free shipping). But I admit to it being somewhat of an experiment. It's possible that you will hit some sort of bottleneck in the system if you're attaching 10 FPGAs to it doing 5 GH/s in total. I know memory bandwidth in particular is pretty poor. But if I were to build a mining rig consisting of only a single or a few FPGAs, I'd definitely go for this. It uses 5-10W, is cheap as hell and stable. Also, I think ARM is cool .

If you are going this in-depth and installing a whole OS, I prefer VIA's Pico-ITX formfactor, in particular the ARTiGO A1150 (Dual core VIA Eden X2, x86_64 compatible, max 4 GB RAM). I've got the single core version (The A1100), and it is damn sweet - just be sure to install the video drivers, otherwise the thing is slower than molasses due to video being processed on the CPU instead of the chipset.

It supports expansion boards, and includes I/O headers for tons of things including serial ports, plenty of USB ports, standalone LCD displays, and tons more. Kind of expensive, but compact and reliable.

I've heard reports of the Pandaboard being unstable. I've heard better things of the Efika MX Smarttop. One user, I remember, reported an uptime of 30 days, which would have been more if he didn't have to reboot the machine for a kernel upgrade. This seems to me to be of chief importance in a mining rig configuration. The Smarttop only sports an 800 MHz Cortex-A8 though. For pooled mining that should be more than sufficient to drive several FPGAs, as far as I know. If you're talking p2pool, where you have to do transaction verification, initial block chain download might be slow (though we can just move the block chian over from another computer), but at the current transaction rate I think it should be able to keep up with incoming transactions. It has an onboard cryptographic HW processor too, but I'm not sure if this does ECC-DSA.
It comes with Ubuntu 10.10 installed by default, but they have a Debian hardfloat image up on their support/community site (http://www.powerdeveloper.org/).

I would love to see your design on this - have you considered making it PoE powered?

My GA-970-D3 with Athlon draws < 25W. My Jetway mini-ITX about 20W.

Using a Raspberry-Pi is more for the novelty of it and just finding a use for one when they finally are available.

To really save power... here's my current project: a PIC18F86J65 (cost $3), a RJ45 with MAGS ($1), a 25MHz crystal ($.50) and a few passive parts. The only real power draw here is driving the ethernet signals. And a simple PIC miner program that relays work to the FPGA cluster of up to 128 devices.

PCBs done, just waiting to send to fab. Overhead cost per FPGA < $21 qty 1, <$16 qty 25.

@ArtForz - I'd be interested in comments on PCB/Design if you were willing to look. I can post PNGs here.