News from the waterfront...
As promised i created a watercoolingblock for the x6500.
Some pixels
I am interested!As promised i created a watercoolingblock for the x6500.
Some pixels
Topic: X6500 Custom FPGA Miner - page 13. (Read 219829 times)
News from the waterfront...
As promised i created a watercoolingblock for the x6500.
Some pixels
I am interested! As promised i created a watercoolingblock for the x6500.
Some pixels
OK, I'm not using X6500 boards, and my array is half the size of yours (25x Ztex 1.15x boards, plus my initial 1.15d dev board - forgetting the 9 Ghash of GPUs, costing me a fortune and to be switched off soon...) - but I'm having aggro with my systems recognising all the USB devices on the bus.
Theoretically, I should be able to connect 127 of the Ztex single boards to one computer, on one USB bus.
However, this just isn't happening... and that is fucked up, fucked up..
(sorry, listening to Thom Yorke)
There's definitely nothing wrong with the Ztex boards - they're performing brilliantly in idiotically insane environments - ambient air 38˚C, no typo, cheap £3 4-port daisy-chained Amazon USB2 hubs, and MUCH WORSE than I imagined... 5-to-1 2.1/5.5mm barrel plug daisy-chain cable rated at 60W but gets very hot to touch under load, and on top of this - unsupported miner platform... I run Mac OS X and recompiled Stefan's entire SDK and bitminer-src for Mac OS X Snow Leopard on both 32-bit (hackintosh - Dell Mini v10 - double unsupported), and 64-bit (proper full-blown Mac Pro workstation loaded to the gills)...
It appears that you have a similarly sized USB hub / chain problem that I do. I need to connect 25 USB cables to one netbook running Mac OS X.
Stefan explained why my cables were hot - and I now have a solution for that problem.
But the issue remains - are these cheapo Amazon 4-port USB2 hubs simply a disaster waiting to happen? What have you used for all the USB connections you need? Or have you used multiple logic boards? One option for me would be to simply plug the USB cables into my modular GPU miners, which have one logic board (with 6 USB sockets, normally) and 4 GPUs. This should work, but I don't want to be running 5 high-power ATX PSUs when I'm intent on selling off the 6950 / 5850 / 5830 / 5770 GPUs (no use to me as a Mac user, other than the 5770s, which I'll keep to make a 4-GPU Hackintosh tower with 16 GB ram and the i5-2500 CPU clocked to the moon, running Snow Leopard of course).
It's overkill to use separate logic boards and CPUs (wasted power). But high quality 10x USB2 hubs cost a lot of £££ and, if they require their own power bricks, then it defeats the object of consolidating points of failure. A single cheap USB2 hub could easily short out (I've taken photos of the internals of these things, and it's embarrasing - I am awful at soldering, but I could do a better job than these chaps).
Also, part of the project goals include the ability to make the entire structure portable, with a single power input and a single Ethernet output. It also has to look good in Jonathan Ive fashion (i.e. the downside of being an Apple hacker and concomitant need for good industrial design...)
Stefan (Ztex) has given me a good wiring diagram to power the FPGAs. But I'm still left with all 25 of the boards needing USB connections. Since USB2 spec allows for 4-port and 7-port hubs (IIRC), any other number (such as the 10-port hubs) is actually a 4-port and 7-port chained together internally.
This is where quality comes in. I've got cheap hubs... but are the crappiest of hubs 'adequate and functional' if the power supply to the FPGAs is clean, low resistance and properly soldered (i.e. no hot spots)?
After all, I'm looking at 12VDC at 5A - and 5 amps is quite a current for a cheap USB2 hub with ethernet-thin internal cabling...
Any top tips? If simply sorting the PSU to present a low resistance to the FPGAs will allow the cheapest of crap USB hubs, then I'll stick with what I've got... but if quality USB hubs are required... I'd appreciate any recommendations for the USB daisy-chain stage. After all, if I ever get to 127 FPGAs on one bus, the USB hubs need to work. I've had problems with even only connecting *5* to a single port...
Theoretically, I should be able to connect 127 of the Ztex single boards to one computer, on one USB bus.
However, this just isn't happening... and that is fucked up, fucked up..
(sorry, listening to Thom Yorke)
There's definitely nothing wrong with the Ztex boards - they're performing brilliantly in idiotically insane environments - ambient air 38˚C, no typo, cheap £3 4-port daisy-chained Amazon USB2 hubs, and MUCH WORSE than I imagined... 5-to-1 2.1/5.5mm barrel plug daisy-chain cable rated at 60W but gets very hot to touch under load, and on top of this - unsupported miner platform... I run Mac OS X and recompiled Stefan's entire SDK and bitminer-src for Mac OS X Snow Leopard on both 32-bit (hackintosh - Dell Mini v10 - double unsupported), and 64-bit (proper full-blown Mac Pro workstation loaded to the gills)...
It appears that you have a similarly sized USB hub / chain problem that I do. I need to connect 25 USB cables to one netbook running Mac OS X.
Stefan explained why my cables were hot - and I now have a solution for that problem.
But the issue remains - are these cheapo Amazon 4-port USB2 hubs simply a disaster waiting to happen? What have you used for all the USB connections you need? Or have you used multiple logic boards? One option for me would be to simply plug the USB cables into my modular GPU miners, which have one logic board (with 6 USB sockets, normally) and 4 GPUs. This should work, but I don't want to be running 5 high-power ATX PSUs when I'm intent on selling off the 6950 / 5850 / 5830 / 5770 GPUs (no use to me as a Mac user, other than the 5770s, which I'll keep to make a 4-GPU Hackintosh tower with 16 GB ram and the i5-2500 CPU clocked to the moon, running Snow Leopard of course).
It's overkill to use separate logic boards and CPUs (wasted power). But high quality 10x USB2 hubs cost a lot of £££ and, if they require their own power bricks, then it defeats the object of consolidating points of failure. A single cheap USB2 hub could easily short out (I've taken photos of the internals of these things, and it's embarrasing - I am awful at soldering, but I could do a better job than these chaps).
Also, part of the project goals include the ability to make the entire structure portable, with a single power input and a single Ethernet output. It also has to look good in Jonathan Ive fashion (i.e. the downside of being an Apple hacker and concomitant need for good industrial design...)
Stefan (Ztex) has given me a good wiring diagram to power the FPGAs. But I'm still left with all 25 of the boards needing USB connections. Since USB2 spec allows for 4-port and 7-port hubs (IIRC), any other number (such as the 10-port hubs) is actually a 4-port and 7-port chained together internally.
This is where quality comes in. I've got cheap hubs... but are the crappiest of hubs 'adequate and functional' if the power supply to the FPGAs is clean, low resistance and properly soldered (i.e. no hot spots)?
After all, I'm looking at 12VDC at 5A - and 5 amps is quite a current for a cheap USB2 hub with ethernet-thin internal cabling...
Any top tips? If simply sorting the PSU to present a low resistance to the FPGAs will allow the cheapest of crap USB hubs, then I'll stick with what I've got... but if quality USB hubs are required... I'd appreciate any recommendations for the USB daisy-chain stage. After all, if I ever get to 127 FPGAs on one bus, the USB hubs need to work. I've had problems with even only connecting *5* to a single port...
News from the waterfront...
As promised i created a watercoolingblock for the x6500.
Some pixels
As promised i created a watercoolingblock for the x6500.
Some pixels
That is a ridiculously awesome system. Although, it seems like an awe fully expensive way to get to that much power.
In the long run, its extremely cheap!
Nobody wants to think about it but ... what happens when / if BTC fails "in that long run"
Loving to see you try sell those FPGAs
For most of the guys with the funds to buy these kinds of setups it's a non issue. Atleast unless there was some reason to believe BTC would fall. The alternaitves are to send another couple grand to ur broker, but for what? None of them are performing for crap right now and are just as likely to lose your money too. Atleast with this, you hold it and control it.
That is a ridiculously awesome system. Although, it seems like an awe fully expensive way to get to that much power.
In the long run, its extremely cheap!
Nobody wants to think about it but ... what happens when / if BTC fails "in that long run"
Loving to see you try sell those FPGAs
That is a ridiculously awesome system. Although, it seems like an awe fully expensive way to get to that much power.
In the long run, its extremely cheap!
Here's my 10GH/s rig I just set up today using 25 x6500 FPGA boards:
A nice setup you have there.
very clean ! 
That is a ridiculously awesome system. Although, it seems like an awe fully expensive way to get to that much power.
Cool! Now the Cognitive setup has a big brother
Here's my 10GH/s rig I just set up today using 25 x6500 FPGA boards:
Think I just snapped up the last X6500 in this batch. It wouldn't let me buy two, and then once I checked out it wouldn't let me buy another single.
I have 2 used ones for sale if anyone is interested: https://bitcointalksearch.org/topic/wts-two-rev2-x6500-sold-79602
Think I just snapped up the last X6500 in this batch. It wouldn't let me buy two, and then once I checked out it wouldn't let me buy another single.
Overall your setup looks good and I don't think you need to be too worried about the invalids but I'm going to leave the electromigration question up to someone from the fpgaminging team. (BTW I plan on running my units for a full 24/7 /365 days a year if I can)
I really don't know about this, and I don't think anyone else really has data on this since mining with the Spartan 6 hasn't been going on very long. I do recommend running the "overclocker" bitstream even if it means losing 5-10% of hashrate because it will be safer. MPBM will automatically reduce the clock if the FPGA starts to heat up (e.g., from a stopped fan) or invalids get over 1%. This peace of mind is worth losing a few BTC for me, in my opinion.
@Fizzisit:
What brand fan is that?
What brand fan is that?
It happens to be this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16835226039
There's also a little bit slower, quieter, and cheaper one that might be worth trying: http://www.newegg.com/Product/Product.aspx?Item=N82E16835226038
Or, even cheaper, but faster and louder: http://www.newegg.com/Product/Product.aspx?Item=N82E16835150103
Not enough air? How about this?
http://www.newegg.com/Product/Product.aspx?Item=N82E16811999613 What is the longest you have had it mining without interruption using the 200mhz bitstream?
I'm using the same bitstream and it usually runs with no invalids for 2-3 days but eventually they show up. I've had one board running for 5 days with no invalids and then on the 6th day each core got 3 invalids within a few hours. The temps were measuring in the same range as before (25-30C) so I assume it's just near the limit at 200mhz.
I'd like to the boards to run reliably for a long time and I'm not sure if a handful of invalids every few days is worth worrying about. I may just run the latest bitstream at 190-195 mhz for a tiny margin of safety as I don't consider an extra 10-20mhash to be worth running on the edge. Are my concerns warranted? Any opinions on the risk of electromigration from running them at this speed over a period of months/years?
I'm using the same bitstream and it usually runs with no invalids for 2-3 days but eventually they show up. I've had one board running for 5 days with no invalids and then on the 6th day each core got 3 invalids within a few hours. The temps were measuring in the same range as before (25-30C) so I assume it's just near the limit at 200mhz.
I'd like to the boards to run reliably for a long time and I'm not sure if a handful of invalids every few days is worth worrying about. I may just run the latest bitstream at 190-195 mhz for a tiny margin of safety as I don't consider an extra 10-20mhash to be worth running on the edge. Are my concerns warranted? Any opinions on the risk of electromigration from running them at this speed over a period of months/years?
@Coretechs:
Longest I've had the Rev 3 running is about 12 days straight (24/7) and no invalids so far (*fingers crossed*). Having said that, are you using the Rev 2 or Rev 3 X6500? I found that my Rev 2 does produce invalids after 1 - 2 days but I'm only seeing 0.5% invalids after having it run for that amount of time. I wouldn't be too worried about the invalids as they are pretty low in number but I would recommend looking at a couple other things:
1.) Power - The Wall wort from cablesaurus is good but I think a decent PSU would be a good replacement (either use the molex for power or splice a barrel connector). The other thing you may want to consider if you haven't already done this is get a UPS to condition the power for your units / Host PC.
2.) Cooling: My X6500 Rev 3 runs at about 30-31C, so you should be good in terms of cooling. The only thing you may want to play with is how close the fan(s) are to your heatsinks. I know the Delta fans has some pretty strong magnets in them that could possibly affect things so that's something to consider.
Overall your setup looks good and I don't think you need to be too worried about the invalids but I'm going to leave the electromigration question up to someone from the fpgaminging team. (BTW I plan on running my units for a full 24/7 /365 days a year if I can)
@Fizzisit:
What brand fan is that?
Finally posting pics / notes of the X6500 Rev 3.0 after what seems like FOREVER after recieving it (like 4 weeks!):
First off I wanted to say: Fizzisist, you are the man! He was super helpful with all my troubles and worked hard to get me my Rev 3.0!
Having said that; here are some pics!
The Naked Rev 3.0 (*fap fap fap*)
With one of the Zalman "Flower" North bridge coolers on it (apologies for the brightness):
The final product with both Zalmans attached and hashing away (Still trying to figure out the best configuration for a final enclosed setup):
Notes on the Rev 3.0:
1.) I've had this thing running for about 4 weeks now with a variety of weird / good / bad fan setups (80mm / 120mm / 2 x 80 mm / 2 x 120mm) and through out all my configurations I've yet to encounter a single invalid share with the 200 mhz ztexmerge bitstream! The push-pin mounts were definitely a great addition to the Rev 3.0!
2.) The "Flower" heatsinks are good but have some pretty big cons to them: A.) They extend over some of the mounting holes and the 3-pin fan power ports. B.) Two fans are needed to maintain a consistent tempurature on both FPGA chip (according to temp readings from MPBM).
3.) This thing ran the 200mhz ztexmerge bitstreams out of the box without any trouble!! I ended up using MPBM v0.1.0 to get both my Rev 2.0 and Rev 3.0 going at the same time and man am I happy so far!
Big props to the fpgamining crew and to my man Fizzist!
First off I wanted to say: Fizzisist, you are the man! He was super helpful with all my troubles and worked hard to get me my Rev 3.0!
Having said that; here are some pics!
The Naked Rev 3.0 (*fap fap fap*)
With one of the Zalman "Flower" North bridge coolers on it (apologies for the brightness):
The final product with both Zalmans attached and hashing away (Still trying to figure out the best configuration for a final enclosed setup):
Notes on the Rev 3.0:
1.) I've had this thing running for about 4 weeks now with a variety of weird / good / bad fan setups (80mm / 120mm / 2 x 80 mm / 2 x 120mm) and through out all my configurations I've yet to encounter a single invalid share with the 200 mhz ztexmerge bitstream! The push-pin mounts were definitely a great addition to the Rev 3.0!
2.) The "Flower" heatsinks are good but have some pretty big cons to them: A.) They extend over some of the mounting holes and the 3-pin fan power ports. B.) Two fans are needed to maintain a consistent tempurature on both FPGA chip (according to temp readings from MPBM).
3.) This thing ran the 200mhz ztexmerge bitstreams out of the box without any trouble!! I ended up using MPBM v0.1.0 to get both my Rev 2.0 and Rev 3.0 going at the same time and man am I happy so far!
Big props to the fpgamining crew and to my man Fizzist!
Awesome, nbtcminer! Thanks for the kind words! And thanks for sharing the pics and your experiences with it so far.
I have one more configuration that might be worth trying: a single 92 mm fan blowing down on the board. The two mounting holes that aren't covered by the Zalman heatsinks are spaced perfectly for this. I actually tried this out myself recently:
Note that the two screws alone are strong enough to support the fan by themselves, so the other two standoffs aren't needed at all. I'd be interested to hear how this works out for you.
1.) I've had this thing running for about 4 weeks now with a variety of weird / good / bad fan setups (80mm / 120mm / 2 x 80 mm / 2 x 120mm) and through out all my configurations I've yet to encounter a single invalid share with the 200 mhz ztexmerge bitstream! The push-pin mounts were definitely a great addition to the Rev 3.0!
What is the longest you have had it mining without interruption using the 200mhz bitstream?
I'm using the same bitstream and it usually runs with no invalids for 2-3 days but eventually they show up. I've had one board running for 5 days with no invalids and then on the 6th day each core got 3 invalids within a few hours. The temps were measuring in the same range as before (25-30C) so I assume it's just near the limit at 200mhz.
I'd like to the boards to run reliably for a long time and I'm not sure if a handful of invalids every few days is worth worrying about. I may just run the latest bitstream at 190-195 mhz for a tiny margin of safety as I don't consider an extra 10-20mhash to be worth running on the edge. Are my concerns warranted? Any opinions on the risk of electromigration from running them at this speed over a period of months/years?
Finally posting pics / notes of the X6500 Rev 3.0 after what seems like FOREVER after recieving it (like 4 weeks!):
First off I wanted to say: Fizzisist, you are the man! He was super helpful with all my troubles and worked hard to get me my Rev 3.0!
Having said that; here are some pics!
The Naked Rev 3.0 (*fap fap fap*)
With one of the Zalman "Flower" North bridge coolers on it (apologies for the brightness):
The final product with both Zalmans attached and hashing away (Still trying to figure out the best configuration for a final enclosed setup):
Notes on the Rev 3.0:
1.) I've had this thing running for about 4 weeks now with a variety of weird / good / bad fan setups (80mm / 120mm / 2 x 80 mm / 2 x 120mm) and through out all my configurations I've yet to encounter a single invalid share with the 200 mhz ztexmerge bitstream! The push-pin mounts were definitely a great addition to the Rev 3.0!
2.) The "Flower" heatsinks are good but have some pretty big cons to them: A.) They extend over some of the mounting holes and the 3-pin fan power ports. B.) Two fans are needed to maintain a consistent tempurature on both FPGA chip (according to temp readings from MPBM).
3.) This thing ran the 200mhz ztexmerge bitstreams out of the box without any trouble!! I ended up using MPBM v0.1.0 to get both my Rev 2.0 and Rev 3.0 going at the same time and man am I happy so far!
Big props to the fpgamining crew and to my man Fizzist!
First off I wanted to say: Fizzisist, you are the man! He was super helpful with all my troubles and worked hard to get me my Rev 3.0!
Having said that; here are some pics!
The Naked Rev 3.0 (*fap fap fap*)
With one of the Zalman "Flower" North bridge coolers on it (apologies for the brightness):
The final product with both Zalmans attached and hashing away (Still trying to figure out the best configuration for a final enclosed setup):
Notes on the Rev 3.0:
1.) I've had this thing running for about 4 weeks now with a variety of weird / good / bad fan setups (80mm / 120mm / 2 x 80 mm / 2 x 120mm) and through out all my configurations I've yet to encounter a single invalid share with the 200 mhz ztexmerge bitstream! The push-pin mounts were definitely a great addition to the Rev 3.0!
2.) The "Flower" heatsinks are good but have some pretty big cons to them: A.) They extend over some of the mounting holes and the 3-pin fan power ports. B.) Two fans are needed to maintain a consistent tempurature on both FPGA chip (according to temp readings from MPBM).
3.) This thing ran the 200mhz ztexmerge bitstreams out of the box without any trouble!! I ended up using MPBM v0.1.0 to get both my Rev 2.0 and Rev 3.0 going at the same time and man am I happy so far!
Big props to the fpgamining crew and to my man Fizzist!
I also get about 1-2% DOA with the X6500 mining with p2pool. This is using a semi-dedicated PC with an i3/ssd to run p2pool/bitcoind and a separate PC controlling the FPGAs.
Jump to: