Author

Topic: Crazyness Build Time (Read 3936 times)

brand new
Activity: 0
Merit: 250
October 23, 2011, 07:48:44 AM
#27
Yes and no.

PCIe has a concept of both lanes and ports.  So the Southbridge has those 16 PCIe lanes configured to a single port.  When you see a board that has 2 slots and it can be 1x PCIe16x or 2 PCIe8x it means the PCIe switch inside the southbridge is capable of dynamically configuring the port to lane assignments. 

Just splitting the lanes physically will do no good.  The PCIe switch inside the southbridge won't "know" there should be 3 or 7 more logical ports.

So what you need is a PCIE switch (a chip which can route PCIe lanes to ports) and that would require a custom PCB.

There are existing solutions which provide PCIe expansion but they are way to expensive.  Here is one example:
http://www.magma.com/expressbox16basic.asp
It adds 16 PCIe slots in an external chasis from a single 16x port.  So hypothetically take a MB w/ 4 PCIe 16x slots connect each one to a 16 bay expansion chasis and gain 64 slots for GPUs.

Two problems.  One the chassis and expander is an insane $4500.  Two AMD stupidly limits you to 8 GPU.

Given the 8 GPU limit I have found the easiest, fastest, and most efficient setup is a motherboard with 3x 16x slots spaced two spaces apart and use 3x 5970s for 6 total GPU.  No other method seems to come close to that in terms of cost.
OK so it's possible, just expensive.

The trouble with *your* solution DAT is that finding 5970 cards has become rather difficult, not to mention rather expensive in hash per £.

The video card vendors, certainly here in the UK, seem to know damn well that bitcoin miners want the 58xx/5970 cards at any cost and are pumping up the prices. The cheapest I've seen for a usable card (over 200 MH/s for me) is £80 for the 5770. At those prices, when the dual-GPU single cards are costing £600 or more, it's almost worth buying two of my favourite Gigabyte 4-slot boards and running 8 of the 5770s (which all reliably clock up to 220 MH/s each - the disadvantage being space, and potentially efficiency.

I must say that I'm surprised at the cost of the 'lane splitter' solutions. Entire logic boards (like my beloved Gigabyte H61M-D2-B3) cost around £55 inc VAT these days. There's a southbridge on there which splits the PCIe bandwidth into one x16 slot and three x1 slots. Hence the chip logic to split PCIe lanes into different configurations can't be *that* expensive, can it?

I've got an Asus logic board which I've given up on - I bought it for its 5 PCIe slots - and three well-spaced x16 slots (it'd be a candidate for your preferred format, since you'd get your three 5970s onto the board with no extender cables required). A feature in its BIOS allows you to split the bandwidth across the physical PCIe slots - you can disable the two x1 slots and give maximum bandwidth (x16, x8, x8) for three GPUs, or you can have two full-fat x16s with the rest disabled, or the 'x1' config I was trying to get to work, where all slots are available at x1 bandwidth.

This board caused me so many problems (trying to get all 5 slots working with GPUs) that I've taken the CPU out and put it in another Gigabyte board - that's four Gigabyte boards I've got now in my farm Smiley They just work.

However, this Asus board suggests that the BIOS can configure which slots get however many PCIe lanes... so if the software and hardware can already split PCIe bandwidth across the available slots, why are these extension systems so damned expensive?

After all, if the solution I want costs thousands, then I can just bite my tongue and order a few of those damn 'extreme gamer' Big Bong boards which already have 7 PCIe slots on the board (and probably waste 50W or so just running the logic board, requiring more powerful PSUs as well, adding to the cost) Sad
vip
Activity: 166
Merit: 100
October 24, 2011, 07:52:36 PM
#25
B) No I do not pay for elec, I have a datacenter, I pay a hosting rate

Is this to say you own a datacenter or are just leasing from one? Since you say your not paying for electricity, leasing has got to be much more expensive?
hero member
Activity: 518
Merit: 500
October 23, 2011, 01:09:30 AM
#24
Don't know if it is still current, but I was looking at this last year (different application) and found some Compaq stuff.  4 16x slots for $3.5k from memory - there were cheaper ways to do this, and having boards with 4 GPUs is ok for what I'm playing with.
donator
Activity: 1218
Merit: 1080
Gerald Davis
October 22, 2011, 05:33:53 PM
#23
Ive been looking into Magma products fir a while now and while I love the companies products, with them you are paying for enclosure and cabling.

A couple other companies make bare backplanes.  Price is still astronomical.  Like >$2K for 16 slots and >$1K for 4 slots.  IIRC Magma doesn't manufacture their own bakcplanes. They are just a Value Added Reseller putting OEM parts together with warranty and service.
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 22, 2011, 12:42:50 PM
#22
Ive been looking into Magma products fir a while now and while I love the companies products, with them you are paying for enclosure and cabling.
donator
Activity: 1218
Merit: 1080
Gerald Davis
October 22, 2011, 08:57:32 AM
#21
Yes and no.

PCIe has a concept of both lanes and ports.  So the Southbridge has those 16 PCIe lanes configured to a single port.  When you see a board that has 2 slots and it can be 1x PCIe16x or 2 PCIe8x it means the PCIe switch inside the southbridge is capable of dynamically configuring the port to lane assignments. 

Just splitting the lanes physically will do no good.  The PCIe switch inside the southbridge won't "know" there should be 3 or 7 more logical ports.

So what you need is a PCIE switch (a chip which can route PCIe lanes to ports) and that would require a custom PCB.

There are existing solutions which provide PCIe expansion but they are way to expensive.  Here is one example:
http://www.magma.com/expressbox16basic.asp
It adds 16 PCIe slots in an external chasis from a single 16x port.  So hypothetically take a MB w/ 4 PCIe 16x slots connect each one to a 16 bay expansion chasis and gain 64 slots for GPUs.

Two problems.  One the chassis and expander is an insane $4500.  Two AMD stupidly limits you to 8 GPU.

Given the 8 GPU limit I have found the easiest, fastest, and most efficient setup is a motherboard with 3x 16x slots spaced two spaces apart and use 3x 5970s for 6 total GPU.  No other method seems to come close to that in terms of cost.
hero member
Activity: 504
Merit: 500
October 21, 2011, 10:51:55 PM
#20
  To add to my post before about external connections.  If your only option is to connect with that express card adapter, there is as I stated an expresscard(universal both EC54 and 34) that will connect to USB.
 Since I did not provide a proper link before I hunted one down. Only $15.99 too.. There is a catch though, and that is the express card adapter you use from the vid card cannot be the pcie type. They say it will jsut not detect. That can likely be hacked if the guys making the other adapters have not already thought of that....

  http://www.amtron.com/expresscard/usbexp54b.htm


 
  I have been searching my nuts off for a straight up PCIe to USB converter. They were seen as useless for the most part whne companies were researching external plugs due to the low badnwidth of USB 2.0 (~490Mbs) verse ExCard or custome solution being very close to PCIe.

  But, we know mining uses very little for badnwidth. And, with USB3.0 having much more badnwidth hopefully some of these options can be reconosidered.

  I wonder how hard would it be for the community here to design a board that converted PCIe to USB and in varying numbr of PCIe slots? I know it would add very little in the way of efficiency to existing PC power consumption. But it would be more efficient none the less. Especially on very low power, underclocked laptops....


  Cheers
hero member
Activity: 504
Merit: 500
October 19, 2011, 06:39:11 AM
#19
  That's pretty damn slick in design and spec, catfish. Mad props! 
hero member
Activity: 504
Merit: 500
October 18, 2011, 09:53:44 PM
#18
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.

The link you provided is for ExpressCard adapter.  ExpressCard is the replacement for PCMCIA but is incompatible.

http://en.wikipedia.org/wiki/ExpressCard

This is why I post. Good catch on that one.


  There is also 2 different size express card slots. I believe ExpressCard 54 and ExpressCard 36(?) Not much a fan of laptops..
  In any effect, there is an adapter that will convert expresscard 54 to usb. Its intended purpose was for laptops with older slots to be able to use expresscard over usb. Can't be assed to find you a link, sorry. ;p

  Big Bang Marshal ftw if cost is no object.
  
  If you have free pci slot and pcmcia as you say that you can use for external there are these here, http://www.adexelec.com/cb.htm#PCICBI      I did not see anything listed but this company has a HUGE selection of pcie risers, etc. Might be worth a call to see if they make a 'splitter' of sorts.
hero member
Activity: 518
Merit: 500
October 18, 2011, 09:33:45 PM
#17
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.
PCMCIA cannot be adapted to PCI Express. Only a purely generic bus or something that is basically already PCI can be adapted to PCI Express.

I thought they were trying to use an interface that is used in some laptop connections to link to PCIe - there are some cumbersome methods to do that.

If money is really no object, they should be adding an external PCIe enclosure.
legendary
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
October 18, 2011, 08:54:11 PM
#16
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.
PCMCIA cannot be adapted to PCI Express. Only a purely generic bus or something that is basically already PCI can be adapted to PCI Express.
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 18, 2011, 08:17:57 PM
#15
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.

The link you provided is for ExpressCard adapter.  ExpressCard is the replacement for PCMCIA but is incompatible.

http://en.wikipedia.org/wiki/ExpressCard

This is why I post. Good catch on that one.
donator
Activity: 1218
Merit: 1080
Gerald Davis
October 18, 2011, 07:59:04 PM
#14
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.

The link you provided is for ExpressCard adapter.  ExpressCard is the replacement for PCMCIA but is incompatible.

http://en.wikipedia.org/wiki/ExpressCard
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 18, 2011, 07:28:42 PM
#13
Ok, well I have 4 pcmcia connectors so Im going to try it out on them when they arrive.
legendary
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
October 18, 2011, 01:37:26 PM
#12
How do you connect a graphics card to a USB port?!
Well, it looks like you can take this 1x to Expresscard item I linked to at the top of my post, and connect it via USB. If thats not the case, you can use a pcmcia interface for sure as that has been confirmed and with the use of CGminer you can OC the cards.
The USB port on the adapter is not usable to connect the graphics card. It only connects the adapter. (Much like the USB ports on many televisions.)
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 18, 2011, 01:20:59 PM
#11
The larger issue is you are paying $55 per card for connectivity (plus existing computer) and there is a limit of 8 GPU in windows (not sure about Linux).

Take a motherboard with 4 PCIe 16x slots say $150 plus $20 RAM + 30 CPU = $200 for 4 slots = ~$50 per slot. 

Your method costs more for more complexity.


As I said, I dont care about cost here. Just looking for if it works.
donator
Activity: 1218
Merit: 1080
Gerald Davis
October 18, 2011, 10:06:43 AM
#10
The larger issue is you are paying $55 per card for connectivity (plus existing computer) and there is a limit of 8 GPU in windows (not sure about Linux).

Take a motherboard with 4 PCIe 16x slots say $150 plus $20 RAM + 30 CPU = $200 for 4 slots = ~$50 per slot. 

Your method costs more for more complexity.
full member
Activity: 168
Merit: 100
October 18, 2011, 12:06:16 AM
#9
I'll ask a question having looked very hard at external PCIe connectors a while ago.  How do you connect your mobo to the pcmcia interface on the ExpressCard which is a laptop centric interface?

(that is, it is not a USB connection from the mobo - as far as I know, they don't exist yet.  there is something like a pcie splitter I think I saw a few months back - cablesarus maybe??)

There is a PCI-E card for expresscard. Tons on ebay.

This should be the least you have to worry about tho
hero member
Activity: 518
Merit: 500
October 17, 2011, 10:16:15 PM
#8
I'll ask a question having looked very hard at external PCIe connectors a while ago.  How do you connect your mobo to the pcmcia interface on the ExpressCard which is a laptop centric interface?

(that is, it is not a USB connection from the mobo - as far as I know, they don't exist yet.  there is something like a pcie splitter I think I saw a few months back - cablesarus maybe??)
full member
Activity: 168
Merit: 100
October 17, 2011, 09:38:30 PM
#7
Using any PCI-E expansion board is not cost effective at all.

Either get a 8 PCI-e slots board or use dual GPU on 4 PCI-E slot board.

The driver limits you at 8 GPUs.
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 17, 2011, 08:36:15 PM
#6
How do you connect a graphics card to a USB port?!

Well, it looks like you can take this 1x to Expresscard item I linked to at the top of my post, and connect it via USB. If thats not the case, you can use a pcmcia interface for sure as that has been confirmed and with the use of CGminer you can OC the cards.

Im looking at the hardware in the pictures and I see a female usb connection (odd that it is female) and I am hoping it will connect via that. I do have PCMCIA -> USB and PCMCIA to PCI adapters at my disposal.

Edit: Thought I linked it up above, LIE
http://www.harmonicinversion.com/index.php?page=shop.product_details&product_id=193&vmcchk=1&option=com_virtuemart&Itemid=3

Want to order a bunch of them.
mmortal03 claims to have them working.
legendary
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
October 17, 2011, 07:25:03 PM
#5
How do you connect a graphics card to a USB port?!
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 17, 2011, 07:15:54 PM
#4
what makes you want to invest a large fund to mining hardware now? Dont you have to pay for electricity?

Shouldnt you just buy BTC out right at the current price?



A) I run Shades Minoco Mining Contractors
B) No I do not pay for elec, I have a datacenter, I pay a hosting rate
C) People keep wanting to buy mining power without the hassle, so I will keep making more rigs.
full member
Activity: 168
Merit: 100
October 17, 2011, 02:45:40 PM
#3
what makes you want to invest a large fund to mining hardware now? Dont you have to pay for electricity?

Shouldnt you just buy BTC out right at the current price?
full member
Activity: 235
Merit: 100
October 17, 2011, 10:37:16 AM
#2
i think USB3 would have more then enough bandwidth.
i just don't know of any USB to PCIE dongles.
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
October 17, 2011, 05:14:08 AM
#1
Ok, so Im now back on the "I will build the most badass hashing machine ever to grace the planet" kick again.
This time I am looking over to this post here https://bitcointalksearch.org/topic/m.578827 to see what is the possibility of adding some more gpus via usb. Now this is where I ask you to all chime in.

Host systems are the following boards.

Msi 890-gd70
Msi 890-gd65
Foxcon POS that I am to lazy to check out the model, but it has 2x 16 and 2x 1 on it.
Msi 790-??? 2x1
Msi 840-g45

Now the way I see it, the power on these things works like via a floppy with a 5v/12v connection to get the juice for the pcie connection. I plan on supplying my 12v 6pin power to either a 5830 or 5770 from an aux psu in the form of a 80+ silver balanced correctly, or running off of a psu like https://cablesaurus.com/index.php?main_page=product_info&cPath=1&products_id=37&zenid=4dc76ae8853451f4a25565fdff6fd727 but is 12v @ v4.? amps and runs on 115v @ 360w (if any one wants to give me a 5830/psu math check, Im thinking 1 or 3 5770s) So by my math, this could be a theoretical system build if this works.

Msi 890-Gd65
2x Thermaltake 750w
6x 5830s on pcie connections (4 1x / 2 16x)
2x 5830s on usb boxy thing with Visontec 450w Juicebox for the last 2 (this will power 2 5830s, I have them, I run them, just dont put a 5970.....rip Sparky EDIT: Sparky was the psu, didnt trash a $500 card)
Ddr 3 ram (box o' ram)
Semp 130/140 or Ath X2ish thingys
Sata hdd because I love Win7

Now my concerns lay with the usb bandwidth (not in use by anything else on this machine) and of course that it really will work over usb. They have connections I guess over mini hdmi to an ExpressCard for laptops on PCMCIA but looking at the hardware, I may not need it.

Outside of that I have been looking at a few other options like the Magma ExpressBox 7 - x4G1 (starts at $2799 and would be gutted on site and fitted to a shelf and have pcie risers plus a psu fitted with 130cfm fans and attached to a 1u server for a control. The only part I would have to buy would be the nearly 3k piece of hardware which may sound crazy, but I found out they go for much cheaper sometimes.

So let me know so I can let my good buddy over at BitEgg and BitPizza order me 10-20 of these things and go nuts!
Jump to: