Pages:
Author

Topic: Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] - page 25. (Read 169524 times)

donator
Activity: 1218
Merit: 1079
Gerald Davis
Seems like you could do 5 5970s in that with 7 single slot cards. 
Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.



Really? Also please don't say virtualization these are non VT-d capable CPUs.
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
Man, I had no clue that showing you this crap would make you buy it. Love of god, you need help, I got your back. This solves alot of problems if it works.

32 cards waterstation he I come!
sr. member
Activity: 406
Merit: 250
QUIFAS EXCHANGE
Seems like you could do 5 5970s in that with 7 single slot cards. 
Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.



And that is?
sr. member
Activity: 378
Merit: 250
hero member
Activity: 533
Merit: 500
  Ok man sorry... forgive me for merely speculating as to what could physically fit on this board.  Of course I know there's the 8 GPU limit now that I've read about this stuff Wink.  Now if or how that can be overcome, I do not know.  Good luck!
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
Seems like you could do 5 5970s in that with 7 single slot cards.  
Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.


Do tell? If it is anything with running multiple X sessions, I am enough of a linux noob to not know how this works or where to begin. If you mean by using VT-d and some virtual machines to overcome the driver limits, sure sounds good but my hardware is not capable (no VT-x or VT-d unless I get a more expensive host board). Or, if you mean modifying the drivers in any way, or using specific versions without a limit, perhaps you will enlighten me in that regard?

EDIT: Spoke on IRC, and it seems that VT-d was the idea, but unfortunately my current hardware doesn't support it Sad
staff
Activity: 4284
Merit: 8808
Seems like you could do 5 5970s in that with 7 single slot cards. 
Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.

hero member
Activity: 518
Merit: 500
Seems like you could do 5 5970s in that with 7 single slot cards. 

Might want to read the thread. We still have to deal with 8 gpu driver limit.

Yeah might want to pressure ATI to remove this stupid ARTIFICIAL limit in their crap drivers.

If more of us do it, then they will probably take their heads out of their asses and listen.
legendary
Activity: 1344
Merit: 1004
Seems like you could do 5 5970s in that with 7 single slot cards. 

Might want to read the thread. We still have to deal with 8 gpu driver limit.
hero member
Activity: 533
Merit: 500
Seems like you could do 5 5970s in that with 7 single slot cards. 
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
I wish. I did some research, and as far as I can determine, I have these processors: http://ark.intel.com/products/33103/64-bit-Intel-Xeon-Processor-2_80-GHz-2M-Cache-800-MHz-FSB

Yeah they support 64-bit processing and hyperthreading, but that is about the only modern (ish?) features that they have.. Definitely no VT-d (needed for Xen VGA Passthrough), and not even VT-x (needed for any kind of virtualization that runs on bare metal).

The cheapest "new" option that supports VT-d is the Intel Xeon E3-1225 - which Trenton quoted me at $1,610 in single quantities.  Angry If any of you have access to TSB or JXT series SHB cards, or know where to get them at a reasonable price, do please let me know...
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
RJK,

Are you just running BAMT native on that Machine or are you virtualizing it as I assume you will have to in order to get beyond the 8GPU limit per OS?


No right now I just got it going really quick yesterday, haven't spent much time on it yet. My plan was to virtualize it if I couldn't make something else work with the drivers. However after a very hasty peek at the BIOS, there doesn't seem to be options for enabling/disabling VT-x and VT-d. However I might have missed something obvious in my haste.

Yep, rjk's Battle of Trenton will be even more epic than the Revolutionary War with General George Washington.

A George Washington smile for rjk.


When the 8 GPU barrier is broken by the diligent open source peeps rjk will be ahead of the game.
Lol!

I need to figure this virtualization situation out. Perhaps I can look up the processors in the Intel Ark. They are 2.8 Ghz LV (Low Voltage) Xeons, but I'm not sure of a model number.

I have also heard someone say that the display drivers in linux are loaded for each X session, and that I could run multiple X sessions? This seems far-fetched though.... anyone tried this before?

Finally, this system has a very intriguing BIOS option: Namely, the ability to switch between 32-bit and 64-bit PCIe resource tables. I think this is to overcome some kind of addressing limitations that apply to having so many slots, but enabling 64-bit mode made it so that neither BAMT nor Windows would load/install. Both reduced the display down to 4 colors (or was it 16?), and Windows wouldn't install without a driver. It didn't tell me what device the driver it needed was for though, so anything I presented to it was rejected.
full member
Activity: 196
Merit: 100
Web Dev, Db Admin, Computer Technician
Yep, rjk's Battle of Trenton will be even more epic than the Revolutionary War with General George Washington.

A George Washington smile for rjk.


When the 8 GPU barrier is broken by the diligent open source peeps rjk will be ahead of the game.
member
Activity: 84
Merit: 10
RJK,

Are you just running BAMT native on that Machine or are you virtualizing it as I assume you will have to in order to get beyond the 8GPU limit per OS?
hero member
Activity: 642
Merit: 500
Definitely watching this.  Smiley
sr. member
Activity: 266
Merit: 250
The king and the pawn go in the same box @ endgame
i must watch rjk's shenanigans, and learn
hero member
Activity: 518
Merit: 500
Well done rjk for investigating this crazy land of expensive backplanes.

We bow down to your awesomeness Shocked

Keep up the good work and us posted !
hero member
Activity: 533
Merit: 500
That's an epic board!  Good luck fitting it out and continuing to make it all work Wink.  Very impressive.
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
Are you being overly dramatic about slot locations and useability for the sake of being dramatic ? .....LOL

Any sane person would integrate this into a custom mining stand using 60-100% PCIe soft risers/cables, no ?

Every 3rd or so card on the board, then risers to attach others above to keep everything spaced and cool ? .....or am I to assume this is going into a rackmount case eventually ?

I must admit....this is awesome and I hope you get it working.
The idea was for it to be rackmount, otherwise you would be able to build the same mining power into a larger space easily. Its all about density here. Wink

The other thing that I mentioned in another thread is that it may be a better idea to build it with an enterprise in mind - not Bitcoins. To that end, it would make more sense to use something like nVidia Quadro cards, or AMD FirePro stuff. For mining though.... 5970s all the way.

Another small hang-up: In researching waterblocks that fit 5870s and 5970s, they all seem to be 0.7 to 0.9 inches thick. Since PCIe slot spacing is 0.8 inches, that wouldn't fit. Does anyone know of waterblocks that are known to fit side-by-side in adjacent slots? Or will I have to get them made custom.... Danger Den, I'm looking at you...
Pages:
Jump to: