Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] - page 10.

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: bulanula on April 12, 2012, 06:31:49 AM

WOW. Those fans are incredible.

Sounds like a jetpack.

But, do they provide enough power so the air goes through all the cards and there are no hot spots ?

Short answer - yes.

Longer answer - I might have to do some fancy ducting to be sure that the air doesn't just blow over the top of the heatsink instead of through it, but for right now the sheer tornado force negates that into a non-issue. The issue will be when I turn them down to a reasonable level. Also, I don't want to mod the existing brackets and remove some resale value, so I'll need to see about getting some custom ones fabricated.

bulanula

hero member

Activity: 518

Merit: 500

WOW. Those fans are incredible.

Sounds like a jetpack.

But, do they provide enough power so the air goes through all the cards and there are no hot spots ?

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: JL421 on April 12, 2012, 12:03:36 AM

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

There's tons of room, and they prevent damage to the back of the board due to clumsy mishandling, which I am prone to. Removing the plastic gave me a huge amount of space, and the only thing I really need to do is mod/remove/replace the front brackets with higher flow pieces.

JL421

hero member

Activity: 812

Merit: 510

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: Spotswood on April 11, 2012, 11:08:52 PM

To properly mount those fans, you could remove one of the "beams" in the top, placing it in the front in between the posts.

Well, the problem is that the fancy wiring job that I did with that terminal strip just coincides with the fan placement. They need to be down low, which is why they are in a stairstep. If I mounted them all at the same height, they would be a little high, and all the air would go over the GPUs instead of through them. But I can't go all they way down since the terminal strip is there. I am going to try to get everything hooked into the terminal strip, and then push it down out of the way since it is sticking up right now.

Vimeo is saying it wants to wait half an hour before my video gets converted, so I have uploaded it to cloudfront here:
I think the Vimeo link is this: http://vimeo.com/40200641 and presumably that link will start working after it is converted.

The cloudfront link expires in a couple hours, and I'll remove it later.

Vimeo link here: http://vimeo.com/40200641

Spotswood

sr. member

Activity: 512

Merit: 250

To properly mount those fans, you could remove one of the "beams" in the top, placing it in the front in between the posts.

DiabloD3

legendary

Activity: 1162

Merit: 1000

DiabloMiner author

Huh, now that I see that coming together thats almost exactly like the rig I always wanted to build but never could afford.

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

OK I got the custom frame in today, and have partially assembled the system in it. Here are some pics, and I'm going to upload a short video soon to demonstrate how loud the fans are. Grin

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: Sherkel on April 11, 2012, 12:11:47 PM

Wow. I just don't know what to say. Cheesy

For 18 GPU's, it would probably be a better deal in the end if you have a good PSU, right?

Done and done. But if I want more than 8 GPUs, I need VT-d, unless anyone has been able to test and confirm otherwise.

Sherkel

full member

Activity: 131

Merit: 100

Wow. I just don't know what to say. Cheesy

For 18 GPU's, it would probably be a better deal in the end if you have a good PSU, right?

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

I just got the custom frame from Spotswood, and it is solid. It was flat packed, and I will have to wait until later this evening after work to put it together and take pics. But +100 for Spotswood for an awesome custom-built open air frame.

Also got fan grilles and screws, and have shipped off a couple fans to Garr255 to develop a custom PWM controller for them. Still wondering whether I should get a VT-d capable host board, especially considering that this rig could be somewhat obsolete soon. Cry

Unless Vladimir develops a PCIe based ASIC card that I can shove in. Grin

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: amazingrando on April 01, 2012, 06:54:26 PM

Quote from: rjk on April 01, 2012, 04:05:30 PM

Quote from: mcorlett on April 01, 2012, 03:32:36 PM

Quote from: amazingrando on April 01, 2012, 03:23:49 PM

If there is a way to use virtualization but still access the GPU hardware, that could allow for some really interesting ways of doing things.

There is, and a lot of commercial virtualization software offers it. It's usually called "GPU pass-through", or something along those lines.

Yes, but I have yet to hear from anyone that has tested it much. Luke-jr is the only one that I know of that has actually mined with a video card in a virtualized OS, using KVM passthough virtualization. But that was only one card, and the system was somewhat unstable. Others have mentioned that since the drivers run in usermode with X, that I might be able to run multiple X instances. But I don't really know enough about how that works to know how to set it up, or even whether it would work at all.

I'm fairly swamped at the moment, but if I can find some time I'll give it a try and see how it works, then report my results.

Did you get a chance to try this? I really would like to know before dropping another grand on this box.

discordian666

newbie

Activity: 53

Merit: 0

I have much respect for the OP and this project, when he gets this fully operational and stable I'd love to do the same thing at my place

Clearly one of my more favourite threads on this forum

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: tnkflx on April 10, 2012, 09:06:21 AM

Quote from: rjk on April 10, 2012, 08:07:20 AM

Quote from: tnkflx on April 10, 2012, 02:32:00 AM

Quote from: rjk on April 09, 2012, 11:37:56 PM

Quote from: ARapalo on April 09, 2012, 11:21:52 PM

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

I am the only one (that I know of) that has tried this. Although, Bigpiggy01 has a different board that works similarly, that he has gotten to work with more than 8 GPUs using IOMMU.

8 GPUs is not a problem. Going above that is

I know, he has it working with 11 GPUs.

Do you have a link to his setup somewhere?

https://bitcointalksearch.org/topic/m.834841

tnkflx

sr. member

Activity: 349

Merit: 250

Quote from: rjk on April 10, 2012, 08:07:20 AM

Quote from: tnkflx on April 10, 2012, 02:32:00 AM

Quote from: rjk on April 09, 2012, 11:37:56 PM

Quote from: ARapalo on April 09, 2012, 11:21:52 PM

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

I am the only one (that I know of) that has tried this. Although, Bigpiggy01 has a different board that works similarly, that he has gotten to work with more than 8 GPUs using IOMMU.

8 GPUs is not a problem. Going above that is

I know, he has it working with 11 GPUs.

Do you have a link to his setup somewhere?

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: tnkflx on April 10, 2012, 02:32:00 AM

Quote from: rjk on April 09, 2012, 11:37:56 PM

Quote from: ARapalo on April 09, 2012, 11:21:52 PM

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

I am the only one (that I know of) that has tried this. Although, Bigpiggy01 has a different board that works similarly, that he has gotten to work with more than 8 GPUs using IOMMU.

8 GPUs is not a problem. Going above that is

I know, he has it working with 11 GPUs.

tnkflx

sr. member

Activity: 349

Merit: 250

Quote from: rjk on April 09, 2012, 11:37:56 PM

Quote from: ARapalo on April 09, 2012, 11:21:52 PM

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

I am the only one (that I know of) that has tried this. Although, Bigpiggy01 has a different board that works similarly, that he has gotten to work with more than 8 GPUs using IOMMU.

8 GPUs is not a problem. Going above that is

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: ARapalo on April 09, 2012, 11:21:52 PM

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

I am the only one (that I know of) that has tried this. Although, Bigpiggy01 has a different board that works similarly, that he has gotten to work with more than 8 GPUs using IOMMU.

Quote from: Spotswood on April 09, 2012, 11:19:08 PM

Quote from: rjk on April 09, 2012, 10:55:13 PM

...
I've been considering that. Not exactly sure how well the angle connectors will conduct current though, some seem to be plastic, and others are bolted with a metal connector. It would suck to have the slightest loose connection, which is why I was considering copper busbar.
...

The t-slot extrusions are anodized. All of the angle brackets are cast aluminum.

Cool. Maybe it will be sufficient then to use as my ground plane.

ARapalo

member

Activity: 93

Merit: 10

This thread/concept is quite awesome. I skimmed some of it, I'll have to go back and try to read every page.

Did majority of people that tried this backplane actually get at least 8 GPUs working?

Spotswood

sr. member

Activity: 512

Merit: 250

Quote from: rjk on April 09, 2012, 10:55:13 PM

...
I've been considering that. Not exactly sure how well the angle connectors will conduct current though, some seem to be plastic, and others are bolted with a metal connector. It would suck to have the slightest loose connection, which is why I was considering copper busbar.
...

The t-slot extrusions are anodized. All of the angle brackets are cast aluminum.

Topic: Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] - page 10. (Read 169522 times)