who has the highest GH for a single machine out there? (GPU Mining)

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: PatrickHarnett on February 08, 2012, 07:33:11 PM

I read this thread expecting some numbers. Where are they?

I'm not a serious miner, so haven't loaded up a 4*5970 board yet, and I run standard clocks so don't get any special speeds. My top rig is currently cruising at a bit over 1.8Ghash (6 * 314Mhash from three 5970's). Maybe I'll crank up the clock speeds temporarily with a fourth card on a different board I have, but that's not the plan at this stage.

Well I haven't tried any hardcore OC with my current rigs, but they both have 6x 5870's, clocked at 940/300 default voltage. About 2550 mhash per machine, 60 degrees or so and about 55% fan. So 5 Ghash across both machines.

PatrickHarnett

hero member

Activity: 518

Merit: 500

I read this thread expecting some numbers. Where are they?

I'm not a serious miner, so haven't loaded up a 4*5970 board yet, and I run standard clocks so don't get any special speeds. My top rig is currently cruising at a bit over 1.8Ghash (6 * 314Mhash from three 5970's). Maybe I'll crank up the clock speeds temporarily with a fourth card on a different board I have, but that's not the plan at this stage.

Edit: slight o/c to 800Mhz/500 gives 2.077

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: P4man on February 08, 2012, 07:09:07 PM

It does support my point there is no driver limitation

. From your link:

Quote

but I really do think you'll hit SBIOS limitations before you hit driver problems. I know we have a test machine set up for many-GPU experiments and with 12 GPUs we ran into numerous system BIOS issues that we had to fix before we could even boot to a console.

Well NVidia has never made any claim >8 GPU will work even without BIOS issues. I took that more as blame deflection rather than an admission that the drivers will work with more than 8. Others have reported inability to get >8 GPU on one rig.

Some time back I looked around and never found anybody on any forum that had 9+ GPU working on a single rig.

Quote

Leaving system BIOS issues aside, the nVidia driver does not support more than 8 GPUs in a single machine in either Linux or Windows. This fact leads one to ponder virtualizing the hardware and running multiple VMs on the machine, each with fewer than 8 GPUs. Unfortunately there is no VM software that I have been able to find that can virtualize GPUs. Xen is working on it for their next release, but because of the way that Xen functions even it may not work for this specific problem. Xen may still be limited by the nVidia driver's 8 GPU limitation, but this is unclear to me as yet. There is also a fair chance that you'll have to use Linux and not Windows on Xen to get virtualized GPUs to work, and that would in turn require using the finicky WINE CUDA wrapper.

http://foldingforum.org/viewtopic.php?f=38&t=9011#p89268

Still it might be possible. I would love to see someone try, I just would they would go into it with understanding there is a good chance it won't work and they will be left with a lot of expensive parts.

Still say 8x 7990 7970s, ~18GH per rig.

P4man

hero member

Activity: 518

Merit: 500

It does support my point there is no driver limitation

. From your link:

Quote

but I really do think you'll hit SBIOS limitations before you hit driver problems. I know we have a test machine set up for many-GPU experiments and with 12 GPUs we ran into numerous system BIOS issues that we had to fix before we could even boot to a console.

Whether or not those BIOS problems will be a factor using those backplanes is anyone's guess I suppose. The only way to found out for sure might be expensive

.

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: P4man on February 08, 2012, 06:45:13 PM

Quote from: chungenhung on February 08, 2012, 06:37:46 PM

nVidia have 8 GPU limit as well.

I dont think they have. Windows, more specifically, WDDM has a 6 or 8 ?GPU limit for display devices. Fiddling with the registry can overcome that, by making some cards not be recognized as display devices (they wont function anymore if you connect a monitor). Thats also why XP doesnt have this problem, it doesnt have WDDM
Anyway, I distinctly remember reading an nvidia engineer describing how to do that, and saying there was no hard driver limit to the amount of GPUs. That was in the context of CUDA, but it shouldnt be different for OpenCL.

Looking at the folding forums it seems pretty consistent that NVidia doesn't support more than 8. Also there are various BIOS issues that make running >8 improbable even if NVidia officially supported it.

One example from NVidia's forums:
http://forums.nvidia.com/index.php?showtopic=90410

P4man

hero member

Activity: 518

Merit: 500

Quote from: chungenhung on February 08, 2012, 06:37:46 PM

nVidia have 8 GPU limit as well.

I dont think they have. Windows, more specifically, WDDM has a 6 or 8 ?GPU limit for display devices. Fiddling with the registry can overcome that, by making some cards not be recognized as display devices (they wont function anymore if you connect a monitor). Thats also why XP doesnt have this problem, it doesnt have WDDM
Anyway, I distinctly remember reading an nvidia engineer describing how to do that, and saying there was no hard driver limit to the amount of GPUs. That was in the context of CUDA, but it shouldnt be different for OpenCL.

cablepair

hero member

Activity: 896

Merit: 1000

Buy this account on March-2019. New Owner here!!

all my rigs have 5 x 5870 in them, I have never been able to get a system to have more than 5 cards stable in windows, im sure someone has done it but I just could not get it to have any stability

chungenhung

legendary

Activity: 1134

Merit: 1005

Quote from: bulanula on February 06, 2012, 04:31:29 PM

Quote from: rjk on February 06, 2012, 03:19:47 PM

Damn if I can get this working, I think I can safely say I have the largest e-peen of all the GPU miners here. Grin

If I want to sell it to an enterprise or government, I'd probably have to load it with "Pro" cards like FireGL shit and Quadros though. I think there are a few FireGL cards that have 1600 shaders each and only take up one slot.

BTW, I know of Quadro-based systems that have more than 8 GPUs - wonder if the "pro" drivers remove that limitation, or whether nVidia just has never had such a limitation.

What cards have 1600 shaders and only one slot except watercooled 5870s ?

AFAIK Nvidia has no limitations on the number of GPUs but I think they may have an artificial one like ATI with 8 ( maybe 16 with Nvidia ? ). Ask the folding guys.

nVidia have 8 GPU limit as well.

The-Real-Link

hero member

Activity: 533

Merit: 500

Quote from: rjk on February 06, 2012, 04:41:15 PM

Quote from: bulanula on February 06, 2012, 04:31:29 PM

Quote from: rjk on February 06, 2012, 03:19:47 PM

Damn if I can get this working, I think I can safely say I have the largest e-peen of all the GPU miners here. Grin

If I want to sell it to an enterprise or government, I'd probably have to load it with "Pro" cards like FireGL shit and Quadros though. I think there are a few FireGL cards that have 1600 shaders each and only take up one slot.

BTW, I know of Quadro-based systems that have more than 8 GPUs - wonder if the "pro" drivers remove that limitation, or whether nVidia just has never had such a limitation.

What cards have 1600 shaders and only one slot except watercooled 5870s ?

AFAIK Nvidia has no limitations on the number of GPUs but I think they may have an artificial one like ATI with 8 ( maybe 16 with Nvidia ? ). Ask the folding guys.

My bad, the only 1600 shader cards are dual slot. However the FirePro V7800P has 1440 shaders and a passive single slot cooling solution designed for servers.

Wow that's pretty cool. Good luck finding them cheap though. Still $1,100+ on Ebay.

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: bulanula on February 06, 2012, 04:31:29 PM

Quote from: rjk on February 06, 2012, 03:19:47 PM

Damn if I can get this working, I think I can safely say I have the largest e-peen of all the GPU miners here. Grin

If I want to sell it to an enterprise or government, I'd probably have to load it with "Pro" cards like FireGL shit and Quadros though. I think there are a few FireGL cards that have 1600 shaders each and only take up one slot.

BTW, I know of Quadro-based systems that have more than 8 GPUs - wonder if the "pro" drivers remove that limitation, or whether nVidia just has never had such a limitation.

What cards have 1600 shaders and only one slot except watercooled 5870s ?

AFAIK Nvidia has no limitations on the number of GPUs but I think they may have an artificial one like ATI with 8 ( maybe 16 with Nvidia ? ). Ask the folding guys.

My bad, the only 1600 shader cards are dual slot. However the FirePro V7800P has 1440 shaders and a passive single slot cooling solution designed for servers.

bulanula

hero member

Activity: 518

Merit: 500

Quote from: rjk on February 06, 2012, 03:19:47 PM

Damn if I can get this working, I think I can safely say I have the largest e-peen of all the GPU miners here. Grin

If I want to sell it to an enterprise or government, I'd probably have to load it with "Pro" cards like FireGL shit and Quadros though. I think there are a few FireGL cards that have 1600 shaders each and only take up one slot.

BTW, I know of Quadro-based systems that have more than 8 GPUs - wonder if the "pro" drivers remove that limitation, or whether nVidia just has never had such a limitation.

What cards have 1600 shaders and only one slot except watercooled 5870s ?

AFAIK Nvidia has no limitations on the number of GPUs but I think they may have an artificial one like ATI with 8 ( maybe 16 with Nvidia ? ). Ask the folding guys.

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Damn if I can get this working, I think I can safely say I have the largest e-peen of all the GPU miners here. Grin

If I want to sell it to an enterprise or government, I'd probably have to load it with "Pro" cards like FireGL shit and Quadros though. I think there are a few FireGL cards that have 1600 shaders each and only take up one slot.

BTW, I know of Quadro-based systems that have more than 8 GPUs - wonder if the "pro" drivers remove that limitation, or whether nVidia just has never had such a limitation.

bulanula

hero member

Activity: 518

Merit: 500

Quote from: rjk on February 06, 2012, 12:13:58 PM

Quote from: eviltt on February 06, 2012, 11:54:07 AM

Quote from: rjk on February 06, 2012, 10:56:14 AM

Quote from: P4man on February 06, 2012, 08:26:51 AM

You guys arent ambitious enough. Get one of these:
http://blog.trentontechnology.com/wp-content/uploads/2012/01/Trenton-Systems-BPG8032-PCI-Express-Backplane.jpg

18 PCIe slots.

http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/video-processing-gpu-backplane-bpg8032

Heh. I just bought one. Actually the one I got is the "server-class" version, not the "graphics-class", because mining does not need anything better than a PCIe 1x link. Its more of a fun exercise to see what will work, not an insane profit machine. Its going to cost quite a lot.

EDIT: Link to the one I got: http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/pci-express-backplane-bpx6806

what kind of price are we talkin for a rig like that? that would be insane..

Well, I'm going to try hooking up used server power supplies, to keep cost down there. I have found some 2360 watt PSUs for less than 100 bucks, and they are 91% efficient at full load! The board costs close to $1200 new, but I got mine for $600. The largest cost will be the Single Board Computer that runs the whole show - this board is only a backplane. New, the SBC (AKA System Host Board or SHB) is between $1500 and $3000. I hope to get some used ones, but I don't yet know the cost.

I am going to be experimenting with VT-d and AMD drivers to see what happens in a virtualized environment. It is going to be a fun experiment, and hopefully I will come out of it with the fastest single system ever built. It probably won't be -the- lowest power usage system, since the SHB has 2 quad core Xeons, but it might rank pretty high, if I can underclock them. (Or remove one and have a single quad core)

Even if it is far more expansive than it should be, I might be able to sell the complete system for uses other than Bitcoin mining, since systems of this sort are usually $40,000 and up. Especially with more than 4 video cards.

Loving people like you ( with server PSUs and server backplanes ) and people like P4Man ( oil submerged rig ) that bring new creativity and ideas into the community and actually carry out those ideas at great risk to themselves and share with us !

Thank you !

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: rjk on February 06, 2012, 12:13:58 PM

I am going to be experimenting with VT-d and AMD drivers to see what happens in a virtualized environment. It is going to be a fun experiment, and hopefully I will come out of it with the fastest single system ever built. It probably won't be -the- lowest power usage system, since the SHB has 2 quad core Xeons, but it might rank pretty high, if I can underclock them.

I am pretty sure Trenton makes single CPU SMB that can use a Core i series CPU. You should be able to find an SMB for $500 (although that is still insanely high for a miner).

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: eviltt on February 06, 2012, 11:54:07 AM

Quote from: rjk on February 06, 2012, 10:56:14 AM

Quote from: P4man on February 06, 2012, 08:26:51 AM

You guys arent ambitious enough. Get one of these:
http://blog.trentontechnology.com/wp-content/uploads/2012/01/Trenton-Systems-BPG8032-PCI-Express-Backplane.jpg

18 PCIe slots.

http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/video-processing-gpu-backplane-bpg8032

Heh. I just bought one. Actually the one I got is the "server-class" version, not the "graphics-class", because mining does not need anything better than a PCIe 1x link. Its more of a fun exercise to see what will work, not an insane profit machine. Its going to cost quite a lot.

EDIT: Link to the one I got: http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/pci-express-backplane-bpx6806

what kind of price are we talkin for a rig like that? that would be insane..

Well, I'm going to try hooking up used server power supplies, to keep cost down there. I have found some 2360 watt PSUs for less than 100 bucks, and they are 91% efficient at full load! The board costs close to $1200 new, but I got mine for $600. The largest cost will be the Single Board Computer that runs the whole show - this board is only a backplane. New, the SBC (AKA System Host Board or SHB) is between $1500 and $3000. I hope to get some used ones, but I don't yet know the cost.

I am going to be experimenting with VT-d and AMD drivers to see what happens in a virtualized environment. It is going to be a fun experiment, and hopefully I will come out of it with the fastest single system ever built. It probably won't be -the- lowest power usage system, since the SHB has 2 quad core Xeons, but it might rank pretty high, if I can underclock them. (Or remove one and have a single quad core)

Even if it is far more expansive than it should be, I might be able to sell the complete system for uses other than Bitcoin mining, since systems of this sort are usually $40,000 and up. Especially with more than 4 video cards.

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: eviltt on February 06, 2012, 11:54:07 AM

so a 4 5970 should net what 3200mh?

More like ~3.0 to 3.1 GH. To get 3200 MH would require 400 MH/s per GPU, and that means running them at 870MHz each. While you can do it keeping them stable and cool would be very difficulty unless you are watercooling.

eviltt

member

Activity: 84

Merit: 10

Quote from: rjk on February 06, 2012, 10:56:14 AM

Quote from: P4man on February 06, 2012, 08:26:51 AM

You guys arent ambitious enough. Get one of these:
http://blog.trentontechnology.com/wp-content/uploads/2012/01/Trenton-Systems-BPG8032-PCI-Express-Backplane.jpg

18 PCIe slots.

http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/video-processing-gpu-backplane-bpg8032

Heh. I just bought one. Actually the one I got is the "server-class" version, not the "graphics-class", because mining does not need anything better than a PCIe 1x link. Its more of a fun exercise to see what will work, not an insane profit machine. Its going to cost quite a lot.

EDIT: Link to the one I got: http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/pci-express-backplane-bpx6806

what kind of price are we talkin for a rig like that? that would be insane..

BTW the reason I am asking is I am looking for the most powerful single dedicated gpu miner i can, i went the 5970 route, even if its less mh than a 4 6990 only due to availability and cost.

so a 4 5970 should net what 3200mh?

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: P4man on February 06, 2012, 08:26:51 AM

You guys arent ambitious enough. Get one of these:
http://blog.trentontechnology.com/wp-content/uploads/2012/01/Trenton-Systems-BPG8032-PCI-Express-Backplane.jpg

18 PCIe slots.

http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/video-processing-gpu-backplane-bpg8032

Heh. I just bought one. Actually the one I got is the "server-class" version, not the "graphics-class", because mining does not need anything better than a PCIe 1x link. Its more of a fun exercise to see what will work, not an insane profit machine. Its going to cost quite a lot.

EDIT: Link to the one I got: http://www.trentontechnology.com/products/backplanes/picmg-13-backplanes/pci-express-backplane-bpx6806

P4man

hero member

Activity: 518

Merit: 500

Quote from: bulanula on February 06, 2012, 09:31:52 AM

You can run a maximum of 8 GPUs using the ATI driver on Linux or Windows ( recent drivers like 11.6 etc. )

Youve said that before. And while its true (I think its actually 6 on windows?), you seem to ignore my suggestion of launching multiple X servers, since each would run its own driver, and each could be limited to less than 8 gpus.

Granted, this is more a theoretical exercise than anything else, because the cost of these boards will likely make it unrealistic anyway.

bulanula

hero member

Activity: 518

Merit: 500

Quote from: Chefnet on February 06, 2012, 08:32:00 AM

and whats with the driver limit?

You can run a maximum of 8 GPUs using the ATI driver on Linux or Windows ( recent drivers like 11.6 etc. )

You can have like 4*5830 and 2*5870 and 2*5850 on one of those boards.

Topic: who has the highest GH for a single machine out there? (GPU Mining) (Read 3779 times)