Mining XMR with Open Compute servers just crushes a GPU rig in price/perf.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: h311m4n on July 13, 2017, 02:15:16 PM

Quote from: Xeon_Xeon on May 27, 2017, 01:56:37 PM

Given the limited availability fo RX580 -570 gpu's and that has put a dent into my GPU rig plans, I decided to try something that I wanted to do for a while and that is to get a couple of open compute servers and play around with them.
So I ordered two off ebay.

Two Servers:
2x Wiwynn open compute server $380.00
4x E5-2660 V1 cpu’s $207.92 (best price found but the seller only had 4)
4x E5-2660 V1 cpu’s $230.80
8 Gb of DDR 8500 8x1 Gb sticks $17.50
4x used hard drives that I already had $00.00
1x New HP PDU $10.00
Power cords to reach unused 250V a/c outlet $20.00
*not including shipping

Total $866.22 gives 3480 H/s

Open compute server is complete except for cpu's, ram or hard drives. Runs on 200-277 volts.

There are two dual socket 2011 nodes per server.

Running xmr-stak-cpu.

Results:

So 868 H/s mining monero on one node.

Cost is $216.55 per node.

So 868H/s for $216.55, not too shabby.

I do not know the power usage yet but I would guess about ~200-220 watts per node (just a guess as the cpu's are rated for 95 w each and are running only 10 of 16 threads per cpu plus what ever the mobo's use.)

And a big plus is that I did not have to dick around with trying to optimize it for the best speed/stability since Intel already did that.

So four nodes in two servers will crank out 870 H/s x 4 = 3480 H/s for roughly 800- 880 or so watts.

And when the faster cpu's come down in price I can easily upgrade for more speed.

Defiantly beats gpu's on price for mining cpu coins.

Yes they are somewhat limited in what to mine but it is another option for mining plus its very cool hardware to use.

Edit: Edited final prices above.

Running Ubuntu 17.04 server and XMR-STAK-CPU

Edit2: Power usage.

I'm just testing one server so for a total i'll just double the readings since they are setup identically.

No cores turned off.

One server plugged in and both nodes powered off: 15w
One node powered on, Ubuntu 17.04 server booted up and idling : 56.5w
One node after 10 minutes mining Monero with 20 threads : 208w @ 868 H/s
Two nodes powered on, Ubuntu 17.04 server booted up and idling : 117w
Two nodes after 10 minutes mining Monero with 40 threads : 401w @ 1736 H/s

So two Open Compute servers running eight E5-2660 V1 cpu's are mining Monero at 3472 H/s using 802 watts.

The efficiency is:

   4.33 H/s per watt.
   4.00 H/s per dollar.

These Open compute servers just are WAY more efficient mining XMR than a GPU rig.

Cool idea, but my 6 RX470 4GB hash at 4200H/s @ 567W with Claymore's. So, I wouldn't say they are more efficient. But in these times of GPU shortage it's definitely not bad if you have access to some of these things!

You are correct if you only talking about efficiency as watts per H/s.

But when you factor the H/s per watt per dollar these servers come out ahead.

2 servers = 3480 H/s at 802w for $866.22

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: ghobson on July 13, 2017, 02:04:24 PM

Quote from: Xeon_Xeon on June 06, 2017, 06:26:01 AM

Final prices and setup and performance.

2x Wiwynn open compute server $380.00 NEW
4x E5-2660 V1 cpu’s $207.92 (best price found but the seller only had 4) USED
4x E5-2660 V1 cpu’s $230.80 USED
8 Gb of DDR 8500 8x1 Gb sticks $17.50 USED
4x used hard drives that I already had $00.00 USED
1x New HP PDU $10.00 NEW
2x Power cords to reach unused 250V a/c outlet $20.00 NEW
*not including shipping

Total $866.22 gives 3480 H/s

I did not include the prices of the video card or pci-e usb card that were installed to install Ubuntu Server 17.04, the video card was an old Dell oem nvidia that came out of a free pc, the usb card was borrowed from my main pc.

To mine XMR that come out to about $1000.00 less than a dedicated 5 GPU rig using RX580 4 gig card with slightly higher performance.

I went with new open compute servers, you could save another $100.00 buy going with two used Quanta open computer servers, I hate buying used motherboards and was willing to spend a little more.

remove the drives, you really dont need them for xmr-stak-cpu, can't you just use a live cd or boot from usb ?
XMR does not use need main RAM, only L3 cache , so 8GB is overkill and consumes power..
Also , doesnt the noise drive you mad ?

what kind of fans are you using ?

I'm running 8 gig's total in both servers so 1 gig for each cpu.

I have plenty of unused hard drives and zero unused cd/dvd drives. Tho I could have used some 16 gig usb flash drives, and still might, but I wanted a more normal Ubuntu install so I could have a minimum of hassle getting them going.

The fans are WAY quieter than I expected as long as the cpu temps are 78c or lower and actually quieter than my youngest kids watercooled FX 8320 gaming pc , but they do crank up to full speed on a cold boot and are kind of loud but ok because they are in a room between my living room and dining room and off to the side. So even at full speed you don't hear them unless you are in the same room.

The factory fans that came with the servers.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: brian_notgrr on July 12, 2017, 09:51:38 PM

If you really want to get a handle on the numbers -- and see why this is a great idea -- you need calculate the net present value of the project. NPV is money in minus money out, discounted to present value. Money in is the real value of monero at the end of the project, when you plan to sell what you mined. Money out is initial investment, plus expenses, less salvage value (when you're done mining and sell your equipment) discounted to present value. Whichever project has the highest NPV wins.

If you run the numbers this way, the open compute project comes out way ahead.

There are two ways to compare an open compute project with a GPU project. The first is to match the initial investment. This would be a risk tolerant approach, where you leverage any appreciation. Meaning if monero appreciates, you come out way way ahead, because you get way more hashes for your initial dollar. The risk averse approach is to match hashrates, spending less upfront on the open compute project. That way if monero crashes you haven't dug yourself into as deep a hole.

This approach tends to keep your power expenses in perspective. It's just one part of the equation.

OP's points about intangibles are well taken. I would add that the open compute project is scalable, if you can find a host for the right $ (they are out there). GPU projects are only so scalable, you run into marginal cost issues as you cross the 15-20a barrier -- where you have to start figuring in the cost of renting and cooling the sort of place that won't ask questions about all those wires everywhere and all those sparks flying -- and the 200a barrier, which God knows what you have to do to make it fly.

Great post and your spot on about scaleability.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: Marvell1 on July 13, 2017, 12:25:44 AM

Quote from: Xeon_Xeon on May 27, 2017, 02:23:42 PM

I will eventually run them all headless and login via the lan to mess with them.

If your wondering why I went with E5-2660 V1 cpu's it is that right now they are the best performance vs price. I paid a little over $50.00 each for them.

The E5-2660's are half the price of E5-2670's and have ~ %85 of the performance.

the main advantage of GPU's is they hold their value long after you are done with them , this is a good project but only for the short term I think due to GPU shortage

I will keep following

Compute servers will have decent resale because there are a TON of other uses for them if you throw more ram in them.

Virtualization, compute, KILLER PfSense box LOL, simulation and on and on.

Theses servers are a bit outdated but would be excellent learning tools for the above for an individuals test lab.

Resale would be a concern if I had a bunch of them but since I only have two I will find a use for them after XMR gets too hard to mine.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: brian_notgrr on July 13, 2017, 10:46:21 AM

Quote

the main advantage of GPU's is they hold their value long after you are done with them , this is a good project but only for the short term I think due to GPU shortage

I don't think so. Run the numbers and you'll see. I think the main advantage of GPU is flexibility, you're pretty much stuck with XMR if you want to do CPU mining. So you're coming in at a high difficulty level with an important inflection point coming up, in terms of issue rate. But yeah, if you're saying we are at a unique time, I agree. Convergence of high XMR value, lots of servers coming off-lease, GPU shortage, and a brief time left to mine up some XMR profitably.

Don't know whether to thank OP, or remind him that the number one rule of fight club is, you don't talk about fight club.

LOL yep.

After I posted this on a couple of forums the cheaper used Quanta's, they were $149.00, sold out and the seller on ebay raised these Wywinn servers up about $45.00 but the cpu's are a little cheaper now so thats a wash.

EDIT: Some uptime, the servers are rock solid 24/7 except that I live in the sticks and about once or twice a month the power goes off for a second or two because of storms, winds or whatever.

The biggest plus is not having to mess with them at all since I finalized the setup and got all four nodes running.

h311m4n

sr. member

Activity: 487

Merit: 266

Quote from: Xeon_Xeon on May 27, 2017, 01:56:37 PM

Given the limited availability fo RX580 -570 gpu's and that has put a dent into my GPU rig plans, I decided to try something that I wanted to do for a while and that is to get a couple of open compute servers and play around with them.
So I ordered two off ebay.

Two Servers:
2x Wiwynn open compute server $380.00
4x E5-2660 V1 cpu’s $207.92 (best price found but the seller only had 4)
4x E5-2660 V1 cpu’s $230.80
8 Gb of DDR 8500 8x1 Gb sticks $17.50
4x used hard drives that I already had $00.00
1x New HP PDU $10.00
Power cords to reach unused 250V a/c outlet $20.00
*not including shipping

Total $866.22 gives 3480 H/s

Open compute server is complete except for cpu's, ram or hard drives. Runs on 200-277 volts.

There are two dual socket 2011 nodes per server.

Running xmr-stak-cpu.

Results:

So 868 H/s mining monero on one node.

Cost is $216.55 per node.

So 868H/s for $216.55, not too shabby.

I do not know the power usage yet but I would guess about ~200-220 watts per node (just a guess as the cpu's are rated for 95 w each and are running only 10 of 16 threads per cpu plus what ever the mobo's use.)

And a big plus is that I did not have to dick around with trying to optimize it for the best speed/stability since Intel already did that.

So four nodes in two servers will crank out 870 H/s x 4 = 3480 H/s for roughly 800- 880 or so watts.

And when the faster cpu's come down in price I can easily upgrade for more speed.

Defiantly beats gpu's on price for mining cpu coins.

Yes they are somewhat limited in what to mine but it is another option for mining plus its very cool hardware to use.

Edit: Edited final prices above.

Running Ubuntu 17.04 server and XMR-STAK-CPU

Edit2: Power usage.

I'm just testing one server so for a total i'll just double the readings since they are setup identically.

No cores turned off.

One server plugged in and both nodes powered off: 15w
One node powered on, Ubuntu 17.04 server booted up and idling : 56.5w
One node after 10 minutes mining Monero with 20 threads : 208w @ 868 H/s
Two nodes powered on, Ubuntu 17.04 server booted up and idling : 117w
Two nodes after 10 minutes mining Monero with 40 threads : 401w @ 1736 H/s

So two Open Compute servers running eight E5-2660 V1 cpu's are mining Monero at 3472 H/s using 802 watts.

The efficiency is:

4.33 H/s per watt.
4.00 H/s per dollar.

These Open compute servers just are WAY more efficient mining XMR than a GPU rig.

Cool idea, but my 6 RX470 4GB hash at 4200H/s @ 567W with Claymore's. So, I wouldn't say they are more efficient. But in these times of GPU shortage it's definitely not bad if you have access to some of these things!

ghobson

full member

Activity: 250

Merit: 100

Crypto Developer

Quote from: Xeon_Xeon on June 06, 2017, 06:26:01 AM

Final prices and setup and performance.

2x Wiwynn open compute server $380.00 NEW
4x E5-2660 V1 cpu’s $207.92 (best price found but the seller only had 4) USED
4x E5-2660 V1 cpu’s $230.80 USED
8 Gb of DDR 8500 8x1 Gb sticks $17.50 USED
4x used hard drives that I already had $00.00 USED
1x New HP PDU $10.00 NEW
2x Power cords to reach unused 250V a/c outlet $20.00 NEW
*not including shipping

Total $866.22 gives 3480 H/s

I did not include the prices of the video card or pci-e usb card that were installed to install Ubuntu Server 17.04, the video card was an old Dell oem nvidia that came out of a free pc, the usb card was borrowed from my main pc.

To mine XMR that come out to about $1000.00 less than a dedicated 5 GPU rig using RX580 4 gig card with slightly higher performance.

I went with new open compute servers, you could save another $100.00 buy going with two used Quanta open computer servers, I hate buying used motherboards and was willing to spend a little more.

remove the drives, you really dont need them for xmr-stak-cpu, can't you just use a live cd or boot from usb ?
XMR does not use need main RAM, only L3 cache , so 8GB is overkill and consumes power..
Also , doesnt the noise drive you mad ?

what kind of fans are you using ?

goccat

newbie

Activity: 37

Merit: 0

I've been too lazy to build the chaincoin cpu miner but that would be worth checking out right about now.

brian_notgrr

newbie

Activity: 26

Merit: 0

Quote

the main advantage of GPU's is they hold their value long after you are done with them , this is a good project but only for the short term I think due to GPU shortage

I don't think so. Run the numbers and you'll see. I think the main advantage of GPU is flexibility, you're pretty much stuck with XMR if you want to do CPU mining. So you're coming in at a high difficulty level with an important inflection point coming up, in terms of issue rate. But yeah, if you're saying we are at a unique time, I agree. Convergence of high XMR value, lots of servers coming off-lease, GPU shortage, and a brief time left to mine up some XMR profitably.

Don't know whether to thank OP, or remind him that the number one rule of fight club is, you don't talk about fight club.

Marvell1

legendary

Activity: 2408

Merit: 1102

Leading Crypto Sports Betting & Casino Platform

Quote from: Xeon_Xeon on May 27, 2017, 02:23:42 PM

I will eventually run them all headless and login via the lan to mess with them.

If your wondering why I went with E5-2660 V1 cpu's it is that right now they are the best performance vs price. I paid a little over $50.00 each for them.

The E5-2660's are half the price of E5-2670's and have ~ %85 of the performance.

the main advantage of GPU's is they hold their value long after you are done with them , this is a good project but only for the short term I think due to GPU shortage

I will keep following

brian_notgrr

newbie

Activity: 26

Merit: 0

If you really want to get a handle on the numbers -- and see why this is a great idea -- you need calculate the net present value of the project. NPV is money in minus money out, discounted to present value. Money in is the real value of monero at the end of the project, when you plan to sell what you mined. Money out is initial investment, plus expenses, less salvage value (when you're done mining and sell your equipment) discounted to present value. Whichever project has the highest NPV wins.

If you run the numbers this way, the open compute project comes out way ahead.

There are two ways to compare an open compute project with a GPU project. The first is to match the initial investment. This would be a risk tolerant approach, where you leverage any appreciation. Meaning if monero appreciates, you come out way way ahead, because you get way more hashes for your initial dollar. The risk averse approach is to match hashrates, spending less upfront on the open compute project. That way if monero crashes you haven't dug yourself into as deep a hole.

This approach tends to keep your power expenses in perspective. It's just one part of the equation.

OP's points about intangibles are well taken. I would add that the open compute project is scalable, if you can find a host for the right $ (they are out there). GPU projects are only so scalable, you run into marginal cost issues as you cross the 15-20a barrier -- where you have to start figuring in the cost of renting and cooling the sort of place that won't ask questions about all those wires everywhere and all those sparks flying -- and the 200a barrier, which God knows what you have to do to make it fly.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: NiHaoMike on June 23, 2017, 11:32:26 PM

Quote from: Xeon_Xeon on May 27, 2017, 02:17:07 PM

Sure, anybody can buy them and run them as long as you can meet the power requirements since these will not run on 110v.

Just as a note, you can get a Quick 220 or (if you're really experienced with electrical work) its DIY version. At 800W per server, you can run 3 on a pair of 15A circuits with plenty of margin to spare.

I'm lucky, I moved a piece of furniture and rediscoverd a 250v 20 amp a/c outlet only about 12 feet from where the servers are. Cool

NiHaoMike

full member

Activity: 236

Merit: 250

Quote from: Xeon_Xeon on May 27, 2017, 02:17:07 PM

Sure, anybody can buy them and run them as long as you can meet the power requirements since these will not run on 110v.

Just as a note, you can get a Quick 220 or (if you're really experienced with electrical work) its DIY version. At 800W per server, you can run 3 on a pair of 15A circuits with plenty of margin to spare.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Updated original post with power usage.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Oh I almost forgot a BIG plus is not having to spend hours and hours with a video card's bios's and all the time testing and optimizing, Intel already did that for me. Grin

The haven't crashed, locked up, dropped a cpu, burned a powersupply or riser power connector or all the stuff that can go wrong on an all GPU rig.

They are %100 SOLID performing.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Final prices and setup and performance.

2x Wiwynn open compute server $380.00 NEW
4x E5-2660 V1 cpu’s $207.92 (best price found but the seller only had 4) USED
4x E5-2660 V1 cpu’s $230.80 USED
8 Gb of DDR 8500 8x1 Gb sticks $17.50 USED
4x used hard drives that I already had $00.00 USED
1x New HP PDU $10.00 NEW
2x Power cords to reach unused 250V a/c outlet $20.00 NEW
*not including shipping

Total $866.22 gives 3480 H/s

I did not include the prices of the video card or pci-e usb card that were installed to install Ubuntu Server 17.04, the video card was an old Dell oem nvidia that came out of a free pc, the usb card was borrowed from my main pc.

To mine XMR that come out to about $1000.00 less than a dedicated 5 GPU rig using RX580 4 gig card with slightly higher performance.

I went with new open compute servers, you could save another $100.00 buy going with two used Quanta open computer servers, I hate buying used motherboards and was willing to spend a little more.

Xeon_Xeon

full member

Activity: 184

Merit: 103

I installed Windows 7 on a spare drive so I could easily update the mb bios.

Oh the freaking updates to win 7 took for ever......

Anyway updated both bios on both nodes on the first server.

Now i'm installing Ubuntu 17.04 server on the second node.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: VyprBTC on May 27, 2017, 03:05:46 PM

How is it that profitable? I have a couple i7 7700k computers I just wanted to see how much they would make and on nicehash they only were making $0.30 a day, so really curious thanks.

An i7 7700k hashes what about 325H/s ?

So two is ~650 H/s?

Its all about scale AND efficiency , your running one cpu per machine. One power supply per cpu.

My servers have four 8 core cpu's with hyperthreading and one 700 w platinum power supply per machine.

In those two servers I'll be running eight 8 core cpu's with hyper threading so 128 threads total with 80 threads dedicated to mining.

So its really hard to compare a couple of i7's to 8 Xeons but here it goes.

Well plugging in my figures on What To Mine's website.

Two servers with four nodes with eight Xeon E5-2660 cpus is 3480 H/s

About 880 watts

My electricity cost is .097 kwh

That comes to $5.86 per day profit.

Two servers $874.00

So 874.00/5.86 = is a little less than 5 months ROI at --todays-- prices, ( prices that are off by %50 from a couple of days ago. Embarrassed

)

I'm guessing on your figures, two i7 at 650 H/s.... ~350 watts power used for both machines....and using my electricity cost because I do not know what yours is comes out to $.71 profit per day.

I cant make a guess on what your ROI is because I just do not know your machines cost.

Oh and i'm into computers and mining as a hobby, and if I have a hobby that pays for itself and allows me to get newer and better hardware then thats all good.

VyprBTC

full member

Activity: 350

Merit: 100

How is it that profitable? I have a couple i7 7700k computers I just wanted to see how much they would make and on nicehash they only were making $0.30 a day, so really curious thanks.

Xeon_Xeon

full member

Activity: 184

Merit: 103

Quote from: CookieMonstaSMTH on May 27, 2017, 02:30:43 PM

Quote from: Xeon_Xeon on May 27, 2017, 02:17:07 PM

Quote from: CookieMonstaSMTH on May 27, 2017, 02:11:18 PM

I have about 35 instances running on amzn ec2. All of them are 32 core. Can I do something like you?

Sure, anybody can buy them and run them as long as you can meet the power requirements since these will not run on 110v.

If your outside of the USA freight would probably kill the deal.

What difference between mining and running nodes?

A node is simply one computer, motherboard, cpu, ram and hard drive in a removable tray.

These have two nodes per server, they share one power supply and chassis.

So basically two computers per server chassis.

You can configure them identically or completely different.

You mine on each node just like you would a regular pc.

Topic: Mining XMR with Open Compute servers just crushes a GPU rig in price/perf. (Read 2548 times)