Topic: Hosts are dropping like flies... or at least dropping me (Read 631 times)

After some consideration we decided to announce availability of miner hosting at our facility again.

https://bitcointalksearch.org/topic/ann-sunbreak-electronics-llc-3-mw-datacenter-colocation-usa-oregon-5404764

Even if you had free power, it probably isn't worth it to install Antminer S9's because the buildout will be expensive. Even if you could get the miners for $120 each, the cooling + electric circuits will easily cost $200+ for each one (depends on labor cost). So each S9 actually costs $320. The Bitmain T15 or S15 would be better.

Agree with you on the overheads of $0.04/kWh. This is over and above electricity cost.

We did a quick back of the envelope calculations. Unless we were getting free electricity, it isn't even worth it to use S9s to mine anymore.

Where I'm at,
Home miners - USD 0.20/kWh + tax
Wholesale rate for farms - >> USD 0.50/kWh

Screws up the mining economics. Plus land prices aren't cheap. There has been an exodus of colo players here, leaving only the most hardcore home miners..

Wow, this is tremendously useful knowledge that is very hard to find. I agree that there's no free lunch when it comes to ASIC mining. Each hosting option has different trade-offs. Rolf Verslusius was right when he predicted back in 2107 that small miners have many advantages. I also did a calculation last week that showed the cost per BTC for a 1MW miner is only 10-20% higher than RIOT or MARA, not to mention the risk of having 100s of employees.

If I had a million bucks to invest in mining, I would go with the small warehouse with $3k/month of overhead, but I would buy reliable equipment like the Whatsminer M30S/Antminer S19 so that labor cost is zero. It is possible to upgrade the power supply past 400A to 1-2MW, but only if the utility company will do it for free/low cost and that area has enough grid capacity. Let's say operating cost remains $3k but power capacity is now 1MW. The overhead is just 0.4¢/kWh. On top of that, there are only 300 ASICs or so, and if they're reliable, only 1 or 2 will break every month. It should take 5 hours a week max to manage this operation. Power cost won't be 3-4 cents like the giga miners, but it can be as low as 5-6 cents in some places at 1MW.

Some small farms only use containers, which are temporary structures, so no property tax is owed.

The residential power rate can easily be double the 1MW commercial rate in some areas, not just 50% more.

What if you made them sign a contract where a piece of the hashrate is diverted to yourself plus the service fee? You would have some insurance if the customer fails to pay their bill every month. If coin prices rise, you get extra revenue so you can at least get compensated for the risk. I wrote software recently that can do this and I'm using it for my own farm.

What if you also had an efficiency requirement, like 40W/TH, so that it's less likely the mining customers want to shut down?

I have to agree. Back in 2017 we started construction on a general purpose datacenter in Oregon rather than a mining specific one. Between 2017-2020, crypto miners were the bulk of our revenue, but it was also the hardest earned and the most inconsistent.

The boom/bust cycle of crypto is difficult for any datacenter to accommodate. Just for historical context, in November 2018 the market dropped below profitability for our customers. Overnight everyone was asking us to turn their machines off, so we did. Unfortunately that also meant nearly 100% of our revenue stopped.

At the time, my colleague argued that if none of our customers were paying, we should evict them, have them collect their equipment and reclaim the space. However it would have been counter productive, there was absolutely zero demand to host new mining equipment. Which was pretty evident as some of our customers tried to sell their equipment with little success.

After 6 months, in May 2019 the price did turn around and all of our customers took control of their machines again. Unfortunately this only lasted 5 months until November 2019 when increasing difficulty forced all of our customers to shut off again. In June 2020 the halving occurred, which pretty much assured that equipment was never getting powered on again.

Over the course of operating 24 months, we only generated a profit for a combined 10 months. The other 14 months with non-paying "idle" customers consumed whatever profit we had managed to make previously and took us negative into the red.

Today we have almost no crypto customers, although we do have a growing number of "traditional" datacenter customers. If we hadn't built the facility with customer diversity in mind we would have perished like everyone else.

Which brings me to my point, the "mining facility" concept has little chance to succeed even with scales 10 MW and beyond. Even when you have low priced power, there are unavoidable overheads. Rent or mortgage payments, construction loans, staffing costs, local taxes, licensing, insurance, accounting fees, property tax, etc.

Obviously there are some economies of scale that can help average these costs out, but we have found that beyond a certain scale the cost actually goes up. There are a lot of cost saving opportunities when you don't have to buy everything new, pay premiums for expediting construction, permits, lawyers, etc.

No matter how you balance it, you always end up paying ~$0.04/kWh in overhead.

- Want to mine at home? Then your stuck paying for retail power with a 50% premium over commercial
- Found a mining facility with super cheap hosting in some remote country? Now your exposed to a very likely risk of losing your equipment.
- Found a warehouse to rent for $2k/month with 400 amps of power? That's $4k/month in power that your paying a 50% premium on thru rent.
- Decided to go big! Get a loan for $3M to build a 10 MW facility. Paying your loan is equivalent to paying a 50% premium on power for 24-36 months.

Ironically, I have been considering hosting mining customers again now that they aren't critical to the success of our business. I just need to find a creative way to more appropriately split the risk between hosting provider and miner.

A month or two ago, some people said that home mining at a rate of 10-12 cents was possible with the most efficient ASIC on the market, which was a Bitmain Antminer S19j Pro. But now, the power cost is half the revenue. Profit is half of what it used to be for that miner.

Now you need at least a Bitmain L7 or Goldshell KD2 for home mining to be viable. The only other option is a GPU rig for altcoins, but even that is not becoming worthwhile for many U.S. households who pay 15-16 cents for power. It's becoming more compelling to just sell the cards and buy the coin, since coin prices are low.

Yes, that would make sense. Where I'm from, its a factor of a few things - garage space is a luxury where 99% of us don't have the luxury of one..

Colo spaces will then make sense. But with the recent macro happenings, home electricity pricing isn't working out either..

I would imagine as the price falls more people like me question running their miners in the garage at residential electric rates and start looking for some place to host their machines instead. Haven’t found any so far only replies that places are at capacity.

Resurrecting this thread..

Very interesting discourse.. I am a miner & colo myself and I totally agree with the "ROI" required. If its too much hassle, we'd rather drop customers. Its easier to deal with 1 customer with 1000 miners than 1000 customers with 1 miner each.

Well at the moment profitability is super high, the only hang-up is that gear is super expensive. 10K to get a single miner these days, but you could run it at 10c power and be only marginally less profitable than the 5c people. So in that sense, it seems there are more possibilities for small operations to start up, just not the home miner that wants to run a miner or two in the basement/garage.

That's the name of the centralization game.  Cheaper and easier always has sacrifices somewhere.  First it was the home miner and they said to find a host.  Then the hosts started dropping customers...  It's a shame nobody has thought of a good way for the network to incentivize decentralization and small miners.  I don't think the blocksize has as much to do with it as the narrative would have one believe.  You saw different pools and companies come and go a lot during the CPU->GPU->FPGA->ASIC evolution which I think helped keep any one operation from grabbing a long term foothold on the market, but I fear that more and more individual miners will only continue to be squeezed out until another evolution hits the reset button and Antpool goes the way of Deepbit (to make a poor analogy).

The answer for me, is to run a small diversified mining operation on renewable energy while trying to stay nimble.

That's just what we have for the servers & networking stuff we use.
Even worse it's @ 110v but we needed rack space for the stuff, none of it is really power hungry and 2KVA is minimum of what they give with racks there. Unless you are taking dozens then it's all negotiable.
The point I was making (poorly) was that if you go into some of these places and take a lot of space and power then it's fairly inexpensive even in a high price / high power cost location like NY.
Asking for 4 racks and a bit of power is a much larger cost for both power and space. To the point of if the paperwork from the other people was accurate, and I have no reason to think it was not, they could have marked up what they are paying by 40%+ and sold the 4 racks / 8KVA to us for less money they we are paying direct.

-Dave

We could host a few s17's (3 or 4)  at our place.  For the winter it would be easy enough. Say til April 1. Or may 1.  After that it gets hot.
What did you pay 8 cents?  We have dead s17 gear that needs repair maybe we can do a deal of some kind.

Both hosts I am/was using were relatively small operations, and the owner/operator of each had lots of their own gear running. You could maybe even call them a mining operation with a hosting side business. If these guys have been stacking sats all this time, after last year they are flush with cash and have plenty of capital to buy machines to fill out their capacity. I know the NY host was paying under 4c for power, so with a 55w/th miner, cost to mine 1BTC is under $10K. If I was looking at those numbers, I'd have done the same thing, boot everyone and bring in all my own new gear. Why settle for a profit of ~4c/kwh for a 1 year hosting contract, when your own 1 gen old miner can make 21c/kwh profit.

I'm not sure about network/server colocation facilities for miner hosting, I would have thought the cost to build out one would be multiples higher than a mining farm. And the power density not as high. Can only run 2 S19s with 8KVA. Maybe if you can find a local one that hasn't built out fully they'd be willing to lease space with only power and allow you to build out your own infrastructure.

What is kind of interesting is at least IMO is that for the server / network hosting kind of thing 3 year contracts and longer are readily available at least here in the US. 3 years does tend to be the best option for pricing in a lot of them.

Looking around at mining stuff it's all month to month or 1yr.
So for mining are these hosting facilities expecting you to fail or don't want to commit for more that that amount of time or

I'm wondering if a lot of them are not "real" hosting facilities, but rather someone who is getting space in a much larger facility then sub-leasing to miners. For what the company I work for pays for 4 racks / 8KVA there is no way we could be competitive to rent space to miners. For what the company in the next area over pays 100+ racks and 900KVA they could (don't leave your paperwork sitting on the table in the common area, I will read it). But they obviously have a much bigger monthly bill then we do....

-Dave

Got a quote for an immersion system including tank, liquid, dry cooler, and pump for a system large enough to handle 28 S17s at 40% overclock. Cost comes in at about $50K. That's roughly 4 months of profit for the 28 S17s at today's conditions.

About half of that cost is the pump + dry cooler. Could probably save >$10K by finding some 2nd hand surplus for those.

Might pull the trigger on it if I can find a good place to run it.

Mercy! I haven't mined in years (2016) so my perspective is totally out of date *apparently that's like 100 in bitcoin years  
It was always as a home/hobby project so it took maxing out my houses ability, increasing difficulty + a cross country move for me to stop "upgrading".

That is definitely not a 'benefit' since it means in any court of law, you own the hardware, not the person sending it to you.

You can explore hosting solution in Georgia (Europe) with us. We at Maas Ltd are creating two mining facilities in Free Industrial Zones in Georgia.

We will be starting our Hosting services from Next Month and have total of 8 MW available which will be scaled to 25 MW by mid-2022. Our pricing is competitive and starts from 0,065 EUR/KWh.

Currently our website is under development (https://maas.ge) though you can email us at [email protected] to inquire more regarding our services.

Some of the benefits of hosting with us:

* NO IMPORT DUTIES (INVOICE REQUIRED TO BE ON OUR NAME, otherwise regular rates of import duties)
* Regulated Entity
* 24x7 Armed Security
* 24x7 Monitoring
* 1 Month deposit only. Discount of 0,005 EUR/KWh on longer contract
* REMOTE HANDS ON DEMAND
* SPARE PARTS READILY AVAILABLE (FANS/PSU/CABLES)
* VPN Access
* MINER REBOOTING

Regards,

Maas Ltd

SETUP YOUR OWN be more expensive but can avoid 1 year BOOT. OR co-exist with your host with higher fees so the host be profitable to keep you.

Being a small player has something to do with it, BUT being seen as unprofitable customer (none sustainable business) may be more likely. If it is a profitable or sustainable cost paying customer, host will probably love you with even 1 unit hosting.

Invest proper cost into your host. Be reasonable with the host for uptime and maintenance performance. Many host seem to be rejecting S17 and T17 who are paying lower hosting fee while these miners required high maintenance and support requests and frequent monthly bill adjustment due to some offline times and maintenance time needed. If host knows you are PAIN IN THE BUTT customer with essentially no gain or possibly the net loss, those customers be the first to go if someone else offer more profitable propositions. Coindesk.com article said it cost publicly traded company to build the mining data center is about $1,000,000 to $1,500,000 per Mega Watt in Texas.

If use the newest S19 as example, it is about 3000W each. Usable 1Mega Watt can service about 330 units of S19. If use 80% load math, about 270 units per $1,000,000 to $1,500,000. This means if hosting customer does not bring $3,030 to $5,555 profit margin, Those newly built data centers probably has 0 interest to offer hosting to us if we have zero intention to pay off that cost in the next few years with long term contract or start with higher hosting fee.

I am sure, there are much cheaper data center host all over the place with less than perfect performance and conditions but it will work with 80% uptime which be acceptable for mining performance. Cost for those data center may be 50% 75% cheaper but the stability may be the trade off.

Special mining container for sale seems to go in the range of $100,000 to $250,000 with about 300-400 units of S19 capacity. This cost seem not include the electrical infrastructure outside,  Electrical transformer to service this may cost $50,000 to $100,000 to include the electrician and material and permit expenses. This means if any lucky hosting company out there that gets free transformer (If small family owned hosting shop may be BUT as industrial size operation, NOT REALISTIC)

Current Covid crisis, the salary seem to be not cheap now days. Owner wants his pay, company needs administrative staffs as well as customer support and technical staffs. That is additional expense on top of the base electricity cost the hosting company has to pay with rent or mortgage of the business location. I speculate, many small size hosting facility who built 5-10 Megawatt size facilities did not have realistic business projection with real operational cost. They are now slowly realizing what the real cost would be as push to expand their facility further. Now days hyper inflation with everything to include the building material cost is getting so expensive, too.

best option be to search for land or building that has enough electricity you need to setup your own container, or find host who will agree longer term service contract with higher monthly fee to be win-win condition.

quick search shows many host offering immediate hosting option from $0.08/kwh but has minimum sign up quantity restriction of 10 units or more. With current bitcoin price and data center demand, $0.1/kwh even seem reasonable, compare to residential or commercial electricity price and warehouse rent to pay if plan to operate less than 1 Mega Watt. Best $5,000 - $10,000 rent with 1000 amp old ice-cream parlor or old rural agricultural site to find near you.

Best solution be set up your mining on your own land it seems or find a host who offers long term stability but to expect higher signup cost or higher monthly fee to avoid early termination or none renewal pain. If you sign up $0.07/kwh hosting now, higher probability of BOOT at the end of the initial contract term or earliest contract termination date. Miner wants ability to cancel hosting contract as short term as possible = you take risk of BOOT when market heats up.

what puzzle me is a guy who spend $100,000 to buy miners from someone on telegram chatroom instead from the store with the brick building. You expect the miner to be usable for 3-4 years or possibly more but only sign up for cheaper short term cancelable contract. When you rent a store space or your office space, you sign up for 3-10 years lease with the landlord.  Why NOT with your expensive mining business enterprise does not do the same?  

Cheaper shorter term contract is similar to 1-year ARM home mortgage loan that you know you have to refinance shortly after the 1-Year ARM term ends or your monthly payment is so painful until you refinance.

Quick google search shows many hosting options but many seem to be reselling larger hosting companies' space with markup, too. I see $0.089/kwh with 10 units minimum. This means, when the distributor price becomes none profitable or distributor demand unreasonable service from the big host, the distributors to be experience the BOOT. Always GO direct with the HOST to lower the risk be 2cents

Many delays are from cost savings. Many difficulties setting up the data center comes from attempt to save cost via engineering, also. Hence, more delay.

Big questions to who are facing possible BOOT. Refused or Decline to pay extra $25-50 usd monthly hosting fee when you are earning $500-$800 usd per month worth it?

Yeah, that's defiantly an issue. Looks like plenty of options are available if you want 1MW plus of capacity. But even at the level, the host could still just decide they want the space and boot you after the contract is done. So buy $3Million worth of miners with a ~10 month ROI, only to have your host boot you after your 12-month contract.... that could hurt. Although it would probably only happen if profitability is high, so the miners could be worth close to or maybe even more than what you paid. All my S17s are worth significantly more than I paid for them 2 years ago (in $, not BTC obviously...).

At that point it's probably better to buy your own container and have that hosted. At least then it would be a little more difficult and costly to decide to boot you when profitability spikes since they'd need to buy their own container and install it.

Would be interesting if someplace would host a mini-container, something with a max of 200kw or so.

GREED - the human nature

miner owner wants the cheapest data center, willing to pay only the getto make shift data center on $10,000 equipment. ASIC miners made in china are known with high hardware failure rate. It is developmental device status and level. ASIC miner is known to be unstable or very sensitive to the environmental changes and it can freeze or may need physical power cycle or some technician's troubleshooting time. When a profit margin per 1 device is about $20-30 usd per month, but receive 5 email requests to reboot the equipment, 3 emails requesting hosting fee adjustment just because the equipment was frozen and not producing hash rate, consume 1 hour of technician's physical labor of dismantling the device and troubleshooting. When 1 device goes offline for 30 minutes, you receive SCAM report online, receive negative complain at BBB and your corporate administrative staffs spend hours communicate with BBB to avoid lower rating to be shown on google search. When BTC price is rising, your data center is near full and require more staffs. Your technician's salary is over $20 per hour, remote customer support agent is $10 per hour. There is not much or no profit margin left.

vs

publicly traded companies offering to buy out your entire hosting capacity with none disclosure agreement that generate 0 public online negative complain when 200 equipment under performing with 2/3 hash card operating, 30 dead equipment exist any given days. You can have a few technicians, only 1 administrative assistance who communicate with the public company to have daily update or small chitchat and the minimum electricity load the data center require to keep consuming 24 hours a day be automatically met. Some big companies are paying short term $50-70 profit margin per equipment to be powered on immediately as well as one time bonus of $100+ usd to get the equipment to be online in a few days of notice.

data center also must have minor improvement to accept newer generation, but the data center can not bill customers for the improvement cost, however, terminating the existing customer and accept new big customers who are willing to pay full facility improvement expenses. To build or expand data center is expensive. Data Center will naturally select customer who will pay for the improvement upfront upon signing up for the service.

If you are the data center boss, what would you pick?

The biggest issue I am seeing with hosting is that we are small players.
With the great miner migration out of China I am seeing data centers that are being taken over by mining operations.
No press, no fanfare, just slowly pushing out those of us hosting servers / routing equipment and the miners moving in.

I guess there is going to be a lot of that in the dedicated miner hosting market.
Would you want to host 5000 miners from 750 different customers or 5000 miners from 1 customer?
You can fire most of your support staff, no dealing with tons of configuration issues.
Get them power, get them cooling and let their staff deal with the rest.

-Dave

when something goes wrong, sadly it can be super catastrophic
highly suggest to use mechanical system to lift the equipment in-out of the liquid and watch your thermostat like a hawk. Inspect item and rehears physical effect on some of the components in mid to long term effect. Do not pinch penny as you may experience $1 loss

great technology to use and super stable with S17 and T17, however, it can turn your millions to a dream of the past 

Had a discussion in discord about it the other day ...
I'd be interested is hearing from someone running one for a while.

There's a bunch of 'not insurmountable' issues I can think of,
but details from someone actually running one with more than just a few miners,
and handling those issues over a period of time would be of most interest.

I'll start a thread if I can compile some useful info. It seems like it should be something that can be DIYed for not crazy $, but just the lower cost dielectric liquid goes for ~$10 per liter.

I've contacted these guys that have been mentioned on the forum before for information, https://www.engineeredfluids.com. They have a bunch of good info and guides on their website.

I'm going to price out a system that will work for 28 S17s.

Sorry to hear the bad news that was brought to you.



I admire your knowledge, I have always been skeptical about the hosting business for many reasons, this one you mentioned being on the top of my list, and then you got nearly the whole world facing power shortage and increased gas/oil prices, it makes things even worse.

By the way, this isn't just you, it has been the "trend" on telegram, many people are facing issues with various hosts all over the globe, I saw someone looking for a new home for a 1000 miners he had somewhere in Russia which the host can't host anymore.



That sounds like a plan, maybe someone should start an " immersion cooling" dedicated topic where we all can collectively collect information about suppliers, the Dos and Don'ts of immersion cooling, and whatnot, something like the difficulty topic we have, let me know what you think.

I just need to send shipping labels to my NY host, they are close so it only costs ~$20 each to ship them back to me.

The Canada host is another issue. They are in a fairly remote city in north-eastern Canada, just the shipping cost is going to be around $150 each. But the bigger issue is the 27% duty since the miners are from China. So I need to either sell them to someone in Canada or pay the duty to get them back. Still might be worth paying the duty though. At least I know these S17s are solid. No issues on any one of them for 2 years. And I'm hoping to be moving in the spring to an area where it looks like I can get cheaper power.

I was bummed when Bitminter and BTC Guild shut down and they didn't even have gear of mine, it was sad to see trusted OGs go.

Maybe you can sell them your hosted gear cheap or are they just going to end up with it?

sad news. This seems to be common occurring now days

Hope you had a good cheap hosting with the NY company so you have already earned handsomely

I have some miners up at Cryptoboreas who announced earlier this year that they will be shutting down and clients will have until April to get their miners out. That was disappointing, and a huge pain in the ass since I'm in USA and would have to pay $1000s in duties to get them back. But at least they gave us plenty of time to get things figured out.

Now I just got an email from another host I use in NY that they will not be renewing my contract and will be shutting down my miners at the end of the month. I've been month-to-month with them for years, but still ... not even 2 weeks notice?

Guess that's just the nature of the business. But when I made the investment in the miners I did not consider the fact that if profitability goes up too far, these hosts will drop you as soon as possible so they can utilize their own capacity. And once you get dropped, good luck finding any place to host your 1 gen old miners in a bull market.

So looks like come April I'll be down to a single S17pro running in my garage, as long as it's profitable to run at 23c/kwh, or until I find a better option. But that option is a lot less likely to be a host. I'm going to try very hard to figure something else out... time to start begging my relatives with cheap power in upstate NY to let me set up on their property... I'm already researching immersion cooling since I know the noise will be an issue. Ugh.