Big post, but I'll try to cover everything, because I think you bring up great questions.
I know the little money you are making monthly doesn't add up to the headache you receive on here. At the end of the day the people that are complaining the most are only paying you 30 dollars per month and try to hold you accountable as if they were paying you 5000. We who get it want to thank you very much because I don't want to manually mine alt coins.
The entitled attitude is always annoying, but I've been on the internet long enough to take it with a grain of salt
The bonus income is great, but you're right, its not nearly enough to do this full time, or hire someone to babysit it 24/7. At MC's 3% or so, its significantly closer, at 1% (minus like half for fees at exchanges), its just not quite there.
Is there any way to protect us from the impact the whales have on the pool?
I'm doing my best to not treat him/(them?) any differently. I've had communication with him over the past few days, and any time he sees something that looks odd (server might be down, etc) he's let me know, but the majority of my decisions have not been based on what whales would prefer, but would be best for the pool overall.
Again this was posted by another person but I never saw you respond. I would greatly appreciate a response to this if possible:
I have to say, I lol'ed reading that because it is true. I mined on MiddleCoin for a long time, the hash rate spiked in early January and H20 went nuts making changes to accommodate and eventually the entire pool suffered and became unprofitable. I split my workers between MiddleCoin and WafflePool and WafflePool won so I switched everything over and was the highest MH/s *38MH/s* until the whale showed up. Since then it is eerily similar to the pre-collapse of MiddleCoin.
Lets me see if I can summarize:
1) WafflePool had vardiff, mined lower difficulty coins, and was profitable in comparison.
Separating the three of them:
Vardiff is awesome, it really is. We get good estimates of hashrates, miners always see shares submitted, etc. The down side of it, is a massive amount of overhead on the server side. We need to track all of the miners, their recently submitted shares, and their difficulties separately (memory). And then every 15-30 seconds or so, we need to calculate some averages (cpu time), push them to the clients (bandwidth), and then constantly check to make sure they're obeying them. The change from vardiff --> 512 was more of a necessity at the time. Tons more hashrate came on (and actually, this is _way_ more influenced by a large number of small miners, than by 1 big one - as we track it on a per connection level, high-hashrate single connection is tons lighter than 500 small guys), and our load spiked, bandwidth spiked (which caused orphaned blocks increase), and the simplest answer at the time was to hard-lock difficulty. After doing it, we noticed our server requirements are about 1/4, our orphan rate dropped significantly (2-3%), and overall there wasn't a _ton_ of backlash (some is to be expected, and I had to explain variance repeatedly).
Mining lower difficulty coins:
This is a major thing, and something that I've almost got implemented (and have tested a few times to no avail). So the issue we're running into with smaller coins, and a larger hashrate is essentially a fundamental limit of networking. The main issue we've faced (and I _assume_ what MC and others have faced) is what I've called (to myself) "internal orphans". These are times that we find a block on one of our clusters (say USWest for example), and find a 2nd block on a separate cluster (EU) quicker than we could get the data from USW --> EU. Essentially EU doesn't know there is a new block yet, and has continued working, and found its own. We now have 2 competing blocks for the same height, and one of them is guaranteed to become orphaned. This all happens in roughly 500ms. From what I've seen, we have about 200ms delay from cluster to cluster, a 30-50ms delay in our processing (daemons, blocknotify, etc), 200ms latency to miners, and an additional some latency for work to start on those miners (50ms).
This becomes more of an issue with lower difficulty coins. For something like dogecoin, where even if we put our full hashrate on it, we have an average block time of around 4.5 minutes, half a second on that means a fraction of a percent of chance of internal orphans. However, for something smaller, say leafcoin (difficulty 13 at time of this post), we have an average block time of 3.5 seconds. Half a second of latency means roughly 15-20% of our blocks have an internal orphan. The correct long term solution (planned), is to divide up hashrate depending on profitability (if a coin is 50% more profitable, a 20% orphan rate is fine), and based on server geography. The short term (implemented) solution is to penalize the larger small coins (10-20 difficulty), and to disable the very small coins (under 10 difficulty).
Profitability:
See above for the most part
2) The whale (and collapse of MiddleCoin) caused the network and server load to spike.
As mentioned above, at least on the vardiff side (which was 90% of it), it was smaller miners jumping on, rather than 1 big miner.
3) The difficulty was locked to 512 (same as MC which eventually raised it to 1024).
See above again
I think its the right answer (at least from my side). Server load, bandwidth, etc all make sense to me. The only downside I can see is smaller miners and a misinterpretation of variance, which definitely has its place, but at this point, there are bigger issues for me to deal with.
4) Lower difficulty coins have been removed (same as MC). MC eventually just settled on LTC and DOGE and ended up not being a "mining the profitable coin" pool because the size of the pool couldn't support that model.
See above again
We've got a short term solution in place that prevents us from losing 20-30% of our hashrate in orphans. And we have a long-term plan, once we can finish the algorithms and get some testing done.
5) Aholes from MC thread have come to this thread and will probably eventually drive WP insane and he'll lurk in to the shadows like H20 did.
Yep, its like a welcome party to the internet.
I've never complained - not about MC or WP but I do think we can learn from history here and see the writing on the wall. I prefer the openness and effort of WP 100000x more than H20/MC so I'm going to stick it out but I think its clear there has to be a shift in direction.
I appreciate your time asking these questions! As I know you know, this came up quickly, and a lot of it came down to "fix it now" or die quickly and watch everyone sprint for the next pool. I made some decisions that I think are the right ones at least in the short term (disabling coins, 512 difficulty), but for most of the ones that I had more than a few hours to get a solution in place, I've tried my best to post here the why's and why nots for different options. Or at least post why I've picked a given option.
A large pool is a sledgehammer on small coins and generates a massive number of orphans but much of the profit can be had in the small coins. I think the opportunity here is in bringing back the small coins because all large pools will eventually come to the conclusion the orphans are too high and lock them out. If you could round-robin rotate the workers to the small coins in comparison to the coins total MH/s(GH/s) then we could mine them where other pools cannot.
I've got something better up my sleeve. I'd prefer not to say the exact plan, as I think its probably better than what other pools are doing on this front, and while I don't think its a huge advantage, it could be 5% or so, and I'd rather keep that
I don't care if my workers hit the small coin everytime its profitable everytime so long as they get the chance. I'd rather hit 40% of the opportunity than 0%.
I know the whale brings you a lot of money so I'm not advocating blocking him out but that whale is going to find that DOGE and the larger difficulty coins are all they'll ever end up making profits from because every pool they go to will have the same challenges. A round-robin worker coin assignment would work for them too though - you could drop in 200-300MH/s of their 6GH/s in to the smaller coins just like the rest of us.
WP: Is something like this doable and if not how do you think you can overcome the challenges of pool size vs coin size (profitability)?
Yep! I've got a good plan here (at least I think so - I've been wrong before), but it will take some time to implement.
Essentially since launch, my days have gone like this: Wake up, realize something is broken (or close to broken), spend 12 hours fixing it, eat dinner while watching charts/logs and think to myself "wow, tonight might be the first night everything works fine overnight, I fixed all the problems, and everything looks good". Get close to bed time, realize something is broken again, coffee, fix fix fix, pass out. Wake up to 30 angry posts as to why things were broken in the middle of the night and I wasn't actively replying (was fixing them). Repeat. And its literally been that, for about 10 days straight now
I've got some stuff that I'd love to get added (above, as well as simple things like per-worker stats - that we should have had weeks ago), but things just keep popping up, and I get an hour a day that I can either reply to people here, or work on the new bits of code.
Its slow going, but I think overall we're doing well (hopefully the fact that we're a large pool now attests to that), and we have a ton of room for growth. This went from a "hey, I could build one of those", to a 16hr/day job in under a week, so theres going to be growing pains
