Yeah, that's a really good idea! I'll toss my S3s behind it too to soak up the low-difficulty work. I wonder what the optimum ratio would be. Obviously optimum is for AM to actually build functional hardware and offer customer support, but barring that. I wonder if the stratum proxy allows per-worker difficulty? So I can set the Prisma worker as 1024 or higher, and let the S3s float. Or is the proxy even smart enough to do QoS and distribute the higher difficulty shares to the Prismas, while relegating everything lower to the S3s?
I've used Slush's stratum_proxy before, but I like bfgminer better, not just because of its ties to Eligius, but because it offers a lot more functionality and settings I can tweak. We're looking to perform load-balancing on specific pools with varying percentages of the overall hashrate, and also looking to feed each worker with different difficulty work.
Re: "distributing" higher difficulty shares.
You are thinking about it wrong. All Bitcoin mining is the same difficulty to begin with. It's no harder to hash a block with a given nonce than any other given nonce. It's only the results that can be given a 'difficulty score' after the fact.
It's like if you are forced to roll dice all day. Say you are told to roll a single die over and over again, and write down your results. But the person only wants you to report rolls of 5 or above, and to discard rolls of 4 or less. So your "dice rolling difficulty level" is 5, and your taskmaster can extrapolate how many times you actually rolled, period, from how many times you rolled 5 or above. (By simply tripling the reported number in this simple case.)
But it's no more "difficult" for your hand to roll a 1 or a 6. It's the same process either way. "Pool difficulty" in Bitcoin is simply a filtering mechanism so that you aren't reporting every single result to the pool, in order to save computing resources and bandwidth. So what the proxy does is actually strip away the 256-difficulty results instead of passing them on to the pool. It only lets the 1024-difficulty (or higher) results pass.
Okay, so perhaps I'm misunderstanding how vardiff works, I thought you could request specific difficulties from the server, or it would provide them until you stopped solving them in a reasonable period of time, and then only issue blocks that are at or below your most reasonable share submission rate. So couldn't the proxy request blocks for which the pool has determined they will represent a higher difficulty? I didn't realize diff was calculated after "the dice roll", I thought it had something to do with how the work was parted out by the pool. It doesn't seem that the Prismas handle Extranonce2_size correctly and when it's too low, it just drops it for some reason, and so the pool never decides to send the higher difficulty shares.
From:
https://www.btcguild.com/new_protocol.phphttp://support.bitcoin.cz/Knowledgebase/Article/View/2/1/how-does-vardiff-workBut I am very much a novice with all this!
Vardiff is just the pool scaling the minimum allowable difficulty for your miners to report to it. It is based on your hashrate, and scales up based on results from a low baseline.
For example, say your initial connection to the pool is at difficulty 16. (I think that Eligius did this earlier this year... not sure what the current initial/minimum difficulty is now.) So the pool only wants you to send back shares that meet the difficulty threshold of 16. You have some pretty fast miners, so you send in a bunch of difficulty 16 (or higher) shares to the pool very quickly. The pool says "woah woah woah, this is too much" and scales up your difficulty to 128. So now you are only sending shares of difficulty 128 or higher. This scales you to the rate that the pool wants you to report (e.g. a few times per minute).
So you are mining along at this rate and then you add a couple of new, powerful miners to your setup. These new machines are kicking out difficulty-128+ shares every second or two. So the pool scales you up again to difficulty 1024. Now you are sending difficulty-1024+ shares a few times a minute.
I used the dice-rolling analogy for a reason. The pool has no idea what the difficulty a given piece of work will be before you crunch it, just as you have no idea how a dice will roll before you roll it. There is no way for the "blocks for which the pool has determined they will represent a higher difficulty" to exist... if they did, then the pool would have already done the hashing and would have no reason to give it to you! Of course, if you have a high hashrate then the pool will give you larger chunks of work to process (bigger nonce range) so you don't run out too fast, but that's different from the difficulty setting.
A nonce is just a random string of characters. The way that hashing works, a change of 1 bit in the input results in greater than half of the bits of the output flipping. So a block with nonce 000000001 will give a hash that looks completely different than the block with nonce 000000002. (More than half of the hash string will be different.) The one with nonce 000000001 might have a difficulty of 1 while the one with nonce 000000002 might have a difficulty of 10 billion. And it is impossible to tell beforehand. That's the nature of hashing (and, BTW, what makes the whole thing secure).
Vardiff is just determining what level of difficulty results will be reported/'shared' back to the pool. If the pool thinks that you are at difficulty 256 but your miners are set at difficulty 1024, then you will be underpaid by 1/4 because you are only reporting about 1/4 of the work that you're doing. If the pool wants you at difficulty 1024 but your miners only go to 256, then (I guess?) the pool will cut you off because it doesn't want you spamming with 256 difficulty shares. (Might be dependent upon specific implementation of the pool.) When you "request a difficulty from the pool" then you are simply requesting it to set your minimum difficulty at a given level. It has nothing to do with the work that you are given.
All of that being said, you don't really have to understand all of the above in order to fix your problem. IMHO Setting up a proxy is the way to go here. It will act as a filter for your miners and hopefully handle the vardiff negotiations correctly. But others will be better informed than I. I never reached the scale of mining where I had to worry about this stuff.