...It then gets set to 211 on kano...
...I've got around 24.7 THs...
Are you talking "211" for initial diff? or running diff?
According to the logs, 211 was set as the initial diff after the zpool disconnect and failing over to kano:
2016-08-25 23:03:15,975 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.pool.Pool - Trying reconnect of pool zpool in 15.0 seconds.
2016-08-25 23:03:16,015 INFO [192.168.80.101-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New WorkerConnection 192.168.80.101 subscribed. 1 connections active on pool Kano.
2016-08-25 23:03:16,020 INFO [192.168.80.102-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New WorkerConnection 192.168.80.102 subscribed. 2 connections active on pool Kano.
2016-08-25 23:03:16,227 WARN [192.168.80.102-Thread]:strat.mining.stratum.proxy.network.StratumConnection - Unknown request type on connection 192.168.80.102. methodName: mining.multi_version, id: 56912, params: [1]
2016-08-25 23:03:16,234 WARN [192.168.80.101-Thread]:strat.mining.stratum.proxy.network.StratumConnection - Unknown request type on connection 192.168.80.101. methodName: mining.multi_version, id: 60712, params: [1]
2016-08-25 23:03:21,962 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - Set difficulty 211.0 on pool Kano.
2016-08-25 23:03:21,967 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - Accepted share (diff: 211.0) from
[email protected] on Kano. Yeah !!!!
Moments later kano's pool starts rejecting shares at that diff:
2016-08-25 23:03:31,438 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.pool.Pool - Testing stability of pool zpool for 30 seconds.
2016-08-25 23:03:31,447 WARN [Pool-zpool-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - Pool zpool is UP.
2016-08-25 23:03:31,527 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - Set difficulty 64.0 on pool zpool.
2016-08-25 23:03:31,530 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New block detected on pool zpool.
2016-08-25 23:03:35,710 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - Accepted share (diff: 211.0) from
[email protected] on Kano. Yeah !!!!
2016-08-25 23:03:37,584 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - Accepted share (diff: 211.0) from
[email protected] on Kano. Yeah !!!!
2016-08-25 23:03:38,646 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New block detected on pool zpool.
2016-08-25 23:03:38,824 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - Accepted share (diff: 211.0) from
[email protected] on Kano. Yeah !!!!
2016-08-25 23:03:46,273 INFO [Pool-Kano-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - REJECTED share (diff: 211.0) from
[email protected] on Kano. Booo !!!!. Error: JsonRpcError [_code=null, message=null, traceback=null]
In around a minute from the disconnect event, the proxy determined that zpool was stable and pointed back at it, so kano's pool never got an opportunity to ramp up. Zpool set the diff back to 64 (ouch) and it starts getting rejects due to "Invalid job id" errors:
2016-08-25 23:04:01,448 WARN [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.ProxyManager - Pool zpool is STABLE.
2016-08-25 23:04:01,450 INFO [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.strategy.MonoCurrentPoolStrategyManager - Switching worker connections from pool Kano to pool zpool.
2016-08-25 23:04:01,452 INFO [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.strategy.MonoCurrentPoolStrategyManager - Close connection 192.168.80.101 since the on-the-fly extranonce change is not supported.
2016-08-25 23:04:01,458 INFO [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.ProxyManager - Worker connection 192.168.80.101 closed. 1 connections active on pool Kano. Cause: Change extranonce not supported.
2016-08-25 23:04:01,460 INFO [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.strategy.MonoCurrentPoolStrategyManager - Close connection 192.168.80.102 since the on-the-fly extranonce change is not supported.
2016-08-25 23:04:01,464 INFO [StratumProxyManagerSeverSocketListener]:strat.mining.stratum.proxy.manager.ProxyManager$1 - New connection on /0.0.0.0:3333 from /192.168.80.101:49375.
2016-08-25 23:04:01,471 INFO [TimerExecutorThread-3593]:strat.mining.stratum.proxy.manager.ProxyManager - Worker connection 192.168.80.102 closed. 0 connections active on pool Kano. Cause: Change extranonce not supported.
2016-08-25 23:04:01,477 INFO [StratumProxyManagerSeverSocketListener]:strat.mining.stratum.proxy.manager.ProxyManager$1 - New connection on /0.0.0.0:3333 from /192.168.80.102:53840.
2016-08-25 23:04:01,517 INFO [192.168.80.101-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New WorkerConnection 192.168.80.101 subscribed. 1 connections active on pool zpool.
2016-08-25 23:04:01,521 INFO [192.168.80.102-Thread]:strat.mining.stratum.proxy.manager.ProxyManager - New WorkerConnection 192.168.80.102 subscribed. 2 connections active on pool zpool.
2016-08-25 23:04:03,276 WARN [192.168.80.102-Thread]:strat.mining.stratum.proxy.network.StratumConnection - Unknown request type on connection 192.168.80.102. methodName: mining.multi_version, id: 57067, params: [1]
2016-08-25 23:04:03,278 WARN [192.168.80.101-Thread]:strat.mining.stratum.proxy.network.StratumConnection - Unknown request type on connection 192.168.80.101. methodName: mining.multi_version, id: 60886, params: [1]
2016-08-25 23:04:06,562 INFO [Pool-zpool-Thread]:strat.mining.stratum.proxy.worker.StratumWorkerConnection - REJECTED share (diff: 64.0) from
[email protected] on zpool. Booo !!!!. Error: JsonRpcError [_code=21, message=Invalid job id, traceback=null]
Zpool started accepting shares after setting a diff of 128. Took about 8 minutes to ramp up to the full difficulty (lower than before the disconnect because one blade had fallen over at some point during the above thrashing).
Note that I've changed both zpool and kano to have a minimum difficulty of 13722. Hopefully this will help with stability moving forward.