Hopefully this information is helpful.
Something I've noticed is that most of the rejected shares I receive on your pool are not stales as everyone seems to be claiming. They may be listed as round stales under My Stats; but they are not. I have 4 rigs and I run both cgminer and bfgminer. When running on your pool all four rigs are seeing an increased unknown work rejection. I have tried running theses rigs on several different pools and never see this error. On your pool my unknown work rejects have hit as high as 25 percent in a round.
I don't claim to be knowlegable of any of these systems and software; but, I am a capable troubleshooter in my day job. As a trouble shooter I try to break problems down into sections to solve them. The fact that my rigs work else where eliminates one of the sections and puts the problem at your end.
Dividing your end into two sections to start:
1- Your server is sending my rigs work that is not legitamate work. My miner solves and sends the work back and it is rejected as unknown work as it should be. It shows up as a stale under My Stats and is rejected.
or
2- The work your server sends is legitimate work. My end solves correctly and sends in the work. Your server rejects this legitimate work. Maybe its the DDos filters. Maybe some other error. It shows up as a stale under My Stats and is rejected.
I hope the above information somehow points you in the right direction. If it was legitimate stales it wouldn't be much of an issue. I'm running on a Stratum pool right now and get some stales there too. Spending time working on Stratum when the problem may lay elsewhere will gain very little for the time being. I'm looking forward to this problem being solved. I like the excellent work you've put into your pool.
Thank you for that, it got me thinking about the block boundary issues we've seen in the past and I realized that the litecoind process that is supposed to send a HUP to pushpool might only be reading the PID file at startup.
I restarted litecoind, which caused it to re-read the PID file, and the stales have dropped significantly now. I thought it would just pick up changes to the PID file as they occur. So now I know that anytime pushpoold is restarted... I have to also restart litecoind. At least until I can alter the custom patch to re-read it on each block boundary.
We still have other issues. Now that I've found this one I need to set the difficulty back where it was. I'd changed the difficulty thinking it'd help matters, but I think in reality it made them a percent or two worse than they were before.
The other thing I can do is I can change the values around for how long work is considered current. Right now you only get two minutes from the time you get the work to when you submit it. If I make that 3-5 minutes then some of the cpu miners will probably see lower stales.
Cheers.