All three OC experiments failed. Both of the other units I tried 250/0750 with looked fantastic for the first hour, then slid into slow decline without any symptoms other than declining hashrate. They bottomed out at about 430 after 12 hours. Reverting to stock and taking a break!
I am not by any means disputing your results as that would be futile, however if I may point out my experience with S3's, it is very rare that you find any two units hashing the same, even when using similar freq & voltage settings, PSU and connected to the same pool.
In your case however, that you have experienced the same wall of performance after the first hour, and that happening on most of your units, leads me to believe the issue must be local to your setup.
My first instinct was a heat / temperature problem, however, you seem to discount this as a possibility, and that the units run as normal on stock frequencies seems to justify that. I am not completely sold though since I have never run an S3 variant at temps as high as yours. Before the season change and my moving my rigs outside, I had them all running the fans at full pelt, aka blue wire hack, which kept the temps low, and now that the seasons have changed and my rigs reside in the garden, my temps are a lot lower.
My second guess is how you are powering your rigs. Though you appear to have the right PSU's, this may be a cause if you have them all plugged into the same extension lead. They may not trip your circuit, but they will compete for power, especially if you are running them off a 110v circuit.
If you choose to attempt the OC again, I wish you luck, but I think in your instance I have given as much help as I possibly can, bearing in mind I am neither an engineer nor an electrician.
Thanks for trying to help! I am both an engineer and an electrician. I do not think I have power issues, as all 6 units are running from Corsair 750 or 800W supplies, with all four PCI-E connectors well seated. Three power supplies are on one 20A 120V circuit, and three are on another. So there is plenty of spare power.
I think what is happening is a slow overheating. Interestingly, the S3s do not appear to respond by increasing the fan speeds, instead the hash rate goes down as we have discussed. All six units are in a well ventilated space that stays around 18C. The S3s report temps around 39-42C, so not very high.
If I find the time, I may try redoing the paste and/or maxing the fan speeds to see if that helps. I will also keep an eye out for any S3 firmware updates that fix something. For now, I have concluded that my six units are not overclockable other than the slight bump I get from setting the frequencies to 225/231/243/237/225/237 for the six.
Very frustrating to see such promising hashrate increases in the first hour, but then nothing sustainable. But overall I am impressed with the units, they are all working to spec and running just fine.
