Pages:
Author

Topic: Why has this site been going so slow? (Read 4024 times)

legendary
Activity: 1302
Merit: 1007
August 11, 2013, 10:46:38 AM
#68
That's weird. It has been running completely fine for a week already - at least for me.
sr. member
Activity: 336
Merit: 250
Cuddling, censored, unicorn-shaped troll.
August 09, 2013, 10:09:53 PM
#67
Confirmed, sluggy again... If we can help you track down the issue, just shoot instructions away!
Good luck.
legendary
Activity: 966
Merit: 1000
- - -Caveat Aleo- - -
August 09, 2013, 10:00:15 PM
#66
Running slow again this evening.  Sad
legendary
Activity: 858
Merit: 1000
August 09, 2013, 08:57:31 AM
#65
Same, I got several Time-Outs when posting. Loading is for the most part fine, though.
hero member
Activity: 927
Merit: 1000
฿itcoin ฿itcoin ฿itcoin
August 05, 2013, 03:09:54 PM
#64
So far it has been fine.  However, occasionally, the site will fail to load at all for minutes at a time, as if it were offline.  Other times, it's just reliably slow, but now, it's speedy.
Noticed the same on my phone, thought it was just my crappy 3G, a quick refresh brings it straight back usually though.
legendary
Activity: 1078
Merit: 1003
August 05, 2013, 02:36:03 PM
#63
So far it has been fine.  However, occasionally, the site will fail to load at all for minutes at a time, as if it were offline.  Other times, it's just reliably slow, but now, it's speedy.
member
Activity: 111
Merit: 10
August 05, 2013, 02:30:45 PM
#62
Running like a top.
hero member
Activity: 784
Merit: 1000
0xFB0D8D1534241423
August 04, 2013, 08:07:36 PM
#61
It seems to be much faster now most of the time, although I did have a few long loading times today.
legendary
Activity: 2324
Merit: 1125
August 04, 2013, 07:22:32 PM
#60
I improved modSettings. It's only updated once every few minutes now instead of once every 5-10 seconds. Hopefully that'll make things faster.
I'm seeing much better results

Confirmed
hero member
Activity: 574
Merit: 500
August 04, 2013, 07:21:02 PM
#59
Yes. Good again. Thanks
legendary
Activity: 2786
Merit: 1031
August 04, 2013, 04:25:37 PM
#58
Just read a bunch of pages, smooth surfing!

Much better.
legendary
Activity: 966
Merit: 1000
- - -Caveat Aleo- - -
August 04, 2013, 11:45:48 AM
#57
Yes.  Quicker today.
hero member
Activity: 927
Merit: 1000
฿itcoin ฿itcoin ฿itcoin
August 03, 2013, 08:56:21 AM
#56
I improved modSettings. It's only updated once every few minutes now instead of once every 5-10 seconds. Hopefully that'll make things faster.
I'm seeing much better results
legendary
Activity: 1302
Merit: 1007
August 02, 2013, 03:47:19 PM
#55
I improved modSettings. It's only updated once every few minutes now instead of once every 5-10 seconds. Hopefully that'll make things faster.

I'm sorry to be the bad news messenger, but it has not improved anything on my side.
I also got a few whole blank pages, with no error, but this was just before your post, so I'm guessing it's related to your fix deployment.
Still the same for me.
sr. member
Activity: 336
Merit: 250
Cuddling, censored, unicorn-shaped troll.
August 02, 2013, 02:56:14 PM
#54
I improved modSettings. It's only updated once every few minutes now instead of once every 5-10 seconds. Hopefully that'll make things faster.

I'm sorry to be the bad news messenger, but it has not improved anything on my side.
I also got a few whole blank pages, with no error, but this was just before your post, so I'm guessing it's related to your fix deployment.
administrator
Activity: 5222
Merit: 13032
August 02, 2013, 02:17:46 PM
#53
I improved modSettings. It's only updated once every few minutes now instead of once every 5-10 seconds. Hopefully that'll make things faster.
Vod
legendary
Activity: 3668
Merit: 3010
Licking my boob since 1970
August 01, 2013, 11:42:02 PM
#52
Or maybe the SSD disks storing the backends are getting worn out? Nearly random and multi-second delays on writes are common symptoms when using flash SSD drives to store databases with write-ahead-logging, which is the worst wear pattern for them: many writes and no reads.

Is there any way to test for this? I thought that SSDs were built to last several years, even under extremely heavy load.

SSDs were built to last for so many write cycles.  If your hard drive is almost full, then the same sectors will get re-written faster, making them fail faster.  SSD capacities are determined giving some lee-way (a 100GB drive might have 120GB of space) for sector failures.

When using SSD's you should make sure your drives never become full - always have lots of free space.
legendary
Activity: 1246
Merit: 1077
August 01, 2013, 10:42:57 PM
#51
Or maybe the SSD disks storing the backends are getting worn out? Nearly random and multi-second delays on writes are common symptoms when using flash SSD drives to store databases with write-ahead-logging, which is the worst wear pattern for them: many writes and no reads.

Is there any way to test for this? I thought that SSDs were built to last several years, even under extremely heavy load.

I do not think a hardware problem should be investigated until all software problems are ruled out. Try attaching counters to suspect functions. Don't bother with making a SQL table for them; just make a bunch of flat files (possibly mounted on RAM, but shouldn't matter that much), one for each function. All that's needed are two calls to system time, a subtraction and addition, and a file read/write (possibly to tmpfs)—all should be extremely cheap, and the data is invaluable.
legendary
Activity: 2128
Merit: 1073
August 01, 2013, 02:57:32 PM
#50
Is there any way to test for this? I thought that SSDs were built to last several years, even under extremely heavy load.
They can last years if you have a decent mix of reads and writes for the flash technology. Typical write-ahead-log is a worst case write-amplification pattern: writes 512-byte blocks, forces physical sync/flush to the drive, which in turn forces full page (about 4096-bytes) relocate and erase in the flash controller. And to add the insult to the injury that data is never read unless the database or the OS crashes.

I don't know of any way to check it while the disks are online, besides continuously monitoring the S.M.A.R.T. parameters with smartmontools and hope that the "old-age" statistics shown by them aren't lying.

The folks I know would simply run a full backup, full low-level security erase of the flash drives and then full restore. The security erase may help fix the bottlenecks in the wear-leveling software in the flash controller that is causing those random delays.

But they also aggresively use the warranties on the flash drives, preemptively returning them before the warranty expires. This is only workable if one orders such drives in quantities large enough to compile some sensible wear statistics.

Due to the extremely competitive nature of this business it is quite hard to get true and correct information about the wear-leveling algorithms used and their interaction with file systems. I'm sorry about not being able to be more specific: on one hand there are NDAs, on the other hand the flash controller firmware changes very frequently.

To get flash SSD last several years you'll need to mix them with normal spinning drives. Since database logs are almost exclusively sequentially written and almost never read they won't bottleneck the whole machine.

Edit: If you are willing and able to rebuild your kernel there ware some patches that use ATA TRIM and optional bitwise negation to significantly reduce the flash wear on some controllers. The frequent operation of "allocate disk blocks sectors and initialize to zeros" is replaced with simply "erase block" which initializes flash to all 0xFF and then stores all data complemented. Folks who created and maintained those patches would presumably know for which controllers they are worth applying and maybe if there are other custom patches for similar circumstances. The other strings worth seaarching may be "Deterministic Read Zero after Trim" DZAT, which basically refers to the same idea implemented closer to the actual hardware.

Edit2: I keep forgetting about the partition table issue. Many OS installers continue to create the "compatible" disk partition layout with integral number of "cylinders" (C/H/S == x/255/63). With 255 "heads" per "cylinder" and 63 "sectors" per "track" you would be assured that the partitions have odd sizes and alignments. Therefore the most common I/O operation 4kB is assured to cross into 2 physical 4kB sectors. Various optimizations then become ineffective and they automatically disable itself for the sake of safety. This is related, but non identical, to the "long sector" "advanced format" on the mechanical drives. Just make sure that you didn't accidently set your system into "512 byte sector emulation" (512e) mode.
member
Activity: 73
Merit: 10
August 01, 2013, 12:18:52 PM
#49
The site is slowing down from all the avalon customers that have it on auto-refresh awaiting word from Yifu. 

Or maybe that's just my contribution  Cry
Pages:
Jump to: