Bitcointalk running slow/crashing (split from What happened to bitcointalk.org today?)

theymos

administrator

Activity: 5166

Merit: 12850

Quote from: drawingthesun on April 13, 2013, 10:02:25 AM

If not, could you tell us the max that was online, I heard it was 7,500! If so that is amazing.

The max number of online users isn't being tracked anymore.

Quote from: Severian on April 13, 2013, 10:10:21 AM

From what I've seen of your bugs, they're better than some peoples' programs. Thanks for your efforts, t.

Thanks.

Severian

sr. member

Activity: 476

Merit: 250

Quote from: theymos on April 10, 2013, 10:15:58 PM

I wrote my own caching system because I didn't want to use memcached, etc. There may be bugs with this.

From what I've seen of your bugs, they're better than some peoples' programs. Thanks for your efforts, t.

drawingthesun

legendary

Activity: 1176

Merit: 1015

Quote from: theymos on April 10, 2013, 10:15:58 PM

Some changes you may notice:
- The users online list at the bottom of the board index is gone. That was too expensive.

Instead of having all the users, could we just have a ticker showing the amount of online users? I really like that info, think its kinda cool.

If not, could you tell us the max that was online, I heard it was 7,500! If so that is amazing.

John Self

member

Activity: 98

Merit: 10

'twas the goxocalypse

zakoliverz

hero member

Activity: 536

Merit: 500

Quote from: theymos on April 10, 2013, 10:44:56 PM

The forum's search offers additional features that Google can't replicate.

yes this forum is the one and only.

Logik

sr. member

Activity: 315

Merit: 255

Quote from: theymos on April 10, 2013, 10:15:58 PM

- If you request pages faster than once per second too many times, your IP will be temporarily banned.

Can you PM me the exact logic of this, so that I can ensure my scraper doesn't get banned.

justusranvier

legendary

Activity: 1400

Merit: 1009

Quote from: theymos on April 10, 2013, 10:44:56 PM

The forum's search offers additional features that Google can't replicate.

There's a feature that makes it annoyingly difficult to refine a search: if you perform a search and don't find what you're looking for, your only option to proceed is to use the back button in the browser which resets all the fields to their default values.

theymos

administrator

Activity: 5166

Merit: 12850

The forum's search offers additional features that Google can't replicate.

🏰 TradeFortress 🏰

vip

Activity: 1302

Merit: 1042

👻

Thanks theymos! Bitcoin is on the front page of reddit along with various other sites, probably legitimate traffic.

Regarding searching, maybe you could point people to "site:bitcointalk.org search terms" on Google when they are being throttled?

theymos

administrator

Activity: 5166

Merit: 12850

I think that it is legitimate traffic, but there is a ridiculous amount of it: ~250 requests per second. I made several changes to speed up the forum, which seems to have helped.

Some changes you may notice:
- The users online list at the bottom of the board index is gone. That was too expensive.
- Searching is now subject to the same spam protection limits as posting. So newbies have to wait 6 minutes between searches and users with more posts have to wait progressively less time.
- If you request pages faster than once per second too many times, your IP will be temporarily banned.

I wrote my own caching system because I didn't want to use memcached, etc. There may be bugs with this. In the future I may need to add more aggressive caching.

Remember remember the 5th of November

legendary

Activity: 1862

Merit: 1011

Reverse engineer from time to time