Holy F***, what is happening? Most Users Online today since 1 and 1/2 Years!

mufa23

legendary

Activity: 1022

Merit: 1001

I'd fight Gandhi.

Quote from: 420 on January 11, 2013, 05:18:06 PM

Quote from: the_thing on January 11, 2013, 05:04:08 PM

It may have something to do with the fact that Moot accepts Bitcoins for his 4chan Pass and says so under every post form. Thus millions of people see the word Bitcoin everytime they write a post on 4chan. CES is nothing compared to that.

well how long has that been there

Late November 2012, I think? I doubt it's related. I remember the passes being on sale, with Bitcoin, right before the holidays.

wtfvanity

hero member

Activity: 504

Merit: 500

WTF???

Quote from: ShadowOfHarbringer on January 13, 2013, 06:18:20 AM

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

He already stated he would eventually move to nginx if apache became the bottleneck.

Think about what servers you are on now.

https://www.ovh.com/us/dedicated-servers/

OVH allows huge bandwidth, you can get a full 100 Mbps line, intel e3 1245v2 processor with 2x SSD 120GB drives for under $100 a month. That would be a heck of a DB server. Mirror raid that for read performance and data security. Have that sit behind the main server as the DB server, and toss another one out in front with nginx as the main server. for $180 a month... you've got a solid set up. Each server has 32 GB of ram in it.

Their virtual rack price is a little bit expensive compared to most solutions (But their servers are much cheaper) Virtual rack you can communicate between the two servers on a local lan and not use the public bandwidth and shield the DB server a little bit. Local connection is at a gig. It appears even without the virtual rack though, that you can communicate with local IPs to other servers. Have not tested that and not as secure as a virtual rack.

If you're hosting with softlayer, you are probably already paying for a similar processor for about $200 a month, and tapped at 5 TB bandwidth with a fraction of the ram.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: rasputin on January 13, 2013, 08:51:51 PM

The right approach to handling load spikes on a forum like this would be full-page caching in the web layer

It is a good idea, however it will not be enough.
There should also PHP-side cache based on Memcached to offload MySQL/MariaDB servers.

I don't even use Varnish and still I can serve hundereds of requests per second on single server. You can get performance increase of thousand percent and more. Been there, done that.

Check this out:

Code:

$ mysqladmin status
Uptime: 3154789 Threads: 16 Questions: 3738552556 Slow queries: 112796 Opens: 27041477 Flush tables: 1 Open tables: 1482 Queries per second avg: 1185.040

This is a single Quad Core server, it contains MariaDB, NGINX, Apache and PHP - all on a single server. It serves total of ~1.400.000 views/day (when you count google/yahoo bots).
And i still think i can squeeze out much more speed using NoSQL.

Also, my MariaDB tables contain ~15GiB of data, with total of 65 million records.

theymos

administrator

Activity: 5222

Merit: 13032

Quote from: farlack on January 13, 2013, 02:34:52 PM

So is there any word on where the hits came from?
Can admin let us know where the hits came from?

reddit and Hacker News.

furrycoat

newbie

Activity: 56

Merit: 0

We need more servers!

rasputin

newbie

Activity: 14

Merit: 0

The right approach to handling load spikes on a forum like this would be full-page caching in the web layer, using a caching reverse proxy in front of whatever webserver you use now. I do not know the guest:logged in user ratios here, but you could probably get away with caching only guest contents (probably less headaches that way).

Look at Varnish, which is written specifically for this job, or at Nginx which should also work. Here is an example and some numbers for you.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: cedivad on January 13, 2013, 08:40:15 AM

Quote from: ShadowOfHarbringer on January 13, 2013, 07:11:48 AM

Quote from: cedivad on January 13, 2013, 06:51:31 AM

Quote from: ShadowOfHarbringer on January 13, 2013, 06:18:20 AM

Quote from: camolist on January 11, 2013, 03:29:04 PM

Quote from: repentance on January 11, 2013, 02:46:14 PM

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software? Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract) (for sale in this thread https://bitcointalksearch.org/topic/litespeed-web-server-2-cpu-owned-licenses-122271)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

I use XtraDB instead of the standard InnoDB. I serve 40 req/s med and 100 req/s top on a single server. It heavily depends on the patter of the visits.

MariaDB contains XtraDB (it replaces InnoDB "drop in", plus it is binary compatibile, or at least MariaDB devs claim so)

farlack

legendary

Activity: 1310

Merit: 1000

So is there any word on where the hits came from?
Can admin let us know where the hits came from?

cedivad

legendary

Activity: 1176

Merit: 1001

Quote from: ShadowOfHarbringer on January 13, 2013, 07:11:48 AM

Quote from: cedivad on January 13, 2013, 06:51:31 AM

Quote from: ShadowOfHarbringer on January 13, 2013, 06:18:20 AM

Quote from: camolist on January 11, 2013, 03:29:04 PM

Quote from: repentance on January 11, 2013, 02:46:14 PM

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software? Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract) (for sale in this thread https://bitcointalksearch.org/topic/litespeed-web-server-2-cpu-owned-licenses-122271)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

I use XtraDB instead of the standard InnoDB. I serve 40 req/s med and 100 req/s top on a single server. It heavily depends on the patter of the visits.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: cedivad on January 13, 2013, 06:51:31 AM

Quote from: ShadowOfHarbringer on January 13, 2013, 06:18:20 AM

Quote from: camolist on January 11, 2013, 03:29:04 PM

Quote from: repentance on January 11, 2013, 02:46:14 PM

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software? Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract) (for sale in this thread https://bitcointalksearch.org/topic/litespeed-web-server-2-cpu-owned-licenses-122271)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

cedivad

legendary

Activity: 1176

Merit: 1001

Quote from: ShadowOfHarbringer on January 13, 2013, 06:18:20 AM

Quote from: camolist on January 11, 2013, 03:29:04 PM

Quote from: repentance on January 11, 2013, 02:46:14 PM

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software? Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract) (for sale in this thread https://bitcointalksearch.org/topic/litespeed-web-server-2-cpu-owned-licenses-122271)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: camolist on January 11, 2013, 03:29:04 PM

Quote from: repentance on January 11, 2013, 02:46:14 PM

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software? Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract) (for sale in this thread https://bitcointalksearch.org/topic/litespeed-web-server-2-cpu-owned-licenses-122271)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

cedivad

legendary

Activity: 1176

Merit: 1001

Quote from: theymos on January 13, 2013, 01:05:02 AM

25% of queries are delivered from the query cache, which seems pretty good.

http://www.mysqlperformanceblog.com/2011/04/10/should-we-give-a-mysqlquery-cache-a-second-chance/

Quote

Over last few years I’ve been suggesting more people to disable Query Cache than to enable it. It can cause contention problems as well as stalls and due to coarse invalidation is not as efficient as it could be.

25% of queries delivered from cache doesn't necessarily means better performances.

I don't know about how optimized the indexes are for this decade-old board, but i'm sure that you would see a huge benefit converting to a 2013 board (xenforo is my bet, IPB of VB if you wish) and using InnoDB.

theymos

administrator

Activity: 5222

Merit: 13032

25% of queries are delivered from the query cache, which seems pretty good.

exxe

full member

Activity: 187

Merit: 100

Quote from: cedivad on January 12, 2013, 02:29:23 AM

I have no words to describe how dumb its to enable the query cache on such a website. Every time there is an update to a table the whole cache is invalidated (removed from the ram). Every time someone access the website the new cache is created, to be invalidate one second after.

From dev.mysql.com: "When tables are modified, any relevant entries in the query cache are flushed." Not the whole cache ... but maybe I'm too dumb to understand your point of a complete cache invalidation when someone just accesses the forum.

However, I would suggest to: Enable it; Check and optimize the hit ratio; See how it performs; Disable it if it makes things worse

Cheers!

cedivad

legendary

Activity: 1176

Merit: 1001

Quote from: theymos on January 11, 2013, 09:05:39 PM

Quote from: exxe on January 11, 2013, 05:57:21 PM

Is mysql query cache enabled? AFAIK it is disabled by default and can really improve DB performance.

You can check with a SQL query:

Code:

Show variables like 'query_cache_size';

(0=disabled)

Thanks, I'll try enabling that!

Disable it.

I have no words to describe how dumb its to enable the query cache on such a website. Every time there is an update to a table the whole cache is invalidated (removed from the ram). Every time someone access the website the new cache is created, to be invalidate one second after.

It's counter productive. It only works with websites with no/infrequent upgrades.

If you are tiny on budget (why the hell are you using softlayer/rackspace?) and you want to improve the database performance, convert everything to InnoDB and set innodb_buffer_pool to your free ram. I suppose that the free ram should be bigger than the size of the database. If not, buy a new dedicated. Even an ovh.com/us should do, you can buy 2 monster servers for the price you pay now and setup a failover to withstand "poor" hardware iusses.

I could do that for you, I enjoy doing these things, but I'm sure you wouldn't trust me enought. So please study and fix this.

Phinnaeus Gage

legendary

Activity: 1918

Merit: 1570

Bitcoin: An Idea Worth Spending

Quote from: molecular on January 11, 2013, 06:06:21 PM

tbh: the forum is blazingly fast at the moment. I had experienced some minor delays loading threads (maybe 1-2 seconds) earlier today, but nothing serious.

How bad is/was it for others?

Most of yesterday, and some this morning, I was getting that Google warning page that states the page took too long to load. I was blaming my computer. Now I have to apology to her.

420

hero member

Activity: 756

Merit: 500