Author

Topic: Redis Down on Bitcoinica (Read 1511 times)

hero member
Activity: 770
Merit: 500
You're fat, because you dont have any pics on FB
January 12, 2012, 11:09:43 AM
#11
You might want to consider having staff on call at all times, as well as failover hardware.  Operating on a shoestring budget isn't going to cut it if you want people to continue to trust you with thousands of dollars.  I'm not upset about the outage, and you responded very well given the situation.  However, one must always work towards reliability when dealing with such large sums.

I agree! Bitcoinica is a mission critical project now. I'll consider your suggestion. Thanks!

I gotta hand it to you Zhou..   your professionalism is something you should be proud of.

Good work, keep it up..
vip
Activity: 490
Merit: 502
January 12, 2012, 08:58:34 AM
#10
You might want to consider having staff on call at all times, as well as failover hardware.  Operating on a shoestring budget isn't going to cut it if you want people to continue to trust you with thousands of dollars.  I'm not upset about the outage, and you responded very well given the situation.  However, one must always work towards reliability when dealing with such large sums.

I agree! Bitcoinica is a mission critical project now. I'll consider your suggestion. Thanks!
legendary
Activity: 1904
Merit: 1002
January 12, 2012, 08:54:15 AM
#9
You might want to consider having staff on call at all times, as well as failover hardware.  Operating on a shoestring budget isn't going to cut it if you want people to continue to trust you with thousands of dollars.  I'm not upset about the outage, and you responded very well given the situation.  However, one must always work towards reliability when dealing with such large sums.
vip
Activity: 490
Merit: 502
January 12, 2012, 08:52:32 AM
#8
Site completely down for 41 minutes. The worst outage ever!!!
vip
Activity: 490
Merit: 502
January 12, 2012, 08:49:41 AM
#7
The Redis server went out of memory due to huge queues of flood of orders.

I have rebooted and the site will be up in 1 minute.

Thank you for the notification and I apologize for the error!

I see the site, but prices are way low compared to what I see from mtgox data.

It was temporary and not tradable.

Cause confirmed: Recently Mt. Gox API had a lot of issues and our Redis server has accumulated all sorts of errors until the memory is full.

You should really have a monitoring system in place to notify you if memory usage is growing without bound.

I have new relic to alert me whenever the site is down or any server has issues. However, I'm in the airport waiting to board my plane. That's why I had such delays just now. So unfortunate.
legendary
Activity: 1904
Merit: 1002
January 12, 2012, 08:41:35 AM
#6
The Redis server went out of memory due to huge queues of flood of orders.

I have rebooted and the site will be up in 1 minute.

Thank you for the notification and I apologize for the error!

I see the site, but prices are way low compared to what I see from mtgox data.

It was temporary and not tradable.

Cause confirmed: Recently Mt. Gox API had a lot of issues and our Redis server has accumulated all sorts of errors until the memory is full.

You should really have a monitoring system in place to notify you if memory usage is growing without bound.
vip
Activity: 490
Merit: 502
January 12, 2012, 08:40:39 AM
#5
The Redis server went out of memory due to huge queues of flood of orders.

I have rebooted and the site will be up in 1 minute.

Thank you for the notification and I apologize for the error!

I see the site, but prices are way low compared to what I see from mtgox data.

It was temporary and not tradable.

Cause confirmed: Recently Mt. Gox API had a lot of issues and our Redis server has accumulated all sorts of errors until the memory is full.
legendary
Activity: 1904
Merit: 1002
January 12, 2012, 08:30:55 AM
#4
The Redis server went out of memory due to huge queues of flood of orders.

I have rebooted and the site will be up in 1 minute.

Thank you for the notification and I apologize for the error!

I see the site, but prices are way low compared to what I see from mtgox data.
vip
Activity: 490
Merit: 502
January 12, 2012, 08:27:39 AM
#3
The Redis server went out of memory due to huge queues of flood of orders.

I have rebooted and the site will be up in 1 minute.

Thank you for the notification and I apologize for the error!
hero member
Activity: 770
Merit: 500
You're fat, because you dont have any pics on FB
January 12, 2012, 07:54:58 AM
#2
Here too.. Sad
legendary
Activity: 1904
Merit: 1002
January 12, 2012, 07:52:55 AM
#1
Connection refused - Unable to connect to Redis on redis-1.bitcoinica.net:6379

Wake up Zhoutong!
Jump to: