Topic: [ANN Mt.Gox] It’s been an epic few days: What happened? (Read 4015 times)

wow, what a bitch!

I'm guessing the PR in your name is there for irony?

Digging up a slightly old thread...

I was curious if those 57,000+ accounts created recently are legitimite.  Obviously this information would inspire the price to go higher (and I believe it did help up to $266).  I could see it as another way to attack Gox by creating a bunch of false accounts and swamp their customer service.

Just a thought..

And it has been actually designed (correctly) and actually implemented (correctly), by people who actually matter and actually do things (unlike you + friends). Read it, because you must read it. Stop with the posturing, you are not part of the cool kids, you're a dork "working on a soft drink distribution system" with "some friends". It's called Coca-Cola, dummy. It's listed under KO. You're embarassing yourself.

I'd also suggest Tor hidden services, and using private hidden service URLs that are different for each partner. The URL's don't give any information about the network topology, and if any individual URL is compromised and DoS attacked the individual URL can easily be taken down. Even URL's for individual high-volume traders would be feasible, although, keep in mind I'm no Tor expert so someone who knows more should weigh in before doing this. In particular what the Tor developers think of the attention and attacks it may attract.

In some cases using Amazon's infrastructure could work too. Amazon Simple Notification Service is essentially a DoS-resistant broadcast medium. Of course, it's central infrastructure, so not appropriate for every use, but  there is a lot of public information like price tickers that could use it. In any case information should be distributed though multiple methods.


Regardless of what is done, an important first step is to sign and timestamp all public information broadcasts so that regardless of where you got the data, you can verify it as being genuine; the current API does not appear to authenticate pricing information other than via https.

+1 to the private UDP broadcast idea.

Further, there has been talk off and on (by myself, and others) about a "backbone network" where Big Players privately and directly interconnect, for reasons similar to this.

Yup as working as in ITer in a DC, if they want to hit you with DDoS a lot of tricks work, till they either flood the lines or bring down the router of the DC, where you are at. Once they hit that point, there is no stopping it. And even though the real big DC's have a dozen of 10gbit lines, at some point it just stops. (A month back we saw an DDoS we saw a DDoS peak of 250gbit hit... what do you do about it... you null the IP at your carriers. (making the site unavailable))
Though when hackers/botnets are attacking in this type of volume, there is nothing that can really save you, the only way to minimize, is host global, all with loads of overcapacity, high performance routers and heavy duty DDoS firewalls. This is the only way to truly mitigate most of them. (if they attack from US, only the US DC will get hit, and EMEA and Asia will hopefully keep running smooth, same goes for both other regions when under attack)

DDoS is one of the most difficult things to combat, purely because its nature is to flood the network, hence extra capacity would be needed to combat this. (you can't buy 100gbit of network speed when doing 1gbit of traffic just because you might get hit with a DDoS up to 100gbit... and what if they DDoS you with 101gbit?) Seeing what DDoS botnetworks have been capable of, these days you are technically not safe with anything under 2tbit.... (highest I've seen was 1.3tbit) well if your in the hosting business, you know what a 10gbit uplink costs, multiply by 200, subtract 30% a month... while it would only take a botnet of half a million with 20mbit connection to push that same DDoS out. (known botnets have been found with more then 300k bots) And then I'm not even talking about some infected webserver, on a gbit connection to the internet, that can start to DDoS as part of a botnet... (only takes a 1000 of these to get a 1tbit)

Still I think Mt.Gox can do better, DDoS can't always be completely prevent, but bot lagg time in triple digit seconds as shown earlier in the thread... that should've been impossible imo. Get that security ramped up and make sure everything is running efficient. Know you get hit by DDoS on regular basis, and apply as much pre-emptive mitigation methods as possible.

You can run java very lean if you know what your doing and not adding a hundred libraries to it. But this is one of those flame wars that will start cause you like those languages better, in any case what ever language would be better.



Again this is what language you like better, and will probably start a flame war. Also C/C++ if they don't know what they are doing can have massive memory leaks with all that data.

These are my thoughts to an extent.
Down with Walmart. *ren and stimpy stinky face*
Anarchy! 

A high performance trade engine should be written in C/C++, simple as that.

This may work, but you will have to make arrangements with the DC hosting, which is something they may or may not be able to do (from the looks of it, more like not).


Ahaha can I have that engraved on a medallion? 

Why?

I would like the trade engine to be written in Haskell, Erlang or some flavor of Lisp. Just because a functional language makes it easier to keep the code clean of unintended consequences. Also, Java is very bloaty, not a good thing for high performance code.

However, I understand sometimes compromise is necessary.

Yes, UDP stands for Useless Deprecated Protocol, it's that protocol that automatically detects and cunningly removes any information and redundancy you add to transmissions to help an application using it detect and recover from losses of datagrams, thereby making it useless as an upstream data source for scripts.  It is also a protocol that is specialized in conveying data that, by virtue of having traveled via UDP, becomes impossible to republish on a TCP stream for the benefit of being consumed by scripts, to help increase the workload of script writers who of course will want to implement a UDP listener in their scripts directly.  It is commonly known as an unreliable protocol, and it gets this reputation by sneakily altering important data in such a manner where it cannot be made reliable through other methods.  In fact, it even warps the minds of people considering using it, such that they cease to even know what UDP is!

Do you know what UDP is? If they switched to UDP now all PHP/Python/java scripts would be useless and take now even more code to just connect. Also trading bots need reliable connections which UDP doesn't support by any means. UDP is great for the same information broadcast over and over, like said above a radio. Trading shouldn't use UDP it makes it very unreliable. They just need either a better network architecture clearly.

I am not sure that trading bots and DDoS are considered the same problem from the view of the consensus here.

UDP isn't going to solve this problem, instead it make it harder for bots to trade. If they really wanted to solve this issue it is so simple. The trading engine should be ran completely offline, and use a database, like redis to store all information so the rest api can still have access to the information. Also the trade engine should be written in java or python.

Before it comes within miles of the host sending it.  After not informing the public who the UDP is coming from.

The UDP sending address doesn't have to be public knowledge, since not anyone can necessarily subscribe to it.  It would be a private UDP feed only offered to specific known sites.  The UDP feed would be used to drive the services of other sites who currently get it via websocket now, who in turn could provide that data to other downstream TCP websocket clients.

They say reading enriches the imagination.


You are rubbing sticks together trying to solve already solved problems. The many ways available for talking to MPEx.


Of course the clueless knowitalls haven't yet answered the

I guess there simply would not be be anything to broadcast during the times when the engine is down or when nobody can make any trades. It would broadcast only silence. Maybe it could broadcast "Help!!! Help!!! We're under attack!".

Thank you for the explanation and terrific post.


This is absolutely true. Some attacks are more application level (synflood, real HTTP requests), but others are of such a volume where the pipe is saturated. You'd pretty much have to have anycasted datacenters and massive pipes to the Internet to absorb large enough attacks. I've seen 3 Gbit/s attacks to minor sites for no apparent reason. I can't imagine what MTGox gets on a regular basis.

Re, the UDP suggestion. That might not be a bad idea at first glance. It'd work if MTGox advertised the price from some mostly unknown IPs and out different routers, out to a list of subscribers. Another option would be to put this data in DNS, maybe in a TXT or SRV record with a TTL of 60. Then the DNS servers might be attacked, which could be a new problem.

In my opinion, MTGox runs a great site. It's a bit tricky to get onto and the interface isn't as sleek as some sites, but ultimately, MTGox has single handedly encouraged a massive growth of adoption. I think ideally trades should be distributed by nature, but MTGox is still (and probably always will be) the benchmark site for Bitcoin trading, especially in bulk.

My hat is off to these guys for how thorough they are, dealing with the past through days, and 57,000 signups in one month. Those are some real challenges.

Why the lag? It's really not that many transactions. It has to be 90% the procedure / architecture of your engine.

Also +1 to UDP