The coming change at mtGox and its implications

hgmichna

hero member

Activity: 695

Merit: 500

Quote from: myself on April 01, 2013, 09:29:45 PM

Quote from: molecular on March 31, 2013, 03:46:22 PM

Quote from: myself on March 31, 2013, 01:51:28 PM

well in stock and forex u cant advertise that you want to buy or sell a currency or a stock if you dont freaking owned it first

what about naked short selling?

you cant do that no broker allow you to do that

Outside bitcoin it is normal within some reasonable margin, so I'd expect it to become normal for bitcoin too, as soon as the bitcoin market is a bit more stable and mature.

myself

legendary

Activity: 938

Merit: 1000

chaos is fun...…damental :)

Quote from: molecular on March 31, 2013, 03:46:22 PM

Quote from: myself on March 31, 2013, 01:51:28 PM

well in stock and forex u cant advertise that you want to buy or sell a currency or a stock if you dont freaking owned it first

what about naked short selling?

you cant do that no broker allow you to do that

hgmichna

hero member

Activity: 695

Merit: 500

Quote from: shmadz on March 31, 2013, 10:59:57 PM

Not familiar with Gox.

Over at virtex, every bid or ask that you place puts that money into escrow. so if you don't have the money (ex. if I want to buy right now, but all my money is tied up in lower bids) I have to cancel some bids to free up the money.

From the sounds of it, this is not the way it works at Gox?

Is the escrow system just dumb? or is it something that Gox will have to implement to get around this problem? […]

From what I have read here, Mt.Gox allows orders without complete funding. In my view this is not a problem, unless users heavily overuse this feature. But Mt.Gox could protect itself against this easily, as they have the whole account history and could easily put up a simple rule that prevents the overuse of this feature.

As I have mentioned before, their real problem is the large number of tiny nuisance orders. I think they should first demand a small fixed fee for each order or for each transaction to get rid of micro-orders.

shmadz

legendary

Activity: 1512

Merit: 1000

@theshmadz

Not familiar with Gox.

Over at virtex, every bid or ask that you place puts that money into escrow. so if you don't have the money (ex. if I want to buy right now, but all my money is tied up in lower bids) I have to cancel some bids to free up the money.

From the sounds of it, this is not the way it works at Gox?

Is the escrow system just dumb? or is it something that Gox will have to implement to get around this problem?

(was thinking of trying Gox out, once coinlab takes over Can/US payments, since virtex % is ridiculous (they started out at 0.6% - now they're up to 3% for low volume accounts)

molecular

donator

Activity: 2772

Merit: 1019

Quote from: myself on March 31, 2013, 01:51:28 PM

well in stock and forex u cant advertise that you want to buy or sell a currency or a stock if you dont freaking owned it first

what about naked short selling?

myself

legendary

Activity: 938

Merit: 1000

chaos is fun...…damental :)

well in stock and forex u cant advertise that you want to buy or sell a currency or a stock if you dont freaking owned it first

dancingnancy

hero member

Activity: 661

Merit: 500

Quote from: molecular on March 31, 2013, 11:38:30 AM

Quote from: hgmichna on March 31, 2013, 11:11:14 AM

Not allowing users to place an order if they do not have the funds at the time the order is placed seems wrong to me. The order needs the funds only when it is executed, not before. Consider that Mt.Gox is there to fulfill its users' needs, not the other way around. Provided, of course, that the users pay appropriately for the service.

The user could have a combination of orders that works, even though not all orders are funded beforehand. There could be sufficient funds coming in before an order is executed. I see no need to check funding before an order is actually executed. I would not complain if they did that though. They can do what they like, as long as they don't alienate their users.

I do see a need though for a fixed base fee for transactions, like every other online broker has. If necessary, there could even be a base fee for placing an order or a punishment fee for trying to execute an unfunded order. It seems that Mt.Gox is wasting their limited computer power on lots of tiny nuisance orders. The whole bitcoin system is not designed for large numbers of tiny transactions. There is a block size limit, for example. Bitcoin is not a micro-payment system. This will become ever more obvious as bitcoin trade volume grows.

I see the need to check funding of order before they are displayed in the public order book. Unfunded order should not be in the book because they cannot be fulfilled.

True. This is what bitcoinica was doing as well, IIRC. You could post anything you wanted to the order book, but it was another thing if you had the money in your account. Seems entirely messed up, not really sure what to think about all that.

molecular

donator

Activity: 2772

Merit: 1019

Quote from: hgmichna on March 31, 2013, 11:11:14 AM

Not allowing users to place an order if they do not have the funds at the time the order is placed seems wrong to me. The order needs the funds only when it is executed, not before. Consider that Mt.Gox is there to fulfill its users' needs, not the other way around. Provided, of course, that the users pay appropriately for the service.

The user could have a combination of orders that works, even though not all orders are funded beforehand. There could be sufficient funds coming in before an order is executed. I see no need to check funding before an order is actually executed. I would not complain if they did that though. They can do what they like, as long as they don't alienate their users.

I do see a need though for a fixed base fee for transactions, like every other online broker has. If necessary, there could even be a base fee for placing an order or a punishment fee for trying to execute an unfunded order. It seems that Mt.Gox is wasting their limited computer power on lots of tiny nuisance orders. The whole bitcoin system is not designed for large numbers of tiny transactions. There is a block size limit, for example. Bitcoin is not a micro-payment system. This will become ever more obvious as bitcoin trade volume grows.

I see the need to check funding of order before they are displayed in the public order book. Unfunded order should not be in the book because they cannot be fulfilled.

hgmichna

hero member

Activity: 695

Merit: 500

Not allowing users to place an order if they do not have the funds at the time the order is placed seems wrong to me. The order needs the funds only when it is executed, not before. Consider that Mt.Gox is there to fulfill its users' needs, not the other way around. Provided, of course, that the users pay appropriately for the service.

The user could have a combination of orders that works, even though not all orders are funded beforehand. There could be sufficient funds coming in before an order is executed. I see no need to check funding before an order is actually executed. I would not complain if they did that though. They can do what they like, as long as they don't alienate their users.

I do see a need though for a fixed base fee for transactions, like every other online broker has. If necessary, there could even be a base fee for placing an order or a punishment fee for trying to execute an unfunded order. It seems that Mt.Gox is wasting their limited computer power on lots of tiny nuisance orders. The whole bitcoin system is not designed for large numbers of tiny transactions. There is a block size limit, for example. Bitcoin is not a micro-payment system. This will become ever more obvious as bitcoin trade volume grows.

SkRRJyTC

legendary

Activity: 1008

Merit: 1000

Quote from: oakpacific on March 30, 2013, 08:42:26 AM

Are there people actively taking advantage of this deficiency? Gox is lagging even if there are only several orders being placed in total, WTF?

Would be silly not to take advantage, since it is extremely easy.

Le Happy Merchant

hero member

Activity: 634

Merit: 500

Quote from: oakpacific on March 29, 2013, 04:43:30 AM

If I just did a calculation with my ass, it's gonna make the complexity roughly O(m*log(n)) instead of O(log(n)), and m could be easily larger than log(n).

I suspect there are many badly-programmed bots relying on this out there.

If m > log(n) > 1 this is bad for the complexity.
If 1 > m > log(n) this is good for the complexity.

What are you trying to say?

oakpacific

hero member

Activity: 784

Merit: 1000

Are there people actively taking advantage of this deficiency? Gox is lagging even if there are only several orders being placed in total, WTF?

bitcoinBull

legendary

Activity: 826

Merit: 1001

rippleFanatic

Quote from: Enigma81 on March 29, 2013, 06:14:10 AM

Re-Calculating an accounts open-orders after each transaction is just NOT that big of a deal. If the site was designed by anyone with even the remotest sense of distributed computing, it would be such a non-issue that this discussion wouldn't even be taking place. A properly designed trading system could handle the current load of bitcoin trading and still twiddle it's electronic fingers 75% of the time.

Scaling a database which needs guaranteed atomic consistency is hard. Nasdaq had a "flash crash" when it couldn't handle the load, and BATS botched their IPO when they debuted a new back-end and their APPL prices went haywire. They had to suspend trading and roll back trades.

I am disappointed that mtgox still lags, but I am glad that MagicalTux is doing more extensive testing. I remember when we couldn't go more than a month without seeing some mismatched rogue trades show up on bitcoincharts outside of the trading range with crazy volume, and not knowing whether they were real trades until they were deleted and removed from the charts. And rememember when trading was "halted" once, for about 12 hours, when someone sent an invalid currency pair to the API? Its more important to prevent those kinds of mishaps than to do realtime processing of peak volume (though i agree, we are paying enough for both).

foo

sr. member

Activity: 409

Merit: 250

This new system is how Bitstamp works already, so you can get a preview of it there.

Enigma81

full member

Activity: 180

Merit: 100

This whole things is either
A ) Complete bullshit
or
B ) A sign of the level of inept that is MtGox

For God's sake - It's NOT that damn hard. It's just not. I promise. They do not need to limit the ability of their customers to do what they can do now (place currently unfunded orders). When I traded at Gox, I did it all the time. If I thought the price might fall, place a sell at -5% and then a buy (unfunded) at -10%.. Or something of that nature. It's a nice "feature".

Re-Calculating an accounts open-orders after each transaction is just NOT that big of a deal. If the site was designed by anyone with even the remotest sense of distributed computing, it would be such a non-issue that this discussion wouldn't even be taking place. A properly designed trading system could handle the current load of bitcoin trading and still twiddle it's electronic fingers 75% of the time.

I'm not a huge Oracle fan.. But jesus Gox.. Call them.. Or IBM.. or one of the other heavy-metal DB-App specialists.. They'll be happy to politely explain how it works.

SOOOOO glad I left Gox. Yeah, the reduced liquidity is a bit of a bummer.. But when things start rolling, I place a trade and its done... No 19 Minute "LAG" shit..

Enigma

molecular

donator

Activity: 2772

Merit: 1019

Quote from: ?? on ??

Quote from: Zomdifros on March 29, 2013, 05:17:09 AM

Quote from: molecular on March 29, 2013, 04:57:16 AM

Quote from: Zomdifros on March 29, 2013, 04:56:38 AM

With the transition of North American customers to Coinlab, are they still going to use the same trading engine (in Tokyo?)? If all these orders will be done on a new engine in the US, this could drastically reduce the pressure on the current system. If so, when will this transition happen?

I can't imagine them running 2 markets. That doesn't make sense.

I guess not, but when Coinlab customers are able to get their dollars on the trade floor within 30 minutes through Silicon Valley Bank, this is gonna weigh on their engine. Then I can imagine that MagicalTux really has to work on a tight schedule. I wish him all the best pulling this of!

They can (and should) just maintain a central order book where all orders present are sure do be fully covered (see my previous comment), and have the two "brokers" independantly insert orders into it.

This (and your previous description about maintianing 2 balances) is effectively what this change is about. No need to demand it, Magicaltux is on it. Wether or not he will use your implementation idea is up to him.

Zomdifros

full member

Activity: 210

Merit: 100

Quote from: molecular on March 29, 2013, 04:57:16 AM

Quote from: Zomdifros on March 29, 2013, 04:56:38 AM

With the transition of North American customers to Coinlab, are they still going to use the same trading engine (in Tokyo?)? If all these orders will be done on a new engine in the US, this could drastically reduce the pressure on the current system. If so, when will this transition happen?

I can't imagine them running 2 markets. That doesn't make sense.

I guess not, but when Coinlab customers are able to get their dollars on the trade floor within 30 minutes through Silicon Valley Bank, this is gonna weigh on their engine. Then I can imagine that MagicalTux really has to work on a tight schedule. I wish him all the best pulling this of!

molecular

donator

Activity: 2772

Merit: 1019

Quote from: damnek on March 29, 2013, 05:06:18 AM

Quote from: Hawkix on March 29, 2013, 05:01:17 AM

Quote from: molecular on March 29, 2013, 04:37:53 AM

Last night I finally understood what's with the gox engine lag:

Quote from: #mtgox

...
molec: we need to change the way users place orders to solve this
MagicalTux, in what way?
disallow users to place orders they can't fill
MagicalTux, +1. oh...
MagicalTux, it's dawning on me
and disallow users to withdraw balance required for open orders
MagicalTux, you have to trickle down the whole order list to set invalid/open status
MagicalTux, go do it!
right now each time a user balance change (trade, etc) we need to recheck all his open orders
...

I think that with proper data structures, one can link together the bids and asks of single user and immediately update "reserve" for the bid/ask after sell or buy. Funding and withdrawing are infrequent compared to trades. The structure can be held in memory, with commits to permanent storage. This should handle hundreds of trades per second, not a few, like it is now.

Of course, this requires to avoid using normal relational databases, queries upon them, PHP code in server, etc.

Also, the socket with trade data is sometimes overloaded, too. I am not a TCP/IP expert, but I cant see problem with cloning the data between multiple (in order of thousands) sockets open. This would solve all those disconnects on ClarkMoody, MtGoxLive, etc.

An exchange, which cannot provide reliable and non-lagged data for its users, is .. unusable. Like that graph at the top of mtgox.com. Isn't it ironic that to trade at MtGox, or just watch it, you need to use third party tools?

Absolutely agree with you. I've been long enough to know that bitcoin and exchanges are not the same thing but I can't help to think that this market got too big for what the infrastructure really is. It's just not there yet.

And that is exactly the reason why this change on the infrastructure is being made.

It's debatable wether or not this could be done with the same order behaviour. mtGox decided to change the order behaviour to make the engine perform better. That's ok with me as long as it actually solves the issue and I know about it beforehand.

molecular

donator

Activity: 2772

Merit: 1019

Quote from: Hawkix on March 29, 2013, 05:01:17 AM

An exchange, which cannot provide reliable and non-lagged data for its users, is .. unusable. Like that graph at the top of mtgox.com. Isn't it ironic that to trade at MtGox, or just watch it, you need to use third party tools?

According to MagicalTux (and I can actually believe what he says there), it's not the socket.io/websocket that lags. It's actually "realtime", it's the engine that can't cope with the amount of trades.

molecular

donator

Activity: 2772

Merit: 1019

Quote from: oakpacific on March 29, 2013, 05:00:59 AM

We need to pay attention especially to a rapid price movement scenario, when people will be trying to issue orders desperately, and may never bother to check on their balances, and Gox's UI irresponsiveness only adds salt to the injury, as people will try to issue more orders when they can't see their orders showing on the list. The more orders people issue, the greater the lag, which would make people more anxious and making more vain efforts, which in turn causes even more lag. This vicious closed-feedback loop causes the ridiculous 10-minutes lag we saw on a weekly basis.

The idea is that this change will do away with the lag problem. So the UI will hopefully be responsive.

Topic: The coming change at mtGox and its implications (Read 5079 times)