Author

Topic: Wall Observer - MtGoxUSD wall movement tracker - page 546. (Read 1811709 times)

donator
Activity: 2772
Merit: 1019
You've never written a relational database application with millions of rows of data that is queried, inserted and updated at a high rate and that needs to be (pseudo) real time, have you?

you're mistaken. I designed and now maintain 2 such dbs. One of them has tens of millions of rows in some tables (this one is mssqlserver on some 2 year old 2 x quadcore). The other has millions of rows in one of the main tables. I run quite complicated search and update queries on that one (based on http requests). Performance simulations showed >40 of the most complicated search/update/insert_into_multiple tables combos per second on. While production is deployed on ok hardware (single quadcore), these performance tests were done on a fucking single ATOM 350 (dualcore) with nfs-mounted volumes (!!). No oracle magic either, just simple mysql innodb.

For reference: mtgox aggregated order book (acquired by fulldepth api call) currently has about 10,000 entries. Can't be more than 50,000 individual orders in the orderbook.

Maybe I'm missing something, but I still think I could do it on my nexus 4.


Anyone knows how Gox moves people's deposits around? If you use http://www.bitcoinmonitor.com/ , you would notice that whenever a trade is executed, a lot of transfers take place on the blockchain, in principle it should be just bookkeeping, no real transaction should happen.  I don't know why and if it has anything to do with their sluggishness.

Anyone who could answer my question?

No. The trade engine doesn't concern itself with the blockchain
hero member
Activity: 518
Merit: 500
I blame Mt.Gox lag for the lack of stickyness of $45! Grrr
newbie
Activity: 12
Merit: 0
If disk is gox's bottleneck, get some damn SSDs already! Yesterday's profits alone would buy a multi-TB RAID

IF Gox is using a Relational Database that will be their bottleneck, I'd hope they are using a more sensible database system though, there are many non-relational systems much more suited to the job.
donator
Activity: 2772
Merit: 1019
I thought the guys at Gox manually audit every trade before it goes through after they got hacked? That's why there is a delay (assuming what I heard is true).



I wrote that once somewhere as a joke.
qwk
donator
Activity: 3542
Merit: 3413
Shitcoin Minimalist
Gox's trade matching engine is fundamentally broken.

Agreed.


They need to either solve this scalability issue by combining multiple DB operations in a single transaction OR (potentially as a stopgap) change their fee structure to discourage sub-dollar transactions.  Especially since it seems like these bids are being used to slow down and obstruct normal market price discovery.  In other words, if you put 10000 .00001 BTC bids at 44 (say) and it takes Gox about 1/10th of a second to process a sale, you've managed to put a 1000 second time wall between 44.1 and 43.9.

Aside from a technical solution, i'd also propose the much quicker to implement, economic solution:
Fixed fee per transaction, 0.001 btc per bid or something like that.

They could even allow bots without fixed fees but skip their transactions once there's too much traffic.

I already suggested that in 2011 and nobody ever listened to me Angry
Well, why would they?
hero member
Activity: 616
Merit: 502
....
In other words, if you put 10000 .00001 BTC bids at 44 (say) and it takes Gox about 1/10th of a second to process a sale, you've managed to put a 1000 second time wall between 44.1 and 43.9.
This is similar to the vulnerability. Hmm, I did not know about it, I think now they have to change a few things on their website ... Undecided
hero member
Activity: 616
Merit: 502
If disk is gox's bottleneck, get some damn SSDs already! Yesterday's profits alone would buy a multi-TB RAID
No! There are significant issues (problems) of data recovery (using SSD drives)
legendary
Activity: 1260
Merit: 1000
Drunk Posts
If disk is gox's bottleneck, get some damn SSDs already! Yesterday's profits alone would buy a multi-TB RAID
legendary
Activity: 1246
Merit: 1010
Gox's trade matching engine is fundamentally broken.  Anyone who has worked with databases under real load knows that as load rises you need to chunk multiple operations in a single transaction since a transaction commit involves a disk seek and write (10ms) and a network operation (to replicate the transaction to the backup(s)).  But Gox persists in matching executing and committing every single tiny btc txn independently (or so it looks from the feed).  

They need to either solve this scalability issue by combining multiple DB operations in a single transaction OR (potentially as a stopgap) change their fee structure to discourage sub-dollar transactions.  Especially since it seems like these bids are being used to slow down and obstruct normal market price discovery.  In other words, if you put 10000 .00001 BTC bids at 44 (say) and it takes Gox about 1/10th of a second to process a sale, you've managed to put a 1000 second time wall between 44.1 and 43.9.
hero member
Activity: 616
Merit: 502
hero member
Activity: 784
Merit: 1000
Hehehe. $44 is dead. Long live $45.

Seems to be hotting up, I wonder how long it will be until MtGox goes bad again.

Not very long at all, it seems. Bitcoinity already has a 25 second lag warning.

There it is... I swear I could reach across the ocean and slap Karpeles across the back of the head!

And when you get back you will find out you trade is just about to be executed. Wink
legendary
Activity: 1106
Merit: 1001
Hehehe. $44 is dead. Long live $45.

Seems to be hotting up, I wonder how long it will be until MtGox goes bad again.

Not very long at all, it seems. Bitcoinity already has a 25 second lag warning.

There it is... I swear I could reach across the ocean and slap Karpeles across the back of the head!
hero member
Activity: 518
Merit: 500
Hehehe. $44 is dead. Long live $45.

Seems to be hotting up, I wonder how long it will be until MtGox goes bad again.

Not very long at all, it seems. Bitcoinity already has a 25 second lag warning.
legendary
Activity: 2097
Merit: 1071
Hehehe. $44 is dead. Long live $45.

Seems to be hotting up, I wonder how long it will be until MtGox goes bad again.
sr. member
Activity: 252
Merit: 250
After the huge dump, it feels easier to convince friends to invest long term. Expectations just got raised.  Smiley
hero member
Activity: 518
Merit: 500
Hehehe. $44 is dead. Long live $45.
hero member
Activity: 1302
Merit: 502
The fiat is back.

How do you mean?



I really don't understand why anyone would sell more than a handful of BTC at a time at the current price.  There is such obvious upwards pressure and willingness to buy at higher prices.

Oh well. Long live the $40 dollar range!

The graph I linked shows the total amount of fiat on the order book and the total amount of Bitcoin on the order book.
The spike down on the end of the blue line shows yesterdays panic, the liquidation took a large chunk out of the available fiat. The fiat was feeding the rally, and it's back to all time high territory.

legendary
Activity: 3122
Merit: 1538
yes
Its slow today in price. May dip to high 30-ies again for some profit taking.
legendary
Activity: 1400
Merit: 1013
legendary
Activity: 1036
Merit: 1000
The price action today was in line with the exponential growth since January. For the first time, the price started growing even faster than that exponential trendline, and it quickly overcorrected to below the exponential trendline, then back onto the trendline. It simply got overexcited, then reverted to the norm.

We remain almost uncannily dead-set on that exponential curve that puts BTC at $50 by late March and $100 by the end of April.
Jump to: