Pages:
Author

Topic: [BETA] MTGox websocket API, testers wanted - page 2. (Read 77671 times)

full member
Activity: 217
Merit: 100
September 13, 2013, 03:38:45 PM
Well Socket.io and Websocket have been down for a while, for all of us it seems.

Except for this guy, older API version or something? Or maybe just lots of HTTP requests?:

http://bitcointicker.co/

EDIT: curiosity got the better of me, wireshark caught this:

GET /socket.io/1/websocket/qRpvu5AQFZvju4nIao6m HTTP/1.1
full member
Activity: 217
Merit: 100
sr. member
Activity: 379
Merit: 250
wait a second, now you have to subscribe on your own?
They really changed the default behavior when on connect you were subscribed automatically?

hero member
Activity: 548
Merit: 502
So much code.
full member
Activity: 217
Merit: 100
And another one (after this I'll just start shouting Bueller?): I don't seem to be able to subscribe to both depth and trade messages. If I do, I get depth but not trade. I can repro this in <50 lines of python; is this normal behaviour?

Normal for us two maybe...

Im trying to get websocket stream working (coded on top of vanilla C socket) and I seem to have same problem?

I am getting lag, depth and ticker but no trades. Huh

One error message comes back, related to the failing trade subscription.

Code:
!!!THANKS TO PROF7BIT FOR SUBCRIPTION INFO IN EARLIER POST!!!

04:25:30 MtGox_Socket::SocketCreate: CONNECTED TO: websocket.mtgox.com:80
04:25:31 MtGox_Socket::DoHandshake_WebSocket: GOOD: NowListening
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "trade.lag"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "ticker.BTCUSD"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "depth.BTCUSD"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "trades.BTCUSD"}

04:25:33
{
    "message":"Unknown channel requested or not a public channel",
    "op":"remark",
    "success":false
}
.....
Now lag, depth and ticker JSON comes streaming in
.....
newbie
Activity: 12
Merit: 0
And another one (after this I'll just start shouting Bueller?): I don't seem to be able to subscribe to both depth and trade messages. If I do, I get depth but not trade. I can repro this in <50 lines of python; is this normal behaviour?
newbie
Activity: 12
Merit: 0
Thread necro! Is there a limit on the number of times you can call private/idkey? My bot asks for it at startup, which is many times a day during development, and towards the end of the day (perhaps past midnight, if that has anything to do with it?), I start getting "invalid call" back instead of the key.

It's trivial enough for me to store the idkey and the time it was requested, but I wanted to know if anyone else has seen this behaviour, or if it's just Broken On My Machine?

Ta.
newbie
Activity: 12
Merit: 0
That works for me, thanks! It's really helpful that the documentation is full of lies  Roll Eyes
hero member
Activity: 938
Merit: 500
https://youengine.io/
1. Once the connection is established I subscribe to the USD trades channel:
Code:
TRADES = "dbf1dee9-4f2e-4a08-8cb7-748919a71b21"
def subscribe( channel ):
output = { 'op': 'mtgox.subscribe', 'channel': channel }
return json.dumps( output )

ws.send(subscribe(TRADES))
...but I get the remark spat back "Unknown channel requested or not a public channel".

Try "mtgox.subscribe" instead of "subscribe" and/or also try "type" instead of "channel" and/or also try using the channel name instead of the GUID.

The documentation seems incomplete or outdated, you need to experiment a bit.

For example this works for me: {"op":"mtgox.subscribe", "type":"trades"}
And also: {"op":"mtgox.subscribe", "channel":"ticker.BTCUSD"}

And for subscribing to the private channel (after receiving my idkey) I'm using the following:
{"op":"mtgox.subscribe", "key":idkey}
newbie
Activity: 12
Merit: 0
Well, it seems that the diagnosis for at least one of these two issues is "I'm an idiot" Smiley I've edited my post above with an update.
hero member
Activity: 938
Merit: 500
https://youengine.io/
2. My websocket connection keeps dropping, for no reason that I can see. This thread's history has some talk about having to send heartbeat messages,

websocket should never disconnect at all, I have it running here 24/7 and it disconnects only maybe once or twice a day, some days it doesn't disconnect at all (and probably my few disconnects here are only due to my very poor wifi connection).
newbie
Activity: 12
Merit: 0
Folks,

I've been reading this thread with interest as I too am writing a client in Python (using websocket-client). I'm connecting over plain old websockets (not socketio) and am mostly working apart from a couple of issues:

1. Once the connection is established I subscribe to the USD trades channel:
Code:
TRADES = "dbf1dee9-4f2e-4a08-8cb7-748919a71b21"
def subscribe( channel ):
output = { 'op': 'mtgox.subscribe', 'channel': channel }
return json.dumps( output )

ws.send(subscribe(TRADES))
...but I get the remark spat back "Unknown channel requested or not a public channel". I seem to be subscribed to trades by default, so this isn't a huge problem - but is there anything I'm doing wrong here?

2. My websocket connection keeps dropping, for no reason that I can see. This thread's history has some talk about having to send heartbeat messages, but from what I understand that's only when you're connected to socket.io, not the plain old websockets I keep on using. I find all clients I run drop at the same time, regardless of when I start them, which makes me suspect the problem is on Gox's end. I can easily reconnect to it when it drops, but it's frustrating because it takes about 90s between the connection being dropped and me getting a new trade message after I've reconnected. EDIT: I discovered a settimeout(5) sitting quietly in the code, which was likely wrecking things... removing it and upgrading websocket-client to 0.11 seems to have done the trick.

Any advice would be much appreciated!
-Floomi
donator
Activity: 2772
Merit: 1019
We do not know what is the cause of the problem. That fact that all separate currency orderbooks are produced from the one majore orderbook is understood.

I mentioned this only as an explanation of why the problem is observed synchronously in all currencies. MT once mentioned in #mtgox that it might be due to occasional database rollbacks that can happen *after* the depth message has been sent out already (its sending out the depth mesages before the database commit happens and no inverse depth mesage is sent when it must be rolled back for whatever reason).

I am observing the same problem in goxtool and have no solution for it. Also fulldepth is cached a few minutes by the server, so simply reloading fulldepth every x minutes will also lead to some sort of inconsistency, especially near the top of the book where most of the action is and where its most important for traders and bots. I have no idea how to properly and cleanly fix this mess client side, probably nothing can be done about it at all.


tbh: the solution is to switch exchange. magicaltux will never fix this. I've given up.

no matter what the problem is: magicaltux should either provide us with a solution to keep a synced orderbook or outsource it if he can't do it himself.
sr. member
Activity: 379
Merit: 250

... MT once mentioned in #mtgox that it might be due to occasional database rollbacks that can happen *after* the depth message has been sent out already (its sending out the depth mesages before the database commit happens and no inverse depth mesage is sent when it must be rolled back for whatever reason).


this is interesting
hero member
Activity: 938
Merit: 500
https://youengine.io/
We do not know what is the cause of the problem. That fact that all separate currency orderbooks are produced from the one majore orderbook is understood.

I mentioned this only as an explanation of why the problem is observed synchronously in all currencies. MT once mentioned in #mtgox that it might be due to occasional database rollbacks that can happen *after* the depth message has been sent out already (its sending out the depth mesages before the database commit happens and no inverse depth mesage is sent when it must be rolled back for whatever reason).

I am observing the same problem in goxtool and have no solution for it. Also fulldepth is cached a few minutes by the server, so simply reloading fulldepth every x minutes will also lead to some sort of inconsistency, especially near the top of the book where most of the action is and where its most important for traders and bots. I have no idea how to properly and cleanly fix this mess client side, probably nothing can be done about it at all.
sr. member
Activity: 379
Merit: 250
As long as gox provides separate http requests for full orderbooks for different currencies and allows socket.io subscribtion to multiple currenclies sending depth messages for each currecy in a separate channel this allows us to support orderbook state for each currency separately as if they are independent.

Observed lost depth messages effect is visible not only for one crrency orderbook but for all of them synchronously. We do not know what is the cause of the problem. That fact that all separate currency orderbooks are produced from the one majore orderbook is understood. The thing is that no matter is there only one major orderbook or not, the mechanism of depth messages itself is broken and not all depth messages reach the client.

This is suported by the obvious neccessity of fixing invalid orderbook state (when best bid price is greater than best ask price) against ticker messages, but also by observing the effect of accumulated old (not removed) orders which lead to increase in orderbook volume through time and significant descrease immediately after orderbooks state was reinitialized.

hero member
Activity: 938
Merit: 500
https://youengine.io/
I agree, and wanted to add that this noticed affect is further suspicious becuase it is visible for all 8 currecly orderbooks at the same moment.

MtGox has only one orderbook. Every order you add in one currency will also appear in every other currency (bids will appear 2.5% lower in other currencies, asks will appear 2.5% higher in other currencies).
sr. member
Activity: 379
Merit: 250
legendary
Activity: 906
Merit: 1002
Most of you are writing a trading bot, so Im sorry, but why do you all need full-depth? depth/fetch works fine and you can pull the data like every 30sec...
sr. member
Activity: 379
Merit: 250
I agree, and wanted to add that this noticed affect is further suspicious becuase it is visible for all 8 currecly orderbooks at the same moment.

Just to remind that we support state of 8 currency orderbooks in the sime time. And this effect can be seen for all of them synchronosouly.

Needless to say that for such currencies like GPB or RUB here are significantly less trading activity which could be account for those simultanous volume decreases.

hm
Pages:
Jump to: