Topic: [BETA] MTGox websocket API, testers wanted - page 7. (Read 77653 times)

Maybe not use SSL, use unencrypted on port 80

here is a complete example as far into the handshake as it is possible with telnet (because after that it switches to binary WS protocol):

--($)-- telnet socketio-beta.mtgox.com 80
Trying 184.169.152.65...
Connected to socketio-lb-1655485032.us-west-1.elb.amazonaws.com.
Escape character is '^]'.
GET /socket.io/1 HTTP/1.1
Host: socketio-beta.mtgox.com:80
Connection: keep-alive

HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 23 Mar 2013 21:06:56 GMT
Connection: keep-alive
Transfer-Encoding: chunked

53
Qi-RuYva5rgLXtXuhsrE:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling
0

GET /socket.io/1/websocket/Qi-RuYva5rgLXtXuhsrE HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: socketio-beta.mtgox.com:80
Origin: socketio-beta.mtgox.com:80
Sec-WebSocket-Key: UWaphFPiSqq3f2gOmaD5Sg==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Bzz8qKJPEMNNOgC4hOZd3iZNb5o=

�1::�2::

the �1:: and the �2:: are the first two websocket frames it sends (websocket is a binary protocol), now it expects "1::/mtgox" and for every "2::" it sends you must reply "2::", of course you need to properly mask and frame what you send as it is described in the Websocket RFC. This is how socket.io over websocket (or websocket over socket.io?) looks like. Unfortunately from here on its impossible to proceed with telnet because of the binary frames.

So this is a problem of certificates and not of socket.io-client version?

Or which version of socket.io-client should I use?

Yes, that's because this server uses cert issued for socketio.mtgox.com, which obviously does not match the different hostname.

becaue its a socket.io server and not a websocket server:

telnet socketio-beta.mtgox.com 80
Trying 184.169.152.65...
Connected to socketio-lb-1655485032.us-west-1.elb.amazonaws.com.
Escape character is '^]'.
GET /socket.io/1 HTTP/1.1
Host: socketio-beta.mtgox.com:80
Connection: keep-alive

HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 23 Mar 2013 20:35:15 GMT
Connection: keep-alive
Transfer-Encoding: chunked

53
W_4my4iKDRSNXtJN89vh:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling
0

then you parse the second line of the response take the ID W_4my4iKDRSNXtJN89vh to form a new URL that look like /socket.io/1/websocket/W_4my4iKDRSNXtJN89vh and then you use the same still open socket (its still open because you sent keep-alive header) and initiate the websocket request:

GET /socket.io/1/websocket/W_4my4iKDRSNXtJN89vh HTTP/1.1
Upgrade: WebSocket
and so on...

If it does not work then make sure that you are sending websocket version 13 headers (see the websocket RFC I posted a few days ago) and not the old version you showed above (that were not version 13 headers), this also means you need to provide Sec-Websocket-Key header and all that. See how goxtool does it (its using https://pypi.python.org/pypi/websocket-client/ with an overridden connect() method to do the initial http for the ID before the actual websocket starts).

You either need to find a complete socket.io implementation for your programming language or a websocket V13 client that you can tweak the same way like I did it with the python client.

maybe too much DNS? try using this as the URL:

socketio-lb-1655485032.us-west-1.elb.amazonaws.com

I still cant hook up thru telnet.

Im getting this error using the beta socketio server with the javascript library:


DEBUG: Error: Hostname/IP doesn't match certificate's altnames
    at SecurePair. (tls.js:1280:23)
    at SecurePair.EventEmitter.emit (events.js:92:17)
    at SecurePair.maybeInitFinished (tls.js:883:10)
    at CleartextStream.read [as _read] (tls.js:421:15)
    at CleartextStream.Readable.read (_stream_readable.js:293:10)
    at EncryptedStream.write [as _write] (tls.js:330:25)
    at doWrite (_stream_writable.js:211:10)
    at writeOrBuffer (_stream_writable.js:201:5)
    at EncryptedStream.Writable.write (_stream_writable.js:172:11)
    at write (_stream_readable.js:547:24)

Using puTTY I tried:

=====================================
websocket.mtgox.com, port 80, RAW

Connection is established, I then type:

GET /mtgox HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: websocket.mtgox.com
Origin: null

Server replied with handshake ACK.

Then I was getting the stream displayed, until server seemed to drop connection. I tried this a few times with success and now cant repeat.

=====================================
socketio-beta.mtgox.com, port 80, RAW

Connection is established, I then type:

GET /mtgox HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: socketio-beta.mtgox.com
Origin: null

No reply, Server remains silent?
=====================================

Does it mean that it is actually not original socket.io client have to be used but rather different client implementation?

Do you have any link to the new socketio-beta project?

or what I did not get?

This also seems to use different server software and it at the moment it seems it is essential that the websocket connection is done on the same open socket that the initial http request came from. So the client has to fetch the session ID (with the header Connection: keep-alive) and then use that same socket to  upgrade the connection to websocket. Otherwise it will fail with reason "client not handshaken". MT said this is a bug and he intends to fix it. Meanwhile I have tweaked my own socketio client to be able to use the same socket for both requests, so for me it works. I don't know how the javascript implementation handles this.

so the only thing that have to be changed is url? Any changes to socket.io lib version necessary?


are you sure you placed correct url? I tried and getting multiple reconnects and errors from socket io client?

can you check?

Yesterday in the #mtgox channel on freenode MT has finally announced the hostname of the new beta server: socketio-beta.mtgox.com

This new server seems much improved, I have not yet seen it disconnecting or acting strange a single time since yesterday (maybe partially also because only few are using it yet).

goxtool now also supports this new server: pull the latest release and then:

i seem to be able to connect to websocket occasionally; but socket.io seems completely non responsive now?

Thank you. Just wanted to be sure that it is not my local problem.

By the way I wanted to thank you for your hint about orderbook state synchronization.

full book stamp and 60 secs depth channel buffer + ticker fixing made my mtgox stream api client work very well
Thank you once more

Total MtGox tease. Usually happens during high-traffic times.

I can see my socket.io client connects to mtgox and connect event is invoked.

and then no any subscribtion message at all nothing more.

this behavior is already for several hours.

restarting client does not help.

Clarks's client shows the same. Connects and then silence.

Does anybody see the same?

My 2 Satoshi:

Really? I find it one of the simpler and better written networking specs out there (assuming you know some HTTP). I wrote a client library from scratch in a few days (~1000 lines of code) that I believe is complete. Don't Firefox and Chrome have complete client implementations with a Javascript API?

I also have a partial server implementation but that's because I don't need a WebSocket server for anything. Most of the work is already done because the framing and handshake code can be shared with the client version.

My only complaints are that it unnecessarily relies on crypto libraries (SHA-1 and base-64 for the handshake is overkill) and that the binary / text message distinction is kind of redundant. Perhaps it makes sense in the context of web browsers and Javascript.

I also did a double-take when I read about the masking. However, I think it might make sense to avoid interference from web proxies. The part of the RFC that justifies masking as a way to avoid 'attacks on infrastructure' is utter bullshit though.

If you're worried about the performance hit then just set the mask to all zeros and have a fast path in the software that doesn't perform the XOR.

Fragmented messages may also be unnecessary but might be useful in low-latency applications. It doesn't add much complexity to handle them.

A real bidirectional TCP connection is a pain when you factor in web proxies, firewalls, NAT, etc. WebSocket is a reasonable alternative with relatively little overhead per message.

Cheers mate! With that I think I will make it work. Your help is much appreciated.
In case you are wondering, this is what Im working on:
http://goxgame.sytes.net/
Click legend to show/hide stuff, click and drag to zoom in.
Warning: buggy, obviously incomplete data, cant keep order book synched  etc,  its definitely still WIP.

The 5360180921750645752 you get from the initial http request is the socketio session id, it belongs to the socketio protocol. It is generated by the server, see the socketio document on github (posted by Fireball a few postings before). its only used to generate the websocket GET request (the websocket url)

The Sec-Websocket-Key belongs to the websocket protocol (see: http://datatracker.ietf.org/doc/rfc6455/?include_text=1) its a random 16 byte you create and then base64 encode (see section 11.3.1 and section 4.1 (page 17, article 7))

_____
PS: The websocket protocol as defined in rfc6455 exists primarily to:
 * ensure that the google engineers who created it have secure jobs until the end of their life
 * ensure that nobody will ever write a *complete* reference implementation of serer and client
 * prove that it is possible to embed utter nonsense that serves no purpose like for example the masking of client-to-server data as specified in RFC6455 in an RFC and nobody recognizing it as what it really is: an april fools joke.
 * [...]
 * to establish a bidirectional connection between A and B

in exactly the above order.

thanks for that. Before I try.. what that websocket key? It doesnt look anything like what I get from
https://socketio.mtgox.com/socket.io/1

I get  5360180921750645752

Is that related to your mtgox api key (and if so, is it safe to publish?) and if so, why doesnt it look anything like mine?
Or is it a fixed key and I can just use it?