[BETA] MTGox websocket API, testers wanted - page 20.

jed

full member

Activity: 182

Merit: 107

Jed McCaleb

Anybody else occasionally miss trade messages?

jed

full member

Activity: 182

Merit: 107

Jed McCaleb

Shouldn't the order_add message tell you if it was a buy or sell order?

MagicalTux

vip

Activity: 608

Merit: 501

-

Quote from: jed on May 04, 2011, 10:29:52 AM

I noticed that sometimes the price is sent in quotes:

Code:

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":"3","type":2,"volume":-6},"op":"private","origin":"broadcast","private":"depth"}
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":1,"type":2,"volume":10},"op":"private","origin":"broadcast","private":"depth"}

Not a big deal but weird that it isn't consistent.

When the price comes from mysql, it is as a string. I could cast it to float to make sure it's a float, but I guess it doesn't change much for anyone (shouldn't, anyway)

jed

full member

Activity: 182

Merit: 107

Jed McCaleb

I noticed that sometimes the price is sent in quotes:

Code:

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":"3","type":2,"volume":-6},"op":"private","origin":"broadcast","private":"depth"}
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":1,"type":2,"volume":10},"op":"private","origin":"broadcast","private":"depth"}

Not a big deal but weird that it isn't consistent.

MagicalTux

vip

Activity: 608

Merit: 501

-

Quote from: 0x6763 on April 29, 2011, 01:35:27 AM

It appears that you fixed the dark orders ok (at least the dark asks...I got an error when I tried to make a dark bid, but I have to play around with that to see what the error was since my GUI currently doesn't display error messages to me, yet.)

The orders with insufficient funds still seem to have some problems, though.

I tried an ask of 2000 @ $4, and I only saw the amount I had funds for come across the websocket, but when I canceled both orders (the one with the funds, and the one without), they both came through the websocket as two negative asks @ $4 totaling to 2000.

I then tried a bid of 2000 @ $1, and I only saw the amount I had funds for come across the websocket, but when I was viewing my orders through the old API, I only saw one order of 2000 @ $1 marked as not having sufficient funds (seems to be an unrelated problem to the websocket...I do have funds to cover part of that order), and when I canceled it, a bid of -2000 @ $1 came across the websocket.

So the depth channel is still causing some inconsistencies, but it's definitely improved. I think there's also the occasional issue where a the price of a trade doesn't exactly match the price of the bid or ask being removed from the depth tables.

Also, I agree with jed that being able to get the current depth table (as well as trade table, and my own full order list) through the websocket instead of the old API would be an eventual nice-to-have.

Thanks for what you've done so far, though!

Fixed notices for orders without enough funds.

As for the trade price, it'll be fixed in a few days when we switch to a new backend~

0x6763

newbie

Activity: 24

Merit: 0

It appears that you fixed the dark orders ok (at least the dark asks...I got an error when I tried to make a dark bid, but I have to play around with that to see what the error was since my GUI currently doesn't display error messages to me, yet.)

The orders with insufficient funds still seem to have some problems, though.

I tried an ask of 2000 @ $4, and I only saw the amount I had funds for come across the websocket, but when I canceled both orders (the one with the funds, and the one without), they both came through the websocket as two negative asks @ $4 totaling to 2000.

I then tried a bid of 2000 @ $1, and I only saw the amount I had funds for come across the websocket, but when I was viewing my orders through the old API, I only saw one order of 2000 @ $1 marked as not having sufficient funds (seems to be an unrelated problem to the websocket...I do have funds to cover part of that order), and when I canceled it, a bid of -2000 @ $1 came across the websocket.

So the depth channel is still causing some inconsistencies, but it's definitely improved. I think there's also the occasional issue where a the price of a trade doesn't exactly match the price of the bid or ask being removed from the depth tables.

Also, I agree with jed that being able to get the current depth table (as well as trade table, and my own full order list) through the websocket instead of the old API would be an eventual nice-to-have.

Thanks for what you've done so far, though!

MagicalTux

vip

Activity: 608

Merit: 501

-

Hi,

Broadcasting of depth changes fixed, I think (mostly).

As for the extra fields, they are there mainly to avoid mistaking the messages for something else, however you can safely ignore them

jed

full member

Activity: 182

Merit: 107

Jed McCaleb

This depth channel is broadcasting the changes to the depth. It would be nice if you could send something in the beginning to get the current depth table in the same format. Then you can use this feed to change the original depth table as time goes on.
I know I can get it with the old API but it would be nice if the messages were in the same format.
Also is there a point to all these extra fields: "op":"private","origin":"broadcast","private":"depth" ?

0x6763

newbie

Activity: 24

Merit: 0

@MagicalTux

When do you think you'll be able to stop broadcasting dark orders and orders with insufficient funds on the websocket?

MagicalTux

vip

Activity: 608

Merit: 501

-

Hi,

I have located the source of the null and fixed it

Mark

0x6763

newbie

Activity: 24

Merit: 0

Now that I've looked at more instances of it, the volumes are not always obviously related to the subsequent trades, though I think they're related at least most of the time, since these nulls only seem to occur right before a trade or throughout a series of trades.

Here are some examples of the "null" depth message with some surrounding depth/trade info (I removed some messages coming in around these same times that did not seem relevant):

Code:

REMOVE BID -10 @ 1.1832
BID 10 @ 1.1832
REMOVE BID -10 @ 1.1832
BID 10 @ 1.1832
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-56},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -56 @ nil
TRADE 10 @ 1.1832
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-46},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -46 @ nil
TRADE 17 @ 1.1827
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-29},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -29 @ nil
TRADE 23.216 @ 1.18151

Code:

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-5.784},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -5.784 @ nil
TRADE 5.784 @ 1.1621

Code:

ASK 0.026 @ 1.1875
ASK 9.974 @ 1.1875
ASK 0.03 @ 1.1875
REMOVE ASK -9.97 @ 1.1875
BID 1 @ 1.18751
ASK 0.026 @ 1.1875
{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-9.974},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -9.974 @ nil
TRADE 1 @ 1.18751

Code:

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-4},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -4 @ nil
TRADE 4 @ 1.18751

Code:

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","depth":{"price":null,"type":2,"volume":-19.966},"op":"private","origin":"broadcast","private":"depth"}
REMOVE BID -19.966 @ nil
TRADE 19.966 @ 1.18751

0x6763

newbie

Activity: 24

Merit: 0

Occasionally the websocket sends a depth message to remove an ask or bid (has a negative volume) with a "null" price. I've only noticed this right before some (not all) trade messages. I haven't yet seen it happen at volumes unrelated to a trade.

Could it maybe be a race condition in your code where most of the time it gets the depth information before the trade clears it, but sometimes it doesn't?

slush

legendary

Activity: 1386

Merit: 1097

Example usage of this API from Python: MtGox->Sierrachart feed

MagicalTux

vip

Activity: 608

Merit: 501

-

Quote from: Clark on April 17, 2011, 07:39:23 PM

Another question: how can I re-subscribe from a previously unsubscribed channel? I have tried sending

Code:

{'channel': 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f', 'op': 'mtgox.subscribe'}

and

Code:

{'key': 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f', 'op': 'mtgox.subscribe'}

but neither works at getting the ticker feed back after I unsubscribe from it.

Just added:

Code:

{"op":"mtgox.subscribe","type":"trades"}

Possible types: trades, ticker, depth

Clark

hero member

Activity: 548

Merit: 502

So much code.

So I've answered some of my own questions:

I grabbed a copy of the pywsc Websockets library for Python and am receiving trades, orders, and ticker events.

I see there is a new market depth channel as well.

Another question: how can I re-subscribe from a previously unsubscribed channel? I have tried sending

Code:

{'channel': 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f', 'op': 'mtgox.subscribe'}

and

Code:

{'key': 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f', 'op': 'mtgox.subscribe'}

but neither works at getting the ticker feed back after I unsubscribe from it.

Here's an idea for an initial documentation page:
List the possible fields contained in the JSON return objects, starting with op values and going from there. This is what I have so far, from seeing different things come across the socket:

Code:

op:remark
message:
success:
op:subscribe
channel:
op:unsubscribe
channel:
op:private
channel:
origin:broadcast
private:depth
depth:{
volume:
price:
type:
}
private:ticker
ticker:{
high:
low:
vol:
buy:
sell:
}
private:trade
trade:{
date:
amount:
type:trade
price:
}
private:order_add
order_add:{
oid:
price:
date:
amount:
status:
darkStatus:<0 or 1>
}
private:order_rem
order_rem:{
oid:
}

memvola

hero member

Activity: 938

Merit: 1002

Quote from: MagicalTux on April 16, 2011, 06:00:45 AM

Might just have been "by chance". There are a lot of ticker events, no need to think too hard about this

Well, anyway, I was actually hoping to be able to keep market depth up to date without polling, like I can do with recent trades using the trades channel, but it seems there's currently no way to do the same with orders. Ticker data doesn't report new or deleted orders, or at least I failed to identify them. I don't want to ask for updates like crazy.

Are you planning to introduce a new channel for this?

EDIT: Oh, Keefe already asked for this but I missed it. Nevermind then...

MagicalTux

vip

Activity: 608

Merit: 501

-

Quote from: memvola on April 16, 2011, 05:56:15 AM

Everything works as expected. Looks like I get a lot of identical ticker frames when I enter an order, I'm guessing this is because there are a lot of bots adjusting orders in response?

Might just have been "by chance". There are a lot of ticker events, no need to think too hard about this

memvola

hero member

Activity: 938

Merit: 1002

Here's the haskell code which connects to the server, subscribes to the private channel and unsubscribes from the ticker. No json parsing or error handling.

Code:

module Main (main) where

import Network
import System.IO
import Control.Monad

clientHandshake :: String
clientHandshake = "GET /mtgox HTTP/1.1\r\n\
\Upgrade: WebSocket\r\n\
\Connection: Upgrade\r\n\
\Host: websocket.mtgox.com\r\n\
\Origin: null\r\n\
\\r\n"

-- Expected server response
serverHandshake :: [String]
serverHandshake = ["HTTP/1.1 101 Web Socket Protocol Handshake",
"Upgrade: WebSocket","Connection: Upgrade",
"WebSocket-Origin: null",
"WebSocket-Location: ws://websocket.mtgox.com/mtgox",
"WebSocket-Protocol: *"]

-- https://mtgox.com/code/getKey.php
key :: String
key = "\"key\":\"00000000-0000-0000-0000-000000000000:0000000000000000000000000000\""

hGetHeader :: Handle -> IO [String]
hGetHeader h = do s <- hGetLine h
if s == "\r"
then return []
else do s' <- hGetHeader h
return (init s : s')

hGetFrame :: Handle -> IO String
hGetFrame h = do c <- hGetChar h
if c == '\xff'
then return "\n"
else do c' <- hGetFrame h
return $ if c == '\x00' then c' else (c:c')

hPutFrame :: Handle -> String -> IO ()
hPutFrame h s = hPutStr h $ '\x00' : s ++ "\xff"

main :: IO ()
main = do h <- connectTo "websocket.mtgox.com" (PortNumber 80)
hSetBuffering h NoBuffering
hPutStr h clientHandshake
hdr <- hGetHeader h
when (head hdr /= head serverHandshake) .
error $ "Invalid server handshake:\n" ++ show hdr

hPutFrame h "{\"op\":\"unsubscribe\",\"channel\":\"d5f06780-30a8-4a48-a2f8-7ed181b4a13f\"}"
hPutFrame h $ "{\"op\":\"mtgox.subscribe\"," ++ key ++ "}"

forever $ hGetFrame h >>= putStr

Everything works as expected. Looks like I get a lot of identical ticker frames when I enter an order, I'm guessing this is because there are a lot of bots adjusting orders in response?

MagicalTux

vip

Activity: 608

Merit: 501

-

Quote from: slush on April 15, 2011, 01:44:23 PM

Is websocket api down?

Code:

$ telnet websocket.mtgox.com 80
Trying 69.64.54.38...
telnet: Unable to connect to remote host: Connection refused

Yep was down, found the reason, fixed the code and restarted it (division by zero when someone sent badly formatted draft-00 headers).

Quote from: slush on April 15, 2011, 11:15:10 AM

Hello, thanks for the API. Few questions/comments:

a) What exactly is the ticker? Why server broadcast so many packets without any changed data? I see few updates per second without any change...
b) Can you please post example of subscribe/unsubscribe command? It's not clear for me how to unsubscribe from ticker.
~~c) Can you include (official) server timestamps to the broadcasts? Client timestamps in trading platforms are hell...~~ I'm sorry, I miss 'date' field Roll Eyes

a) The ticker is the current stats about bitcoin. It's sent once each time it have a chance to be changed, which includes when orders are posted, removed or completed.
b) Here's an example:

Code:

{"op":"unsubscribe","channel":"xxx"}

Quote from: mndrix on April 15, 2011, 11:29:20 AM

Quote from: MagicalTux on April 14, 2011, 10:48:21 PM

Additionally each user has a "own" channel which streams informations about orders (new order, deleted order, etc) and trades only the user's trades).

Does a user's "own" channel include information about when BTC/USD funds post to his account?

Not at this point.

Quote from: jed on April 15, 2011, 02:24:56 PM

why are you using UUID's to name the channels rather than just something human readable like: "trades" ?

Because it was easier since I intend to use this system for many other sites, without having to do something to isolate users of a specific system. Also there are user-specific channels and many other stuff which makes use of uuid a bit easier (for me).

jed

full member

Activity: 182

Merit: 107

Jed McCaleb

why are you using UUID's to name the channels rather than just something human readable like: "trades" ?

Topic: [BETA] MTGox websocket API, testers wanted - page 20. (Read 77653 times)