Nitrous,
Many thanks for the comprehensive reply. A great help. Yes, caching seems to be the issue (I am in Australia) as I can definitively answer that I could not get anywhere near 20 updates as fast as you can..
I had had some issues with the websocket so (for the time being went with the V2 API (did everything using V2 API so was interested to read you referring to V1) but will take a look at the V1 stuff.
Also, yes the whole trade data history thing is an issue but it looks like it will be sorted soon on the face of it.
Regards,
Steve.
Topic: *GOX v2 API Drawbacks* (Read 834 times)
Hi Steve,
I believe that the restrictions actually originate with cloudflare, a service used to protect websites by regulating traffic and preventing DDoS attacks which serves as a 'middle-man' between a website (ie, data.mtgox.com) and you. I don't think there is actually a definite limit of 10 requests per 10 seconds, I've ran an app before at roughly 20 per 10s period and it's tended to function fine, the limits are more recommendations - levels at which you run no risk of being blocked.
Cloudflare also introduces a lot of caching, which seems to be giving you problems. The strategy I would recommend is a mix of the websocket and http api. The websocket shouldn't suffer from any caching (at least from cloudflare, there is still going to be some internal caching for the ticker, but that should be much more frequent), however I have heard of trouble and uncertainties in performing trades. I suggest that you therefore use sockets for all requests which don't actually do any actions, those which just get information such as ticker price, order info, account info, etc. All critical requests, such as placing orders, cancelling them, etc, should be run through the http api.
Addressing some of your points more specifically: There seems to be a lot of variance in caching for people, generally my own requests seem to be very current, with ticker price updating at least every 1s, sometimes even better. Some however experience much less frequent updates. This is probably due to cloudflare identifying certain regions, ISPs and IP ranges as more active than others, and so their different CDN servers probably cache at different rates, depending on this factor. Hence why for these information requests it's better to use the socket api.
I believe the socket API also, by default, pushes you trades as they happen on the "trade.BTC" channel which you are autosubscribed to, so this should hopefully fill in your trade data, if I understand you correctly. There are also ticker and depth channels which regularly push you up-to-date info. Please note that when using HTTP methods over the socket API you need to use the v1 methods (info on implementation here).
If you need historical trade data though, please avoid the API and have a look at the new (soon to be official) method and my tool for more info. These are still in development and the data is not current yet, but in the near future the mechanism and the tool should be ready.
I hope this answers all your questions, let me know if you're still unsure about anything.
Thanks,
nitrous
I believe that the restrictions actually originate with cloudflare, a service used to protect websites by regulating traffic and preventing DDoS attacks which serves as a 'middle-man' between a website (ie, data.mtgox.com) and you. I don't think there is actually a definite limit of 10 requests per 10 seconds, I've ran an app before at roughly 20 per 10s period and it's tended to function fine, the limits are more recommendations - levels at which you run no risk of being blocked.
Cloudflare also introduces a lot of caching, which seems to be giving you problems. The strategy I would recommend is a mix of the websocket and http api. The websocket shouldn't suffer from any caching (at least from cloudflare, there is still going to be some internal caching for the ticker, but that should be much more frequent), however I have heard of trouble and uncertainties in performing trades. I suggest that you therefore use sockets for all requests which don't actually do any actions, those which just get information such as ticker price, order info, account info, etc. All critical requests, such as placing orders, cancelling them, etc, should be run through the http api.
Addressing some of your points more specifically: There seems to be a lot of variance in caching for people, generally my own requests seem to be very current, with ticker price updating at least every 1s, sometimes even better. Some however experience much less frequent updates. This is probably due to cloudflare identifying certain regions, ISPs and IP ranges as more active than others, and so their different CDN servers probably cache at different rates, depending on this factor. Hence why for these information requests it's better to use the socket api.
I believe the socket API also, by default, pushes you trades as they happen on the "trade.BTC" channel which you are autosubscribed to, so this should hopefully fill in your trade data, if I understand you correctly. There are also ticker and depth channels which regularly push you up-to-date info. Please note that when using HTTP methods over the socket API you need to use the v1 methods (info on implementation here).
If you need historical trade data though, please avoid the API and have a look at the new (soon to be official) method and my tool for more info. These are still in development and the data is not current yet, but in the near future the mechanism and the tool should be ready.
I hope this answers all your questions, let me know if you're still unsure about anything.
Thanks,
nitrous
Hi All,
I just wanted to check that I wasn't going mad on a few GOX related API (v2) 'issues' (as i see them) if you could perhaps provide some knowledge / experience ? I have developed a desktop trading client for GOX and BTCE but for GOX I have issues with the request throttling by the GOX servers. What I mean by this is the max frequency of requests if you will. Currently I have:
1. Ticker updates on a 1 second update timer (although the server just seems to cache my last request and resend it to me until their refresh throttle period has been reached (10 seconds) ?
2. Account updates I also want to have on a 1 or maybe 5 second update but naturally these requests 'overlap' with the ticker requests and are throttled likewise to the (assume) 10 second rule.
3. Trades - you would think that GOX would be keen to have trades come through and perhaps not impose as stringent as above) restrictions on trade instructions but seemingly they do for this also.
Has anybody else experienced the above ? What is the throttle 'policy' ? How have you overcome it ? Is it per account or per APIKey (eg will 3 Keys and Secrets circumvent it) ?
Understand that API 2.0 is the latest but in all seriousness (unless I am missing something) what is the point in using it for a serious application at all ? It stands to reason that I want ticker data to come in, very shortly thereafter (based off this data) make a trade, as soon as the result of the trade is back in, update the users' account details.
Seems to me this is only 'fluid and workable' if we use 3*throttle period (assumed to be 10s), therefore the entire process really needs to be across 30seconds. We all know what can happen in 30 seconds right !?
Any and all input appreciated...
Steve.
I just wanted to check that I wasn't going mad on a few GOX related API (v2) 'issues' (as i see them) if you could perhaps provide some knowledge / experience ? I have developed a desktop trading client for GOX and BTCE but for GOX I have issues with the request throttling by the GOX servers. What I mean by this is the max frequency of requests if you will. Currently I have:
1. Ticker updates on a 1 second update timer (although the server just seems to cache my last request and resend it to me until their refresh throttle period has been reached (10 seconds) ?
2. Account updates I also want to have on a 1 or maybe 5 second update but naturally these requests 'overlap' with the ticker requests and are throttled likewise to the (assume) 10 second rule.
3. Trades - you would think that GOX would be keen to have trades come through and perhaps not impose as stringent as above) restrictions on trade instructions but seemingly they do for this also.
Has anybody else experienced the above ? What is the throttle 'policy' ? How have you overcome it ? Is it per account or per APIKey (eg will 3 Keys and Secrets circumvent it) ?
Understand that API 2.0 is the latest but in all seriousness (unless I am missing something) what is the point in using it for a serious application at all ? It stands to reason that I want ticker data to come in, very shortly thereafter (based off this data) make a trade, as soon as the result of the trade is back in, update the users' account details.
Seems to me this is only 'fluid and workable' if we use 3*throttle period (assumed to be 10s), therefore the entire process really needs to be across 30seconds. We all know what can happen in 30 seconds right !?
Any and all input appreciated...
Steve.
Jump to: