Pages:
Author

Topic: SierraChart bridge - Realtime Bitcoin charts [v0.5] (MtGox, Intersango, ...) (Read 142274 times)

full member
Activity: 368
Merit: 100
JIC anyone is interested, the live feed from Bitcoincharts just started working for me this morning after a 2+ month hiatus.

Update: Live feed gone again. DC, if you're listening, any ideas on why this is happening? TIA.
newbie
Activity: 41
Merit: 0
I have install but can not start this software. Is that available to use in Window server 2008 ?
newbie
Activity: 26
Merit: 0
This is a great software for monitoring the crypto market. I am downloading and installing this software onto my laptop. I would like to thank all of you who have built this software for sharing to the trader.
newbie
Activity: 8
Merit: 0
I would also consider possibility that a globe organisation lik IMF ill utimately regulariee the forex and provide much more support for coping with it. If they do not get in agreement with the community they might think about spinning their ownwlz
newbie
Activity: 3
Merit: 0
the feed just close by itself all the time ...................
legendary
Activity: 1512
Merit: 1036
I've got it working again. Huzzah. I just need to iron out some crap data from bitcoincharts CSVs, like this:

1387430609,560.640000000000,0.356700000000
1387430826,667.500000000000,0.299600000000
1387430714,554.260000000000,0.036100000000
1387431207,692.500000000000,0.151600000000
1387431697,1000.000000000000,0.150000000000
1387431246,612.700000000000,0.100000000000
1387431516,540.290000000000,0.029000000000
1387431550,583.000000000000,0.514600000000


Those are timestamps in seconds (in this case from localbtcUSD). You can see that at least a 10 minute range of trades are all randomized, which really pisses off SierraChart. I'll need to sort them efficiently without impacting 500MB+ of exchange data that only has a minor problem (and so far, they all do to some extent).

Hint: to get it working faster once I release the code, start downloading your exchange's CSV files now, and put them in the C:\SierraChart\Data directory (don't worry, SierraChart feed will download it and keep it up to date if you don't...)  I will be ignoring previous SCIDs by using new filenames.

DONE: moving this to a new thread so OP doesn't have non-working code and links: https://bitcointalksearch.org/topic/sierrachart-feedbridge-reborn-realtime-bitcoin-charting-377649
wev
newbie
Activity: 19
Merit: 0
https://mega.co.nz/#!0xtRUKJL!fXZX-R3ZkGSo9sf-lJDQbncWcCVWRC5guQNEr_cIx-o

Let me know if it works for you.
full member
Activity: 124
Merit: 100
Can somebody share his bitstampUSD.scid?
There's something wrong with mine..
sr. member
Activity: 336
Merit: 250
Greetings,

Forex broker www.FXOpen.com has BTCUSD data.  You will have to enable it in the "symbols"  section. [ right click on any currency pair and select symbols -> crypto ]

have fun!

legendary
Activity: 2618
Merit: 1007
Any chance of using the protocol in https://github.com/SierraChart/GSP (ideally a python implementation of it) in the future instead of SCID files? This could enable the bridge to allow trading via API keys too, you'd basically host your own broker server.
legendary
Activity: 1512
Merit: 1036
The history stream API IP address has no IPv4 DNS right now, but for those with native IPv6 it has an AAAA address. It also has been half-returned to working status, there is five days of history (instead of all history available), with date ranges working the way it used to. This should allow you, if your backup/remove old SCIDs if you have been down a while, to run with --history=6 and see some perspective history.

I'm going on holiday for a bit, hopefully the IP and full history availability will be restored to the API.
legendary
Activity: 1778
Merit: 1008
The ISP of bitcoincharts was hit with a DDOS, and it blackholed many sites to mediate the problem. The site is back up.

I have opened a dialog with the site admin, hopefully this will result in an API that will continue to work for applications such as sierrachartfeed.

Awesome. I was worried id be stuck without... My second monitor just wouldn't be the same.
full member
Activity: 124
Merit: 100
legendary
Activity: 1512
Merit: 1036
The ISP of bitcoincharts was hit with a DDOS, and it blackholed many sites to mediate the problem. The site is back up.

I have opened a dialog with the site admin, hopefully this will result in an API that will continue to work for applications such as sierrachartfeed.
legendary
Activity: 1778
Merit: 1008
alas, the streaming api is down either, clearly we need a p2p solution.

agreement.
full member
Activity: 124
Merit: 100
alas, the streaming api is down either, clearly we need a p2p solution.
legendary
Activity: 1778
Merit: 1008
so, in short, we're screwed?
legendary
Activity: 1512
Merit: 1036
I started working on sierrachartfeed to see what can be done.

The bitcoincharts history API is purposely borked. It's been modified to only have five days of trades available regardless of the query.

The HTML wrap was removed, but the ~ 1 request a minute limitation is still there.

The only way to fix this is to get the complete uncompressed CSV download (http://api.bitcoincharts.com/v1/csv/, 300MB for just mtgox) if SCID data is older than five days. I think I tried HTTP byte range requests on these CSV files before and the server didn't support it; there's no Accept-Ranges header. If it did, I would still have to make requests "blind", since I could only approximately index them to find where to continue.

....or have a charting app that only goes back five days.

Edit: the full history file.csv server supports byte ranges. I wrote a library that will get chunks by ticker...

Gameplan:
resume history from csv to 1 day old
slowly catch up last day from crappy history API

Hurdles:
-Efficiently index and seek for resume timestamp in csv, without local copy
-1 minutes between requests means naive history to live transition will miss trades, will need to buffer live stream until stream trades are also in history, and merge ticker lists.
legendary
Activity: 1512
Merit: 1036
It looks like some more undocumented unannounced changes to break things.

1. Per-IP request limiting to 1 per 60 seconds or so
2. history API no longer works like it did.

A "start" only request seems to be the only type of history API request that returns data:

A request like:
http://api.bitcoincharts.com/v1/trades.csv?symbol=mtgoxUSD&start=1385000001

dumps about 1MB of now HTML-header-wrapped trades, but with 0D 0A (linefeed) between trades instead of any HTML:

1385000001,635.500010000000,0.299400000000
1385000008,635.500010000000,0.250000000000
1385000050,635.500010000000,1.000000000000
1385000063,635.500000000000,1.802060090000
1385000090,635.500000000000,1.197939910000
1385000091,635.500010000000,0.368739030000
1385000092,635.500010000000,0.945989800000
1385000092,635.640000000000,0.756793140000
...


These calls were previously documented to not consistently return the correct time periods.

For now, you can temporarily backup and remove your C:\SierraChart\Data\*.scid files, and start with the --disable-history and at least see live trades. I will try to correspond and see if it is fruitless to try to write any API interface that is expected to work for more than a few months.
legendary
Activity: 1974
Merit: 1030
Unfortunately bitcoincharts is returning 404 since yesterday.
Pages:
Jump to: