Pages:
Author

Topic: SierraChart bridge - Realtime Bitcoin charts [v0.5] (MtGox, Intersango, ...) - page 6. (Read 142193 times)

full member
Activity: 124
Merit: 100
Are there any Fibonacci plugins for Sierra charts? (Beyond the Fibonacci fan)
Beyond? Like what? I've been thinking about phi spiral recently... Fibo fan is great fun, sometimes..
legendary
Activity: 1876
Merit: 1003
Are there any Fibonacci plugins for Sierra charts? (Beyond the Fibonacci fan)
legendary
Activity: 1512
Merit: 1032
Every attempt I make to download the whole history, it often gets through a month or two but always ends up crapping out with a 502 error.  This is in Win 7 Ultimate 64.  Tried running as admin, increasing the buffer and running in XP SP3 compatibility mode all to no avail.

I can't make bitcoincharts not go down. http://www.tomshardware.com/forum/237521-49-what-gateway-error
Is there a particular exchange/currency that it is always getting or date you never get past when you see this error?

The history download will continue where it left off when the bridge program is restarted - the program SHOULD stop if it gets website errors, otherwise it will skip history downloading and get the live trade stream. After the live ticker data is added to the database with the current date and time, you will never get data older than that.
full member
Activity: 135
Merit: 107
Every attempt I make to download the whole history, it often gets through a month or two but always ends up crapping out with a 502 error.  This is in Win 7 Ultimate 64.  Tried running as admin, increasing the buffer and running in XP SP3 compatibility mode all to no avail.

I'm just going to leave this here:

http://we.lovebitco.in/schart/sierrachartfeed.exe (windows 32 bit standalone exe)

or

http://we.lovebitco.in/schart/sierrachartfeed.py (python 2.7 source, replace old file)

Yes, it's fixed in a big way.


NOTE: rename or delete your old C:\SierraChart\data\*.scid files if you've been using the broken sierrachartfeed version, or want older history than you previously retrieved.


Code:
Usage: sierrachartfeed.py [options]

Options:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir=DATADIR
                        Data directory of SierraChart software
  -y, --disable-history
                        Disable downloads from bitcoincharts.com
  -p PRECISION, --volume-precision=PRECISION
                        Change decimal precision for market volume.
  -s SYMBOLS, --symbols=SYMBOLS
                        Charts to watch, comma separated. Use * for streaming
                        all markets.
  -l HISTORY, --history=HISTORY
                        Number of days of history to retrieve (default=30).

I set the default "symbol" to only be mtgoxUSD. The symbols are the file names for the exchange you can see here: http://api.bitcoincharts.com/v1/csv/ - for multiple exchanges, use the command-line option like:
sierrachartfeed -smtgoxUSD,bitfloorUSD,btceUSD

This is my own code, you can examine how I solved the problem of reading from old to new without an external database to sort and with 0 chance of missing or duplicate trades being downloaded (although it does inevitably waste some bandwidth).

Update 1: smarter download size adaptation, days of history option, actual source comments
Update 2: fixed duplicate trade bug I created in update 1
legendary
Activity: 1778
Merit: 1008
ah, got it. all working now though.
legendary
Activity: 1512
Merit: 1032
trying out the new feed from deepceleron. looks to be good. it's currently grabbing the history for me. to make sure i'm doing this right:

for the windows exe, the multiple exchanges option should look like this, right? "sierrachartfeed.exe -s mtgoxUSD,bitstampUSD,btceUSD"


There should be no space between the -s and the option.

You can also use sierrachartfeed.exe -s* if you want to download EVERY ticker, which is about 40 of them.

For those of us lacking programming expertise, is there a compiled version incorporating the recent changes discussed here? TIA.
Yes. 10 posts back.
legendary
Activity: 1778
Merit: 1008
trying out the new feed from deepceleron. looks to be good. it's currently grabbing the history for me. to make sure i'm doing this right:

for the windows exe, the multiple exchanges option should look like this, right? "sierrachartfeed.exe -s mtgoxUSD,bitstampUSD,btceUSD"
full member
Activity: 361
Merit: 100
Will someone sum up what exactly this does or can be used for in laymans terms?

For those of us lacking programming expertise, is there a compiled version incorporating the recent changes discussed here? TIA.
legendary
Activity: 938
Merit: 1000
chaos is fun...…damental :)
on SC data feed since the data is properly formatted and you can use tick charts for example the only problem is they dont have data since day 1



here a sample BTCUSD.7z
full member
Activity: 124
Merit: 100
Will someone sum up what exactly this does or can be used for in laymans terms?
It's bridging, feeds almost realtime data into one of the most powerful charting platform..
member
Activity: 98
Merit: 10
Will someone sum up what exactly this does or can be used for in laymans terms?
full member
Activity: 124
Merit: 100
SierraChart recently added their own mtgox data feed (and a $10/month bitcoin-only subscription).
Before that, and again now, the trial/free version of SierraChart can use data downloaded from bitcoincharts/mtgox when processed by the feed/bridge.
The bitcoin only subscr is a very promising move forward..

Yes, but our bridge stream seems a bit more delayed..
Afaik there are two different apis/data sources. Maybe that's why..
legendary
Activity: 1512
Merit: 1032
sierrachartfeed.exe
Great. That way i'll learn python pretty soon..

Are you going to up it into the github repo?
I think I've got everything done worth doing, and it shouldn't break or have any more buffer overflow vulnerabilities than it did before. If bitcoincharts is down or 500, it errors out. If the data API changes to something else or is blank, just the current ticker works. If there are periods of data without trades in history (like from ddos) the logic doesn't care.

I'll just refactor to minimize diff lines before I put a pull request in (and create an account, install git, etc). Someone else could check it in; slush could look if I broke his functions, etc; it's public domain.


I changed mtgox feed to native SC data source: https://www.sierrachart.com/SupportBoard.php?ThreadID=527
Why there's a difference in feeds? Different apis? Websockets vs socketio?
SierraChart recently added their own mtgox data feed (and a $10/month bitcoin-only subscription).
Before that, and again now, the trial/free version of SierraChart can use data downloaded from bitcoincharts/mtgox when processed by the feed/bridge.
full member
Activity: 124
Merit: 100
I changed mtgox feed to native SC data source: https://www.sierrachart.com/SupportBoard.php?ThreadID=527
Hell, today it's good.. A bit faster than the bridge..
Very much like bitcoinity.. But slightly different somehow..

I'm sure quality service and good support worth it.. But we need more testing and feedback first..
I bet they are gonna be the first broker providing the btc datafeed and perhaps even trading.
Should be an option anyway, a good option.. 'Cause what really matters is the choice..

Why there's a difference in feeds? Different apis? Websockets vs socketio?

Great. That way i'll learn python pretty soon..

Are you going to up it into the github repo?
legendary
Activity: 1512
Merit: 1032
I'm just going to leave this here:

(old windows 32 bit standalone exe removed)

or

http://we.lovebitco.in/schart/sierrachartfeed.py (python 2.7 source, replace old file)

Yes, it's fixed in a big way.


NOTE: rename or delete your old C:\SierraChart\data\*.scid files if you've been using the broken sierrachartfeed version, or want older history than you previously retrieved.


Code:
Usage: sierrachartfeed.py [options]

Options:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir=DATADIR
                        Data directory of SierraChart software
  -y, --disable-history
                        Disable downloads from bitcoincharts.com
  -p PRECISION, --volume-precision=PRECISION
                        Change decimal precision for market volume.
  -s SYMBOLS, --symbols=SYMBOLS
                        Charts to watch, comma separated. Use * for streaming
                        all markets.
  -l HISTORY, --history=HISTORY
                        Number of days of history to retrieve (default=7).

I set the default "symbol" to only be mtgoxUSD. The symbols are the file names for the exchange you can see here: http://api.bitcoincharts.com/v1/csv/ - for multiple exchanges, use the command-line option like:
sierrachartfeed -smtgoxUSD,bitfloorUSD,btceUSD

This is my own code, you can examine how I solved the problem of reading from old to new without an external database to sort and with 0 chance of missing or duplicate trades being downloaded (although it does inevitably waste some bandwidth).

Update 1: smarter download size adaptation, "days of history" option, actual source comments.
Update 2: fixed duplicate trade bug I created in update 1.
Update 3: include volume git pull (tick accurate data now), undo precision change (back to real volume = 0.01 in Chart->Chart Settings->Volume Multiplier), show both current time & trade time on console.
Update 013: tolerate the random HTTP 502 Gateway errors, adaptive data request rate to mitigate, default 7 days of history
Update 017: refine rate adaptation, enforce max history when catching up scid, give a user-agent
full member
Activity: 124
Merit: 100
Is this any more accurate? You can get 1min resolution on bitcoincharts
1min is not enough. We need tick data..

dkasak, kind soul, has made changes to the source code.. need testing..
https://github.com/slush0/sierrachartfeed/pull/10
member
Activity: 98
Merit: 10
Why not just use bitcoincharts? Is this any more accurate? You can get 1min resolution on bitcoincharts
newbie
Activity: 58
Merit: 0
I have read back through posts and see that many are having problems downloading historical data.

I had a gap in my data file during the last weeks' problems and so deleted my mtgoxUSD.scid.  Unfortunately, it will not pull all the historical data to create a new, complete scid file.  All I get now is the last 2 days of data. 

Could someone direct me to the instructions for fixing this by downloading all data or said time blocks of data?

Thanks!
legendary
Activity: 1512
Merit: 1032
sr. member
Activity: 462
Merit: 250
Clown prophet
Sierra accepts bitpay btw
Pages:
Jump to: