a script which:
1. downloads all available trades from bitcoincharts.com csv api
2. downloads all missing (the most recent which were not yet cached by bitcoincharts csv api) trades from mtgox api directly
3. integrate data from two sources
4. dump to csv or sqlite db
https://bitcointalksearch.org/topic/mtgox-trades-history-download-r-script-286755
open source
Topic: Where can one easily download the Mt Gox transaction history? (Read 1751 times)
whydifficult, Thank You!
MtGox is offering all trades through a google BigQuery database (not realtime data), Nitrous has written some python code that you can download which will download all those trades in a local (sqlite) database. I have written a small (nodejs) script that can calculate candles based on this local database (you can specify what dates you want candles and the candle size in whatever you like). If you want to use this you have to do a couple of dev things (signing up for google BigQuery, installing python, nodejs, running scripts from command line etc.)
If you are interested in MtGox candles in csv format you can download a lot of them directly in my topic.
If you are interested in MtGox candles in csv format you can download a lot of them directly in my topic.
Where can I get all mtgox trade data in csv format?
...I want the historic data in text or spreadsheet file form...
To answer your original question, I saw somebody in #mtgox recently posted a bittorrent magnet link of the entire Mt.Gox trade history:
magnet:?xt=urn:btih:175a394988a0d48bdc4d4ea4d85d34c8ab55ba52&dn=MtGox%5Ftrade%5Fhistory-20130429&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce
No guarantee that this torrent will always be working, but it worked for me (and it may save poor Mt.Gox on some bandwidth).
The demo at https://github.com/knowitnothing/btcx/blob/master/demo_2.py downloads every trade from MtGox serially and stores all them in a sqlite database. You can easily stop and resume as needed, as it will take a relatively long time to download all of it. It should be possible to create another program that doesn't work completely in serial mode, but maybe MtGox imposes some kind of limitation per IP address or API key.
To use this you will need to download https://github.com/knowitnothing/btcx as well any dependencies you don't have, which is not necessarily trivial. Also, currently it does everything through the MtGox's Websocket API, so calling into the HTTP API to request data requires defining an API key/secret. That can be done by running the program with the "-n" option.
To use this you will need to download https://github.com/knowitnothing/btcx as well any dependencies you don't have, which is not necessarily trivial. Also, currently it does everything through the MtGox's Websocket API, so calling into the HTTP API to request data requires defining an API key/secret. That can be done by running the program with the "-n" option.
When my windows trading application is finished it will be able to:
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
Hi Yucca, this sounds interesting. Please repost to this thread when you are complete, and I (and I'm sure others) will certainly take a look.
To bitcoinbitcoin113, many thanks for the links. Exactly what I was after at this stage!
OK, the free beta version now has ability to harvest trade history for BTC currency of choice, this gets every single trade for the last year.
The trade history is saved as a CSV file to "C:\ProgramData\Cortex7"
Every time you open up the application it gets the new trades from MtGox and appends them to the file, so you always have a complete local history file that grows without gaps.
This is the CSV format:
UNIXTIME,PRICE,VOLUME
1366249897,101.51,0.01
1366249897,101.37,3.69
1366249904,102.71,0.10
1366249989,106.42,0.80
1366249989,106.42,0.01
It gets the data via HTTP directly from MtGox. This requires many requests; each one downloading one block of trades, then it gets the timestamp of the oldest trade in the block, then it requests another block with that as start time. Repeats until no more data available.
For USD this takes hours the first time as MtGox serves it in a relaxed manner, each subsequent append goes fairly quickly though.
Check my siggy for the latest download link.
When my windows trading application is finished it will be able to:
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
Hi Yucca, this sounds interesting. Please repost to this thread when you are complete, and I (and I'm sure others) will certainly take a look.
To bitcoinbitcoin113, many thanks for the links. Exactly what I was after at this stage!
When my windows trading application is finished it will be able to:
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
>>>
Download complete trade history (ala bitcoinbitcoin113's link above)
>>>
Parse entire trade history into one hour bins.
>>>
Display hourly OHLCV data in the applications chart.
>>>
Export binned hourly data as a CSV spreadsheet file.
Functionality above will be in the free version.
Click weblink below my avatar for more info.
http://bitcoincharts.com/charts/mtgoxUSD#igDailyztgSzm1g10zm2g25zv
Click "Load raw data" then copy paste into excel
The all time chart won't give you higher resolution than 1-day. So you can either go through shorter time periords and copy-paste each (pain), or run a script.
You can get all the trades (actually I think it is down to one second) from here, but I wouldn't recommend opening that in a browser:
http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD&start=0
see this link for just the most recent trades which you can open in a browser:
http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD
But that data is formatted like this, I'm not sure of a good way to get it into a table just using excel:
If you are willing to install R ( http://www.r-project.org/ ) I can give you a script I use that will create a csv or text file you should be able import into excel. There is alot of data though, for me its using 500 mb of RAM, but all you would have to do is run it.
Click "Load raw data" then copy paste into excel
The all time chart won't give you higher resolution than 1-day. So you can either go through shorter time periords and copy-paste each (pain), or run a script.
You can get all the trades (actually I think it is down to one second) from here, but I wouldn't recommend opening that in a browser:
http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD&start=0
see this link for just the most recent trades which you can open in a browser:
http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD
But that data is formatted like this, I'm not sure of a good way to get it into a table just using excel:
Code:
364001159,70.050000000000,0.255309620000 1364001188,70.050000000000,4.150000000000 1364001192,70.040000000000,0.174419680000 1364001218,70.040000000000,0.050000000000 1364001218,69.999900000000,0.350000000000 1364001218,70.050000000000,2.391435110000 1364001229,70.000000000000,0.854495010000 1364001235,70.000000000000,94.145504990000 1364001235,70.048900000000,0.000439330000 1364001235,70.050000000000,5.854055680000 1364001246,69.815010000000,1.451343700000 1364001246,69.815000000000,0.638656300000 1364001249,70.040000000000,0.010000000000 1364001256,70.000010000000,0.024174560000 1364001260,69.815010000000,0.875817030000 1364001262,70.000000000000,0.010000000000 1364001271,70.000000000000,1.000000000000 1364001275,70.000000000000,2.100000000000 1364001360,69.815010000000,1.294111240000 1364001361,69.765300000000,12.744054520000 1364001364,69.999990000000,0.010000000000 1364001396,69.999990000000,1.000000000000 1364001426,69.815030000000,1.204892400000 1364001426,69.815010000000,5.325000000000 1364001426,69.780000000000,18.478026080000 1364001426,69.765300000000,34.205201170000 1364001427,69.660000000000,0.700000000000 1364001427,69.520000000000,4.516700000000 1364001427,69.500000000000,2.477076830000 1364001427,69.460000000000,2.462214370000 1364001428,69.500000000000,1.807338600000 1364001438,69.500000000000,2.096007900000 1364001448,69.500000000000,2.805452540000 1364001464,69.500000000000,0.428383000000 1364001467,69.500000000000,10.000000000000 1364001473,69.500000000000,0.101886060000 1364001480,69.500000000000,0.133700000000 1364001481,69.500000000000,0.114739850000 1364001486,69.500000000000,9.122000000000 1364001489,69.500000000000,0.331857520000 1364001498,69.500000000000,0.636506130000 1364001499,69.500000000000,0.133700000000 1364001501,69.500000000000,5.310900000000 1364001517,69.500000000000,10.000000000000 1364001531,69.500000000000,5.207921600000 1364001532,69.500000000000,5.207921600000 1364001534,69.500000000000,9.122000000000 1364001546,69.500000000000,13.727036830000 1364001547,69.500000000000,10.000000000000 1364001560,69.500000000000,1.113617120000 1364001580,69.500000000000,5.282000000000 1364001582,69.500000000000,5.000000000000 1364001584,69.500000000000,1.835999690000 1364001584,69.500000000000,0.025800000000 1364001600,69.500000000000,2.991999500000 1364001608,69.500000000000,9.556335580000 1364001621,69.817060000000,0.219315010000 1364001665,69.817070000000,2.627984990000 1364001701,69.922750000000,0.365525020000 1364001755,69.960000000000,0.050000000000
If you are willing to install R ( http://www.r-project.org/ ) I can give you a script I use that will create a csv or text file you should be able import into excel. There is alot of data though, for me its using 500 mb of RAM, but all you would have to do is run it.
I've seen lots of charts of Mt Gox trades, but I want the historic data in text or spreadsheet file form for analysis. Where can a lay person obtain this without requiring coding skills? Much appreciated if anybody can help.
Jump to: