Topic: Cryptopia trading simulator development (Read 353 times)

Entry post was updated. Updated demo builds of trading bot (t_bot)

core was updated to 1.1.06b
changelog:
- workaround for issue when "GetBalance" request does not work for some c-currencies
- enhanced "scalping2" strategy

Entry post was updated. Added demo build of trading bot (t_bot)

t_sim core was updated to 1.1.04b
changelog:
- refactoring
- added new trading strategy : "scalping2", configured by "scalping2StrategyConfig" struct in config.json (format of strategy settings is pretty similar to "scalping" config )

t_sim and history_collector were updated to 1.0.16b
changelog:
- MacOSX: fixed random crash during trading with several strategies simultaneously
- All platforms: fixed several potential memory leaks

Entry post was updated. MacOSX build of tSim was added. Tested with MacOSX 10.13 (High Sierra)
HistoryCollector's version was changed to 1.0.15b. Both projects uses same codebase so I do not see reasons to keep separate versioning.

Entry post was updated - MacOSX build of History collector 1.0.15b was added. Tested with MacOSX 10.13 (High Sierra)

tSim was updated to 1.0.15b
- now Backtesting mode disabled by default. (You do not need declare ["isBacktester" : false] anymore for realtime simulation)
- couple minor improvements

Entry post was updated. Linux builds of tSim and History collector was added. Compiled with gcc 5.4, tested with Ubuntu 16.04.

tSim was updated to 1.0.14b
changelog:
 - backtesting was optimized.

tSim was updated to 1.0.13b
changelog:
 - added backtesting mode.

To enable backtesting mode you should add this line to main config file:
   "isBacktester" : true,
to specify database file with historycal data you should add:
   "backtestingDataBase" : "cryptopiahistory.sqlite",
Where "cryptopiahistory.sqlite" - name of the SQLite database file. It should be located in the same folder as tSim.
You can create/fill such file with HistoryCollector1.0.1b.

HistoryCollector was updated to 1.0.1b (link and description was moved to entry post)
changelog:
 - Fixed premature program stop
 - Added ability to specify timeframes to store in database. If skipped no candles will be stored
 
 Example:
   "historyCollectorConfig" :
   {
      "timeframes" : [
      {
         "symbol" : "m5"
      },
      {
         "symbol" : "m15"
      },
      {
         "symbol" : "h1"
      }
      ]
   }
   
Supported timeframes: m5, m10, m15, m30, h1, h2, h4, h8, h12, h16, d1, w1

P.S.: tSim + HistoryCollector = seems now I can add support of some kind of backtesting

tSim was updated to 1.0.12b
changelog:
 - fixed premature stop of some strategy due to condition of "allowable losses" if several different strategies run simultaneously
 - now the profit will be shown only after the completion of all strategies.
 
 Work on tSim project is temporary suspended (for several weeks, I believe)
 
 Reason:
 Cryptopia's Public API uses strange and uncommon way to provide historical data. GetMarketHistory method gives list of closed trades (1000 maximum).
 In practice this means about 2 hours of trading history on high loaded market (like BTC_USDT). Thus, there is no possibility to implement some classic trading strategies based on indicators based on analysis of "candlesticks".
 Seems like I have to implement separate service(project) to collect historical data from Cryptopia and build "candle" streams for several timeframes (m5, m10, m15, m30 etc), and keep them in local storage.
 Of course, I could use third-party source of such "candle" history, but in this case we will have to trust one more participant in trading process. I prefer to minimize count of dependencies.

Next update of this thread will relate to new project.

tSim was updated to 1.0.11b
changelog:
 - strategy logging moved to separate log files (scalping.txt, pingpong.txt). [Not sure about it, could be reworked]
 - couple minor bug fixed (logging)

Very interesting project.

If you had a simulator module supporting multiple exchanges, then I definitely see the use-cases.

The thing i hate most about making trading bots is the fact that you have to write the logic, and then test that logic thoroughly with real-money trades. I believe it was Binance that had a test_trade call that would validate your params and not execute trade.

Also a very helpful tool for beginners who want to dip their toes i crypto-trading.

I hope you will get plenty support and users for your module Alal!

tSim was updated to 1.0.10b
changelog:
 - Improved stability (on client side) during API failures (on server side)

tSim was updated to 1.0.9b
changelog:
 - added new trading strategy - "scalping".
 - added settings for new strategy - "scalpingStrategyConfig". format same as "pingpong" settings.
 - now you can use several strategies. They will run simultaneously. (Do we need parameter for serial/parallel running?)
 Example:
      "includeMarkets" : [
   {
      "symbol" : "BTC_USDT",
      "tradingStrategy" : "scalping"
   },
   {
      "symbol" : "ZEC_USDT",
      "tradingStrategy" : "pingpong"
   }
   ]
   Note: logging still goes into one file log.txt, Probably I should split logging by strategies.
   
Due to Cryptopia team's experiments with private API(or issues) last days, I could not check real profitability of "scalping", but simulation shows much better profit than with "pingpong"(or maybe I just got lucky)

tSim was updated to 1.0.8b
changelog:
 - optimized preparation for trading (less http requests)
 - updated format of trading strategy settings : "totalBaseAmountToTrade" and "orderBaseAmount" are lists now.
 Example:
...
   "totalBaseAmountToTrade" : [
   {
      "symbol" : "USDT",
      "amount" : 10
   },
   {
      "symbol" : "BTC",
      "amount" : 0.10
   }
   ],
   
   "orderBaseAmount" : [
   {
      "symbol" : "USDT",
      "amount" : 2
   },
   {
      "symbol" : "BTC",
      "amount" : 0.001
   }
   ],
...

I choose Cryptopia mostly because I have an Cryptopia's account, so i can compare simulation of autotrading algorithm and autotrading on real account with same algorithm.
Support of new API not so big problem in scale of whole project, i can add another exchange API at any moment(not literally, of course - it can take several days).
All last updates are not directly related to API, they more affect usability and bug fixing. Adding support of another exchange does not make sense since "b" in version mean "beta"

Probably, yes. But I do not worry about my coding skills too much, I have enough experience in software development.

Very good work OP. But: is there any need of simulating trading on cryptopia where the volumes are low and no many trades will trigger?
Or are you practicing your coding skills to make something more valuable in the future? Good luck.

tSim was updated to 1.0.7b
changelog:
 - added exploring status of currency/market before trading start.
 Warning is displayed when currency/market can be delisted/closed.

Example:
http://i65.tinypic.com/2v0k591.jpg

tSim was updated to 1.0.6b
changelog:
- added command line parameter "-s, --sequences" - path to sequences folder (by default: "./sequences")
- added new feature - custom orders sequence processing.

How it works:
You should create .json file with order sequence in it and move it in "/sequences" folder.
Example of custom order sequence is located in /sequences folder in tSim1.0.6b.zip archive:
exampleSequence.json:
{
"orderSequence" : [
   {
      "marketSymbol" : "BTC_USDT", - market symbol
      "type" : "buy",           - type of order ("buy" or "sell")
      "amount" : 40,           - amount of base currency you want to spend
      "price" : 8573.72865433,     - price of trading currency you want to buy
   },
   {
      "marketSymbol" : "BTC_USDT", - market symbol
      "type" : "sell",          - type of order ("buy" or "sell")
      "amount" : 0.00466541,       - amount of trading currency you want to sell
      "price" : 9000          - price of trading currency you want to sell
   }
   ]
}
This sequence means: I'm going to buy "40/8573.72865433 - fee" BTC (= 0.00466541), and then sell 0.00466541 BTC by price 9000 USDT
You can create orders for different markets in one sequence.

After that you should add parameter "runOrderSequence" in main config file and fill it with name(s) of order sequence(s):
Like that:
"runOrderSequence" : [
   {
      "name" : "exampleSequence.json" - name of the order sequence you created in ./sequences folder and want it to run
   },
   {
      "name" : "exampleSequence2.json"
   }
   ],

"runOrderSequence" have higher priority than "includeMarkets".
That means what if "runOrderSequence" is presented in config then "includeMarkets" ignored by tSim - only custom order sequence will be processed.

Example of tSim processing example sequences:
https://i.imgur.com/TnUBzdB

As you can see, tSim simulates filling of each order immediately - no need to wait. Fees is taken into account too.
After each sequence processing resulting balance is shown.

You can use custom order sequence processing feature as profit calculator.

tSim was updated to 1.0.5b
changelog:
- added support for command line parameters.
Currently available parameters: "-h, --help" - params description, "-c, --config" - path to config file

tSim was updated to 1.0.4b
changelog:
- added "stop loss" mechanics. Controlled by "allowableLosses" parameter in trading strategy settings.
Example:
"pingPongStrategyConfig" :
{
    "allowableLosses" : 0.05 - maximum allowable losses = 5%
    ...
    
}

All trading platforms is facing issues of scamming/hacking/bad maintenance etc.
Bad reputation of Cryptopia is more explained with a lack of work with the community than some unsolvable problems on exchange.
I had not any serious troubles for half a year of using Cryptopia.
Many of the complaints that I have seen are created by people who do not understand the technical part of the work of the exchange and / or cryptocurrencies.
Or just they themselves violate basic informational safety rules and compromising their passwords/keys.
Cryptopia definitely has things that should be improved but this platform is not hopeless.

About my project: its kernel is created platform-independent and not tied to Cryptopia API.
And in general, to test the trading strategy, only the source of prices is needed - the balance is virtual, so reputation of the exchange does not really matter in that case.
I conceived this project as a simulator/trading robot is more flexible than typical "magic button" trader which can trade only BTC or ETH with one single strategy.
Cryptopia API was chosen just because I have an account on it.
I plan make project more modular in next development iterations and add API of some other exchange(s), so there's a lot of work waiting for me.
Even if this project does not become popular, as you said "there's plenty of trading simulation", then in any case I'll get a good line in my resume.

tSim was updated to 1.0.3b
changelog:
- trading strategy settings added in config file - "pingPongStrategyConfig"
- sessionDuration parameter moved to trading strategy settings

"sessionDuration" - trading session duration (in hours)
"pricesUpdatePeriod" - periods when orders prices updates to top buy/sell prices(in milliseconds)
"orderBaseAmount" - strategy uses this amount of coins in base currency to create buy orders
"totalBaseAmountToTrade" - maximum of coins in base currency which can be used by strategy

Example:
"pingPongStrategyConfig" :
{
   "sessionDuration" : 24,
   "pricesUpdatePeriod" : 1800000,
   "orderBaseAmount" : 2,
   "totalBaseAmountToTrade" : 10
}

tSim was updated to 1.0.2b
changelog:
- Updating orders' prices to current top prices now set to every 30 minutes
- In simulation mode added network latency imitation. Now simulation more relevant to real autotrading.

tSim updated to 1.0.1b
changelog:
- Reduced frequency of annoying "until the end of the session left" messages and fixed time format if session duration more than 24 hours

Project is frozen, same as Cryptopia.

Added demo builds of trading bot (t_bot):

Main restriction of "demo" build - you can trade only on BTC_USDT market()
t_bot_demo1.1.06b_windows_x86_32:
https://mega.nz/#!3vwVFCSC!rfmhRkASI0C3Dza5Y2Ou4tVNEKYSiOjYXBPscXowRSg
md5:
0af986c919164854afb9671fa2f55fe9

Main restriction of "demo2" build - you can trade on several markets but with minimal allowed amount of base currency (for USDT = 1 USDT)
t_bot_demo21.1.06b_windows_x86_32
https://mega.nz/#!Ci5hDArD!HDivqnT3JUHRV_B4FtxdtZGQXM0yA1uIZ8S9Dqo5B-g
md5:
c9b231ff60bb2a3402745445f5987e5c

General advises:
- try t_sim first
- do not use it on your main account
- do not use autotrading strategies with big amount of coin, remember about "totalBaseAmountToTrade" and "orderBaseAmount" parameters
- check how it's going periodically - there is still "b" in version number (beta)

To switch t_bot from/to simulation mode use "isSimulator" parameter:
"isSimulator" : false,  //trading on real account
"isSimulator" : true,  //simulation

For authorization set fields "publicKey" and "privateKey" in config.json:
"publicKey" : "yourapipublickeylookforitinaccountsettings",
"privateKey" : "yourapiprivatekeylookforitinaccountsettings",

links to t_sim1.1.06b:
windows x86_32
https://mega.nz/#!PmpVjIBQ!Zv1sydc3xWtYkUCUoVgTeCr9zHAbuaspHv6g67KidFw
md5:
0005e2855fedd69e07271b0a78f0d716

links to t_sim1.0.16b:
windows x86_32
https://mega.nz/#!vzgCyaxY!oX2C217dU5XFTOOwOIP1ulj99agLQTNGI9kufagjKlM
md5:
f5891759fdffad0e1cbe57bf7b0d0f4a

linux x86_64
https://mega.nz/#!H6ggHC4S!BKeiVQbFySQdOOTGBt_Q9uDXuHgztjDaEEOi4alW5r8
md5:
3110fd81a5c907cb11cc07de6c105df6

macosx x86_64
https://mega.nz/#!rrxSzCTT!srCE58T8zSDSpiazlMOtVSNuq5QuhrmI54a61HDG49E
md5:
6cac6efcec7ce021fd0a6afdd4a913a2

Always check md5 after downloading archive, always check extracted files with antivirus.

Trading simulator is a console application
Run t_sim.exe to start simulation.

Configuration of simulation is done through editing config file - config.json. It is in the same folder as tSim.exe
parameters:
    "isSimulator" - if true - simulation mode, otherwise - real trading(disabled in this version)
    "simulatedBalance" - you can specify amount of coins for simulation, it is a list - you can use several currencies. (Don't forget about comma after list element)
    "runOrderSequence" - list of custom orders sequences to run
    "includeMarkets" - list markets for trading, if you miss this parameter then simulator will use several markets with highest volume
    "excludeMarkets" - list markets you do not want to use in trading. It has higher priority than "includeMarkets" - if same market specified in both "includeMarkets" and "excludeMarkets", it will be excluded from
trading.
    "pingPongStrategyConfig" - trading strategy settings

Archive with t_sim contains example config file (config.json) and example of custom order sequence(exampleSequence.json). Edit them before use (or leave as is if you just want to see how it works).

If you run autotrading simulation:
- By default simulation lasts 24 hours, also you can specify your virtual balance in config file. Orders filled according to real situation on exchange. Resulting balance will be shown after simulation session expired.
If you run custom order sequence processing:
- Orders filled immediately. Resulting balance and profit will be shown after sequence  processed.

//////////////////////////////////////////////////////////////////////
history_collector
Periodically sends getTradeHistory requests to Cryptopia and saves data to SQL database (SQLite).

links to history_collector1.0.16b:
windows x86_32
https://mega.nz/#!uypCWISR!lkh7iDewnzzyFwN-ROqYfUIJUBcQAYLCvMznDhAthoA
md5:
771cd5a4a5b0c15e10eb2e6a4d998d5a

linux x86_64
https://mega.nz/#!juoA3AyL!Tl0l2fyTjU_akUmuREEuQRIuPJSGZKrNZhm64Yx7Igs
md5:
0469b6d1b5af8630b4b664e75872f8f6

macosx x86_64
https://mega.nz/#!OrwyUIqK!UB5rCaHW-p3rMWg7qcNhTIx5IBVL11IhTbTupGerJOw
md5:
8951368632489de497b95c484a88bdcf

How to use:
0. Edit config.json
1. Run history_collector
2. Wait...
3. Get trade history from database - "cryptopiahistory.sqlite". No need to stop history_collector process - let it fill database all time.

history_collector also builds and stores "candles" (OCHLV) for "m5" timeframe. (I know Cryptopia uses "m15" as base timeframe, but sometimes trading can be very intense, especially during pumps/dumps)

Config file format and log output are pretty similar to tSim.
To specify markets to collect history use "includeMarkets" with "historycollector" as trading strategy.
Example:
{   
   "includeMarkets" : [
   {
      "symbol" : "BTC_USDT",
      "tradingStrategy" : "historycollector"
   }
   ]
   
}