Topic: FOSS Algo Trading Platform (Read 1219 times)

Sounds all pretty good. Delighted that you're choosing such a model and hope to collaborate.

On the term exchange vs broker dealer: if Alice and Bob exchange digital cash for digital cash, this is indeed an exchange settled through blockchains as soon as the money moves outside the exchange. This definition depends on the soundness of the settling algorithm - for example see the malleability issue. Bitcoin does not have a full accounting system, and the semantics are not clear. If an exchange XYZ settles trades between members that is not a true settlement via blockchain, just an internal settlement. But we can imagine a model where XYZ opens up the books, so to speak. That's what I call a distributed or transparent exchange (as opposed to P2P, which does not work IMO). But digital cash for fiat goes through central banks and therefore in that case the fiat institutions clear the trades, and impose restrictions through AML and other rules. Doing that as P2P is certainly not going to work.

About polling: we use a Token Bucket algorithm to minimize query latency.  If a site allows 600 requests in 10 minutes, we actually will make up to 600 requests in the first instant, then wait for the 10-minute window to expire before issuing the next 600 requests.  This approach gives much lower data latency than a simple rate like "1 per second."  We still need to do some work here to space out redundant queries.

Also, the event architecture allows us to share tick data with any number of strategies, so the polling process only happens once no matter how many strategies are active.  The model here is to accumulate real time data, not to poll for conditions.  Instead of saying "fetch some data then analyze it", you write a strategy to accept new data and incrementally update its signals.  Then it can react as soon as new data is received, instead of being on a polling timer.  There are a couple markets now providing streaming data (finally) so again, writing your strategy to work incrementally will pay dividends as the data providers get more sophisticated and start to implement push feeds.

I also like antlr and could definitely see the usefulness of a DSL once we have the baseline system working.

I eschew the term "exchange" because none of the providers are actually exchanges– they are broker-dealers with deposit accounts.  Instead, Coin Trader uses the term "Market" which generically may be a broker-dealer or an exchange (although no true exchanges exist yet, AFAIK).

Coin Trader allows you to work with either generic Listings (BTC.USD) or what we call MarketListings which are tied to a specific Market (BITSTAMP:BTC.USD).  In your use case, you will write Esper to capture all Trades which are joined to one of five specific MarketListings, then you will get the individual prices for each Market.  You may then place an Order by specifying a specific MarketListing to trade on a particular Market, or you can place an Order for a Listing which may be spread across Markets, as the order routing engine may decide.

Cryptocoin Partners will be a commercial organization with additional features & services on top of the open source platform, so the packages were migrated to .ORG to emphasize the open-source nature of the released code.  cointrader.org is already taken by someone else, so it is unavailable.  The commercial Cryptocoin Partners company will offer market making and private investment opportunities using our proprietary order routing & machine learning.  We have no intention of crippling the open-source code; we just do not want to make public our own order routing and trading strategies.

The license is Apache 2.0.  I tagged the github project as such but forgot to add a LICENSE.txt file... will do now.

More documentation about the whole system will follow in the next couple weeks as I prepare for the presentation and launch on the 23rd.

Yeah, I'm about to add HyperSQL as the persistence layer to my app. But the lack of it was one of the reasons, why I run mainly strategies that don't need long term data. And then I still want the option to run bots on hardware like smartphones etc, which just don't have a lot of memory.

Yes, you want local copies of remote state. All exchanges currently use HTTP as a transport protocol which is incredibly inefficient. If you look at some of data loads of the requests there is a lot of room for optimization. Standardizing all of this is some hard work. It would also be nice to have a public data store for historic (tick) data, as well as some standardized performance measurements. The only project which has ever done opensource Algo trading, at least that I know of, is marketcetera. They had 300kLOC at some time, although I suspect the quality of the code base is mediocre (http://code.marketcetera.org/ ). Traditionally this kind of software costed mid 6 figures. Doing proper strategy containers, performance reporting, order execution, ... is not easy to get right.

@TimOlson: I got the same problem, too. I can execute drools rule sets with included java to access a depth, or so. But I'd like to get rid of this java for several reasons. The syntax (especially for the BigDecimal based prices and amounts) is ugly. I wanted use (and started just a bit) an antlr grammar for a trading language. I thought the advantage would be, that it's not just a definition of a language, but you can use it immediately to create your own compiler just by adding statements to the production to create code for esper or drools. I have to admit, that I've not heard about esper before your posting, so I cannot really comment yet how good it is for this purpose.

@benjyz: in my solution this class is a TradeSite

https://github.com/ReAzem/cryptocoin-tradelib/blob/master/core/src/de/andreas_rueckert/trade/site/TradeSite.java

but: you should never access a trade site directly! Most of them have strict rules how often you are allowed to make requests per second/minute/. Problem is, that you might have x bots running and you don't know what bot requests which data at which time. That's why I have a central ChartProvider class (not complete though), that controls those requests and also caches the results, so 2 bots requesting the depth for pair ab might actually only trigger 1 request to the actual exchange.

I also have situations in which a strategy applies to x exchanges or loops over all the exchanges to check for a certain condition.

This is very cool. Thanks. Some comments:

* as far as I can see there isn't an exchange class. An important and difficult use case is handling N exchanges at the same time. For example say a trader has 1 BTC and wants to exchange for x$, where x = current last price on average over 5 exchanges. Then he wants to issue an order depending on the market rates through an abstractInterface ("routing"). basically the platform provides an abstraction for MTF's. An even more abstract pattern would be routing to exchanges with certain parameters (buy 25% at 4 exchanges each, if they are reliable).

* naming of same packages on this level [1] - what is cryptocoinpartners? what is "module", "bin"?

* I have developed my own semantics which I will publish in 1-2 months. Interoperability will become more important over time. There is certainly a lot to be done in this area, and I'm glad there are these opensource projects. There were are a couple of closed source ones, which misses the point.

* License?

[1]
https://github.com/timolson/cointrader/tree/master/src/main/java/org/cryptocoinpartners

I like rules engines too, and have used Drools in the past, but for this application you really gotta check out Esper http://esper.codehaus.org/ which is at the core of Coin Trader.  Esper is like a rules engine for time series, and in combination with Coin Trader's schema, you can create all kinds of triggers and signals on market events:

@When(" select avg(price) from Trade.win:time(30 sec) where symbol='BTC.USD' ")
void averagePriceChanged( double avgPrice ) {
  if( avgPrice < triggerPrice )
    placeMarketOrder( Listings.BTC_USD, Side.BUY, 1.0 );
}

But it's still Java and Esper– not for beginners.  Our goal is to support machine-learning based trading, so anyone who could make strategy writing more accessible would give a lot of value to the project.  From an operational point of view also, a simple deployment / configuration language might make a lot of sense.  I'm not sure what you have built or want to build, but if you connect with me on github or LinkedIn or email me at shamme/gmail-com, I'd be glad to discuss it more discretely.

Thanks for the great comments.  ATP gets the job done in a very direct way, too ad-hoc for my liking.  Coin Trader's biggest contribution so far is the schema and architecture.  Our polling implementation offers superior reactivity, and the Esper engine is perfect for time-series analysis.  Esper also allows us to easily record and replay events exactly, whether in simulation, paper-trading, or live trading.

We have some better documentation now on the Readme:
https://github.com/timolson/cointrader

I'm getting interest from devs here in San Francisco will be making an intro presentation on June 23rd to the SF Bitcoin Devs group:
http://www.meetup.com/SF-Bitcoin-Devs/

BTW: take a look at the various ATP* projects. Also a lot of java trading code there.

Awesome. Good to see a f/oss project that has to do with trading. I hope enough people see this and you get enough volunteers to finish it up.

Sounds like a very cool project. I'm working on java trading code, too and might be somewhat further in _some_ aspects, since I have a running bot for a couple of months. I might be willing to donate some code, if you can specify, where you need help. I'm also interested to share some code/development in some areas, where I'm not that far. Like a simple trading language to define loadable strategies. I wanted to use drools for that.

Announcing Coin Trader, an open-source, Java-based backend engine for algorithmically trading cryptocurrencies.  It integrates with existing libraries for market data and order execution, plus it provides persistence and an event-based (Esper) architecture for backtesting, algorithm design, and live trading.  It is not quite ready for live trading, but with a few additional contributions we will have a robust algo trading platform.  See the github site for documentation, and I am happy to provide support via email: shamme – gmail/com

https://github.com/timolson/cointrader

Tim Olson
http://linkedin.com/in/olsontim