Topic: PROFITS - ATPx [Automated Trading Platform eXtended] - page 3. (Read 7655 times)

cant wait
 

I've solved the problem by revising the design.  It seems to be working fine now.

Let me try and explain what the problem was more succinctly and then you'll understand the solution.

You had 2 components at play here.  The ExchangeModule and the UIComponent.
I originally wanted the exchange module to run as a standalone service and just have the UI subscribe to a datafeed that would be streaming off the exchange module.
This design was unworkable because the UI isn't particularly interested in the exchange data so much as the signals data being generated by the other modules.
Because of the complex interplay between signals and exchanges, it made more sense to have the UI marshall the exchange and signals modules and for expediency they should all share a static informational nexus point.

The exchange module(s) update the nexus with the most current exchange data and nexus notifies the signals modules and ui modules with the latest data.

The problem was that the UI, because it was using JavaFX was forcing all javascript to run through the Event Dispatch Thread.
There is a lot of javascript at play.  A lot of it is just hooks to allow scripts to run and implement some custom functionality at key points, but the fact is there is a lot of it.

JavaFX for some reason is forcing all contexts onto a single context and a single thread.  This means that any data no matter what thread it was running on would have to wait for the EDT to fire an action before it could be processed.  This of course caused a back-up and even a halting to occur.
 
The solution was to drop JavaFX as the UI toolkit.  It was causing problems and I'm just lucky I caught it early enough.

The new design, which appears to be working, uses the NetBeans Platform and nbm bundles in place of what we were calling modules before.
The current UI is mostly swing, it really didn't take that long to re-implement and I think I like the netbeans gui designer a lot more than the JavaFX gui designer

I'll be posting some screenshots soon.  Also I'm waiting on the logo to come back then the site expressvest.com will be up later today or early tomorrow.

I could be missing the point as well. I like the UI idea. I love the limit orders and BTC-e support, MTGoX streaming ticker etc. Is there a way to make a  small change to have to UI only send data and get replies?

What I mean is maybe instead of having the modules as part of the UI apparently only have the UI send the module when it periodically checks the bots. Like send trading logic, request balances or send some specific trade related action command. If the Bot sees something have it act then send the results to the UI. That way the UI could try working on them whenever but the bot can still fire fast when a trade arises. I would worry that the trade might be held up waiting for a UI especially if different machines are used for bots with one UI.

I am betting it would require a fair amount of rewriting though.

I was thinking the exact same thing.  The "engine" and the way you "talk to the engine" are completely different animals (should be?) but I have not looked at any of the code or tried to use the software so my opinion means little in this context.  My sense, again not looking at any other this beyond the posts in this thread, is that you should be talking to the engine with a script language of some kind.  Lisp, for example.  Why do you need any user interface at all at this point?  (again, forgive my lack of knowledge on this specific software).

I don't really understand your overall design, but it sounds to me, like your UI is tied too closely to the API implementations.

I wouldn't even tie any UI to the trading engine. Was one of my mistakes in my app design.

Ok so this might be a design breaker.
I've been slow on release because I wanted to make sure everything works correctly the first time for 99% of you with minimal setup and watching.

After I completed the MtGox module I noticed that it would run for awhile and then it would just sort of drop out.
It was working fine before I implemented the JavaFX based UI, and so I stripped the UI out and tried again, sure enough it worked just fine.

This of course begs the question, what the heck is the UI doing that's causing the MtGox module to choke up especially when the BTCe module seems to be alright.

As it turns out it has to do with the way the MtGox module works.  Since the MtGox module uses the streaming websockets API, it has an event driven architecture and consumes an "OnMessage" event each time a message arrives.  This message is passed to the scripting engine for further processing because frankly that's the easiest way to deal with it.

The problem is that JavaFX appears to be overriding the scripting API or forcing it to run on the event dispatch thread even though the MtGox connector is actually running on it's own thread.  This means that the UI has to fire an event for all the backed up MtGox events to be processed.  I'm assuming that JavaFX forces the scripting engine onto the event dispatch thread because it uses a lot of javascript itself, trying to be an HTML5 compliant tech and all.  However it's unacceptable for ALL javascript to be running through the UI.

The obvious solution is to completely refactor the MtGox connector so that it doesn't rely on the scripting engine at all.  However the scripting engine is used in numerous places throughout the code.  Not the least of which is the custom programmable trading logic module.  Since it's all event driven, having the system hold off on processing events until the UI fires an event onto the Event dispatch thread is not going to work out well at all.

I'm going to put more thought into this.  I don't want to chuck out all the work that's been done on the UI so far.  However it may be that I end up dropping JavaFX in favor of Swing if I can't figure out any other way around it, but generally speaking I'd prefer not to do so.

Has anyone worked with JavaFX and run into similar problems?  Thanks!

Good news everyone!

As you may or may not be aware, I really do suck badly at art.
I wanted this product to have a fun, fresh, clean & professional feeling to it.  I didn't want it to be loaded down with unprofessional programmer art, so I set out to find an artistic director.  I needed someone with a strong portfolio of artwork that would lend itself to exactly the look and feel I wanted to give the product.
Someone who can be responsible for the artistic aspects of the site, the documentation and the product.

I found one and boy what a catch! 
Everyone please welcome aboard Alex Wendzel

Alex is probably best known for his webcomic Out At Home, www.out-at-home.com
A fantastic little comic about a retired baseball allstar, his family & friends and their adventures.

I was inspired to ask him to join us after seeing this comic http://www.out-at-home.com/archives/2940 which pretty much sums up my own fears about how complex this product is starting to get. 
So everyone please welcome Alex aboard and check out his site, it's a lot of fun and he does a great job.

Your description appears to be correct for open net trades.

Your bot would just participate as a regular trader on the p2p network and function as a filter against the exchange's pricing.
It would read in the current pricing, add in the profit that it wants to see, and handle the negotiations on both sides.

Effectively shouting in a room either
"Hey guys, I got BTC, fresh hot BTC only $105 USD, I take paypal, okpay, or sepa, come an get 'em!"
or
"Hey guys, I got USD, warm, smooth and easy on the digestion, I'll give ya 105 of 'em for just 1 BTC, that's right folks I said 1 BTC for 105 of these tasty USD's, come over and see me!"

The answer is, it depends on your configuration.

There is only a single bot per exchange.  This is to reduce system load.
However there is a highspeed, hightrust option that you can configure.  
This sets up up a direct connection between your bots and causes the bots to be intimately aware of each other's internal state.
Currency balances are considered internal state.
In this setup the bot's represent a currency as source.symbol
e.g. MTGOX.BTC or BTCE.BTC
It also causes them to preference the desires of the other bot.

This configuration is intended specifically for exchange to exchange arbitrage on a single pair.  BTCE.USD/MTGOX.BTC
However I haven't finished the closed loop solver on it so it can only arb as long as there are funds available at the source.
When the closed loop solver is complete it will seek to match the exit method of one exchange to a supported entry method on another exchange.
Example (note, that : means transfer and | means trade in this example)

The closed loop solver will ONLY be enabled in for owned bot arbi mode.
It's not helping any that OKPAY doesn't want to do business with exchanges anymore.  I expect the exchange modules are going to be the biggest upkeep responsibility here.

Not as much as know what it is willing to pay to keep a trade going. Yes likely it would to some extent either learn or at the very least catch up to where no one would sell at a price the bot would pay.
As long as BTC-e bot says buying 100$ at 100$ per BTC and the MTGox bot knew GoX price was 107 the trade would likely continue until the bots had a very similar price.
I do believe there is an overarching control system that likely would know what tickers values are at the exchanges you have bots at. But it wouldn't require a bot to make money from it as long as your bot is buying when its profitable for your bot.

EDIT:
Likely scenario would be reversed. Gox bot being higher price would post an ask for USD at say 105 with the current price at 107. BTCe bot would jump on it noting that it would make a 5% profit.....

But the btc-e bot must stop the order, if there's no profit! Not just buy the btc at any price! It has to know the data from the gox bot?

Your BTC-e Bot can place an order to the whole network. An order asking to buy some amount of BTC for some other currency. Another bot, yours or someone else's, can take part of your order and complete the trade. You wouldn't need a Gox account to Arbitrage MTGoX. Your MTGox bot could help with or complete the arbitrage for you depending on size. It sees an order for 100USD for some 1 BTC say. The gox bot sends some amount of that order. Lets say the price on Gox is 107, the bot jumps on the order noting a 7% profit. So on Gox some slightly under 1 BTC is sold and 100$ is sent to your BTC-e account. If there isn't enough interest in the order or profit the Gox Bot would take a pass and not trade. if there isn't enough funds then the Gox bot would trade part of the order.

If you are using the darknet option and depending on how the upper controls work it is possible that a total price and account values would be available at your controller. It sends some signal to your bots and they handle it. Initially there isn't a darknet option working so I don't know for sure. I would know more when it gets released.

Uhm...sorry, I don't get it...

Let's say, I want to do arb from btc-e to gox. I buy btc at btc-e and sell at gox. What matters is the difference of the price at the 2 exchanges, so no bot can place an order without knowing about the other bot?

Or did I misunderstand the whole concept?

Pardon my ignorance...

Judging by the description I would say they don't. One bot puts up an ask another if there is enough money bids for part of the transaction. No bot needs to know about an opportunity directly. If there is an opportunity the ask will be filled.

How do these bots communicate the total funds to know if arb is feasable?

I promise to get a proper architecture document up when I release this thing to testing.  I'll probably make a wiki and link it to the javadoc or something. 
Arbitrage is just 2 (or more) bots talking to eachother, which is why we have the expressnet p2p network.  If you're going to arbitrage 2 or more exchanges you just setup 2 or more bots and have them connect to eachother (you can darknet this by preloading their keys which will mean they don't talk to other bots).

Sorry no you didn't say anything like that. I was just thinking out loud on the one per pair. I was thinking of having one specific host per bot per exchange. Then I thought maybe there would be even less latency if it was one bot per pair. Sadly at some point the connection would saturate leaving the same latency issue just farther along. That and having a number of bots all on one API and keypair likely wouldn't help either setting up multiples would work but even then its more load on the server for maybe a small boost in trading speed.

Anyways on ATPx once more exchanges are added it would be fairly simple to add another bot to handle say MTGoX then have them able to arbitrage the difference.

If I understood you correctly you are going to have a streaming ticker on ATPx as well.

FYI folks, I'll be changing nicks again soon.
I get too many inquiries about NovaCoin which has nothing to do with me.  However I am working on a new coin too (not coming very soon so don't hold breath) and the email traffic makes it hard to sort who's asking about what.

I've always liked A Midsummer Night's Dream, so I'll be likely be taking up the name Auberon here shortly (unless of course it's taken already too).  If they don't let me change nicks here, I may end up noobifying myself again like I did after the Isis incident.  Sure wish I could hold onto a Nick for awhile.

On another note.
Come the 15th of July, I'll be opening boards for ATPx on the expressvest.com site.
If you click before then, you'll notice the distinct absence of anything resembling a site.

I just re-secured the rights to the expressvest name.  It was originally for a fully licensed banking/trading/investment site and part of the OpenPay & OpenBank framework, but the investors fled the minute MtGox got their US assets seized.  I just managed to finally get the keys to the domain name handed back to me from the dissolution of the partnership.

We'll be adopting the ExpressVest name for the new business that will be sponsoring this project.  The software will be re-branded expressvest and whispernet will be probably be re-branded to something like ExpressNet.  Here's hoping Citizens Bank doesn't send me a C&D for trademark infringement.  I don't see them as closely related, but it wouldn't be the first time I've had to change direction due to a legal filing.

I'm open to other names for ExpressNet, if anyone feels like racking their brain on it for awhile.

Yeah it's one bot per exchange by default.  However there is nothing preventing more than 1 per exchange, nor multiple exchanges per bot.  It's whatever you have resources for.
Not sure where you got the idea about a single bot per currency pair.  That was never my intent and I hope I didn't say anything like that.  The closest I can think of is the older ATP design where it was one pair per thread.  But that had serious limitations.

We're using an observer pattern with publish & subscribe as well as DSP.

Nova! what would be better from a personal setup type system. I am thinking having one bot per exchange would make each bot more responsive to that exchange. I would assume that the benefit to one bot per currency pair would be limited. I also assume one bot per user limits its speed. I am not sure that I have it right but I think I do.

I am excited about this release. It seems to have a lot of potential