Topic: RollerBot - Alpha testing begins soon! - page 3. (Read 18583 times)

3-16-2014

The backend services are pretty much complete, leaned out and optimized!
I'm not going to let too much out of the bag, but I'm currently load testing the daemon system and the results are simply blowing me away!

Currencies supported: 164
Markets being queried:729
API calls being cached: 1,948 @ 20 second refresh

There is still a ton of headroom left on the backend server, the database server and the redis cache also.  All of these small optimizations are really adding up!

Those stats look awesome:
(1077 API calls are being cached @ a 20 second refresh).

Keep up the good work and don't forget to rope me in for the beta program.

3-15-14

Bittrex is fully verified and tested now, as is ANX (ANXBtc/ANXPro)
I've been working on the daemon system quite a bit to make it lighter on resources and give it more control over spawning and tracking threads.  The results are excellent.  In my load tests right now, 7 exchanges querying/caching all trade and depth data for every market in each of these exchanges is in the 35%-40% load range on the backend server (1077 API calls are being cached @ a 20 second refresh).  I still have some optimizations to make, but these results are beyond what I was expecting already.

I have about one more day of backend work to do, then I can concentrate on the website stuff.

all sounds promising must be getting close for release well beta testing anyway.  I know i have not been on much of late but im still here.
keep up the good work.

Very exciting times ahead

I'd like to test your app. It looks much better then most of the existing bots.

Updated my signature with a BTC donation address.

Thanks for the suggestions guys

I've actually been asked about donations a few times now.  I honestly didn't think there would be anyone willing to donate..  I'll get a donation address set up for those interested and post back here.

Regarding infrastructure - I have found at least part-time help with this recently and it has been a tremendous help.  We're actually in talks about the legalities and specifics about retaining him - If we work something out I'll make an announcement.

Regarding other help, there may be a possibility to bring in some limited PHP help just for supporting new exchanges (writing the exchange class files, most of it would be copy and paste)- though to expose even this little bit of the source code to an individual would require some good references and standing in the community.

Another area of help would be someone fluent in the Chineese language (or any foreign language where exchange platforms exist) to help me get accounts set up at these exchanges and do some test trades etc.

Thanks for the great Ideas!

Great update! I was very curious how much processing power would be required. Sounds like you need an infrastructure guy to worry about all the easy stuff so you can get back to the coding and development. If you have any immediate needs or desires you should outline them. Perhaps donations to fund more development servers? Would you be interested in having help beyond all us pestering you with "let me be a part of the alpha test team so i can make easy money with all your hard work?"

3-12-14 Swinging in for a quick update

Getting the infrastructure set up has turned out to be a much larger task than I expected.  Almost the entire last week has been spent testing configurations, running load tests, reconfiguring, testing again, etc.
I'm happy to say that I have finally found the mix of performance, redundancy and cost needed to efficiently and effectively run the platform.  I would even venture to say that I've spent more in server infrastructure so far in development than most (if not all) bots spend in a year.  I'm not quite sure anyone really understands the amount of power it takes to run such a large trading platform (currently 8 servers in total to get through development and alpha testing, and even more when it comes to full release).  I'm beginning to see why something of this scale has not been done yet!

With all that said, things are still moving forward, albeit a bit slower than I originally expected.  Due to costs, RollerBot will launch supporting 10 exchanges.  As a userbase builds, I'll be able to afford more infrastructure to enable support for even more exchanges (in blocks of 10, to keep costs reasonable).

I'll be able to get back into the programming that needs done over the next couple of days now that the final layout for the infrastructure is now set!

Killer day today
I did a lot of work tightening up the backend code.  The cache daemons are running fast and smooth, and the backend plugin system is fired back up (just storing trade data ATM)
Made some changes that reduces the number of MySql calls per second dramatically as well.  Letting it run all night to see how things do

Just wanted to drop in with some progress updates.

I didn't get as much done as I wanted to over the weekend.  Most time was spent on getting the rest of the infrastructure set up on AWS, which included making some important decisions that took some time to thoroughly evaluate.  I'm not going to release too many details, but as of right now RollerBot consists of 5 ec2 server instances (with more to come) and the important backend services are islanded and isolated from the Internet.  The only server attached to the Internet is the web server, which it's self doesn't even have database access.  Many steps have been taken and decisions made to ensure a secure platform.  Now with that out of the way, I can continue marching forward with the code.

Code-wise, I spent a lot of time yesterday cleaning out the source tree of old/antiquated files and making an effort to improve the documentation (through phpdoc). 
Today will be spend finishing off some of the backend services (daemon spawning and load balancing) and then working on some of the user services (private API call daemons, etc)

Thank you everyone for your continued interest and support.  Things are starting to get exciting as everything is slowly starting to pull together

I'm interested to test it as I have many alt-coins as well as an active trader on many exchanges plus an linux administrator. So this fits very well. I've use AWS and Windows Azure somewhat, just let me know if you need anything.

3-8-2014

No code changes today.  All of today was spend in AWS with configuration.
I've recruited the help of an absolutely awesome system administrator to help guide RollerBot in the best direction possible when it comes to the underlying infrastructure.  Most of today was spent discussing future direction and getting things set up to provide the scalability and reliability needed for the trading platform.  A lot of discussion revolved around security and many things were put in place today to ensure a secure platform to build on.  I wish I could share more, but this is leading to the possibility of an exciting announcement in the near future.

I'll have more updates tomorrow

I've hear back from them once and sent another email.  So far they have been great to work with

I know the ANX team personally. Buzz me if they don't get back to you.

I am glad it is going so well.
Also glad to hear you are feeling better.
Sounds like you are really rolling along.

Today was a great day.  A lot of progress is being made on final decisions when it comes to the actual specifics behind RollerBot deployment.  I've brought aboard someone with extensive knowledge in system administration and deployment, especially when it comes to AWS.  This weekend will be a pivotal point for RollerBot as the full groundwork is put into place that will empower the scalable architecture that I dreamed for this platform.  There should be some exciting news in the coming weeks as things are nearing being ready for alpha testing.

Stay tuned

Today I'm mainly going to be working in Amazon AWS console getting things organized (VPC, elastic IP addresses, load balancers, etc), then work on the website starts.  I'll update the thread later with how things go.

Not a lot to report today

I'm still waiting to hear back from some of the exchanges in which I've identified bugs in their API (currently I've hear back from Bittrex with indication of a fix on the way, and ANX (anxbtc/anxpro) with interest in working on solutions).  Therefore there are no new significant changes for today.  I've also contacted some other potential industry partners in order to open clear lines of communication.

Tomorrow I will mostly work on web frontend stuff, and will report back here tomorrow with my progress