Pages:
Author

Topic: [BETA] MTGox websocket API, testers wanted - page 5. (Read 77653 times)

sr. member
Activity: 379
Merit: 250
I seem to be receiving trade tickers with following unix timestamp:

1365219272

which is a future date:

2013-APRIL-06 04:34:32 UTC

How is the bitcoin price next week? Is it above 100 already?
I knew that C++ code can be quite fast but this seems to be a remarkable breakthrogh Wink

It is just hilarious Smiley
hero member
Activity: 938
Merit: 500
https://youengine.io/
I seem to be receiving trade tickers with following unix timestamp:

1365219272

which is a future date:

2013-APRIL-06 04:34:32 UTC

How is the bitcoin price next week? Is it above 100 already?
I knew that C++ code can be quite fast but this seems to be a remarkable breakthrogh Wink
full member
Activity: 217
Merit: 100
I seem to be receiving trade tickers with following unix timestamp:

1365219272

which is a future date:

2013-APRIL-06 04:34:32 UTC

both "tid" and "date" entries have this wrong time?
full member
Activity: 217
Merit: 100
Here is the corrected rfc6455 framing/masking func in C.

C code! Things got serious in here...  Grin

lol, now someone do an optimised X86 assemler version.

Great site you made by the way Clark! Smiley
full member
Activity: 217
Merit: 100
great work I think!

any palns to publish your socket.io-client implementation on github for the benefit the public?

thanks. with any vanilla socket you just need to use the posted framing func for sending data after upgrade. that's the trickiest part.

that func could even be used to make an arduino into a socket.io client with good performance.

i have a standalone visual studio project (threaded, well encapsulated, no lib dependencies, just MFC) that has just the socket.io logging the json stream to a window if anyones interested.
sr. member
Activity: 379
Merit: 250
great work I think!

any plans to publish your socket.io-client implementation on github for the benefit the public?
full member
Activity: 217
Merit: 100
sorry, can you please explain what you are actually developing?

A realtime charting/trading platform with neural networks for windoze: http://www.cortex7.net/forum/showthread.php?tid=87

So you implement socket.io-client in C? And there was no any existing implementation already in the net?

There seems no existing implementation in C or C++. Closest is objective C. But I chose only to reference only RFC6455 for frame/mask.

I referenced prof7bits python code for the handshake.

I already had socket working in my codebase for mtgox websocket, but as you know its not a reliable data stream.

I just had to add a frame/mask func per RFC6455, this was the only really tricky bit, and I was happy to receive help here with that.

I was using an asynchronous socket with message callbacks, things became alot simpler when I switched to blocking socket. I over-ride base class so it also has timeout function, then I can reset connection if line goes quiet for 1 minute.

I've also coded a JSON parser, not to full spec, it just finds a val string by braching into braces according to passed key path.
sr. member
Activity: 379
Merit: 250
sorry, can you please explain what you are actually developing?

A realtime charting/trading platform with neural networks for windoze: http://www.cortex7.net/forum/showthread.php?tid=87

So you implement socket.io-client in C? And there was no any existing implementation already in the net?
hero member
Activity: 548
Merit: 502
So much code.
Here is the corrected rfc6455 framing/masking func in C.

C code! Things got serious in here...  Grin
full member
Activity: 217
Merit: 100
I think you mean:
Code:
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)(uiByteLen>>24);
*pucOutput++ = (unsigned char)(uiByteLen>>16);
*pucOutput++ = (unsigned char)(uiByteLen>> 8);
*pucOutput++ = (unsigned char)(uiByteLen    );
instead of:
Code:
*pucOutput++ = (unsigned char)(uiByteLen>>24);
*pucOutput++ = (unsigned char)(uiByteLen>>16);
*pucOutput++ = (unsigned char)(uiByteLen>> 8);
*pucOutput++ = (unsigned char)(uiByteLen    );
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;

 Grin

you know... i think you're right!  Embarrassed Cheesy

corrected above codepaste (again Roll Eyes ).
newbie
Activity: 17
Merit: 0
I think you mean:
Code:
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)(uiByteLen>>24);
*pucOutput++ = (unsigned char)(uiByteLen>>16);
*pucOutput++ = (unsigned char)(uiByteLen>> 8);
*pucOutput++ = (unsigned char)(uiByteLen    );
instead of:
Code:
*pucOutput++ = (unsigned char)(uiByteLen>>24);
*pucOutput++ = (unsigned char)(uiByteLen>>16);
*pucOutput++ = (unsigned char)(uiByteLen>> 8);
*pucOutput++ = (unsigned char)(uiByteLen    );
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;

 Grin
full member
Activity: 217
Merit: 100
Here is the corrected rfc6455 framing/masking func in C.

Corrected, as @writhe said to do payload length correctly:

edit: Corrected AGAIN, thx @writhe!!

Code:
///////////////////////////////////////////////////////////////////////////////////////////////////
// CREATE A MASKED DATA FRAME FROM THE PASSED BYTES.
//
// NOTES:
// This implementation has max payload size of 0xFFFFFFFF == 4 GB
// v.pucFrameData is a pointer used to read the data from outside this func.
// v.uiFrameBytes is a variable used to read frame size from outside this func.
// REF: (http://tools.ietf.org/html/rfc6455#section-5.3)
///////////////////////////////////////////////////////////////////////////////////////////////////
void THR_MtGox::CreateFrameData(unsigned char* pucInput, unsigned int uiByteLen, bool bText)
{
const bool B_MASK_ENABLED = true;
const int  MAX_HEADER_BYTES = 14;

///////////////////////////////////////////////////////////////////////////////////////////////////
// ALLOC NEW MEMORY FOR DERIVED FRAME
if(v.pucFrameData)
delete v.pucFrameData;
v.pucFrameData = new unsigned char[MAX_HEADER_BYTES + v.uiFrameBytes];
unsigned char* pucOutput = v.pucFrameData;//temporary pointer, can increment to build the frame

///////////////////////////////////////////////////////////////////////////////////////////////////
// FIRST BYTE FLAGS ("FIN" BIT ALWAYS SET BECAUSE THIS FUNC SENDS A COMPLETE MESSAGE)
if(bText) //TEXTUAL PAYLOAD 10000001
*pucOutput++ = 0x81;
else //BINARY PAYLOAD  10000010
*pucOutput++ = 0x82;

///////////////////////////////////////////////////////////////////////////////////////////////////
// STORE ADDRESS OF BYTE THAT HOLDS MASK BIT
unsigned char* pucMaskByte = pucOutput;

///////////////////////////////////////////////////////////////////////////////////////////////////
// PAYLOAD BYTE SIZE:
// [1] payload length is < 126 bytes,
// the length can be encoded entirely within the 7 bits of the payload length field.
if(uiByteLen < 126)
{
*pucOutput++ = (unsigned char)uiByteLen;
}
// [2] payload length is < 65536 bytes,
// Payload length field is set to 126, next 2 bytes are payload length (big-endian).
else if(uiByteLen < 65536)
{
*pucOutput++ = 126;
*pucOutput++ = (unsigned char)(uiByteLen>>8);
*pucOutput++ = (unsigned char)(uiByteLen   );
}
// [3] payload length is upto pow(2,64)-1 bytes.
// payload length field is set to 127 and the next 8 bytes are the payload length (big-endian).
else
{
*pucOutput++ = 127;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)0x00;
*pucOutput++ = (unsigned char)(uiByteLen>>24);
*pucOutput++ = (unsigned char)(uiByteLen>>16);
*pucOutput++ = (unsigned char)(uiByteLen>> 8);
*pucOutput++ = (unsigned char)(uiByteLen    );
}
///////////////////////////////////////////////////////////////////////////////////////////////////
// MASK ENABLE BIT
if(B_MASK_ENABLED)
*pucMaskByte |= 0x80;
else
*pucMaskByte &=~0x80;

///////////////////////////////////////////////////////////////////////////////////////////////////
// MASKING KEY
unsigned int uiMaskingKey = (unsigned int)rand();
if(B_MASK_ENABLED)
{
unsigned int uiMask = uiMaskingKey;
*pucOutput++ = (unsigned char)(uiMask & 0xFF);
uiMask >>= 8;
*pucOutput++ = (unsigned char)(uiMask & 0xFF);
uiMask >>= 8;
*pucOutput++ = (unsigned char)(uiMask & 0xFF);
uiMask >>= 8;
*pucOutput++ = (unsigned char)(uiMask & 0xFF);
}
///////////////////////////////////////////////////////////////////////////////////////////////////
// ADD PAYLOAD AND XOR WITH MASK IF NEEDED
if(B_MASK_ENABLED)
{
for(unsigned int uiByte = 0; uiByte < uiByteLen; uiByte++)
{
int iMaskOctet = uiByte%4;
unsigned char ucMaskOctet = (unsigned char)(uiMaskingKey >>(iMaskOctet*8) & 0xFF);
*pucOutput = *pucInput ^ ucMaskOctet;
pucOutput++;
pucInput++;
}
}
else
{
for(unsigned int uiByte = 0; uiByte < uiByteLen; uiByte++)
{
*pucOutput = *pucInput;
pucOutput++;
pucInput++;
}
}
///////////////////////////////////////////////////////////////////////////////////////////////////
// CALCULATE RESULTING FRAME LENGTH BYTES
v.uiFrameBytes = pucOutput - v.pucFrameData;
}
full member
Activity: 217
Merit: 100
sorry, can you please explain what you are actually developing?

A realtime charting/trading platform with neural networks for windoze: http://www.cortex7.net/forum/showthread.php?tid=87
full member
Activity: 217
Merit: 100
here is a C mask/frame function that is tested and working, again thanks guys for the help:

The way that you're encoding the payload length doesn't look right to me. There are basically 3 cases:
  • payload length is < 126 bytes,
  • payload length is < 65536 bytes,
  • payload length is upto pow(2,64)-1 bytes.

In the first case, the length can be encoded entirely within the 7 bits of the payload length field.

In the second case, the payload length field is set to 126 and the next 2 bytes are the payload length (using big-endian ordering).

In the third case, the payload length field is set to 127 and the next 8 bytes are the payload length (again, in big-endian).

Woops! I only tested with short message so only tested the 7bits version. Will fix that and post code below.
newbie
Activity: 17
Merit: 0
here is a C mask/frame function that is tested and working, again thanks guys for the help:

The way that you're encoding the payload length doesn't look right to me. There are basically 3 cases:
  • payload length is < 126 bytes,
  • payload length is < 65536 bytes,
  • payload length is upto pow(2,64)-1 bytes.

In the first case, the length can be encoded entirely within the 7 bits of the payload length field.

In the second case, the payload length field is set to 126 and the next 2 bytes are the payload length (using big-endian ordering).

In the third case, the payload length field is set to 127 and the next 8 bytes are the payload length (again, in big-endian).
sr. member
Activity: 379
Merit: 250
sorry, can you please explain what you are actually developing?
full member
Activity: 217
Merit: 100
here is a C mask/frame function that is tested and working, again thanks guys for the help:

edit: See a few posts on for the func....
full member
Activity: 217
Merit: 100
Now I can set about debugging my framing/masking function as i have a working reference frame to verify against.

There are some examples in the RFC that might be useful to check against.

Also, using all zeros for the masking key is valid (you still need to set the masking bit though). This should make it easier to debug as the payload won't be modified by the XOR operation.

Thanks, i missed that, (note to self: READ specs before coding!).

So now i have realised that the payload length bytes are dynamic and are only used if payload size needs them. ahhh so thats why the table has dotted lines for some cell walls. doh!

edit: and thx for the zero mask tip, sure will make things simpler to get going!
newbie
Activity: 17
Merit: 0
Now I can set about debugging my framing/masking function as i have a working reference frame to verify against.

There are some examples in the RFC that might be useful to check against.

Also, using all zeros for the masking key is valid (you still need to set the masking bit though). This should make it easier to debug as the payload won't be modified by the XOR operation.
full member
Activity: 217
Merit: 100
I got the socket.io ticker stream coming in... weeeeee! Cheesy

Thanks to prof7bit, few satoshis coming your way!!!!

I used a hardcoded packet for first send after socket upgrade ala prof7bit's python debug output.

Code:
unsigned char pBuffer[15];
unsigned char* pBuild = pBuffer;
*pBuild++ = 0x81;
*pBuild++ = 0x89;
*pBuild++ = 0x9f;
*pBuild++ = 0x99;
*pBuild++ = 0x10;
*pBuild++ = 0xa1;
*pBuild++ = 0xae;
*pBuild++ = 0xa3;
*pBuild++ =  '*';
*pBuild++ = 0x8e;
*pBuild++ = 0xf2;
*pBuild++ = 0xed;
*pBuild++ =  'w';
*pBuild++ = 0xce;
*pBuild++ = 0xe7;
Transmit_Raw(pBuffer, 15, false, true);

I cant reply to its echo requests yet because my frame/mask func has gremlins. But the data is coming in well until echo request.

Now I can set about debugging my framing/masking function as i have a working reference frame to verify against.
Pages:
Jump to: