pushpool's client developing issues | Bitcointalksearch.org

ius

newbie

Activity: 56

Merit: 0

Really looks like you're the only one to have ever used the binary protocol - anyway, submitted as well. https://github.com/jgarzik/pushpool/pull/40

Thanks.

elmigranto

newbie

Activity: 8

Merit: 0

Quote from: ius on June 30, 2011, 09:40:48 AM

You seem to be right. I have submitted a pull request for Jeff

Could you submit another one (I'm pretty new to CVS, and have sadly too little free time to dig in)?

cjson_encode(), server.c:

Code:

233 /* fill in compressed length */
234 *obj_clen = htole32(comp_len);

It's little mismatched with whole idea. Instead of comp_len, unc_len should be there.

Must be too late in night, when this part was written.

ius

newbie

Activity: 56

Merit: 0

You seem to be right. I have submitted a pull request for Jeff: https://github.com/jgarzik/pushpool/pull/39

(I passed unc_len instead of unc_len + 1, as the last byte is used for null termination and not part of the compressed data)

elmigranto

newbie

Activity: 8

Merit: 0

Turns out it was server-side bug. In cjson_decode() function located in server.c unpacking code contains following lines:

Code:

176 /* decompress buffer (excluding first 32 bits) */
177 comp_p = buf + 4;
178 if (uncompress(obj_unc, &dest_len, comp_p, buflen - 4) != Z_OK)
179 goto out;
180 if (dest_len != unc_len)
181 goto out;
182 memcpy(obj_unc + unc_len, &zero, 1); /* null terminate */

This call of uncompress() isn't completely correct according to zlib docs. dest_len should contain actual size of buffer for uncompressed data before calling uncompress(), but it's never initialized.

This should be corrected (e.g. dest_len = unc_len + 1).

elmigranto

newbie

Activity: 8

Merit: 0

uncompress() actually returns Z_BUF_ERROR indicating the output buffer was too small.
I don't understand how it can be since server allocating uncompressed_length + 1 bytes which obviously enough.

I'm pretty sure that sent uncompressed_length is correct.

elmigranto

newbie

Activity: 8

Merit: 0

By printing out some debug info I've find out that's problem is uncompressing on server-side.

I'm trying to do the following using Qt-built-in zlib (highlighted version):

Code:

char json[] = "{\"version\":1,\"user\":\"test\"}";

std::auto_ptr message(new Bytef[ // allocate memory for:
sizeof(ubbp_header) // + ubbp_header
+ sizeof(uLongf) // + uncompressed length
+ strlen(json) // + message
+ SHA256_DIGEST_LENGTH // + sha256
+ 64]); // + reserve (if compressed size > uncompressed size)

uLongf unc_len = strlen(json);
uLongf comp_len = unc_len + 64;

Bytef* pHeader = message.get(); // header
Bytef* pLen = pHeader + sizeof(ubbp_header); // length of uncompressed data
Bytef* pCompDat = pLen + sizeof(uLongf); // compressed data
Bytef* pSha256; // digest

if (Z_OK != compress2(pCompDat, &comp_len, (Bytef*)json, unc_len, 9))
{
qDebug("Compression failed");
return false;
}

/* calculating message digest */
char pass[] = "test";
uchar md[SHA256_DIGEST_LENGTH];
SHA256_CTX ctx;
SHA256_Init(&ctx);
SHA256_Update(&ctx, pLen, comp_len + sizeof(uLongf));
SHA256_Update(&ctx, pass, strlen(pass));
SHA256_Final(md, &ctx);

/* building message */
uLongf msg_len = sizeof(ubbp_header) + sizeof(uLongf) + comp_len + SHA256_DIGEST_LENGTH;
header.op_size = UBBP_OP_SIZE(BC_OP_LOGIN, msg_len - sizeof(ubbp_header));

pSha256 = pCompDat + comp_len;
memcpy(pHeader, &header, sizeof(ubbp_header));
memcpy(pLen, &unc_len, sizeof(uLongf));
memcpy(pSha256, &md, SHA256_DIGEST_LENGTH);

/* sending message */
socket.write((char*)message.get(), msg_len);
socket.waitForBytesWritten();

And still server-side uncompress() (located in cjson_decode()) doesn't returns Z_OK.
Where's mistake?

elmigranto

newbie

Activity: 8

Merit: 0

Developing pushpool client

Hello,
I'm really interested in jgarzik's pushpool (it's binary protocol in particular) and writing Qt-based client.

Problem is that I can't make it through LOGIN step.
As far as I undestand from forum's threads and (mostly) pushpool's source code, I need to send ubbp_header with BC_OP_LOGIN operation code and message's length in bytes. Message is SHA256 of compressed json updated with userpass following compressed json.

Json looks something like this (protocol is always 1, config is always null):

Code: (json)

{"protocol":1, "user":"username", "config":null}

And I'm doing following (higlighted version):

Code:

ubbp_header header;
   strcpy((char*)header.magic, PUSHPOOL_UBBP_MAGIC);

   QSocket socket;
   socket.connectToHost("my.pushpool.host", 8336);
   socket.waitForConnected();

   // add compressed json to data
   dataToSend = json.toUtf8();
   dataToSend = (qCompress(dataToSend, 9));

   // Since qCompress() adding length (first 4 bytes in dataToSend) in Big-Endian,
   // but server uses le32toh() to get it, change endianness to Little-Endian:
   dataToSend.remove(0, 4);
   uint32_t len = json.length();
   dataToSend.insert(0, (char*)&len;, sizeof(uint32_t));

   // add compressed json digest
   char pass[] = "test";
   uchar md[SHA256_DIGEST_LENGTH];
   SHA256_CTX ctx;
   SHA256_Init(&ctx;);
   SHA256_Update(&ctx;, dataToSend.data(), dataToSend.size());
   SHA256_Update(&ctx;, pass, strlen(pass));
   SHA256_Final(md, &ctx;);
   dataToSend.append((char*)md, SHA256_DIGEST_LENGTH);

   // Send operation code & message size
   header.op_size = UBBP_OP_SIZE(BC_OP_LOGIN, dataToSend.size());
   socket.write((char*)&header;, sizeof(ubbp_header));
   socket.waitForBytesWritten();

   // Send message
   socket.write(dataToSend);
   socket.waitForBytesWritten();

   // Here's a problem:
   // after message sent, I get
   // QAbstractSocket::error() with 'Remote host closed connection'

My assumptions:

JSON compressed by qCompress() cannot be uncompressed by zlib (even after my manipulation)
I'm calculating wrong SHA256
Something wrong with json (i.e. config must not be null)

After puzzling for almost whole day, I'm writing here.
Guys, please help.

Topic: pushpool's client developing issues (Read 1932 times)