ec.PointFp.prototype.getEncoded = function (compressed) {
var x = this.getX().toBigInteger();
var y = this.getY().toBigInteger();
if (compressed) {
var PC;
}
var len = this.getX().getByteLength();
var enc = ec.integerToBytes(x, len);
if (compressed) {
if (y.testBit(0)) {
enc.unshift(0x02);
} else {
enc.unshift(0x03);
}
} else {
enc.unshift(0x04);
enc = enc.concat(ec.integerToBytes(y, len));
}
return enc;
};
Note the code is pulling len from X, and using that length for both X and Y. This is definitely going to be the problem. The length needs to be tested for each coordinate separately (Y should not be using X's length), and zero-padded if it's less than 32. (Will integerToBytes behave properly if you just throw 32 as the second parameter, rather than len?) (Also the possibility of having more than one zero byte needs to be considered. Rare, but the consequences are disastrous if it happens, so it must be handled properly... if throwing 32 in there works, then that should do it, but if not... needs to be checked)
The part that tests compressed can be chucked - Bitcoin doesn't support compressed encoding.
Thank you SO much for this analysis! Thanks to TTBit as well for discovering these two issues.
I can confirm that passing 32 as the length for X and Y will give the proper public key for this private key: 5Je7CkWTzgdo1RpwjYhwnVKxQXt8EPRq17WZFtWcq5umQdsDtTP
It also is working for other private keys who's two points are 32 bytes in length (obviously).
So, seems we have solved one nasty bug. I will do some more testing and investigation of the private key issue where the WIF starts with a 'y'.