One more thing:
Additionally I think that in PHP, result of float <-> integer calculations may differ on 32Bit & 64Bit platforms, however i may be wrong (and I am too lazy to check with Google).
There may be some bugs or php-specific "features" involved.
Topic: why JSON RPC values not use INT64 instead of float string? - page 5. (Read 18438 times)
This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.
Why can't Bitcoin return strings?
Why can't Bitcoin return strings?
+ 1
Floats are a royal pain in the ass. Every bank application programmer will probably tell you that.
Especially RPC-like services should operate on strings - it makes a lot of stuff easier and allows infinite precision.
According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).
For operations of extreme precision, PHP has many sets of mathematical libraries which also operate on strings, not floats.
For example, BC-Math or GMP.
http://php.net/manual/en/book.bc.php
http://www.php.net/manual/en/book.gmp.php
PHP solution: write/maintain my own JSON parser.
Why can't you just multiply the numbers by 1.0e8 and then round to the nearest integer? That integer WILL ALWAYS BE EXACTLY RIGHT (assuming you're not running PHP on some really weird hardware).
According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).
I added a Python JSON-RPC library example on the Proper Money Handling wiki page.
This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.
Why can't Bitcoin return strings?
You're deliberately breaking with all the languages (python JSON-RPC library, PHP JSON-RPC and Perl JSON-RPC). Using floats anywhere in financial transactions is unacceptable.
php's json_decode DOESN'T support anyway to return floats as strings. The option doesn't exist.
PHP solution: write/maintain my own JSON parser.
Neither does Python's JSON-RPC. Have to write my own JSON-RPC lib using json module instead of using the one that already exists.
Same for Perl.
Really, why is it such a big deal? Bitcoin is broken and this needs fixing.
Quote
Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error progragation must be considered when several operations are compounded.
Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....
So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.
Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....
So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.
Why can't Bitcoin return strings?
You're deliberately breaking with all the languages (python JSON-RPC library, PHP JSON-RPC and Perl JSON-RPC). Using floats anywhere in financial transactions is unacceptable.
php's json_decode DOESN'T support anyway to return floats as strings. The option doesn't exist.
PHP solution: write/maintain my own JSON parser.
Neither does Python's JSON-RPC. Have to write my own JSON-RPC lib using json module instead of using the one that already exists.
Same for Perl.
Really, why is it such a big deal? Bitcoin is broken and this needs fixing.
That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh? See that 'import json' statement at the top? That would be the standard (as of python 2.6) JSON parsing library.The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal. If you are doing monetary calculations in python, then you should be using Decimal. That is what it is for.
Not json but json-rpc as recommended by json-rpc themselves.
http://json-rpc.org/wiki/python-json-rpc
That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh? See that 'import json' statement at the top? That would be the standard (as of python 2.6) JSON parsing library.The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal. If you are doing monetary calculations in python, then you should be using Decimal. That is what it is for.
That's code to parse JSONs. There's a Python library to work with JSON-RPC. There's also a PHP library to do JSON-RPC. Both use floats.
Solution A: everybody that wishes to interface with Bitcoin in Python/PHP must write their own (potentially buggy) RPC http code because the default libs for those languages uses floats.
Solution B: a small change is made to Bitcoin.
B is a much better solution
Solution A: everybody that wishes to interface with Bitcoin in Python/PHP must write their own (potentially buggy) RPC http code because the default libs for those languages uses floats.
Solution B: a small change is made to Bitcoin.
B is a much better solution
genjix: here is how to do it right in Python2.6 :
import decimal
import json
# From http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
class DecimalEncoder(json.JSONEncoder):
def _iterencode(self, o, markers=None):
if isinstance(o, decimal.Decimal):
return (str(o) for o in [o])
return super(DecimalEncoder, self)._iterencode(o, markers)
decimal.setcontext(decimal.Context(prec=8))
print json.dumps(decimal.Decimal('10.001'), cls=DecimalEncoder)
print json.dumps({ "decimal" : decimal.Decimal('1.1'), "float" : 1.1, "string" : "1.1" }, cls=DecimalEncoder)
print json.loads('{"blaa": 0.333331}', parse_float=decimal.Decimal)
Produces output:
Note that EVEN IF YOU PASSED THE 'WRONG' strings to Bitcoin, Bitcoin would do the right thing. That is, these two are equivalent once they are parsed by bitcoin:
On the bitcoin side, this is a non-issue. And if code on the other end of the JSON-RPC connection does the wrong thing (truncates values like 10.000999999999999 instead of rounding them to the nearest 8'th decimal place) then that's a bug in that code.
Code:
import decimal
import json
# From http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
class DecimalEncoder(json.JSONEncoder):
def _iterencode(self, o, markers=None):
if isinstance(o, decimal.Decimal):
return (str(o) for o in [o])
return super(DecimalEncoder, self)._iterencode(o, markers)
decimal.setcontext(decimal.Context(prec=8))
print json.dumps(decimal.Decimal('10.001'), cls=DecimalEncoder)
print json.dumps({ "decimal" : decimal.Decimal('1.1'), "float" : 1.1, "string" : "1.1" }, cls=DecimalEncoder)
print json.loads('{"blaa": 0.333331}', parse_float=decimal.Decimal)
Code:
10.001
{"decimal": 1.1, "float": 1.1000000000000001, "string": "1.1"}
{u'blaa': Decimal('0.333331')}
{"decimal": 1.1, "float": 1.1000000000000001, "string": "1.1"}
{u'blaa': Decimal('0.333331')}
Note that EVEN IF YOU PASSED THE 'WRONG' strings to Bitcoin, Bitcoin would do the right thing. That is, these two are equivalent once they are parsed by bitcoin:
Code:
sendtoaddress FOO 10.000999999999999
sendtoaddress FOO 10.001
... because bitcoin does proper rounding.sendtoaddress FOO 10.001
On the bitcoin side, this is a non-issue. And if code on the other end of the JSON-RPC connection does the wrong thing (truncates values like 10.000999999999999 instead of rounding them to the nearest 8'th decimal place) then that's a bug in that code.
Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.
It's one of the many JSON-RPC design flaws. Instead of trying to fix it, I have moved on to working on a new protocol to address all the problems: https://www.bitcoin.org/smf/index.php?topic=3757.0
Yep, just noticed that.
But then the Python JSON-RPC library does not and I couldn't find one for the PHP RPC library either.
BTW, in that wiki page why did you use those lambdas instead of simply using decimal.Decimal? Multiplying by e8 will cause everything like version numbers or difficulty to be multiplied.
But then the Python JSON-RPC library does not and I couldn't find one for the PHP RPC library either.
Code:
>>> from jsonrpc import ServiceProxy
>>> access = ServiceProxy("http://user:[email protected]:8332")
>>> type(access.getbalance())
>>> access = ServiceProxy("http://user:[email protected]:8332")
>>> type(access.getbalance())
BTW, in that wiki page why did you use those lambdas instead of simply using decimal.Decimal? Multiplying by e8 will cause everything like version numbers or difficulty to be multiplied.
Wiki page created: https://en.bitcoin.it/wiki/Proper_Money_Handling_(JSON-RPC)
genjix: You should be calling json.loads(..., parse_float=decimal.Decimal) and use a custom JSON encoder class to convert decimals to JSON strings with no loss of precision...
genjix: You should be calling json.loads(..., parse_float=decimal.Decimal) and use a custom JSON encoder class to convert decimals to JSON strings with no loss of precision...
Python uses floats for it's JSON library. Herein lies the problems.
$ python
>>> import json
>>> json.dumps(10.001)
'10.000999999999999'
>>> json.loads('{"blaa": 0.333331}')
{u'blaa': 0.33333099999999999}
>>> type(json.loads('{"blaa": 0.333331}')['blaa'])
This is unacceptable.
$ python
>>> import json
>>> json.dumps(10.001)
'10.000999999999999'
>>> json.loads('{"blaa": 0.333331}')
{u'blaa': 0.33333099999999999}
>>> type(json.loads('{"blaa": 0.333331}')['blaa'])
This is unacceptable.
Can we not beat this dead horse?
I think there are MUCH more important things to work on / worry about than whether or not "send 1 BTC" is expressed as "sendtoaddress FOO 1.00" or "sendtoaddress FOO 100000000" in the JSON-RPC.
How about we (I'll start) write a "Proper Money Handling" page for the Wiki that discusses the issue and gives code example of how to convert to/from JSON double-precision floating point and 64-bit integer?
I think there are MUCH more important things to work on / worry about than whether or not "send 1 BTC" is expressed as "sendtoaddress FOO 1.00" or "sendtoaddress FOO 100000000" in the JSON-RPC.
How about we (I'll start) write a "Proper Money Handling" page for the Wiki that discusses the issue and gives code example of how to convert to/from JSON double-precision floating point and 64-bit integer?
all Numbers in JavaScript are double-precision floating point
Fortunately for Bitcoin, double-precision floating point represents integers exactly up to 9,007,199,254,740,992 which is above the number of bitcoin base units i.e. 2,100,000,000,000,000.
Because JavaScript doesn't have a 64-bit integer type (all Numbers in JavaScript are double-precision floating point).
This would solve a bunch of problems.
Why not just return the int64 and let the client cast it to a float & divide 10^8 for display?
Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.
Also by returning the value as int64, it will be enforcing good practice on clients instead of them unwittingly using floats.
Why not just return the int64 and let the client cast it to a float & divide 10^8 for display?
Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.
Also by returning the value as int64, it will be enforcing good practice on clients instead of them unwittingly using floats.
Jump to: