Developers - Best practises for decimal handling

Xenland

legendary

Activity: 980

Merit: 1003

I'm not just any shaman, I'm a Sha256man

Quote from: deepceleron on February 19, 2013, 08:28:44 PM

Your site may need more integer math accuracy than 1 satoshi = 1

example: 1 satoshi - 1.3% fee = .987 satoshis

>>> print int(.987)
0
>>> print int(round(.987))
1

However, even using arbitrary precision, these libraries don't necessary give you the correct answer anywhere near the magnitude of the float precision defined.

>>> getcontext().prec = 6
>>> Decimal (.00000001) * Decimal (.987)
Decimal('9.87000E-9')

>>>getcontext().prec = 2000
>>> Decimal (.00000001) * Decimal (.987)[/b]
Decimal('9.87000000000000009104248083235174268563047313600349515597682205532708579013502123676582211686536 538763903081417083740234375E-9')

It's close enough though - you'd get an extra 9 satoshi if your transaction was 10 billion BTC.

Assuming the total BTC in circulation limit was increased this could ever actually happen yes.

TheButterZone

legendary

Activity: 3038

Merit: 1032

RIP Mommy

Just going to leave this here, since I'm not a dev: https://github.com/zootreeves/blockchain.info/issues/5

deepceleron

legendary

Activity: 1512

Merit: 1036

Your site may need more integer math accuracy than 1 satoshi = 1

example: 1 satoshi - 1.3% fee = .987 satoshis

>>> print int(.987)
0
>>> print int(round(.987))
1

However, even using arbitrary precision, these libraries don't necessary give you the correct answer anywhere near the magnitude of the float precision defined.

>>> getcontext().prec = 6
>>> Decimal (.00000001) * Decimal (.987)
Decimal('9.87000E-9')

>>>getcontext().prec = 2000
>>> Decimal (.00000001) * Decimal (.987)[/b]
Decimal('9.87000000000000009104248083235174268563047313600349515597682205532708579013502123676582211686536 538763903081417083740234375E-9')

It's close enough though - you'd get an extra 9 satoshi if your transaction was 10 billion BTC.

davout

legendary

Activity: 1372

Merit: 1008

1davout

Quote from: Zeilap on February 19, 2013, 02:03:46 PM

They're not built-in types, they're types provided by a library.

I call built-in what's part of the standard library of a language. So to me it's built-in the languages I listed but not C++ or PHP for example, because these require an external library.

But yes, I don't know of any language that will natively interpret a numeric string as a decimal type without explicitly telling it to.

Zeilap

full member

Activity: 154

Merit: 100

Quote from: davout on February 19, 2013, 12:39:52 PM

Quote from: Zeilap on February 19, 2013, 12:07:11 PM

no languages (that I can think of) provide built-in type support for arbitrary precision arithmetic

Quote from: Zeilap on February 19, 2013, 12:07:11 PM

and for good reason - it is expensive in terms of computation.

If I follow you, languages shouldn't provide standard interfaces to do network I/O, because you know, that's computationally expensive and slow. I guess I won't follow you...

They're not built-in types, they're types provided by a library. What the original poster seems (to me) to be confused about is that they can write code similar to

Code:

var x = 1.234

in some language and it will magically make x into a 'Decimal' object, and that only javascript will 'convert' this into a float/double. This is not true - they all will, as I say, because no language has a built-in type representing decimals, 1.234 is already a float before it gets assigned to anything. If you want decimals, then you have to specifically create one like

Code:

var x = new Decimal("1.234")

or however your library requires.

davout

legendary

Activity: 1372

Merit: 1008

1davout

Quote from: Zeilap on February 19, 2013, 12:07:11 PM

no languages (that I can think of) provide built-in type support for arbitrary precision arithmetic

Quote from: Zeilap on February 19, 2013, 12:07:11 PM

and for good reason - it is expensive in terms of computation.

If I follow you, languages shouldn't provide standard interfaces to do network I/O, because you know, that's computationally expensive and slow. I guess I won't follow you...

Zeilap

full member

Activity: 154

Merit: 100

Quote from: Xenland on February 13, 2013, 08:04:45 PM

Quote from: Seal on February 13, 2013, 07:23:13 PM

Quote from: davout on February 13, 2013, 01:44:06 PM

Any decimal number can be exactly represented this way.

Unless you are using javascript which treats decimals as floats Huh

I was waiting for someone else to point that out as well that there was no language being referenced.

No, this is not specific to javascript - no languages (that I can think of) provide built-in type support for arbitrary precision arithmetic, and for good reason - it is expensive in terms of computation.

What people are trying to say is that you should be using a library which provides a Decimal class which does provide this functionality (or a subset), and not use floating point values at all for computation.

davout

legendary

Activity: 1372

Merit: 1008

1davout

Quote from: Seal on February 13, 2013, 07:23:13 PM

Unless you are using javascript which treats decimals as floats Huh

That just means javascript doesn't support decimals, only floats.
Using floats for display is fine in most cases, using floats for accounting is terribly wrong.

Seal

donator

Activity: 848

Merit: 1078

Quote from: Xenland on February 13, 2013, 08:04:45 PM

Quote from: Seal on February 13, 2013, 07:23:13 PM

Quote from: davout on February 13, 2013, 01:44:06 PM

Any decimal number can be exactly represented this way.

Unless you are using javascript which treats decimals as floats Huh

I was waiting for someone else to point that out as well that there was no language being referenced.

Yes, the only way I thought of getting around this is to store multiple values in different formats for different languages. ie. store both integer and decimal or float.

It would be a pain otherwise to have a website which depends on javascript to manage the correct decimal display output. Every single value return or shown would require the use of javascript.

Xenland

legendary

Activity: 980

Merit: 1003

I'm not just any shaman, I'm a Sha256man

Quote from: Seal on February 13, 2013, 07:23:13 PM

Quote from: davout on February 13, 2013, 01:44:06 PM

Any decimal number can be exactly represented this way.

Unless you are using javascript which treats decimals as floats Huh

I was waiting for someone else to point that out as well that there was no language being referenced.

Seal

donator

Activity: 848

Merit: 1078

Quote from: davout on February 13, 2013, 01:44:06 PM

Any decimal number can be exactly represented this way.

Unless you are using javascript which treats decimals as floats Huh

TheKoziTwo

legendary

Activity: 1552

Merit: 1047

Quote from: Xenland on February 13, 2013, 12:17:13 PM

Quote from: TheKoziTwo on February 13, 2013, 10:44:40 AM

Quote from: Xenland on February 12, 2013, 11:46:54 PM

Quote from: TheKoziTwo on February 12, 2013, 10:36:48 AM

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

Probably the way binary is done(hint). Binary doesn’t know a "decimal" and computers(CPU) can only "auto" guess the required precision or their calculations could take as long as infinity to compute(hence why we have a math symbol for repeating decimals...) thus most "Test" cases will look like its working correctly but it will break at some point.

BCMath uses arbitrary precision and will never fail as long as you round and use a big enough scale. If you think otherwise I would love to see proof / example of how it can fail.

davout

legendary

Activity: 1372

Merit: 1008

1davout

Some folks still don't get it apparently.

Floats are represented with a sum of powers of 2, any integer can safely be represented in binary since 2^0 = 1.

So for example 0.75 can be exactly represented in floating point representation as : 2^-1 * 2^-2

But some other numbers can not be represented exactly in floating point format, no matter how much bits you use.

Use a decimal type, it's internally represented by an integer times a negative power of ten.

0.75 in decimal is represented as 75 * 10^-2.

Any decimal number can be exactly represented this way.

Xenland

legendary

Activity: 980

Merit: 1003

I'm not just any shaman, I'm a Sha256man

Quote from: TheKoziTwo on February 13, 2013, 10:44:40 AM

Quote from: Xenland on February 12, 2013, 11:46:54 PM

Quote from: TheKoziTwo on February 12, 2013, 10:36:48 AM

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

Probably the way binary is done(hint). Binary doesn’t know a "decimal" and computers(CPU) can only "auto" guess the required precision or their calculations could take as long as infinity to compute(hence why we have a math symbol for repeating decimals...) thus most "Test" cases will look like its working correctly but it will break at some point.

TheKoziTwo

legendary

Activity: 1552

Merit: 1047

Quote from: Xenland on February 12, 2013, 11:46:54 PM

Quote from: TheKoziTwo on February 12, 2013, 10:36:48 AM

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

Xenland

legendary

Activity: 980

Merit: 1003

I'm not just any shaman, I'm a Sha256man

Quote from: TheKoziTwo on February 12, 2013, 10:36:48 AM

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

TheKoziTwo

legendary

Activity: 1552

Merit: 1047

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

davout

legendary

Activity: 1372

Merit: 1008

1davout

Quote from: Seal on February 12, 2013, 08:28:41 AM

Javascript treats floats and decimals as the same type. When performing number manipulations for visual display then some issues might arise. The tricky aspect is... how do i use an Integer number format and have javascript run calculations whilst returning numbers in correct format? Can any javascript developers out there share their thoughts?

Interesting.

If it's purely for display purposes I'd say a float is an acceptable approximation.
But of course it really depends on the specifics of your problem, what works for me might not work for you.

You'd probably better in-depth advice if you gave a bit more context

Seal

donator

Activity: 848

Merit: 1078

Quote from: davout on February 12, 2013, 05:33:01 AM

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

Don't ever listen to anyone suggesting floating point for financial calculations, they don't know what they're doing, no matter how much precautions and care you take floats are and remain approximations that are not suited for monetary amounts.

If you know the difference and know what you're doing there's really no need to use integers as storage type and handling type. Using an integer to store a decimal value in a high level language would show that you don't know exactly what you're doing.

The reason MtGox will return integers in their API responses is because there is no way to discriminate between decimal and float types in JSON, they are represented the same way. Therefore, if your parsing lib parses these as floats instead of proper decimals you're boned. But if it parses them as integers you have an opportunity to post-process these as proper decimals before actually using them.

Understood davout. Most languages these days can handle the different number types, decimal or float won't be an issue. Certainly in my specific cases using MySQL and PHP, these numbers are handled very well and very precisely.

The issue arises with Javascript which is widely used these days.

http://stackoverflow.com/questions/588004/is-javascripts-floating-point-math-broken

Javascript treats floats and decimals as the same type. When performing number manipulations for visual display then some issues might arise. The tricky aspect is... how do i use an Integer number format and have javascript run calculations whilst returning numbers in correct format? Can any javascript developers out there share their thoughts?

davout

legendary

Activity: 1372

Merit: 1008

1davout

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

Don't ever listen to anyone suggesting floating point for financial calculations, they don't know what they're doing, no matter how much precautions and care you take floats are and remain approximations that are not suited for monetary amounts.

If you know the difference and know what you're doing there's really no need to use integers as storage type and handling type. Using an integer to store a decimal value in a high level language would show that you don't know exactly what you're doing.

The reason MtGox will return integers in their API responses is because there is no way to discriminate between decimal and float types in JSON, they are represented the same way. Therefore, if your parsing lib parses these as floats instead of proper decimals you're boned. But if it parses them as integers you have an opportunity to post-process these as proper decimals before actually using them.

Topic: Developers - Best practises for decimal handling (Read 3217 times)