Topic: weird hex calculation (Read 234 times)
use gmpy2 lib to large number multiply or divide etc
Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process 
use math library
Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process
If you have any fixed point library, it is not hard to create an arbitrary-precision calculator. All you have to do is code an AST (abstract syntax tree) to arrange the operators in a tree form according to their order of precedence, and this supports operators with any operant count, and cool stuff like parentheses and even nested functions.
Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of.
Hi PowerGlove and thanks for your reply. Yes, that's why I wrote on my last post. It was due to precision. Didn't know that I can modify gnome-calculator's precision. Changed it to maximum possible value = 100 and now the result is displayed correct. One can use gnome-calculator in "programming mode" to perform arithmetics with hex values.
Nice to have, now I don't need to open a terminal shell and use bc for my calculations.
Thank you.
@citb0in: You ran into a pretty interesting situation (3 different answers, in 3 different tools), here's what happened:
The first attempt
It looks like you used GNOME Calculator for this attempt, and the reason you got the limited-precision answer that you did is because of the default preferences in that tool:
With that level of precision: 67890123456789012345 + 35000000000000 = 67890158460000000000 = 0x3ae2a4b6c0147d800
Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of.
The second attempt
This time you used bc, which supports arbitrary precision, so it gave you the correct (i.e. fully precise) answer: 67890123456789012345 + 35000000000000 = 67890158456789012345 = 0x3ae2a4b6b41e40f79
The third attempt
This time you used an online calculator, which did its job in JavaScript, so the calculation was carried out in double precision: 67890123456789012345 + 35000000000000 = 67890158456789008384 = 0x3ae2a4b6b41e40000
I hope that fully solves the mystery for you!
The first attempt
It looks like you used GNOME Calculator for this attempt, and the reason you got the limited-precision answer that you did is because of the default preferences in that tool:
With that level of precision: 67890123456789012345 + 35000000000000 = 67890158460000000000 = 0x3ae2a4b6c0147d800
Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of.
The second attempt
This time you used bc, which supports arbitrary precision, so it gave you the correct (i.e. fully precise) answer: 67890123456789012345 + 35000000000000 = 67890158456789012345 = 0x3ae2a4b6b41e40f79
The third attempt
This time you used an online calculator, which did its job in JavaScript, so the calculation was carried out in double precision: 67890123456789012345 + 35000000000000 = 67890158456789008384 = 0x3ae2a4b6b41e40000
I hope that fully solves the mystery for you!
That anyway, yes. But the problem in my case was not the missing specification of the base the problem, but the fact that gnome-calculator cannot handle large numbers. I just found after the fact the function where you can set gnome-calculator also in "programmer mode". There you can simply add hex values. Also here the error occurs, see the following screenshot which makes this clear.
Everything after B4 is just simply omitted and replaced by zeros.
Everything after B4 is just simply omitted and replaced by zeros.
There you go 
I used the calculator on my Linux desktop and when adding those two numbers, the result was displayed as:
6,789015846×10¹⁹
so I just adjusted it myself to show the whole number without expression in power of. The thing is: my 'real' numbers were not as clear as this example here. So I just copied and pasted from one tool to another. Conclusion is: some tools are not capable in calculating and displaying very high numbers. I guess the same is true for the mentioned online hex calculators.
Lesson learned --> I will stick with 'bc' toll for further calculations.
Thanks for pointing out!
I used the calculator on my Linux desktop and when adding those two numbers, the result was displayed as:
6,789015846×10¹⁹
so I just adjusted it myself to show the whole number without expression in power of. The thing is: my 'real' numbers were not as clear as this example here. So I just copied and pasted from one tool to another. Conclusion is: some tools are not capable in calculating and displaying very high numbers. I guess the same is true for the mentioned online hex calculators.
Lesson learned --> I will stick with 'bc' toll for further calculations.
Thanks for pointing out!
You are adding the two decimal numbers together incorrectly.
67890123456789012345 + 35000000000000 = 67890158460000000000
67890123456789012345 plus 35000000000000 doesn't equal 67890158460000000000. It equals 67890158456789012345. That's 3AE2A4B6B41E40F79 in hexadecimal format and has been calculated correctly by the tool you used on Linux.
I have two numbers which I want to do simple addition math with.
Dec / Hex
Number 1 is:
67890123456789012345 / 3AE2A2B962F52DF79
Number 2 is:
35000000000000 / 1FD512913000
If I add decimal values of nr.1 and nr.2 the result is:
67890123456789012345 + 35000000000000 = 67890158460000000000
When I convert this decimal value to hex I get as result:
3AE2A4B6C0147D800
Now I try the same thing but only using the hex values. I use 'bc' tool on Linux for this calculation:
As you see the result differs from the 1th method.
3AE2A4B6C0147D800 != 3AE2A4B6B41E40F79
Then I thought I check online with any of the available hex calculators
I enter HexValueA=3AE2A2B962F52DF79 plus (+) HexValueB=1FD512913000 and the
Result = 3AE2A4B6B41E40000[/color]
This same result was output by another online hexcalc (https://purecalculators.com/de/hex-calculator)
I got three different results. What the #!R(T$F2M?!?
Dec / Hex
Number 1 is:
67890123456789012345 / 3AE2A2B962F52DF79
Number 2 is:
35000000000000 / 1FD512913000
If I add decimal values of nr.1 and nr.2 the result is:
67890123456789012345 + 35000000000000 = 67890158460000000000
When I convert this decimal value to hex I get as result:
3AE2A4B6C0147D800
Now I try the same thing but only using the hex values. I use 'bc' tool on Linux for this calculation:
Code:
$ bc
Quote
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
obase=16;ibase=16
3AE2A2B962F52DF79+1FD512913000
3AE2A4B6B41E40F79
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
obase=16;ibase=16
3AE2A2B962F52DF79+1FD512913000
3AE2A4B6B41E40F79
As you see the result differs from the 1th method.
3AE2A4B6C0147D800 != 3AE2A4B6B41E40F79
Then I thought I check online with any of the available hex calculators
I enter HexValueA=3AE2A2B962F52DF79 plus (+) HexValueB=1FD512913000 and the
Result = 3AE2A4B6B41E40000[/color]
This same result was output by another online hexcalc (https://purecalculators.com/de/hex-calculator)
I got three different results. What the #!R(T$F2M?!?
Jump to: