Trying to use RX precision = double for 64-bit math . . .

I am attempting to get my RX621 configured to perform 64-bit math.

I have included <math.h> and <mathf.h>, changed C89 to C99 in both "C/C++" and "Standard Library" tabs, and on the "CPU"tab inside the "Details" section, I have selected "Double precision" for the Precision of double field.

My processor is performing the following :

5803636 x (5803636 / (55669978 / 10000)) = 404249823

The answer is actually :  6050333057

My code snippet follows :

    dTemp = ((double)Numerator * ((double)Numerator / ((double)Denominator / 10000)));

dTemp is a double

Numerator and Denominator are both unsigned long

Am I missing something on the compiler settings?

Lane

  • A quick check using Windows calculator says 6050333057 is the correct result. A rough estimate also says the processor is right:

    (6*10^6)*(6*10^6)=36*10^12

    (36*10^12)*10^4=36*10^16

    (36*10^16)/(6*10^7)=6*10^9

  • Wait, Windows Calculator and an estimate yield >>>>>    6*10^9, while the processor yields >>>>>  4*10^8 ???? That is not even close.

    6050333057 = 0x168A0C181 which is a 33-bit answer.

    404249823 = 0x18185CDF which is a 29-bit answer

  • Sorry, my mistake.

    The calculation on my RX62T gives the correct 6050333057.52267

    What you give as "result" is a double float interpreted as long. You use only the first 4 byte of the result and use this as simple hexadecimal value.

    The result is 0xDF 0x5C 0x18 0x18 0x0C 0x8A 0xF6 0x41

    Please assign this result to a longlong variable and you will see the result you expect.

  • That seems to do it.  Thank you for getting me through that layer of the onion.