Ruud H.G. van Tol on Thu, 22 Feb 2024 14:11:47 +0100


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: Numerical instability in extracting base B digits of the fractional part of real 


On 2024-02-22 13:35, Georgi Guninski wrote:

Bugs me both constants have exact decimal representation and when I
display them on screen I get the exact decimal value with ?zero?
epsilon:
? 0.33
%1 = 0.33000000000000000000000000000000000000

Does this mean I can't trust the decimal form?



Be aware that they are of different types:

? 0.33 == 33/100
%1 = 1

? 0.33 === 33/100
%2 = 0


? 33/100
%3 = 33/100

? 33/100.
%4 = 0.33000000000000000000000000000000000000

-- Ruud