Georgi Guninski on Fri, 16 Feb 2024 18:01:14 +0100


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: Numerical instability in extracting base B digits of the fractional part of real


Thanks.
I think you are not computing the same algorithm, since you
hardcode $d=3$:

? A=0.33;B=10;
? C=A*B;d=floor(C);A=C-d
%98 = 0.30000000000000000000000000000000000000
? C=A*B;d=floor(C);A=C-d
%99 = 0.99999999999999999999999999999999999998
? d
%100 = 2