You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An f32 (32 bit floating point value) can represent the number 2147483904 exactly. Ryu formats this number incorrectly. Ryu formats the number as the string 2.147484E9. Ryu should format the number as the string 2147483904.
This is a bug in Ryu because it violates correctness criteria 3 from the paper, "correct rounding". Both strings satisfy criteria 1 and 2 but the string 2147483904 is closer to the original number. It has a distance of 0 to the number. Ryu's output has a distance of 96.
The text was updated successfully, but these errors were encountered:
For further confirmation, I hand-picked some test samples and used the Dragonbox algorithm for confirmation:
godbolt online playground: https://godbolt.org/z/6vqn74jMY
An f32 (32 bit floating point value) can represent the number
2147483904
exactly. Ryu formats this number incorrectly. Ryu formats the number as the string2.147484E9
. Ryu should format the number as the string2147483904
.This is a bug in Ryu because it violates correctness criteria 3 from the paper, "correct rounding". Both strings satisfy criteria 1 and 2 but the string
2147483904
is closer to the original number. It has a distance of 0 to the number. Ryu's output has a distance of 96.The text was updated successfully, but these errors were encountered: