Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: json!(f32) cannot get right result #14180

Closed
xxhZs opened this issue Dec 25, 2023 · 3 comments
Closed

Bug: json!(f32) cannot get right result #14180

xxhZs opened this issue Dec 25, 2023 · 3 comments
Labels
type/bug Something isn't working
Milestone

Comments

@xxhZs
Copy link
Contributor

xxhZs commented Dec 25, 2023

Describe the bug

We used json!() in some places to convert f32 to json(such as sink's JsonEncoder). It will convert the f32 to f64 and then transform it into JSON, which may result in inaccurate values. (https://stackoverflow.com/questions/73871891/how-to-serialize-a-struct-containing-f32-using-serde-json)
example:
f32: 1.1 -> json: 1.100000023841858

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

@xxhZs xxhZs added the type/bug Something isn't working label Dec 25, 2023
@github-actions github-actions bot added this to the release-1.6 milestone Dec 25, 2023
@xiangjinwu
Copy link
Contributor

It is "right" as defined by IEEE 754 and RFC 8259:

  • First, 1.1::float4::float8 in PostgreSQL also produces 1.100000023841858. It is at the natural of IEEE 754 binary floats.
  • But to_jsonb(1.1::float4) in PostgreSQL produces 1.1 because it casts to numeric rather than float8. According to RFC 8259, json number only has good interoperability within IEEE 754 double precision. So it is acceptable to just use float8 rather than numeric. This will not be an issue when we support numeric in json as well.

@xiangjinwu xiangjinwu closed this as not planned Won't fix, can't repro, duplicate, stale Jan 9, 2024
@xxhZs
Copy link
Contributor Author

xxhZs commented Jan 9, 2024

It is "right" as defined by IEEE 754 and RFC 8259:

  • First, 1.1::float4::float8 in PostgreSQL also produces 1.100000023841858. It is at the natural of IEEE 754 binary floats.
  • But to_jsonb(1.1::float4) in PostgreSQL produces 1.1 because it casts to numeric rather than float8. According to RFC 8259, json number only has good interoperability within IEEE 754 double precision. So it is acceptable to just use float8 rather than numeric. This will not be an issue when we support numeric in json as well.

The main issue found is in the sink JSON format, where we first convert to JSON and then switch to the type of the target library, which may result in the RW being real1.1 and downstream being f32 1.10000023841858. Is this expected

@xiangjinwu
Copy link
Contributor

xiangjinwu commented Jan 9, 2024

which may result in the RW being real1.1 and downstream being f32 1.10000023841858. Is this expected

If the downstream type is also f32 (rather than f64 / numeric / string), it would be converted back to 1.1 rather than 1.100000023841858 (note that number in your question has one less 0). The next representable number after 1.1 (3f8ccccd) is 1.1000001 (3f8cccce):

test=# select 1.100000023841858::float4;
 float4 
--------
    1.1
(1 row)

test=# select float4send(1.1);
 float4send 
------------
 \x3f8ccccd
(1 row)

test=# select float4send(1.1000001);
 float4send 
------------
 \x3f8cccce
(1 row)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants