Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sink): fix sink serde json f32 #14498

Closed
wants to merge 1 commit into from
Closed

fix(sink): fix sink serde json f32 #14498

wants to merge 1 commit into from

Conversation

xxhZs
Copy link
Contributor

@xxhZs xxhZs commented Jan 11, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

#14365
Because we have some downstream sinks that save as JSON strings instead of f32, it causes the error mentioned in this issue.
Related sinks: Redis, Elasticsearch,

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@xxhZs xxhZs requested a review from xiangjinwu January 11, 2024 04:37
@xiangjinwu
Copy link
Contributor

xiangjinwu commented Jan 11, 2024

See #6412

It is WRONG EXPECTATION for f32 / f64 to be lossless, for values like 1.1 that cannot be represented by limited number of bits in binary. If the user do need precise number, they should be using the decimal type.

The arbitrary_precision feature of serde_json, as well as a future similar option in our own jsonbb, is also meant for decimal. It is just an unfortunate fact that json only has a single number type, which according to RFC 8259, may only have good interoperability within IEEE 754 double precision range.

@xiangjinwu xiangjinwu closed this Jan 11, 2024
@xxhZs xxhZs deleted the xxh/fix-json-f32 branch January 22, 2024 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants