-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
json-valid PPL function #3230
json-valid PPL function #3230
Conversation
Signed-off-by: Kenrick Yap <[email protected]>
9cdfd71
to
70152eb
Compare
core/src/main/java/org/opensearch/sql/expression/json/JsonFunctions.java
Outdated
Show resolved
Hide resolved
@kenrickyap can you please add the relevant documentation for this new function? |
Signed-off-by: Kenrick Yap <[email protected]>
f84e21d
to
76c3995
Compare
Signed-off-by: Kenrick Yap <[email protected]>
Added doctest, integ-test, and unit tests |
core/src/main/java/org/opensearch/sql/expression/json/JsonFunctions.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Outdated
Show resolved
Hide resolved
integ-test/src/test/resources/indexDefinitions/json_test_index_mappping.json
Outdated
Show resolved
Hide resolved
Signed-off-by: Kenrick Yap <[email protected]>
Signed-off-by: Kenrick Yap <[email protected]>
result = | ||
executeQuery( | ||
String.format( | ||
"source=%s | where json_valid(json_string) | fields test_name", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what if the json_string is null
? Could you add a test case for null
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming null should return false
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming null should return
false
?
@acarbonetto yes I think it makes sense
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If indeed null
return false, then we should specify this behaviour on documentation to avoid any confusion,
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
null
returns null
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update: you are correct that null should return valse.
I've updated the tests and logic to match.
Signed-off-by: Kenrick Yap <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing license header
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually why do we need 2 separate JsonUtils
& JsonFunctions
?
would it make sense to unify into a single class ? (inside maybe the different namespaces)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent question. I dislike "utility classes" (what is the responsibility of a utility class?). But it gives us a central class to put all the json
business logic. This seems to be how we do things (date time and IP address business logic also lives in util classes).
As for the JsonFunction
class, it provides an integration layer between the language parser's function expressions and the json business logic. The casting expressions will be another class that will access the json logic.
This class could very well be named Json
or JsonMapper
. But if we do this, we should rename all the util classes in the utils
package.
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Show resolved
Hide resolved
result = | ||
executeQuery( | ||
String.format( | ||
"source=%s | where json_valid(json_string) | fields test_name", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If indeed null
return false, then we should specify this behaviour on documentation to avoid any confusion,
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few minor comments.
} | ||
|
||
try { | ||
objectMapper.readTree(jsonExprValue.stringValue()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we extract the ObjectMapper as a static final class member?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not waste the memory unless a user is making json calls.
I think it's better to construct it each time its used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm. What about the initialization-on-demand holder idiom? Agreed that it's probably better to avoid using the memory if it's not needed, but probably also good to avoid creating the same object hundreds of times if there are hundreds of calls to any JSON function? No need to address now either way, but maybe something to consider as JSON work progresses?
core/src/test/java/org/opensearch/sql/expression/json/JsonFunctionsTest.java
Show resolved
Hide resolved
integ-test/src/test/java/org/opensearch/sql/legacy/SQLIntegTestCase.java
Show resolved
Hide resolved
Signed-off-by: Andrew Carbonetto <[email protected]>
Description
Based on this feature request: #3207
Added json_valid request.
Related Issues
Resolves #3207
Check List
--signoff
.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.