Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add new stdlib functions #2199

Merged
merged 7 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Main (unreleased)
- Change processlist query to support ONLY_FULL_GROUP_BY sql_mode
- Add perf_schema quantile columns to collector

- Add three new stdlib functions to_base64, from_URLbase64 and to_URLbase64 (@ravishankar15)

### Bugfixes

- Fixed issue with reloading configuration and prometheus metrics duplication in `prometheus.write.queue`. (@mattdurham)
Expand Down
37 changes: 36 additions & 1 deletion docs/sources/reference/stdlib/encoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,48 @@ The `encoding.from_base64` function decodes a RFC4648-compliant Base64-encoded s

`encoding.from_base64` fails if the provided string argument contains invalid Base64 data.

### Examples
### Example

```text
> encoding.from_base64("dGFuZ2VyaW5l")
tangerine
```

## encoding.from_URLbase64

The `encoding.from_URLbase64` function decodes a RFC4648-compliant Base64 URL safe encoded string into the original string.

`encoding.from_URLbase64` fails if the provided string argument contains invalid Base64 data.

### Example

```
> encoding.from_URLbase64("c3RyaW5nMTIzIT8kKiYoKSctPUB-")
string123!?$*&()'-=@~
```

## encoding.to_base64

The `encoding.to_base64` function encodes the original string into RFC4648-compliant Base64 encoded string.

### Example

```
> encoding.to_base64("string123!?$*&()'-=@~")
c3RyaW5nMTIzIT8kKiYoKSctPUB+
```

## encoding.to_URLbase64

The `encoding.to_base64` function encodes the original string into RFC4648-compliant URL safe Base64 encoded string.

### Example

```
> encoding.to_URLbase64("string123!?$*&()'-=@~")
c3RyaW5nMTIzIT8kKiYoKSctPUB-
```

## encoding.from_json

The `encoding.from_json` function decodes a string representing JSON into an {{< param "PRODUCT_NAME" >}} value.
Expand Down
27 changes: 24 additions & 3 deletions syntax/internal/stdlib/stdlib.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,12 @@ var file = map[string]interface{}{
}

var encoding = map[string]interface{}{
"from_json": jsonDecode,
"from_yaml": yamlDecode,
"from_base64": base64Decode,
"from_json": jsonDecode,
"from_yaml": yamlDecode,
"from_base64": base64Decode,
"from_URLbase64": base64URLDecode,
"to_base64": base64Encode,
"to_URLbase64": base64URLEncode,
}

var str = map[string]interface{}{
Expand Down Expand Up @@ -313,6 +316,24 @@ func base64Decode(in string) (interface{}, error) {
return decoded, nil
}

func base64URLDecode(in string) (interface{}, error) {
decoded, err := base64.URLEncoding.DecodeString(in)
if err != nil {
return nil, err
}
return decoded, nil
}

func base64URLEncode(in string) (interface{}, error) {
encoded := base64.URLEncoding.EncodeToString([]byte(in))
return encoded, nil
}

func base64Encode(in string) (interface{}, error) {
encoded := base64.StdEncoding.EncodeToString([]byte(in))
return encoded, nil
}

func jsonPath(jsonString string, path string) (interface{}, error) {
jsonPathExpr, err := jp.ParseString(path)
if err != nil {
Expand Down
3 changes: 3 additions & 0 deletions syntax/vm/vm_stdlib_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ func TestVM_Stdlib(t *testing.T) {
{"encoding.from_yaml nil field", "encoding.from_yaml(`foo: null`)", map[string]interface{}{"foo": nil}},
{"encoding.from_yaml nil array element", `encoding.from_yaml("[0, null]")`, []interface{}{0, nil}},
{"encoding.from_base64", `encoding.from_base64("Zm9vYmFyMTIzIT8kKiYoKSctPUB+")`, string(`foobar123!?$*&()'-=@~`)},
{"encoding.from_URLbase64", `encoding.from_URLbase64("c3RyaW5nMTIzIT8kKiYoKSctPUB-")`, string(`string123!?$*&()'-=@~`)},
{"encoding.to_base64", `encoding.to_base64("string123!?$*&()'-=@~")`, string(`c3RyaW5nMTIzIT8kKiYoKSctPUB+`)},
{"encoding.to_URLbase64", `encoding.to_URLbase64("string123!?$*&()'-=@~")`, string(`c3RyaW5nMTIzIT8kKiYoKSctPUB-`)},

// Map tests
{
Expand Down
Loading