Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/yaml: add options to omit empty or null documents in streams #3608

Open
mvdan opened this issue Dec 3, 2024 · 1 comment
Open
Labels

Comments

@mvdan
Copy link
Member

mvdan commented Dec 3, 2024

As #3464 points out, it is possible for YAML files to include empty or null documents:

exec cue import --outfile=- --list input.yaml

-- input.yaml --
---
kind: Deployment
---
---
kind: Deployment
---
kind: Other
---
null

Currently, both are mapped to a null value:

> exec cue import --outfile=- --list input.yaml
[stdout]
[{
	kind: "Deployment"
}, null, {
	kind: "Deployment"
}, {
	kind: "Other"
}, null]

This is good as a default, as it doesn't lose any information, and one can filter out those null top-level values if they wish to. However, when dealing with other tools which produce YAML like this, it can be a pain point to have to do this filtering as an extra step.

We propose adding options to the YAML encoding: omitempty would skip over documents in a stream which are empty (or only comments), and omitnull would skip over documents in a stream which are empty or the null value, as they are both equally null.

Then, one could do, for example:

> exec cue import --outfile=- --list yaml+omitnull: input.yaml
[stdout]
[{
	kind: "Deployment"
}, {
	kind: "Deployment"
}, {
	kind: "Other"
}]

It is possible that such options could also be added to other streaming encodings, although right now I can't think of any. JSONL or NDJSON, for example, does technically map each line to one JSON value - but we currently skip over empty lines due to how Go's encoding/json decoder works, so it's as if omitempty were always set, as we don't fail on empty lines.

@mvdan mvdan added the encoding label Dec 3, 2024
@mvdan
Copy link
Member Author

mvdan commented Dec 3, 2024

Worth noting that this is not urgent given that we are resolving #3464 with a change to the cue import --path flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant