Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(whiskers): support alternative template file encodings #179

Merged
merged 1 commit into from
Apr 27, 2024

Conversation

backwardspy
Copy link
Member

@backwardspy backwardspy commented Apr 10, 2024

currently supports whatever encodings the encoding_rs_io crate supports

there are test cases for UTF-8, UTF-8 with BOM, UTF-16 LE, and UTF-16 BE.

resolves #178

BEGIN_COMMIT_OVERRIDE
feat(whiskers): support alternative template file encodings
END_COMMIT_OVERRIDE

@backwardspy backwardspy changed the title feat(whiskers): support alternative template file encodings support alternative template file encodings Apr 10, 2024
@nekowinston
Copy link
Contributor

nekowinston commented Apr 10, 2024

Did you consider an alternative of keeping the target file encoding in the whiskers metadata?
From my own experience where encoding mattered (Hearts of Iron IV modding, lol), keeping track of the correct encoding can be a tad annoying.
e.g.

whiskers:
  encoding: utf8-bom
  version: "2.1.0"

instead of requiring the input to have the same encoding as the output.
Edit: and as this PR diff shows, it would help with git diffs in the be edge cases.

@backwardspy
Copy link
Member Author

backwardspy commented Apr 10, 2024

whiskers:
  encoding: utf8-bom
  version: "2.1.0"

i hadn't considered that, but i like the idea! i think i'll tackle encoding in another PR as the choice of crates for this stuff seems to be split around decoding & encoding. i imagine i'll need to bring in another dependency for it.

instead of requiring the input to have the same encoding as the output.

right now we only care about the file encoding on input, all outputs are utf-8 per rust's defaults. those utf-16 files in the diff are fixtures for tests that ensure we can read & render those files correctly.

@nekowinston
Copy link
Contributor

right now we only care about encoding on input, all outputs are utf-8 per rust's defaults. those utf-16 files in the diff are fixtures for tests that ensure we can read & render those files correctly.

oh I misunderstood this PR, I guess this would've been a better comment to make in #178 🙃
seems fair to split it in two PRs then 👍

@backwardspy
Copy link
Member Author

to be fair, i did imply on that issue that i could carry the encoding through to writing the file too. i much prefer your solution.

currently supports whatever encodings the encoding_rs_io crate[1] supports

there are test cases for UTF-8, UTF-8 with BOM, UTF-16 LE, and UTF-16 BE.

[1] https://github.com/BurntSushi/encoding_rs_io
@backwardspy backwardspy merged commit a966a83 into main Apr 27, 2024
3 checks passed
@backwardspy backwardspy deleted the feat/character-encoding branch April 27, 2024 09:37
@backwardspy backwardspy changed the title support alternative template file encodings feat(whiskers): support alternative template file encodings Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Whiskers doesn't read frontmatter in files encoded as UTF-8 with BOM
2 participants