Add post sync ability to rename files

[Boojapho]

Describe the problem/challenge you have
Sometimes syncs include files that have versions in the names of the files, which may be undesirable. If I sync a githubRelease with sample-v1.2.3-linux-amd64.tar.gz and extract a file named sample-v1.2.3, I may want to rename this to just sample so my binary matches what I had previously.

Describe the solution you'd like
In the spec, I'd like to see an option for renaming files after they are downloaded, and use wildcards/grouping for the source

renamePaths:
- from: sample-v*
  to: sample

This would take all files named sample-v* and rename it to sample. If more than one file matches, the behavior is indeterministic

Anything else you would like to add:
For flexibility, the from: should support globs and regex so the user could craft it to match what is needed. In addition, you may want to include group substitution in the to: so you could replace with a match in the regex.

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[neil-hickey]

hey @Boojapho , thanks for submitting!

I think what you are describing is currently where vendir stops and expects a following tool to do what you are describing. At least how I view vendir's role in a workflow.

The purpose of vendir is to sync a directory to match a remote / local location. The functionality you are describing moves slightly beyond that, to modifying the syncronized bits to rename them or modify them in some way.

Could you describe why another tool that takes this directory as an input could not have this wildcard like behaviour instead of vendir?

[frimik]

I had similar needs, and have also had needs to apply a local patch to certain things while waiting for upstream to bring on certain contributions. I "wrap" it in a Makefile... so Makefile does the vendir sync followed by whatever is needed. Vendir does what it does quite well and should continue to do so.

[reegnz]

I'd also welcome it if there was a way to apply patch files through a post sync hook. Ideally it should just take regular patch files and apply them, ux could ideally be compatible with how https://linux.die.net/man/1/patch takes patches.
I'm using vendir with renovate bot, so using a make target to wrap it with is not a viable path forward.

[mnaser]

I'd also welcome it if there was a way to apply patch files through a post sync hook. Ideally it should just take regular patch files and apply them, ux could ideally be compatible with how https://linux.die.net/man/1/patch takes patches. I'm using vendir with renovate bot, so using a make target to wrap it with is not a viable path forward.

I'm on the same boat. Have you been able to make some sort of progress or workaround on this?

[reegnz]

We've managed to add full vendir lock file maintenance support to renovatebot. Renovate now runs all of our vendir sync commands and commits the vendored files and opens PR-s (renovatebot/renovate#25113).

Executing patches to the vendored files could probably be done with a renovate post upgrade task. But we haven't gotten around to experimenting with that part yet.
Given that we mainly vendor helm charts that we run helm template on, and for those we have other established toolsets to do patching already (kustomize).

But all of that doesn't mean that an integrated patch file support in vendir wouldn't be welcome still, to come full-circle with vendoring.

[dtomasi]

I've a similar need and would love to see the ability to define hooks that run shell commands or scripts.

Something like this would be awesome:

apiVersion: vendir.k14s.io/v1alpha1
kind: Config

directories:
## ...

hooks:
  post-sync:

   # Define a script
    - 'my-post-sync-script.sh'
    
   # Or any other command
    - 'ln -s vendir/confshare/.pre-commit-config.yaml .pre-commit-config.yaml'

Currently we have comments in your vendir.yml files that let people know to not use vendir sync directly, but use the wrapper, that we created in your task runner. Very recently I noticed, that it might happen that symlinks are broken when dependencies are updated by renovate. So I need to write some awkward postUpgradeTask configs to fix this there as well. I would really appreciate a solution that works natively with vendir. And I think this does not conflict with that single, simple purpose idea of vendir, as this is a general purpose enhancement that lets you integrate with almost every tooling. Looking forward to hear what you guys think about this.