Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: filesystem directory backup #27

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

feat: filesystem directory backup #27

wants to merge 9 commits into from

Conversation

a-mesin
Copy link
Member

@a-mesin a-mesin commented Nov 12, 2023

closes #16

Source directory of the backup is getting archived in a tar archive and compressed with gzip resulting in a .tar.gzip file at the configured destination.

Still missing:

  • check if tests are needed and if yes implement them

@a-mesin a-mesin changed the title Filesystem directory backup feat: filesystem directory backup Nov 12, 2023
@a-mesin
Copy link
Member Author

a-mesin commented Nov 12, 2023

Caveat: filename in destination config needs to have the file type .tar.gz. This will not be needed once #26 is implemented

@iabudiab
Copy link
Member

Hey there, two questions / points:

  • Why not filepath.WalkDir?
  • What about symlinks? They are not followed, but the callback is still called for symlinks, if I'm not mistaken. For example, rsync -a preserve symlinks among other things. lifeboat should offer this option too.

And on the topic of rsync, maybe it can be used as an external tool if available. You know, reinventing the wheel and yada yada 😉

@a-mesin
Copy link
Member Author

a-mesin commented Nov 12, 2023

Hey there, two questions / points:

  • Why not filepath.WalkDir?
  • What about symlinks? They are not followed, but the callback is still called for symlinks, if I'm not mistaken. For example, rsync -a preserve symlinks among other things. lifeboat should offer this option too.

And on the topic of rsync, maybe it can be used as an external tool if available. You know, reinventing the wheel and yada yada 😉

Hey,

first of all thanks for having a look :)

  • looks like filepath.WalkDir is indeed the better choice here as appearently it is a more performant/efficient version of filepath.Walk - thanks for pointing it out
  • Symlinks are a complicated one in our use case as we want to support abitrary destinations for a backup. What happens f.e. when the file should be "backupped" into an S3 where the Symlink might point to a non-existing file? One idea could be to actually include the followed file into the final tar?

As for using rsync: the idea was to depend on as few external tools as possible so it can be used in as many environments as possible without the need for other tools.

@a-mesin
Copy link
Member Author

a-mesin commented Nov 12, 2023

Just checked the symlink issue: looks like resolving the symlink is quite trivial - therefore I would just add them...

@iabudiab
Copy link
Member

Regarding the symlinks, I would do something similar to the tar utility itself, i.e. include the symlinks as soft-links in the archive. Here we have two cases:

  • either the links are relative and point to some file inside the archive => they would work after unpacking
  • or they are absolute and point to something outside => they would be broken after unpacking

With tar -h such links can be dereferenced

@a-mesin a-mesin added the enhancement New feature or request label Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement backup of an directory
2 participants