Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial backup fails as files are not added to commit #129

Closed
1 task done
bliepp opened this issue Sep 19, 2024 · 20 comments
Closed
1 task done

Initial backup fails as files are not added to commit #129

bliepp opened this issue Sep 19, 2024 · 20 comments
Labels
bug Something isn't working

Comments

@bliepp
Copy link

bliepp commented Sep 19, 2024

Code of Conduct

  • I promise that i have used markdown syntax to make the code more readable which helps to fix the problem and that I have tested the --fix command mentioned above without success.

What happened

After I installed Klipper-Backup according to the docs I tried to do an initial backup using ./script.sh. However, an error message appears.

What did you expect to happen

I expected it to confirm that my config has been uploaded to GitHub

How to reproduce

Install first, then run ./script.sh

Debug

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username=bliepp
github_repository=Ender3-Klipper-Config
branch_name="main"
commit_username="bliepp"
commit_email="[email protected]"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 8
drwxr-xr-x  2 bliepp bliepp 4096 Sep 19 14:40 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 14:40 ..
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

./script.sh: line 144: /home/bliepp/config_backup/.git/config: No such file or directory
------------------------------

Initialized empty Git repository in /home/bliepp/config_backup/.git/
Skipping symbolic link: printer_data/config/mainsail.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 16
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 14:40 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 14:40 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 14:40 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 14:40 printer_data
------------------------------

error: 'printer_data/config/' does not have a commit checked out
fatal: adding files failed
On branch main

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	README.md
	printer_data/

nothing added to commit but untracked files present (use "git add" to track)
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
From https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git
error: src refspec main does not match any
error: failed to push some refs to 'https://github.com/bliepp/Ender3-Klipper-Config.git'

Additional information

I'm aware that it says "Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)". However, changing it to public didn't change anything (the message became "The GitHub repo bliepp/Ender3-Klipper-Config exists (public)" but the error didn't go away, though).

@bliepp bliepp added the bug Something isn't working label Sep 19, 2024
@bliepp bliepp changed the title Initial Backup fails Initial backup fails (seems not to add/commit the files) Sep 19, 2024
@bliepp bliepp changed the title Initial backup fails (seems not to add/commit the files) Initial backup fails as files are not added to commit Sep 19, 2024
@Tylerjet
Copy link
Collaborator

i see towards the bottom that git prints out your token as https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git can you double check your .env file to make sure that the token is actually set?

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

i see towards the bottom that git prints out your token as https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git can you double check your .env file to make sure that the token is actually set?

Thanks for the quick reply. Yes, it is actually set. I removed my token it from the output of `script.sh --debug" before posting.

@Tylerjet
Copy link
Collaborator

ahh ok, when installing did you use install.sh to configure settings or did you edit the .env manually?

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

I used install.sh. However, it doesn't seem to be an issue with the settings but with my printer_data directory. When I change script.sh on line 253 (where files are actually added) from git add . to git add README.md everything (i.e. only the README.md) commits just fine. If I change it to git add printer_dataor git add printer_data/* it only removes the previously commited README.md but does not add the directory (however, it doesn't fail). So I guess the error is that it does not recognize printer_data as addable.

It's still strange as it at least should commit the README.md, shouldn't it?

@Tylerjet
Copy link
Collaborator

could you run with --debug again and post that, as i see in the initial one it didn't print out things like your .git/config file, etc... since it didn't get created until the script ran.

@Tylerjet
Copy link
Collaborator

Yeah my wondering is that there are some addtional git errors about pushing/accessing the remote repo so if those resolve the rest might too

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

Okay, here's what I found:

  1. If I push to an non-empty repo (e.g. with an existing file) the commit and push always succeeds, no matter if I only add the README.md via modifying line 253, only printer_data, or use the original script. It always succeeds as in something happens without an error.
  2. If I push to an empty repo (i.e. freshly created without any contents) it fails if printer_data is in the list of files to add (via a dot like in the original script or by explicitly stating printer_dataas the git add argument).
  3. printer_data never gets added, commited or pushed to remote, no matter if a error occurs or not. If no error occurs, printer_data is simply ignored.

Very strange

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

could you run with --debug again and post that, as i see in the initial one it didn't print out things like your .git/config file, etc... since it didn't get created until the script ran.

Sure. Here's the output for modifying it to explicitly only add README.md to the commit:

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script2.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username="bliepp"
github_repository="Ender3-Klipper-Config"
branch_name="main"
commit_username="bliepp"
commit_email="[email protected]"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 16
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 15:19 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 15:20 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 15:19 .git
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

[init]
    defaultBranch = main
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[user]
	name = bliepp
	email = [email protected]
[remote "origin"]
	url = https://********@github.com/bliepp/Ender3-Klipper-Config.git
	fetch = +refs/heads/*:refs/remotes/origin/*
------------------------------

Skipping symbolic link: printer_data/config/mainsail.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 20
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 15:20 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 15:20 ..
drwxr-xr-x  7 bliepp bliepp 4096 Sep 19 15:20 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 15:20 printer_data
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------

[main (root-commit) 7a4fbd7] New backup from 19/09/24 - 15:20:56
 1 file changed, 4 insertions(+)
 create mode 100644 README.md
From https://ghp_<access_token>@github.com/bliepp/Ender3-Klipper-Config.git
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 378 bytes | 378.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/bliepp/Ender3-Klipper-Config.git
 * [new branch]      main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.
bliepp@ender3:~/klipper-backup $ 

@Tylerjet
Copy link
Collaborator

ok and then can you do one with klipper-backup reset to the latest commit so we can see the error when it tries to add and push the printer_data

@Tylerjet
Copy link
Collaborator

also after line 221 can you put

echo "Backing up: $file"

that should let us see what files it actually tries to copy over to the backup_config folder

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

ok and then can you do one with klipper-backup reset to the latest commit so we can see the error when it tries to add and push the printer_data

Oh okay now I get what you mean.

also after line 221 can you put

echo "Backing up: $file"

Sure, here it is:

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script2.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username="bliepp"
github_repository="Ender3-Klipper-Config"
branch_name="main"
commit_username="bliepp"
commit_email="[email protected]"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 16
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 18:43 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:43 ..
drwxr-xr-x  8 bliepp bliepp 4096 Sep 19 18:43 .git
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

[init]
    defaultBranch = main
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[user]
	name = bliepp
	email = [email protected]
[remote "origin"]
	url = https://********@github.com/bliepp/Ender3-Klipper-Config.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main
------------------------------

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
From https://github.com/bliepp/Ender3-Klipper-Config
 * branch            main       -> FETCH_HEAD
Already up to date.
Backing up: /home/bliepp/printer_data/config/axes.cfg
Backing up: /home/bliepp/printer_data/config/bed.cfg
Backing up: /home/bliepp/printer_data/config/display.cfg
Backing up: /home/bliepp/printer_data/config/extruders.cfg
Backing up: /home/bliepp/printer_data/config/features.cfg
Backing up: /home/bliepp/printer_data/config/.git
Backing up: /home/bliepp/printer_data/config/macros.cfg
Skipping symbolic link: printer_data/config/mainsail.cfg
Backing up: /home/bliepp/printer_data/config/mcu.cfg
Backing up: /home/bliepp/printer_data/config/moonraker.conf
Backing up: /home/bliepp/printer_data/config/.moonraker.conf.bkp
Backing up: /home/bliepp/printer_data/config/printer.cfg
Backing up: /home/bliepp/printer_data/config/probe.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 20
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 18:43 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:43 ..
drwxr-xr-x  8 bliepp bliepp 4096 Sep 19 18:43 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 18:43 printer_data
-rw-r--r--  1 bliepp bliepp  181 Sep 19 15:19 README.md
------------------------------

error: 'printer_data/config/' does not have a commit checked out
fatal: adding files failed
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	README.md
	printer_data/

nothing added to commit but untracked files present (use "git add" to track)
[main e75f0f7] New backup from 19/09/24 - 18:43:32 - No new changes pushed
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 232 bytes | 232.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/bliepp/Ender3-Klipper-Config.git
   4a196da..e75f0f7  main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.

So this means klipper-backup reads and copies the files correctly but git doesn't seem add them to the commit, right?

@Tylerjet
Copy link
Collaborator

okay can you do

cd ~
rm -rf config_backup

and then run script.sh again

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

Already tried that (and tried again after you said it). Nothing changed. However, if I run the script again after removing config_backup the last bit changes as it then pushes an empty commit to the repo (see below). Here's the output:

● Checking for installed dependencies Done!

Klipper-Backup is up to date


------------DEBUG:------------
Command: ./script2.sh --debug
------------------------------


------------DEBUG:------------
github_token=****************
github_username="bliepp"
github_repository="Ender3-Klipper-Config"
branch_name="main"
commit_username="bliepp"
commit_email="[email protected]"

# All information regarding .env can be found here:
# https://klipperbackup.xyz/configuration/

# Backup paths
# Note: script.sh starts its search in $HOME which is /home/{username}/
# The array accepts folders or files like the following example
# 
#  backupPaths=( \
#  "printer_data/config/*" \
#  "printer_data/config/printer.cfg" \
#  )
#
# Using the above example the script will search for `/home/{username}/printer_data/config/*` and `/home/{username}/printer_data/config/printer.cfg`
# When backing up a folder you should always have `/*` at the end of the path so that files insde the folder are properly searched 

backupPaths=( \
"printer_data/config/*" \
)

# Array of strings in .gitignore pattern git format https://git-scm.com/docs/gitignore#_pattern_format for files that should not be uploaded to the remote repo
# New additions must be enclosed in double quotes and should follow the pattern format as noted in the above link
exclude=( \
"*.swp" \
"*.tmp" \
"printer-[0-9]*_[0-9]*.cfg" \
"*.bak" \
"*.bkp" \
"*.csv" \
"*.zip" \
)
------------------------------


------------DEBUG:------------
curl: (22) The requested URL returned error: 404 
Error: no GitHub repo bliepp/Ender3-Klipper-Config found (maybe private)
------------------------------


------------DEBUG:------------
$HOME: /home/bliepp
------------------------------


------------DEBUG:------------
$backup_path: /home/bliepp/config_backup

Content of $backup_path:
total 8
drwxr-xr-x  2 bliepp bliepp 4096 Sep 19 18:51 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:51 ..
------------------------------


------------DEBUG:------------
$backup_path/.git/config:

./script2.sh: line 144: /home/bliepp/config_backup/.git/config: No such file or directory
------------------------------

Initialized empty Git repository in /home/bliepp/config_backup/.git/
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 12 (delta 7), reused 5 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (12/12), 1.22 KiB | 56.00 KiB/s, done.
From https://github.com/bliepp/Ender3-Klipper-Config
 * branch            main       -> FETCH_HEAD
 * [new branch]      main       -> origin/main
Backing up: /home/bliepp/printer_data/config/axes.cfg
Backing up: /home/bliepp/printer_data/config/bed.cfg
Backing up: /home/bliepp/printer_data/config/display.cfg
Backing up: /home/bliepp/printer_data/config/extruders.cfg
Backing up: /home/bliepp/printer_data/config/features.cfg
Backing up: /home/bliepp/printer_data/config/.git
Backing up: /home/bliepp/printer_data/config/macros.cfg
Skipping symbolic link: printer_data/config/mainsail.cfg
Backing up: /home/bliepp/printer_data/config/mcu.cfg
Backing up: /home/bliepp/printer_data/config/moonraker.conf
Backing up: /home/bliepp/printer_data/config/.moonraker.conf.bkp
Backing up: /home/bliepp/printer_data/config/printer.cfg
Backing up: /home/bliepp/printer_data/config/probe.cfg

------------DEBUG:------------
Content of $backup_path after rsync:
total 16
drwxr-xr-x  4 bliepp bliepp 4096 Sep 19 18:51 .
drwxr-xr-x 15 bliepp bliepp 4096 Sep 19 18:51 ..
drwxr-xr-x  8 bliepp bliepp 4096 Sep 19 18:51 .git
drwxr-xr-x  3 bliepp bliepp 4096 Sep 19 18:51 printer_data
------------------------------

error: 'printer_data/config/' does not have a commit checked out
fatal: adding files failed
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	README.md
	printer_data/

nothing added to commit but untracked files present (use "git add" to track)
From https://ghp_<token>@github.com/bliepp/Ender3-Klipper-Config.git
Branch 'main' set up to track remote branch 'main' from 'origin'.
Everything up-to-date

Executed a second time:

nothing added to commit but untracked files present (use "git add" to track)
[main 77e3919] New backup from 19/09/24 - 18:51:49 - No new changes pushed
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 232 bytes | 232.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/bliepp/Ender3-Klipper-Config.git
   888ff80..77e3919  main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.

@Tylerjet
Copy link
Collaborator

hrmm something is not right as it shouldn't be trying to check out a commit in printer_data/config

@Tylerjet
Copy link
Collaborator

can you completely remove the klipper-backup folder and config_backup.

cd ~
rm -rf config_backup
rm -rf klipper-backup

and then redownload and rerun install.sh

@Tylerjet
Copy link
Collaborator

I'm wondering if something didn't clone properly or copy over during the install.sh process

@Tylerjet
Copy link
Collaborator

also i would use a fresh git repository that is completely empty on this next test just to eliminate the old one having issues as well

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

That was my idea, too, so I'm already doing this right now.

@bliepp
Copy link
Author

bliepp commented Sep 19, 2024

Oh my f'ing god. The answer was so simple. I actually had a local git repository initialized in my config folder. I can't remember doing that but it I may have done that in the past by accident. I checked multiple times but didn't add the -a flag to my ll command so I didn't see it (all my other machines are configured to alias ll to ls -la so I naturally assumed I was shown all file).

Sorry for this and thank you very much for your effort in finding a solution!

@bliepp bliepp closed this as completed Sep 19, 2024
@Tylerjet
Copy link
Collaborator

Ahhhh dang that was going to be my next question about doing ls -la in there, I'm working (albeit slowly) on some stuff in the restore branch for catching any .git repos and popping up some warnings for instances like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants