priors and git

[j3mdamas]

Hi,

I've seen some changes to the priors, which are binary files. Even though GitHub shows small changes in bytes on the commits, that's not what happens in practice, because Git does not know how to deal with those binaries the same way it deals with text files. So in a commit like this one, the total size change is 200 Mb (a delete of 100 Mb and a creation of 100 Mb). These type of changes can make the repository very slow to clone. A clone of the current HEAD is 2 Gb even if the priors are only 600 Mb. The rest 1.3 Gb is on historical objects.

Of course, I can solve this with shallow clones or using the released .tar.gz files, to have a smaller downloads, but I still wanted to highlight it.

I also saw that maybe you've implemented some Git LFS now (ad08ebc), but historically it's hard to change the size now without rebooting the entire repository.

Cheers
João

[halx]

I am thinking to switch over to LFS but the practical problem is forks with local changes. Tools like bfg may be helpful to cut the size down. At the end of the day we do not carry valuable history in the public repo.

[j3mdamas]

bfg should change commit history, commit hashes change, releases are impacted by it, all forks are impacted by it. Maybe no historical value, but for sure repository integrity issues.
I understand this is an overhead. I'm not cloning anymore, I take the code from the releases, so as long as the releases make sense, all is good from my side. I wanted mostly to call attention to the issue.