Syncing the lockfile

[shaneseaton]

A quick question @DavidVujic. Are there any commands that sync the lockfile versions in a project, with the lockfile version in the development project (root pyproject)?

Is seems to me the worst part about the existing patterns is having to manually update the dependency list in the pyproject of each projects. In a perfect world using something like mypy would do be able to work out the library dependencies, and then it could link to the version in the root pyproject file. I also feel like the versions in ALL the project lockfiles should be the same as the root one? Am I wrong about this?

I can see the libs command get us half way there with a visual check, but I am curious in what cases we would ever want them to be different?

[shaneseaton]

After finding poetry lock --no-update, this is manageable by a human, though still not ideal.

I can:

• use poetry poly libs to visually check
• manually update the pyproject.toml files
• write a script to run poetry lock --no-update in each project directory.

Would really nice to automate step 1 and 2 though. I would have thought this is what the sync command would do, but it is limited to the bricks, which is fair enough...be nice if there was a --libs option or something.

Also, wondering if it would be useful for the poetry poly libs command to return an error code saying not all projects have the same version tags. This could help with pre-commit hooks etc?

[DavidVujic]

You can also use the Poetry option --directory and run the lock command(s) without the need to navigate down to each sub directory.

[shaneseaton]

--strict is a nice way to do it.
My poly libs command already return an error code of 1, even though visually I have no issues. Not sure if thats meant to be the case???

There are two sync parts for dependencies in each project:

• The pyproject.toml files
• The lock files

I can see how poetry can be used to update the lock files. Is there a way poetry can sync the versions in a projects pyproject.toml file with the root pyproject.toml? I dont think so. I think this is a missing step that would be handy to have as part of the poly tool.

Though would be happy to be convinced not to if you have a reason why those dependencies might need to be different????

Perhaps the flags --pyprojectdeps and --lockfiles could be added to the poly sync command:

• --pyprojectdeps: ensures any dependencies specified in a a projects pyproject.toml file have the same versions constraints as the root pyproject.toml
• --lockfiles: updates all project lockfiles to match the current python environment (venv). This should be the one defined in the root pyproject.toml

Also, I really appreciate all the work that goes into this project (from you). Slightly embarrassed that I am unable to contribute to the codebase to assist, but hopefully some of these ideas help improve the tool. Never know, might find some time to dabble one day.

[DavidVujic]

Thank you for your feedback and the ideas about new features, much appreciated!

I'm experimenting with this currently: poly check opitionally report errors (exit code 1) if dependency versions are different.

I will have a look at poly sync to optionally sync dependencies in the pyproject.toml file too, to investigate if it is the way to go for the Polylith tool. I don't think I want to tamper the lock files though, that is usually are in a tool-specific format and also contain all the sub-dependencies of each direct third-party dependency. I would rather let Poetry (or Hatch, PDM, Rye) do that.

I agree about the versions and that the expected way is to have them all the same, but I can imagine use cases where a third-party dependency is updating frequently and the minor/patch versions quickly could differ. I wouldn't want the tool to break the CI builds by then, but an optional flag to the poly check command would be a way to go forward.

[shaneseaton]

I think writing a tool to update the lock files when there are existing tools to do it would be overkill, but you could write a wrapper around those tools. ie The implementation of poly sync --lockfiles could simply call the Poetry (or Hatch, PDM, Rye) lock file updater (exec("python lock --no-update --directory ./project/a") ie python for each project. That way you don't have to maintain that functionality.

[DavidVujic]

I'll give it some thought! Thanks for sharing ideas and solutions ⭐

[DavidVujic]

@shaneseaton the first step can now be automated. I have extended the poly check command to return an exit code 1 for library version strings mismatch. This new feature is enabled when using the --strict command.

Announcement: #235

[shaneseaton]

This is cool. Will definitely look to put this into my pre-commit stuff