Desire to relate version in charmstore to VCS revision/branch

[xavpaice]

When we reference a version in the charmstore, e.g. cs:nagios-18, it's really hard if not impossible to know which branch and/or tag or revision that version relates to in the source - hence we can't know which version of the code to look at if we're troubleshooting things.

[mitechie]

Since the charm is uploaded you can include things like version/vcs info into your build steps before pushing up to the store. You can then use the hook for application-version-set [1] to make it obvious in status. The hook doesn't seem to be well documented at the moment but it could be the thing to read a version file from the built charm and output about it in status. The other thing you can do is manage things that are pulled in via charm resources.

1: https://goo.gl/idAvTb

[xavpaice]

Setting something to mark the application version is great, and I really wish more charm devs knew about that and actually made use of it. The doc you linked specifically says that version shouldn't be confused with the charm revision, although the revision in the charm store doesn't relate to the dvcs revision at all.

Having people set a build step would work nicely because that would allow many VCS platforms, and present the version of the charm code itself somewhere the store can access it. We would need to raise a bug against every charm we want the info for, but long term it could work particularly if it became a requirement to allow charms to be promulgated.

If the version info for the charm code was made available via, say, metadata.yaml, as part of a build step, we would still want a way to access that information from the charm store and probably from 'juju status' in order to make it useful. Any thoughts there?

[mitechie]

So the issue with metadata.yaml is that it's a file that needs to be written by a user/committed in and such. I'm thinking more of a make build step that just writes out a version file in the charm directory before running charm build and charm push. These files are listed in the file listing on the charm details page but are not easily accessible from the cli or other UX easily.

So then it comes down to how that's exposed out in a better way. We could standardize on some sort of version file, but that's been done in the past and can be confusing. Since revisions are affected by things like which channel you're in and can be moved forward/backward versions don't always play nice and in a straight line.

Hmmm, this is really like debugging information. It's generic metadata and there's an extra-info endpoint in the charmstore api. Perhaps we just need to allow a CI workflow to set the extra-info on charm push.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md#put-metaendpoint

That actually walks through an example API call to the store that sets a revision-info string. This is able to be retrieved with charm get ...

Ok, I did this and it works out pretty well:

charm set cs:~rharding/gypsy-danger-4 extra-info=`git rev-parse --short HEAD`
charm show cd:~rharding/gypsy-danger-4 extra-info

How's that work out for you and your process?

[ghost]

Setting the extra-info seems like a good way to keep track as the owner of the charm.

However I'd really like to see an easy way to display such a field (with commit tag/hash) this on jujucharms.com when looking at a charm. There was a box for revision history previously but that seems to be removed.

[xavpaice]

extra-info looks really helpful, and it would be really handy to see that exposed on the charm store gui.

This is part-way answered with repo-info in some charms, and that's good to know exactly how a charmstore version ties to source commits. See https://api.jujucharms.com/charmstore/v5/keystone/archive/repo-info for an example.

That part answers the question, but the question evolves as we get closer to the solution.

I'm currently working on a customer site where there are a few customisation to some charms, meaning that for their OpenStack deployment we're limited to using the 17.02 release of the OpenStack charms (current release is 17.08). If I use the cs:keystone source, I get 17.08, the current release, but what I am having to do here is download and build the sources for the stable/17.02 branch, to ensure I get the most recent 17.02 rev. The only other way I can see to get that release, is to go back over each revno of the charm, checking the repo-info against what's in the vcs logs, until I find the right one. Unfortunately, if the devs for that charm backport a patch to their stable/17.02 branch, we won't see that on the charmstore at all.

I'd really be keen to see, if releases are going to be a thing with charms, us going down the path of being able to use cs:charm for the 'current' version, and cs:/release/charm for a particular release, in the same way we can use cs:charm-123 for a particular revision. Release, branch, whatever it might be. If releases aren't supposed to be used like that, then we need to communicate that better with charm devs who are making pretty clear statements about releases and support.

One option I'd like to explore in the meantime is have our team get a charmstore account (bit like the openstack-charmers have one) and then we could just use cs:~bootstack/charm-123 until some other option comes up. We'd just need to maintain docs for what each of our revs are.

[mitechie]

I can appreciate your situation @xavpaice but as the charms are in source control with branches, pull requests, etc. the charmstore is not meant to be a source control tool. I feel like the best thing for you and your customers is to engage with the openstack team and have your patches in source control off their branch for 17.02 https://github.com/openstack/charm-keystone/tree/stable/17.02 and tracking your patches maybe from a fork of the charm or such. The charmstore is for built software and upstream is meant for tracking source changes.

[chris-sanders]

@mitechie To be clear, you're telling us you recommend we do not use the charm store but instead pull from up stream source with something like https://github.com/slank/codetree? Am I understanding you correctly?

[mitechie]

@chris-sanders in speaking with @xavpaice I think the best thing would be to work to be able to track stable upstream charms as they're getting the testing and attention and are the best thing for users to use.

However, if you're forced to stick off a branch of the 17.02 end I'd suggest forking their repo, applying your patches, building the charms, and uploading them to the charmstore in a team namespace. In this way you're able to track changes to the charms that might come about due to bugs/etc and keep your patchset against them in a clean way. You also get to rev the charms and leverage Juju and the Charmstore as a clean way to put together a tested "release" of those patched charms for the team.

[chris-sanders]

@mitechie Yes we are telling you there are current and future cases where sticking to release branches is mandated and outside of our control. I think you should re-think this stance as it doesn't seem sustainable for us to maintain forks of all charms when we could instead use the development teams repositories and not duplicate QA testing.
I want to use the charm store, you seem to be taking that option off the table with this stance. Not temporarily either, but never planning to cover this explicit and common (for us) use case.

[chris-sanders]

@mitechie Thanks for the clarification I'll chase name spaces that might solve this for us