Output build artifacts so that Builds can reliably be repeated

[deldrid1]

One of our issues with using Builder for production with remote includes has been getting a repository back to the exact same state that it was in originally, in order to be able to reproduce "distribution" code (i.e. agent.nut/device.nut) from source so that we can then bug fix.

The @include's in source code act similar to an npm package.json or cocoapods Podfile. However, to get to repeatable builds, it would also be useful to have the equivalent of a package-lock.json or Podfile.lock, especially when (lazily) referencing remote repos without specific commit SHAs or when using custom builder directives.

I'd like to suggest that Builder produce some optional build artifacts. Specifically, I'd like a dependencies.json and a directives.json to be emittable from the tool.

dependencies.json is the more complicated of the two - basically, it would produce a hierarchy similar to a lock file and would use specific git SHA's for all remote includes (as opposed to #master, #develop, etc). Builder would also need to be able to ingest this file and leverage its mapping over the #master/#develop things that it might encounter when attempting to rebuild code many months or years later (when #master/#develop has inevitably been updated with several breaking changes).

directives.json should be pretty straight forward - it would contain all of the builder variables that were used at build time.

[deldrid1]

@ppetrosh - thoughts?

[ppetrosh]

Yes, makes sense. Agree that would be nice to have.

[deldrid1]

Is this something that imp would consider doing the development on?

Or could you at least point me to the right files/place to inject this kind of functionality? Is there any kind of design documentation that walks through the builder source code at 10,000 foot level?

[ppetrosh]

Unfortunately, there is no design documentation other than the source code ) But let us look into this.

[ppetrosh]

One quick question. How would you define version or guarantee a repeatable build for a resource, that's available over http? So I believe this is only feasible and specific for github resources.

[deldrid1]

Tagged git repos (whether Github or Bitbucket - see #43) are really the only things I care about so I’d be fine with that for my use case

[ppetrosh]

@deldrid1 then I believe it should be a github and bitbucket specific dependency files with versions of all the components listed. Will it be enough to generate this file.

Or you also want it to be "replayable" (via an option to specify the dependencies)? In this case, anything not listed in the dependency file will use the latest version available from the VCS.

[deldrid1]

Yes I would be fine with that approach as well.

[betzrhodes]

fixed in release v2.7.0