Skip to content

2021 Developer Meetings

Jump to bottom
David Allsopp edited this page Jul 13, 2021 · 63 revisions

16 Jul 2021

Agenda

Remaining issues to look at for 2.2.0 planning:

  • #4031 - system compilers upgrade behaviour
  • #3997 - OPAMPRECISETRACKING issues. @dra27 thoughts (possibly posted elsewhere, or maybe I just mused):
    • We could write a file in the opam root which is required to have a non-integral of nanoseconds. Reading that file could confirm to opam that nanoseconds haven't been confirmed. Pros: really simple. Cons: bit hacky.
    • Most files are installed now using opam .install files - i.e. we do read them. So we can compute checksums nearly for free. Why don't we always write checksum and timestamp? In this case, if the timestamp doesn't match then we use the checksum. If that does match, and we have a write lock, we update the timestamp in the changes file? Pros: could be turned on by default. Cons: bit fiddlier to implement.
  • #3983 - behaviour of opam pin when the pinned version matches upstream
  • #3962 and #3928 - issues with non-root use of install.sh. Is that worth looking at in more detail?
  • #3876 - is there anything to do here?
  • #3850 - @avsm's revulsion to {build}. Decision needed!
  • #3849 - csh specific issue. Close?
  • #3833 - I think the use-case is effectively a mixed pin, and so not likely to be supported. Close?
  • #3793 - do we really want to do this?
  • #3737 - is this still an issue?
  • #3586 - another system compilers one
  • #3577 - signed sources

8 Jul 2021

Agenda

2 Jul 2021

Agenda

25 Jun 2021

Agenda

  • 2.1.0 / 2.0.9 status 🥳
  • Version policy on branches
  • External contributions
  • Issues
    • Tracker items tagged Bug
    • Issues opened this year:
  • Goals

Notes

  • 2.1.0: @rjbou and @AltGr will be on holiday at various points until during w/b 12 Jul. Assuming nothing explodes, we'll tag 2.1.0 next week before @rjbou is off with binaries and release announcement following during w/b 12 Jul.
  • 2.0.9: @dra27 forgot to ask!
  • Version policy for branches. The PR containing commits for releases has been quite useful. The only problem is that not all of us can push to OCamlPro/. Plan: 2.2.0~alpha~dev version bump on master (as done) but no direct pushes or intermediate merges to maintenance branches (apart from unrelated things such as CI fixes). The first person who wishes to cherry-pick to an old branch prepares a PR with the version bump and the cherry-pick and opens the PR; all subsequent cherry-picks are pushed to that PR until release (i.e. the maintenance branch is a series of merges).
  • External contributions/issues: PRs from external contributors all pinged/closed as appropriate. Issues down to #4528 above triaged/reviewed.

18 Jun 2021

The emergency of #4713 stalled announcing the RC this week. The fix in #4715 is passing CI - @rjbou just checking a few more things with registering local switches (e.g. via --switch) but once that's in we can tag rc2 and, in a nod to Groundhog Day, repeat!

Brief discussion on mechanising the binary build process - @AltGr has done something similar with GitHub Actions for Learn OCaml, and is looking at using that for the bulk of the builds for opam as well. Some of the platforms may be trickier (OpenBSD) and either need doing manually, or possibly with custom runners in GHA?

10 Jun 2021

Agenda

  • 2.1.0: confirm release schedule; double-check release criteria
  • 2.0.9: remaining plans
  • LTS, minimum OCaml version and release speed
  • Project Goals
  • 2.2.0: collecting specs
  • New issue: #4690 - tracking issue for Windows-specific problem.
  • New issue: #4688 - opam switch - as restoring the previous switch.
  • New issue: #4686 - when should opam lint warnings be being displayed to users.
  • New issue: #4705 - spaces in parent directory

Notes

2.1.0

  • @dra27 looking at a couple of minor Windows nits - PRs to come.
  • #4705 is serious (and has been present since OPAM 1.1.0!): @rjbou looking at it; @dra27 will sync up with Windows side of it
  • Small display issue with opam lint --help and the lint codes which we happened to spot discussing something else!

2.0.9

Minimum version / LTS / Release cadence

  • opam-core and opam-format to 4.03
  • Rest to 4.08
  • opam-file-format can stay at 4.02.3, but it's on final warning... as soon as there's an issue, it gets bumped

LTS ideas

  • @rjbou: maintaining last release and a nominated LTS release
  • @rjbou: Bug fixes then cause a decision - is master ready to be spun into a release; or do we back-port to last
  • LTS: gets critical bug fixes only
  • Previous release: all bug fixes
  • Nominate at 2.1 as being an LTS, with a view to ensure a release prior to the next Debian freeze (in 2024ish)

Goals

  • Aim to come up with some individual ideas and then go from there
  • If there's good agreement, we may even publish them!
  • Muriel had suggested last year having a poll of users
    • The idea was discussed, but no questions determined
    • We could piggy-back on the OCaml Survey for 2021 - sync with Gabriel Scherer to see if there is going to be one

2.2.0

New issues

  • https://github.com/ocaml/opam/issues/4686
  • Should we be linting when pinning?
    • @altgr - original intention was that the developer would be pinning; perhaps with CI it's less necessary
    • @rjbou - could have an environment variable to list the lint warnings which are disabled (dune-release and opam-publish would need to explicitly disable this)
    • @dra27 - wondering if we could make it that just pin-depends and --dev-repo don't lint?
    • @kit-ty-kate - could display just the errors, not the warnings at pinning time?
    • @altgr - could consider just displaying the errors on opam update (@rjbou - but a bit surprising if you pin in order to test everything's OK!) @dra27 - or could have the linting for local repos only?

21 May 2021

Agenda

Misc

  • #4672 - switches to the dev profile in Dune for CI. Only questions are whether we use the release profile for the builds which run the testsuite and whether everyone's happy to go with it!

The nearly there

The still-to-decide

  • #4668 - 2.0.9+, 2.1.0+ or 2.2.0+?
  • #4673 - confirm that the original command was wrong (i.e. should it have worked)
  • #4662 - original issue addressed by #4671. MacPorts issue described at #4662 (comment)

On the subject of depexts and interactive prompts

  • #4582 - PR itself is just about ready to go, modulo a couple of help messages and a function name
  • Are we strictly implementing the spec properly?
    • In particular, --yes runs sudo commands (as opam depext --yes used to) but is this contrary to Spec for opam depext integration?
    • Should --yes and--confirm-level <> ask imply non-interactive? i.e. do they answer no to the unsafe questions?
    • (this was already an issue with depext and with --unsafe-yes, so it doesn't block #4582 itself)
  • #4669 - does this count as a regression?
  • #4670 - does this count as a regression?

New issues

  • #4674 - proposal to allow packages to request sandbox disabling

Notes

  • 4672: yes, and with running opam-rt and other tests with a release profile build of opam
  • 4663: is ready to merge
  • 4667: @rjbou to review
  • opam-file-format#45: good to go and then release 2.1.3
  • dune-release#343 - @emillon will have a look at the mdx problem; other than that just needs dune build @fmt
  • 4668: OK for 2.0.9
  • 4673: reopened; it is inconsistent that detection works with opam repo add but not with opam switch create; however not for 2.1
  • 4662: original issue is that the depext on "gcc" was incorrect for Macports. Apparently you truly do have to specify the version now. --assume-depexts was the wrong switch to advise. Clarified that --assume-depexts still requires the depext to be listed it just doesn't go ahead and install it. For example, this you allows you to build and make available your own libgmp, but the depext on gmp must still be correct in the opam file.
  • 4671: worth advising there might be a problem with the package itself (i.e. that it's requesting the wrong thing) in addition to suggesting --no-depexts

depext prompting and other interaction

@AltGr and @dra27 in complete agreement: sudo should not be run without a prompt! It's possible that there may be prompts (e.g. sudo prompting for password, etc.). Shouldn't apt be running explicitly with -i if unsafe-yes is not enabled? Conclusion was no: the package manager will prompt if necessary, opam should pause before invoking it and that's enough.

@AltGr that there's an additional confirmation level which is triggered by stdin being closed - it's then no to anything which wasn't being auto-answered. This seems to resolve the question of what should happen for --yes with unsafe questions. For now, they should be interactive, and you can explicitly close stdin (as you can in 2.0) to answer no to those questions. Interaction and confirmation level are not related features - it just happens to be that --no implies there will be no interaction. In 2.2 we might add a flag --non-interactive to allow triggering this behaviour without closing stdin. The suggestion was made that --yes --no actually means "answer yes to safe questions and no to everything else", but even the Windows user vetoed this!

Alpine and EPEL

Alpine issue - let's fix it (@kit-ty-kate looking into it); the alternate command is fast and stops what otherwise looks like a regression (since it works with opam depext).

EPEL. We should view this as a regression simply because there was a hack in opam-depext 2.0 to deal with this. However, there's a simpler and cleaner solution for 2.1: we refine the error message so that if epel-release is mentioned and not installed then we can give me a more precise error message that the user needs to enable the EPEL repo by running dnf install epel-release and then running the opam command again.

This at least limits the special-casing. Users on CentOS 8 already need to have enabled the PowerTools repo manually (even it was done at installation time or by upgrading), and we can both document clearly that EPEL may be required for some depexts and also the error message guides the user to the correct action.

14 May 2021

  • opam-root-version change is nearly ready
  • #4660 opened to track the issue of 2.1 CLI environment variables leaking into package builds (@dra27 will have a look)
  • @kit-ty-kate will rebase and update ocamllabs/dune-release#343 which should mean that both publishing tools are 2.1-ready

7 May 2021

dra27 was a little lax on the note-taking - discussion of active PRs for the 2.1 release candidate

30 Apr 2021

dra27 was a little lax on the note-taking - discussion of active PRs for the 2.1 release candidate

23 Apr 2021

Agenda + notes

Remaining 2.1~rc issues

  • #4640 😱
    • @AltGr live-coded in the meeting...
  • #4638 (closes #4636)
    • See also opam-root-compatibilities
    • Agreeing on the which fields and where is the larger part of this - implementing it should be straightforward!
  • --confirm-level (#4582)
  • Release opam-file-format and update opam to use it (#4639 closes #4394)
    • Two issues being worked by @dra27 (PR by end Friday):
      • It should be possible for opam 2.4, introducing new fields but no new syntax, to use opam-file-format 2.3 for its parsing (i.e. have no 2.4 release), but at present the future best-effort parser would mean that opam 2.4 wouldn't be able to tell the difference between a syntactically valid opam-version: "2.4" file and one which had a parsing/lexing error part-way through. Solved by inserting a sentinel group which can be detected as a syntax error (and has the right location information)
      • The present implementation doesn't return the opam-version line if there's a lexing/parsing error very close (grammatically-speaking) to the version - i.e. opam-version: "2.4" <.
  • #4641 (closes #4619)
  • #4642 (addresses #4554
    • Include in 2.1? Doesn't address the issue with opam-state 2.0.x and opam binary 2.0.x, but it would mean that plugins and tools using opam-state 2.0.x and opam binary 2.1.x won't see the same issue
    • Agreed - low-risk internal change, which will stop the appearance of a performance regression (if opam-state 2.0.x starts fighting with opam 2.1 over the cache)
  • ocaml-mccs release done - need to update lib-ext
  • #4643 - OK to include, but make cold shouldn't include the solver, just as we're quite close to rc. No problem to add an additional cold target (e.g. cold-0install) then it can be added via that.

Release (candidate) steps

  • dune-release is not yet compatible with opam-state 2.1 - ocamllabs/dune-release#343
  • opam-publish is not yet compatible with opam-state 2.1 - no PR?

16 Apr 2021

Agenda

Remaining 2.1~rc issues

  • --confirm-level (#4582) or --unsafe-depext-yes (#4563)
  • #4636 needs implementing and back-porting for 2.0.9
  • Release opam-file-format and update opam to use it (closes #4394)
  • Implement plugin symlink erasing at upgrade (closes #4619)
  • Fast install (#4494) just needs the the conflict resolved, I think?
  • @AltGr/@kit-ty-kate time to rebase (closes #4545)
  • An ocaml-mccs release is advisable to pick up #30 (4.12 support)

Release (candidate) steps

Other news

  • @AltGr - some work with the Software Heritage Foundation has been agreed (with funding for OCamlPro) archiving all packages from opam-repository there (not just the GitHub repos). Also looking at support for referencing SHF copies in the opam files themselves (i.e. with an ID) - could include falling back to their caches.

1 Apr 2021

Agenda

New PRs

#4619 - plugins and opam-state and upgrades

  • Plugins symlink reset at upgrade (together with #4621 this fixes the issue)
  • Tweaking opam-state to allow read-only access to new opam roots, when possible. Suggested in #4266 but dropped by the implementation in #4280 (following discussion at 17 Jul 2020 dev meeting, based on time stamps of messages!). As we encourage more plugins to incubate ideas, I think we should revisit this: idea is that the config files in the root (global, switch and repo) all have opam-root-version: "2.1" and opam-version: "2.0". The idea is simple: opam-root-version is checked for taking out write locks, opam-version is checked for taking out read locks and opam-state 2.0.x should be updated to internally recognise opam-root-version (i.e. no change in API in OpamFile) and ignore fields it doesn't recognise. The changes we've made for 2.1 mean that it should "just work" to have readable roots to 2.0.

26 Mar 2021

Agenda

Notes in italics

Issues & PRs

No new issues! \o/

2.1.0

  • opam-file-format#43
    • @dra27 to get 4.02 working, but otherwise ready for review
  • opam-file-format#32 - needed/safe for 2.1 release?
    • @rjbou - not urgent for opam 2.1; can double-check with @NathanReb whether it's still urgent for dune-release. Primary problem is that it keeps the "generated by dune comment". Given that it's possible to do it from the lexer, should this code go to dune-release and potentially be spun out further? Further discussion - potentially moving OpamPrinter.Preserved to a separate library.
  • #4616 (not yet opened!) (@dra27) - cold compiler to 4.12.0 to fix Cygwin (and some src_ext updates)
  • #4612 - replaces #4599
    • @AltGr to review
  • #4607 (@AltGr)
  • #4606 (@rjbou)
  • #4591 (@AltGr)
  • #4582 (@rjbou) - alternative to #4563
    • blocked on various CLI versioning PRs
  • #4581 (@rjbou) - blocked on #4606
  • #4580 (@rjbou)
  • #4563 (@rjbou)
  • #4548 (@kit-ty-kate)
    • move to next
  • #4494 (@kit-ty-kate)
  • #4438 (@rjbou)
  • #3897 (@dra27)

2.0

  • #4547 (@rjbou) - 2.0.9 release
  • #4538 (@kit-ty-kate) - CUDF trimming for 2.0

19 Mar 2021

Agenda

PR Sweep

Notes in italics

macOS testing CI problem with opam-rt - a sandboxing script update seems to have broken macOS testing?

2.1.0

2.0

External contributors

Misc. / next

12 Mar 2021

Agenda

Notes

  • Folding the ocaml-option-* packages into ocaml-base-compiler. @kit-ty-kate - concerned potentially at ocaml-option-vanilla not being default. Let's discuss further on a PR.

  • Using 4.13.trunk - @AltGr should be double-checked just in case the letters are treated separately. @kit-ty-kate - the version numbers do work.

  • For ocaml-src.4.13+trunk, @kit-ty-kate - what would happen if you wanted to use the dev branch after 4.13.0 is released? @dra27 - surely that wouldn't be needed for this package, though; the package is there so that there's one for the release.

  • OK to merge the 2020 Dev Meetings into a single page? The only downside would be the loss of linking. @rjbou - use a 2021 page straight away; @dra27 - with anchors for the dates.

26 Feb 2021

Agenda

19 Feb 2021

Present: @avsm (notes), @kit-ty-kate, @rjbou

  • all focus on rc next week -- looking at blockers.

  • start pushing out stuff to 2.1.1 milestone as we need to cut the RC.

  • https://github.com/ocaml/opam/pull/4542 is going to go in shortly.

  • https://github.com/ocaml/opam/pull/4311 is a bad performance regression, but we could punt this to 2.1.1 if we dont get a fix in time.

  • https://github.com/ocaml/opam/pull/4514 needs a decision on which variables we should take. Pushing out to 2.1.1

  • https://github.com/ocaml/opam/issues/4321

    • cant design a confirm-level feature in time for 2.1.0 - defer to 2.2.0 and a design spec
    • rename the cli option to explicitly note that it is dangerous, so it doesnt show up in tutorials etc: opam install --yes --unsafe-depext-yes

  • Minimum OCaml version for 2.x

    • bump to 4.06 as the minimum version for the 2.x series
    • we have bootstrap from a C compiler for any distros that are < 4.06 and not upgrading.

12 Feb 2021

Notes

@dra27 - Confirming that semver changes will wait until post 2.1, please - just for safety! @samoht has a point that we should discuss whether it's what's wanted

@emillon mentioned that Dune 3 development is starting. Since it's major, there are features slated for removal, including dropping support for OPAM 1.x - mainly finding the switch. @dra27 - it sounds fine (@AltGr agrees too) - maybe use opam --version in the error message to make it friendly that it's OPAM 1.2.2

@emillon - what should {with-doc} should do. @AltGr it's simply a variable, from opam's perspective. At present, many projects do @doc {with-doc} but this builds documentation without installing (@dra27 - which is of course wasted effort!).

@kit-ty-kate - does opam know when a package was installed using with-doc? @AltGr - no, it's not recorded @dra27 - although it's likely to change with package variables

@kit-ty-kate - discussions with @samoht on the difference in semantics between opam list --with-test and opam install --with-test. @dra27 - needed for 2.1? @kit-ty-kate - no, too big. @dra27 - let's ensure it's tracked to be picked up later. @kit-ty-kate - opened #4543

  • Various issue and PR triages - directly commented on the issues themselves

29 Jan 2021

Notes

Kate

  • Welcome to maintainership! @avsm will sort out GitHub permissions later.

Minimum supported version?

  • Dose6 is only just out - let's stick both 2.0 and 2.1 with Dose5, but switching to ocamlgraph 2 should be done (@AltGr agrees - it would be worth double-checking changes)
  • We may need to carry a few patches for ocamlgraph 2 in lib-ext
  • The release of extlib 1.7.7-1 unblocks 2.0.8 for OCaml 4.12, so that should be good to go

New issues

  • All in hand!

2.1 roadmap to RC

  • Ticking forward
  • @emillon note for 2.2 for removing unused fields from the format (Slack discussion this week - will move to an issue in the 2.2 project so we track it).

22 Jan 2021

Agenda

Notes

opam 2.0.8

  • Tests all working - backport PR is ready to merge
  • Should be working with OCaml 4.12~beta1 - @rjbou to check before release

New issues

  • #4505 - @rjbou to determine whether to fix for 2.1
  • #4507 - @rjbou problem would be calling the solver each time to check the dependencies, so the call would be much slower. @AltGr - another possibility would only to check the direct dependencies with the new flag. @dra27 - a third possibility would be to note the predicate values and see that they've changed and so interpret that as a reinstall. This is related to package parameters and so might be "fixed in passing". @dra27 to update issue
  • #4503 - restore opam config var for the RC; but ensure PRs open to use opam var for the identified tools

opam 2.1 rc checks

opam-file-format

  • @dra27 to have a look at #4394 (ensuring that opam-version: "2.1" and greater is the first non-whitespace/comment line)
  • @rjbou will ensure ocaml/opam-file-format#32 is merged for 2.1.3
  • ocaml/opam-file-format#25 is non-critical

mccs

  • AltGr/ocaml-mccs#31 may be fixed by a newer GLPK
  • Should check that GLPK 5.0 works, since Debian will package it at some point

src_ext

  • Dose 6 is nearly released - @AltGr
  • ocaml/opam#4470 adds required support for ocamlgraph 2.0

opam-lib users

  • opam-publish is updated
  • dune-release - @kit-ty-kate has a branch (https://github.com/kit-ty-kate/dune-release/tree/opam-2.1) but not yet a PR which has been being tested
  • Camelus - doesn't appear to be updated (@ThomasBlanc is away for a while) - but this is perhaps less critical. @AltGr to have a quick look and check the status?

CI using opam 2.1

  • ocaml-ci is using 2.0.7 - but it's based on docker-base-images which does have a 2.1 binary in it. Kate will test.
  • opam-repo-ci is using 2.1 with no problems
  • ocaml-ci-scripts - @dra27 will do a PR to set OPAMCLI=2.0
  • setup-ocaml - @dra27 will check it's uses

opam-repository-mingw

  • @dra27 + @MisterDA will look into that

opam upgrade

@emillon - it's presently difficult for both developers to test out the betas. We should have some more visible documentation on the process and potentially a tool to help. @AltGr - the install.sh script is supposed to do backups and other things. @AltGr - see "Try it!" at end of https://opam.ocaml.org/blog/opam-2-1-0-alpha/ @rjbou - there is also https://github.com/ocaml/opam/wiki/How-to-test-an-opam-feature (but it's a bit hidden away).

@dra27 - the wiki suggests having an alias for the new binary, but perhaps it would be better to have one for the old binary. So you have a backup of the old opam root and also a way to get back to the old root if necessary.

@emillon - what about local switches, as they're outside the root. @AltGr - the local switches are listed in the root (and this survives the upgrade). @emillon - what about if the local switch is used with 2.1, can it then be used with 2.0? @AltGr - almost certainly not, because the format may be upgraded.

@dra27 - rather than backing up local switches, perhaps we should just warn that that change is one way, so you might have to remake the local switch if you go back

@emillon - yes, the main point is to be clear exactly what backing up means and we do and don't get.

15 Jan 2021

Agenda

Notes

Present: @AltGr, @dra27, @kit-ty-kate, @emillon

beta4 -> rc

@AltGr: various things moving around with the conflict messages which are nearly there.

  • #4344 - @AltGr reckons this is closable.
  • #4489 - docs update; can be pushed to 2.2 if necessary
  • #4381 - @AltGr - almost certain a 1.2.2 upgrade hangover (initialise the defaults) so could be reclassified as a bug - you should just end up with empty selections. @dra27 will update the issue.
  • System compiler upgrading - this should be working; there should be a workflow for it to test it @dra27 will open and issue
  • Setting up switches for "OCaml 4.12" or "OCaml 4.11" with automatic upgrade to point releases. @AltGr - default at the moment will not do this. It's not obvious what the change would look like - let's add this to the considerations for opam 2.2.
  • #4188 - @rjbou to check and update; @AltGr - this may well have been fixed already (the problem was the code to infer the new switch invariant triggering in a read-only context)
  • #4172 - issue is still present; performance improvement bump to 2.1.1 @emillon - may be able to improve the performance of this by changing opam show slightly. For example, local switches are often unnecessarily scanned.
  • #4311 - slowdown appears to be caused by the switch invariants (@AltGr so it's slower "by design"!). @dra27 - can something be done either with two solves; one to get and fix the invariant and then with that list of packages in the base and then a second solve. @AltGr - for --installable the external solver isn't involved; this is actually Dose's SAT solver. @kit-ty-kate - the switch invariant in these cases is ocaml-base-compiler (it's come via an upgrade). @kit-ty-kate - --co-installable-with ocaml-base-compiler.4.11.1 speeds things up considerably. @AltGr - the universe is encoded differently which affects this, too (although it's not clear why that would be responsible for this slow down). @dra27 - does this slowdown still occur even when the switch invariant is exactly the list of base packages from the 2.0 switch? @kit-ty-kate - nope! @AltGr - it sounds as though --co-installable-with is usually what the user wants. @dra27 - in fact, given that --no-switch was needed in 2.0, would it be better for --installable in 2.1 to be assuming the base packages.
  • #4305 - @dra27 will update issue - definitely to be fixed; but not 2.1
  • #4313 - @AltGr will look into it - the two sequences of commands should be equivalent.
  • #4201 - remove from the 2.1 milestone
  • #4245 - remove from the 2.1 milestone
  • #4333 - link to the new issue @kit-ty-kate added
  • #4400 - @AltGr testing
  • #4389 - not critical for 2.1
  • #4454 - @rjbou to determine
  • #4455 - not critical for 2.1
  • #4485 - remove from the 2.1 milestone
  • #4490 - remove from the 2.1 milestone
  • #4297 - @rjbou to determine

8 Jan 2021

Agenda

  • opam-monorepo demo and discussion for metadata with @NathanReb
  • opam-file-format 2.1.2 & opam 2.1 beta4
  • Switching off Travis - is everything migrated?
Clone this wiki locally