Fix compatibility with recent rebar3

[lukateras]

@Lucus16 wrote on Jul 11, 2019 8pm UTC:

Since nixpkgs switched to rebar 3.9, mixToNix is broken for rebar dependencies which have their own dependencies. Rebar deprecated hex registry v1 (https://github.com/hexpm/specifications/blob/master/registry-v1.md), so the fake hex registry no longer works, rebar doesn't even look at it anymore. It doesn't seem like they've switched to v2 either, which uses protobufs. They seem to be using a file called .cache/rebar3/hex/packages.idx instead, which is still in ETS format.

The nixpkgs commit where mixToNix broke is d4b243905f2181111b52d5a58b2119469ed689fc. It also removes a patch for rebar3 which seems to disable the update code. I tried applying it to rebar 3.11, but it still tries to talk to repo.hex.pm.

I think it's best to start using the custom fetchHex to place the sources in _checkouts since it was made for sandboxed builds. See https://www.rebar3.org/docs/dependencies#section-checkout-dependencies for more information. This only provides sources though, so I'm not sure independent compilation for each library is still possible. If the sources are provided in _checkouts, rebar might be convinced of using the artifacts in _build. If that's not possible, rebar dependencies may need to be built flat, without first building its recursive dependencies. I couldn't figure out how to add stuff to the _checkouts directory because the control flow of mixToNix is pretty messy.

I tested all this with a basic elixir project with a dependency on certifi and a recent nixpkgs unstable with 2c2fe05ed5a585194fdc98f71823d9937965a7ca reverted. Rebar dependencies that don't have dependencies themselves still seem to work.

[lukateras]

@yorickvP wrote on Jul 11, 2019 8:57pm UTC:

PR for build-mix fixing: NixOS/nixpkgs#64884

[lukateras]

Status update: I couldn't make _checkouts work without reducing functionality, so I've been working on a rebar3 package index generator:

defmodule FakeRebar3PackageIndex do
  @package_index :package_index_v4
  @package_index_version 4

  def dependency(name, version) do
    [package: name, requirement: "~> #{version}"]
  end

  def package(p) do
    [key: {p["name"], p["version"], ""},
     checksum: p["sha256"],
     retired: false,
     dependencies: (for d <- p["deps"], do: dependency(d["name"], d["version"]))]
  end

  def main([out]) do
    packages = Jason.decode!(IO.read(:all))
    table = :ets.new(@package_index, [:named_table, :public, :ordered_set, {:keypos, 2}])
    :ets.insert(table, {@package_index_version, :package_index_version})

    for p <- packages, do: :ets.insert(table, package(p))
    :ok = :ets.tab2file(table, String.to_charlist(out))
  end
end

This doesn't work just yet, I think I'm getting there.