Would you be interested in a binding to the Julia Language

[freemin7]

Hello,

i could imagine writing a binding for the Julia language which would make the QP solver usable from the JuMP modeling language.
Is that something you would like to see?
I'm not unaware that you removed Julia code in 2019.

[giaf]

Hi, thanks for the interesting proposal.

Personally I'm not very familiar with Julia and I only use it exceptionally.
That also explains the fate of the Julia code you mention.
The Julia interface was added by a colleague of mine, and it just ended up being unmaintained and eventually removed after it was too broken due to some big code change.

Now, the C solver interface has not changed in quite some time now and I don't foresee needs to further change it (except maybe at some point deprecating the set_all routines such as https://github.com/giaf/hpipm/blob/master/include/hpipm_d_ocp_qp.h#L90), so this plays well for a new attempt.
Unless you also volunteer to maintain it, in order to increase the likelihood of its long-term survival it would be important to have something concise and self-explanatory enough for me to easily and quickly fix it assuming close to no Julia proficiency, and a set of tests (preferably performed automatically at each commit, as the current tests on Travis CI).

How does it sound for you? Do you already have some kind of plan in mind?

[freemin7]

I'm actually not sure maintaining this binding as part of this repository would be the best idea. Here is what i have in mind:

• Make use of the Julia build system for non Julia code to have reproducible binaries Julia can link against
• Have this binary wrapper (Probably Hpipm.jll) be tested/hosted by the the Julia binary wrapper organization
• Write a wrapper of the C language interface in Julia (or code that generates such a wrapper).
• Write test for C interface called from Julia
• Make a new repository (probably HPIPM.jl or some caps variation of that)
• Make that a Julia package, so it can be installed vie the Julia package manager easily
• Write code that satisfies high level interfaces in the Julia ecosystem (MOI/JuMP) to make it interoperable with other optimizers from a modeling language
• Write tests for that higher level interface
• Have the JuMP organization host and test the wrapper.
• Register those packages

This would be the ideal way to do this from a Julia standpoint, as it would make installation using the Julia package manager trivial and reproducible. I'm not to familiar with the first two steps and i am currently a bit busy with other projects. My proposal would be a sustainable, discoverable and reproducible solution. However your proposal would be faster to implement and involve:

• Write C wrapper See CBindingGen.jl
• Write test for that C wrapper (It's quiet easy to translate existing C tests to Julia, i could help with that)
• Setup test for Julia in your repository (Also doable)

Since there is no time pressure to have this wrapper i would encourage whoever implements it to pick the first approach as it would be more sustainable. This person might be very well me be but i currently don't see HPIPM on the critical path of any of my projects.

I would also like to keep this issue open for future reference in case anybody is interested.

[giaf]

Ok I see, thanks for pointing out the good way of doing this in Julia, as I said I'm not very familiar with it.
One question about maintainability/sustainability: your proposed approach involved several external steps performed by different Julia organizations. In case of large-ish changes in the solver, or alternatively simply in case of new HPIPM releases, what is the workflow to update the binding?

Unfortunately in this period I don't have enough time to invest in this, but obviously I would provide support to whoever wants to take over the effort.
And surely we can keep this issue open for reference.

[freemin7]

To produce a Julia binding for your solver no changes to your code should be required.
The only changes that i can possible think are:

• Include the Julia binding in your repo (which i would not recommend, putting it an a stand alone package would be nicer).
• Update the readme or documentation on how to use the Julia binding

I am wrapping an proprietary NLP solver with an C interface in Julia. In this case they distribute a shared library and some header files. I generate a header file for this C library and tell Julia where to find the shared library and that's about it and have an interface in Julia that uses C types which i then wrap in more user facing Julia code. In this project I'm currently at the stage of making a JuMP (modelling language) binding for the solver.

The workflow to update an existing binding would be:

• Open an issue with the user facing repo that a newer version is available.
• Depending whether the 1 the C interface stayed the same, 2 new functionality was added or 3 backwards incompatible changes were made these actions would be required to be performed by someone.

1. In Hpipm.jll: Change download location to new version of source code and maybe update build directions, have the Julia binary wrapper organization automatically compile a new version for a variety of architectures. In Hpipm.jl: Update Julia compatibility information in user facing package, commit changes to main branch and make a new release of user facing package.
2. Same as 1 but in Hpipm.jl the C binding would need to be extended (this can happen from the header files or manually) and whether the functionality is needed, new wrapper code would need to be written which might require changes in the Julia facing data structures.
3. Depending on how large the change is a complete rewrite of Hpipm.jl might be faster, the alternative is to do 2 and reflect the changes of underlying library in the Julia user facing code. Mark new version as incompatible with previous one.

For clarification Hpipm.jl is the user facing repo which has Hpipm.jll as dependency which builds the platform specific shared libary for the users Julia installations calling conventions and standard C/Fortran libraries in so far as that is relevant.