Porting python-based llama example to c++.

[dougdew64]

Has anybody attempted a port of the python-based llama example to c++?

My understanding is that because the python and c++ apis are very similar, it's straightforward to port from python to c++. Is my understanding correct?

If my understanding is correct, then I'm going to get started on a c++ port.

[dougdew64]

D'Oh! I had misunderstood the mlx docs. The higher level stuff such as NN layers are only implemented in python and not in c++. Or at least not yet. So, when porting to a 100% c++ solution, it's necessary to port not only the example code but also some of the stuff such as NN layers.

[dougdew64]

Is higher level stuff such as NN layers going to eventually be made available in c++?

[awni]

Yes, that's definitely a possibility and something we originally intended. Prioritizing depends on how useful it would be.

The C++ nn API could be pretty similar with maybe some slight changes to be more C++ friendly.

[awni]

I agree with you. We are actively prioritizing a Swift API which sounds like it would suite your needs (?).

As for the C++ NN module, this hasn't been a big priority but it's also not a lot of work so it's something we can try to get in sooner rather than later.

[dougdew64]

If I correctly understand the implications of your priorities, you would likely implement NN and other higher level stuff in Swift instead of C++. Perhaps only later might you implement NN and other such higher level stuff in C++. But given that there would already be the higher level stuff in both Python and Swift, there probably wouldn't be much demand for also having higher level stuff in C++.

I'm guessing that having the higher level stuff in both Python and Swift would satisfy both the model development folks and the app development folks.

Thank you again for all that you are doing. Very much appreciated!

[douglasdew]

It seems that ultimately, for app developers, mlx will be something like an open source alternative to the CoreML stack. That seems like it would be a very big improvement for those folks who want or need to have some understanding of what is happening underneath their app code.

Again, thank you very much for all that you are doing!

[dougdew64]

For today, what I'd really like is for this repo to provide an Xcode project which could be used to build mlx as a fully-debuggable and fully profilable (sp?) library target within an Xcode workspace.

And, given that the examples repo seems to be 100% python, I'd like for there to be a 100% Swift examples repo which assumes Xcode. In that repo, all of the examples would be provided as Xcode workspaces which include/make-use-of the mlx Xcode library project.

Initially, the Swift examples could be lower level. For example, linear regression. As more higher level Swift-based/Swift-supporting functionality (e.g. NN stuff) is added to mlx, then additional Swift examples could be added to the Swift examples repo to make use of the higher level mlx functionality. Eventually, there would be a Swift llama example, just as there is currently a python llama example.

[dougdew64]

By the way, sorry for posting as two different accounts (work and personal).

[alikhan-tech]

Porting a Python-based example to C++ can indeed be feasible, especially considering the similarities between Python and C++ APIs in certain contexts. While the process may involve translating the logic and structure from Python to C++, leveraging equivalent functionalities, it's essential to note that direct translation might not always be one-to-one due to language differences.

Starting a C++ port based on your understanding could be a great initiative. Just be prepared for potential nuances and differences between the languages that might require adjustments in the translation process. Utilizing the similar logic and APIs can certainly expedite the transition, but some adaptations may be necessary for optimal functionality in C++. Best of luck with the porting process!

[dougdew64]

Thanks @alikhan-tech.