Bring Snapchat things to our main repository

[paulocoutinhox]

Hi,

I talked to "Li Feng", the engineer from Snapchat that is responsible for Djinni on Snapchat to understand all modifications that they do in Djinni source and all the new features added.

I think that it is too much relevant, because will speed up somethings (like strings), have some general fixes and will add support for more field types (like transfer pointers/ownership between languages).

I will bring the source code and the cross-language-cpp guys can check the pull-request slowly as i bring it to our repository.

The Snapchat repository is here:
https://github.com/Snapchat/djinni

The Snapchat article is here:
https://eng.snap.com/improving_djinni

Some modifications that i collect from their README:

• Replaced sbt and gyp with Bazel
• Added move assigment operator to GlobalRef in all djinni_support.hpp files
• Made JniClassInitializer constructor public to allow arbitrary additional initialization in JNI_OnLoad()
• Speed up string passing between Java and C++ (about 5-10x faster depending on string size)
• Eliminate CppProxy finalizers (better stability)
• Injecting code with DJINNI_FUNCTION_PROLOGUE
• Option to generate ObjC protocols
• Option to generate function prologue
• Option to omit objc helper methods
• Option to disable exception translation in ObjC
• array<> type support
• outcome<> type support
• Protobuf type support
• Local flags with @flag directive
• DataView for copy free data passing
• DateRef for copy free data passing with ownership
• Bug fixes

What do you think?

[jothepro]

I think that's a good approach, migrating the features one by one in small PRs. You'll not have to do this on your own, I do have a great interest in the improvements that Snapchat made as well and probably will contribute some PRs as well :)

Support for bazel is already WIP (there is no PR yet): #96 (comment)

[paulocoutinhox]

The problem is separate each part from code.

I will focus on code, instead of Bazel part, that is not my best (nothing is my best, but i will try rsrsrsrs).

In my mind what i will do is compare the two repositories and bring all code changes that is necessary in one time.

What do you think?

[jothepro]

yup, extracting the features will not be easy :(
My worries is that if you try doing all at once this will become a huge PR that has no chance of getting merged because discussions will go on forever and we'll have quality issues afterwards... Apart from that I think that it will not be possible to simply copy stuff over, we'll also have to add/adapt tests, which would make the PR even bigger? 😟

But feel free to give it a try! 💪 If you migrated all features at once we can still split the PR if we are getting overwhelmed :)

[a4z]

Good list, thanks a lot Paulo!

We will get bazel, that is sure, if it will be a replacement , or an option we will see, but atm it looks more like an option.

About the rest, we need to look at it one by one, and maybe we can prioritize the list.
I like some things a lot, to some I have no opinion, and I am unsure if we ever need others.

Also, keep in mind our limited resources, so we need to move slowly, and capture everything with good testing.

The main goal must be avoiding complexity that will kill us over time but grow nevertheless. But we must keep in mind that as an OSS project by voluntaries we play with other roles and need to apply some principles to manage the growth sustainable.

I understand entusiasm is great now, but lets try to order the list to see where everyone has a personal favorite, and then get a consensus in what we can do, and what we will do.

So let's try to see what are the various favorites, how they relate, what depends on what, and how could one be added after one with tests that guarantees sustainability over time in an project like ours.

[a4z]

I will transfer this issue into a discussion item, since there is no definition of done possible, and I see that there are various details to discuss for some points on the list.
So we will keep that as a discussion, from which issues to implement can arise .

[mutagene]

Made PR #157 attempting to bring in snapchat web assembly support, which I see as the "biggest" feature from the snapchat repo. Lots of other nice things too, but I could really take or leave bazel, and I feel like some of the optimizations can be made piecemeal - but we either have web assembly or we don't.

[mutagene]

FWIW, if we merge the web assembly support, my interest would be to next focus on

• array<> type support
• outcome<> type support
• DataView for copy free data passing
• DateRef for copy free data passing with ownership

For my purposes, at least, these are the next most interesting on the list.