Include Swift in Supported Programming Languages

[joelchen]

https://github.com/mzaks/FlatBuffersSwift is bringing FlatBuffers to Swift.

[ghost]

Yup, once Maxim feels the port is solid, he should create a PR to have it merged into the main project (if he wants). @mzaks ?

One problem I see is that the building API seems to require you to create objects beforehand. The building API for all other languages avoid creating garbage objects by constructing objects in-place. For a high performance API that is kind of important.

[mzaks]

Will be glad to create a PR, when most of the Issues I defined in my repo are resolved.
One of them is generating API which let user generate binary array without creating objects beforehand.

[ghost]

Sounds good.

[ghost]

@mzaks: what's the status of this port?

[mzaks]

Sorry for being under the radar for that long. Moved the project to support Swift3. Didn't battle tested it yet, but the unit tests are green :)
I still have some features which has to be implemented to reach the feature parity though.
Like:

• Support "id: n" attribute mzaks/FlatBuffersSwift#9
• Support required attribute mzaks/FlatBuffersSwift#10
• Support 'bit_flags' attribute mzaks/FlatBuffersSwift#11
• Support namespaces mzaks/FlatBuffersSwift#15
• Support include of other fbs files mzaks/FlatBuffersSwift#48

Than FlatBuffersSwift can generate invalid buffers as I implemented support for cycles.
I also provide users with the possibility to not nil terminate the strings.
This is however a small thing. I could remove it from the BuildConfig and force null termination.

To be honest I am also puzzled, how it would be merged as the only thing you need is the code generator fbsCG.jar. I guess it is similar to C port which has a separat repository as well.

[ghost]

It would be great if you could prioritize strong compatibility with the other FlatBuffer implementations. If you do things differently, and people exchange buffers between Swift and other languages, you're causing a world of pain for no good reason. In fact, something that is not compatible with FlatBuffers should not say it is an implementation of FlatBuffers, to avoid confusion.

If you can make it entirely compatible, we can work on integrating it into the main project, if you want.

[mzaks]

The main reason why I implemented cyclical graph support, was because it conforms with object graph mapping.
Given following fbs

table Person {
  name : string;
  friends : [Person]
}

The graph mapping strategy let's user encode following graph.

let p1 = Person()
p1.name = "Maxim"
p1.freinds = [p1]
let data = try?p1.toData()

The last statement produces the buffer. Which will be valid iff Object offsets are allowed to be signed 4 bytes integers. As for example vTable offsets are. But we already had this discussion :)
What I could do is following:
The toData method receives a config parameter, which has a default definition.

public struct FBBuildConfig {
    public let initialCapacity : Int
    public let uniqueStrings : Bool
    public let uniqueTables : Bool
    public let uniqueVTables : Bool
    public let forceDefaults : Bool
    public let nullTerminatedUTF8 : Bool
    public init(initialCapacity : Int = 1, uniqueStrings : Bool = true, uniqueTables : Bool = true, uniqueVTables : Bool = true, forceDefaults : Bool = false, nullTerminatedUTF8 : Bool = false) {
        self.initialCapacity = initialCapacity
        self.uniqueStrings = uniqueStrings
        self.uniqueTables = uniqueTables
        self.uniqueVTables = uniqueVTables
        self.forceDefaults = forceDefaults
        self.nullTerminatedUTF8 = nullTerminatedUTF8
    }
}

I could flip nullTerminatedUTF8from false to true and add an allowCycles property which will default to false
This way: p1.toData() will throw an exception. The user will have to explicitly call:

p1.toData(config: FBBuildConfig(
  initialCapacity = 1024, uniqueStrings = true, uniqueTables = true, uniqueVTables = true, forceDefaults = false, nullTerminatedUTF8 = true, allowCycles = true
))

In order to build buffer which contains cycles and there for is incompatible with standard FlatBuffers implementations. I understand that it is not ideal, but it will be users conscious decision to go with a solution which is incompatible with other implementations.

In my current iOS project (http://resiapp.io) I use FlatBuffers for app state persistence. By enabling cyclic graph I get a persistence strategy which is better than any other technology out there. It also implies that I don't really care about compatibility with other languages, because the data does not leave the device. Removing the cycle support completely feels like putting a tempo limit on an engine which can go faster because of some public regulations :)
I understand the reasons behind the "public regulation" but it still hurts

[ghost]

@mzaks : I understand you can do some cool stuff with cycles, but we've discussed at length in the past why FlatBuffers doesn't support them.

This is a free world, and FlatBuffers is open source, so you're welcome to fork the format in whatever way works for you. But please make sure your users are aware that this is incompatible with FlatBuffers, and it isn't an implementation of FlatBuffers.

For most users, there's not a lot of gain of cycles, yet the downsides are huge.

It be great if you could make your work conformant however, and make Swift officially a language we support.

[m339606]

We are a team that would like to use this library, but we are still afraid of the incompatible problem. Therefore, we would like to know if the official would support Swift, or if it is already in your schedule. Thank you.

[mzaks]

@m339606 Which incompatibility problem do you have in mind?

Generally it would be cool if we had a formalised test of suits.

1. Multiple schemas to verify IDL support for different code generators.
2. Schemas and binaries to verify if the platform specific implementation can encode and decode data represented with such binary.
3. A few use cases for bench mark tests, where every platform implementation performance characteristics is listed.

I am willing to contribute in creation of such test suit.

[aardappel]

@m339606: We would like to support Swift in the main FlatBuffers project, but as discussed above, @mzaks's implementation is incompatible with the other language implementations. So either those need to be addressed, or maybe someone else will make a compatible implementation. If that doesn't happen, I can't promise when or if I can make a Swift implementation myself.

@mzaks: please re-read the discussion above. Support for cycles will need to be removed for your implementation to be compatible with FlatBuffers.

[jnordberg]

I would love to see official Swift support for FlatBuffers!

@aardappel are you saying that cycles needs to be completely removed from the implementation for it to be compatible, it's not enough to have a configuration option that defaults to off like @mzaks suggests?

[aardappel]

An option could work, if it is indeed off, and it comes with a very clear disclaimer that this is a feature currently a) unique to swift, b) not part of the official FlatBuffers spec, and c) generates incompatible binaries that may generate verifier failures or crashes on current and future implementations depending on language.

Feel free to make a PR for Swift, that will make discussing changes easier :)

[Dev1an]

What is the current status on this topic?

[aardappel]

I have not seen any progress in making Swift officially supported in the main FlatBuffers project. PRs still welcome :)

[eduardbosch]

@aardappel @mzaks Is this swift lib secure if it's used only to read Flatbuffers files generated with the official Flatbuffers library?

I generate my files with the official JS and I just want to read those files from an iOS application. I think that if I don't generate any file with this swift library, nothing can break as I won't introduce any incompatible thing in my Flatbuffers files.

[mzaks]

@eduardbosch it should be safe to read.
https://github.com/mzaks/FlatBuffersSwift/blob/master/FlatBuffersSwift/FlatBuffersReader.swift Contains a file and memory reader. So you can read from Data or from FileHandle.
You can use fbsCG.jar to generate Swift classes. However there could be some warning now, as it was built for Swift3, I worked on a pure Swift parser and code generator on following branch. https://github.com/mzaks/FlatBuffersSwift/tree/code_gen it suppose to be a console application written in Swift which can read fbs files and generate swift classes. It is currently on hold because I don't use FlatBuffers in current projects and there were not much request help from the community. I also think I saw other ports. Haven't tried them though.

[eduardbosch]

Thanks @mzaks

I finally used this library https://github.com/TonyStark106/SwiftFlatBuffers

Maybe this library could be used to add support to swift if it follows all the same official FlatBuffers specifications.

It's working good and let me use it with Carthage.

[aardappel]

Did anyone look at the differences between @mzaks and @TonyStark106 implementations? Looks like the latter actually works with flatc, so would be easier to integrate into this repo.

[hassila]

One reflection is that @mzaks Have spent fair amount of time on optimization and benchmarking - couldn’t find any such numbers for the other implementation.

[eduardbosch]

I've moved to https://github.com/mzaks/FlatBuffersSwift only to read FlatBuffers files.

The lib https://github.com/TonyStark106/SwiftFlatBuffers has this bug TonyStark106/SwiftFlatBuffers#2 that causes nearly all my FlatBuffers files unreadable. Seems that it does not handle correctly optional values.

[stale]

This issue has been automatically marked as stale because it has not had activity for 1 year. It will be automatically closed if no further activity occurs. To keep it open, simply post a new comment. Maintainers will re-open on new activity. Thank you for your contributions.