Working on Swift support

[Dan Appel]

Hi! I've been working on a Swift compiler plugin for the past week or so, and have been making decent progress. I've run into a few blind spots in the encoding spec (i.e. unions) but seemed to have gotten over the major humps.

Just wanted to let people know in case somebody has tips or concerns.

I'll also use this thread as a place to ask any future questions that I may have :)

[David Renshaw]

Cool!

Do you take advantage of Swift's pattern matching in your support for Cap'n Proto unions?

How do you manage the lifetime of a message's underlying buffer? Is it like in capnproto-c++, where the burden is on the programmer to ensure that references to a message's interior do not outlive the message itself? Or is it more like capnproto-java, where that burden is taken up by automatic memory management, at the cost of some run time performance? It's totally unclear to me which would be the better choice for Swift.

- David

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

[Dan Appel]

>Do you take advantage of Swift's pattern matching in your support for Cap'n Proto unions?

Yes! A union is generated as just an enum with associated values :)

>How do you manage the lifetime of a message's underlying buffer?

Swift's native memory management model is ARC, so I'm going with that. Once the message loses all references to it, the buffer gets deallocated. Ideally each object the user makes has some kind of reference to the message, so it would work right out of the box (this is the idea, but I have not finalized the design of who references who, so it may not behave correctly just yet).

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

Visit this group at https://groups.google.com/group/capnproto.

--

Dan Appel

[Dan Appel]

Okay, I have a question. In my structs I'm running into pointers that have the value of 0 (the UInt64 pointer itself). What do I do with these? Is it an error on my end that I'm running into these in the first place? I've been trying to debug this for a few hours now and I can't find the source of the issue. 

Here's some logs that I got from reading CodeGeneratorRequest.ptrs[0] (nodes @0 :List(Node)) - link.

I'm leaning on this being my fault somewhere in the parsing but just wanted to make sure I'm not missing something before I dive deeper.

[David Renshaw]

Those are null pointers. A `has()` accessor would return `false` for them, and a `get()` accessor would return the default value. See https://capnproto.org/faq.html#how-do-i-make-a-field-optional

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.

[Dan Appel]

Oh, well now I feel silly :) Thanks for pointing that out!

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

Visit this group at https://groups.google.com/group/capnproto.

--

Dan Appel

[Dan Appel]

Does this mean that every field (including lists) is nullable? That's quite a shame. Looks like all my generated fields are going to be implicitly-unwrapped-optional, then (reminds me of objective-c translated into swift).

--

Dan Appel

[Ross Light]

Just pointer fields. Data section fields are always "non-optional".

[David Renshaw]

On Tue, Sep 20, 2016 at 9:48 PM, Dan Appel <dan.a...@gmail.com> wrote:

Does this mean that every field (including lists) is nullable? That's quite a shame. Looks like all my generated fields are going to be implicitly-unwrapped-optional, then (reminds me of objective-c translated into swift).

All pointer fields can be null. Because all types in Cap'n Proto have a default value, you don't need to return optionals; you can return the default value when a `getFoo()` accessor reads a null. You should additionally provide `hasFoo()` accessors for when the user actually wants to distinguish the null case.

However, if you were decoding Cap'n Proto messages into stand-alone Swift-native plain-old-data structs, then yes, you would in general need to make pointer fields optional, because otherwise values of a recursive type (e.g. `struct Bar { bar @0 :Bar; }`) would be infinitely large.

- David

 

[Dan Appel]

>All pointer fields can be null. Because all types in Cap'n Proto have a default value, you don't need to return optionals; you can return the default value when a `getFoo()` accessor reads a null. You should additionally provide `hasFoo()` accessors for when the user actually wants to distinguish the null case.

I'm a little confused by the documentation on this one. What is the default value of a struct? This section seems to only be about structs inside lists, but even so I'm struggling to understand where I should be looking. If I have an all-zero pointer, where is the default value stored? Where would I store the value a user sets?

>However, if you were decoding Cap'n Proto messages into stand-alone Swift-native plain-old-data structs, then yes, you would in general need to make pointer fields optional, because otherwise values of a recursive type (e.g. `struct Bar { bar @0 :Bar; }`) would be infinitely large.

Gotcha. Unrelated note: making the properties optional does not work in swift, the following does not compile (recursive values need to be wrapped in a reference type, optionals are still a value type. You would need a `class Box` of some sort, which is messy)

struct Foo {

    let foo: Foo?

}

Dan

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

--

Dan Appel

[David Renshaw]

On Wed, Sep 21, 2016 at 2:44 PM, Dan Appel <dan.a...@gmail.com> wrote:

>All pointer fields can be null. Because all types in Cap'n Proto have a default value, you don't need to return optionals; you can return the default value when a `getFoo()` accessor reads a null. You should additionally provide `hasFoo()` accessors for when the user actually wants to distinguish the null case.

I'm a little confused by the documentation on this one. What is the default value of a struct?

The default value of a struct type is a struct whose fields all contain their default values.

 

This section seems to only be about structs inside lists, but even so I'm struggling to understand where I should be looking. If I have an all-zero pointer, where is the default value stored?

There's no data to store. It's as if the struct has an empty data section and an empty pointer section.

 

Where would I store the value a user sets?

 

Ah, so we're considering mutations now, and hence Builders, which differ significantly from Readers. If the user calls `get()` on a null pointer field in a Builder, you do need to actually allocate space for that field, in preparation for the user possibly setting values on it. Note that once you do this, the field will no longer be null.

- David

[Kenton Varda]

FWIW, it's on my wishlist to add Maybe(T) as a built-in type. Then, we can get rid of "has" methods -- Maybe(T) would generate getters which use the language's preferred "maybe" idiom, whereas non-maybe T would generate a getter which returns the default value if the pointer is null (with no way to actually test if the pointer is null).

BTW, note that while the "default default" value of a struct is the zero-sized struct, it is possible for a schema to specify some other default for a specific struct field. Example:

    struct Foo {

      i @0 :Int32 = 123;

    }

    struct Bar {

      baz @0 :Foo = (i = 456);

    }

A typical implementation is to store "baz"'s default value as a static byte buffer somewhere, and then when getBaz() is called and the pointer is null, act like it somehow pointed to that static buffer (or in the case of builders, initialize it to a copy of that static buffer).

It's pretty unusual for people to use this feature in practice, though.

-Kenton

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.

[dean...@yelp.com]

I'm considering using Cap'n Proto for a Swift project. I wouldn't mind picking up a partial implementation and completing it. Dan, or anyone, did you end up committing anything?

Dean

[Dan Appel]

I left it incomplete - I had decoding implemented but did not finish working on traversing the schema and codegen. I could clean up what I have and push it on GitHub if you'd like a starting point :)

Dan

On Mon, May 1, 2017 at 8:12 AM deansher via Cap'n Proto <capn...@googlegroups.com> wrote:

I'm considering using Cap'n Proto for a Swift project. I wouldn't mind picking up a partial implementation and completing it. Dan, or anyone, did you end up committing anything?

Dean

[Dean Thompson]

The way my project has shaped up, I won’t immediately be working on Cap’n Proto over Swift. I’d love to know it’s out there, if you feel like cleaning it up and pushing it to GitHub, but I have no near-term momentum to offer as incentive.

[deansher...@gmail.com]

Dan, I’m now at a point where I’d work on this. Would you be up for throwing what you have into GitHub?

Dean

[dent...@gmail.com]

Did this code ever go anywhere, or should I consider it dead for Swift?

I have a mixed product in Swift and C++ so am looking at mutually-compatible exchange options.

[Dan Appel]

Dean, sorry I missed your email! I just threw it up, though I realize it's been a couple months and it may be significantly less valuable now. The runtime code should have stuff of interest, but the generator code can largely be ignored.

Andy, there's nothing production ready available AFAIK, but if you're willing to take on the weight of implementing it yourself there's a bunch of people willing to help!

What I have so far: link

Dan

On Sat, Feb 10, 2018 at 8:53 PM <dent...@gmail.com> wrote:

Did this code ever go anywhere, or should I consider it dead for Swift?

I have a mixed product in Swift and C++ so am looking at mutually-compatible exchange options.

--

[dent...@gmail.com]

Dan

Thanks for the source link. I doubt I will find the time to do much for a month as I'm off to a US conference (Social Media Marketing World in San Diego) with a killer schedule before but am very interested. My first product release is just Swift/SpriteKit in iMessage so I can defer getting the packet format solid and just use Codable.

I have a long background in this kind of stuff, mostly in C++, with my old OOFILE framework providing high speed mapping direct to the ISAM data store buffers. (I still remember fun times with the SparcStation users running into offset-based crashes.) 

I've done a lot of code generation work in the past, using languages ranging from a weird TCL variant to Ruby and XSLT. 

My Realm experience may also help - I spent nearly two years on the C# team, with our SDK mapping to the C++ core. We used Fody, an IL-generator, to do compile-time reflection of RealmObject subclasses and generate schema and inject mapping calls.

One strategy for the Swift parsing side I'm musing is using Codable itself and writing the generator as a custom encoder. Mike Ash wrote an [article about his binary encoder](https://www.mikeash.com/pyblog/friday-qa-2017-07-28-a-binary-coder-for-swift.html) which is on my list to play with too.