Typed arrays via Lightweight Generics

[Rory Sinclair]

Hi,

We've been using j2objc for a number of years to share a large amount of code (mainly API client code and models) between our iOS and Android apps, to varying degrees of success.

One thing thats always been an annoyance is the fact that Java arrays are mapped to IOSObjectArray. I suspect this is due to the fact that Objective-C arrays are not typed, whereas Java arrays are, so the wrapper object is needed to retain that type information.

Obviously, most iOS teams these days are working with Swift primarily, and while Obj-C <-> Swift interoperability is generally good, it feels like a bit of a shame that we can't deal with native Swift typed arrays in our transpiled code.

I was reading about Objective-C lightweight generics, which apply to NSArray, NSDictionary and NSSet. When using generic NSArrays in Obj-C code, Swift can correctly ascertain the type of the member objects - e.g. NSArray<NSDate *> is treated as [Date] in Swift.

This is obviously extremely desirable for Swift use-cases and anyway desirable even if working purely in Obj-C, as the compiler can warn you about unsafe or wrongly typed access to the array (or Dictionary, or Set).

Is this something the team have considered? To have array types map to lightweight-generic NSArray instances rather than IOSObjectArray?

Am I missing some development that supercedes my thinking above, or is it otherwise wrong?

If it sounds like it would work and nobody has yet tackled it, i'm tempted to say i'd give it a shot myself, though its potentially a bit of a rabbit-hole... :)

Thanks!

- Rory

[Rory Sinclair]

'Annoyance' is a bad choice of word there... I totally get why its needed, its a limitation of Obj-C that it lacks typed arrays. :) Mea culpa, I should have said 'frustration'.

[Tom Ball]

You're right about a Java array having a type, but it also has an immutable length with mutable elements, and unlike lightweight generics, substituting an incompatibly-typed element throws an ArrayStoreException. NSArray is fully immutable, while NSMutableArray is fully mutable, so neither is a perfect match, making it difficult to just replace IOSObjectArray with NSArray.

You bring up a good discussion point, though. What if we somehow enhanced IOSObjectArray so it extends NSArray? The NSArray doc suggests this would be straightforward, but IOSObjectArray already extends IOSArray. Maybe that wouldn't be a problem, though: primitive arrays could support NSArray's objectAtIndex: method with a wrapped instance. For example, a float[] array can have its objectAtIndex: method return a java.lang.Float. Since NSArray requires subclasses to manage their own backing store, IOSFloatArray would still use its jfloat[] buffer_ field, so there's no performance hit or app size increase. There's other issues to investigate, like how NSCoding would be supported.

Is this something you'd consider working on with us? I'm not a Swift developer, and hope to retire before having to fully learn the language. 😀 If you were the first user of such a feature before it was released, we can much more quickly address any problems the Swift importer has with IOSArrays that support generics.

--
You received this message because you are subscribed to the Google Groups "j2objc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to j2objc-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/j2objc-discuss/ecd10904-b57d-4218-b35e-10a7d3817379n%40googlegroups.com.

[Rory Sinclair]

Hi Tom,

Absolutely! Though I took a look at the j2objc code with a view to tackling this myself and immediately went cross-eyed... I think this would be beyond my abilities / time availability to tackle solo, thats for sure. But i'm more than happy to help development in terms of trying things out on our project and delivering feedback / bug reports etc, and maybe small contributions code-wise, if I can.

I suspect we are one of the longest-running users of j2objc in a production environment, for many years now, and would relish the chance to help! Generics support would be a boon for all j2objc users, as mentioned, but especially for developers mainly looking to use j2objc-transpiled code in Swift projects.

[Rory Sinclair]

Not sure if this may spark any thoughts too, but lately we've taken to also auto-generating Swift extensions to our transpiled Obj-C classes, to allow more natural access to fields on the objects via custom getter / setter logic.

We're already introspecting the Java source to swap in annotations and generate custom mappings for a library we're using to map JSON responses to instances of the transpiled classes, called KeyValueObjectMapping, so while we're doing so we now also generate these Swift extensions too

Here's an example using our transpiled ASWv1Privilege domain object, with a simple String array field, as well as a complex (also transpiled) object array:

```
extension ASWv1Privilege {

    var types_ : [NSString] {
        get {
            if let array : Any = self.types {
                let val: [NSString] = LibASWCoreUtils.objArray(for: array as! IOSObjectArray)
                return val
            }

            return [NSString]()
        }
        set {
            if let array : IOSObjectArray = IOSObjectArray.init(nsArray: newValue, type: IOSClass.forName("NSString")) {
                self.types = array
            }
        }
    }

    var tags_ : [ASWv1Tag] {
        get {
            if let array : Any = self.tags {
                let val: [ASWv1Tag] = LibASWCoreUtils.objArray(for: array as! IOSObjectArray)
                return val
            }

            return [ASWv1Tag]()
        }
        set {
            if let array : IOSObjectArray = IOSObjectArray.init(nsArray: newValue, type: IOSClass.forName("ASWv1Tag")) {
                self.tags = array
            }
        }
    }
```

The 'LibASWCoreUtils.objArray' method is defined:

```
    class func objArray<T>(for: IOSObjectArray) -> [T] {
        var array = [T]()
       
        if (`for`.length() > 0) {
            for index in 0...(`for`.length() - 1) {
                array.append(`for`.object(at: UInt(index)) as! T)
            }
        }
       
        return array;
    }
```

[Tom Ball]

Rory,

Step one down the generics path is up for review at https://github.com/google/j2objc/pull/1876. It makes the IOSArray* types subclasses of NSArray, and adds generic declarations. In theory (I'm not a Swift developer), an IOSObjectArray<JavaUtilDate*> instance will show up in Swift as [JavaUtilDate], and dates[3] returns a JavaUtilDate without an optional type.

Primitive arrays will return wrapped elements when using Swift array syntax, for example, with a float[] array in Swift array[3] will return a JavaLangFloat instance. Since java.lang.Number extends NSNumber, that shouldn't be an issue.

Please review this pull request, and ideally try it with a simple Swift test.

To view this discussion on the web visit https://groups.google.com/d/msgid/j2objc-discuss/76af21fa-4b23-4dea-b0bb-e7b887755c16n%40googlegroups.com.

[Rory Sinclair]

Tom,

That is a thing of beauty! I'll try to put together a test build and report back. (Apologies for the delay in replying, I never seem to get notifications for updates from this group..)

Regards

[Tom Ball]

It’s not complete, as testing found that NSArray’s isEqual: calls isEqualToArray:, which tests each array element for equality. That’s what most devs want, but Java arrays’ equals() methods are supposed to only test for array identity (like Object.equals()). The test didn’t pass with by simply overloading the method, so I need to dig deeper.

In your code, it’s worth verifying that you’re using Arrays.equals() is used if you want the the NSArray equality test. 

To view this discussion on the web visit https://groups.google.com/d/msgid/j2objc-discuss/3c515d20-fe2a-41a3-b7a3-99fde7fdcb11n%40googlegroups.com.

[Rory Sinclair]

Hi Tom,

Our code currently does not depend on Arrays.equals(), so not a huge concern for us, thanks. Haven't had a chance to try it out yet, but plan to do so soon.

Thanks!

[Rory Sinclair]

Hi Tom,

After a very long period of being busy with various other parts of our platform, i've finally had a chance to find some time to go digging in the guts of our iOS client library once more and took the opportunity to compile the latest j2objc from source and give it a whirl.

I found that the type information for arrays in e.g. method arguments does not come through the transpiler intact - is this expected at this early stage?  For example:

Java:

Request replaceTags(final ImageTag[] tags)

becomes in Obj-C:

- (ASWRequest *)replaceTagsWithASWv2ImageTagArray:(IOSObjectArray *)tags {

rather than:

- (ASWRequest *)replaceTagsWithASWv2ImageTagArray:(IOSObjectArray<ASWv2ImageTag *> *)tags

I haven't had a chance yet to do a standalone test with Swift, as i'll need a fresh project set up - our existing project has a million errors related to non-generically-typed access to IOSObjectArray currently with the new j2objc build :)

I did a quick translation test of a simple java class Monkey.java with array arguments, and again observed that the type information is not passed along, as above, eg:

public Monkey(String[] sounds) 

becomes 

- (instancetype)initWithNSStringArray:(IOSObjectArray *)sounds

rather than

- (instancetype)initWithNSStringArray:(IOSObjectArray<String *> *)sounds

Note the method name is interesting - how does it know that it should be an NSStringArray? Hmm...

While I did have more success with:

void sort(int[] numbers)

becoming:

- (void)sortWithIntArray:(IOSIntArray *)numbers

Where IOSIntArray obviously is generically typed, which is great.

Thanks for any wisdom you can impart...

[Rory Sinclair]

Another potentially interesting thing:

In a method which takes as an argument the following:

array: IOSObjectArray<ASWv2Attachment>? 

I tried to assign it as a Swift typed array:

let attachments2: [ASWv2Attachment] = array!

But the compiler errored with: "'IOSObjectArray<ASWv2Attachment>' is not convertible to '[ASWv2Attachment]'"

This surprised me a little as IOSObjectArray<T> is an IOSArray<T> which is now an NSArray<T> - I think?

Thanks again in advance for any wisdom you can impart... :) 

[Rory Sinclair]

Last update before I hit the hay...

let attachments2: [ASWv2Attachment] = array as! [ASWv2Attachment]

The compiler was happier with this, but i'm not sure if thats just bending the pointers to my will... :)

[Tom Ball]

Are you building with the new --objc-generics flag? If so, I'm sorry you're hitting these errors; the engineer working on it left and apparently it's not finished yet. I'm on vacation for the next two weeks (my wife just retired 🎉), so please file issues (ideally with code snippets I can reuse in unit tests), and I'll work on them when I get back.

FWIW, my plan after generified arrays works is to try translating all Java generics and see what 

To view this discussion on the web visit https://groups.google.com/d/msgid/j2objc-discuss/249bd876-62c2-4797-97b6-a5db87b509a6n%40googlegroups.com.

[Tom Ball]

... doesn't map between the two generics designs. If ObjC lightweight generics only makes sense for collection classes, we might fall back to using a new annotation to mark those classes that can be supported. 

Any feedback from interesting developers is of course welcome and appreciated.

[Rory Sinclair]

Congratulations to you and your wife! Wishing you both a long and happy retirement together when the time comes :)

I assume you're referring to https://github.com/google/j2objc/blob/9fa90434206327afe97b974a595ef645a1b655d3/translator/src/main/java/com/google/devtools/j2objc/Options.java#L537-L538 which is a switch passed to the j2objc translator, right? There isn't some compile-time thing I need to set when building the distribution itself? 

If that is the case then it appears that it does not yet work outside of the primitive array classes.

I'll raise some issues for you if I get a chance, thanks!

[Rory Sinclair]

Hi Tom,

I hope you had an enjoyable break. I've raised an issue about this here: https://github.com/google/j2objc/issues/1947

Please let me know if i'm missing a step or misunderstanding how this should work :)

Regards