code design question – best ideomatic way to define nested types?

[Michael Borregaard]

Hi,

I am defining a set of types to hold scientific data, and trying to get the best out of Julia's type system. The types in my example are 'nested' in the sense that each type will hold progressively more information and thus allow the user to do progressively more. Like this:

type Foo
  bar
  baz
end

type Foobar
  bar  # this
  baz  # and this are identical with Foo
  barbaz
  bazbaz
end

Thus, you can do anything with a Foobar object that you can with a Foo object, but not the other way around. The real example is much more complex, of course, with levels of nestedness and more fields of complex types.
There are several ways I could design this:

1. I could make all objects be of type Foobar, but make the barbaz and bazbaz fields Nullable. I don't think that is ideal, as I would like to use dispatch to do things with object Foobar that I cannot do with Foo, instead of constantly checking for isnull on specific fields.
2. I could keep the above design, then define an abstract type AbstractFoo and make both Foo and Foobar inherit from this. Then AbstractFoo can be used to define functions for everything that can be done with the fields that are in Foo objects. The downside is that the types become really big and clunky, and especially that my constructors become big and tricky to write.
   
3. I could use composition to let Foobar contain a Foo object. But then I will have to manually dispatch every method defined for Foo to the Foo field of Foobar objects.To make it clear what I mean, here are the three designs:

# Example 1:

type Foobar{T<:Any}
  bar
  baz
  barbaz::Nullable{T}
  bazbaz::Nullable{T}
end


# Example 2:

abstract AbstractFoo

type Foo <: AbstractFoo
  bar
  baz
end

type Foobar <: AbstractFoo
  bar
  baz
  barbaz
  bazbaz
end

# Example 3:

type Foo
  bar
  baz
end

type Foobar
  foo::Foo
  barbaz
  bazbaz
end


I do realize that the easy answer to this is "this depends on your use case, there are pros and cons for each method". However, I believe there must be a general ideomatic solution, as this issue arises from the design of the type system: because you cannot inherit from concrete types in Julia, and abstract types (which you can inherit from) cannot have fields. In C++, where you can inherit from concrete types, this would have an ideomatic solution as:

class Foo{
  protected:
    int bar, baz;
};

class Foobar: public Foo{
  int barbaz, bazbaz;
};

I have been struggling for days with different redesigns of my code and I really cannot wrap my head around it. I appreciate the help!

[Stefan Karpinski]

I would probably go with approach #2 myself and only refer to the .bar and .baz fields in all of the generic AbstractFoo methods.

[Michael Borregaard]

Thanks for the prompt response, I will go with that then :-)

I actually thought later that I may avoid some of the clutter in approach 2 by adding another level of indirection:

abstract AbstractFoo

type FooData
  bar
  baz
  #... several other fields
end

type FoobarData
  barbaz
  bazbaz
  #... several other fields

type Foo <: AbstractFoo
  foodata::FooData

  Foo(bar, baz) = new(FooData(bar, baz))
end

type Foobar <: AbstractFoo
  foodata::FooData
  foobardata::FoobarData

  Foobar(bar, baz, barbaz, bazbaz) = new(FooData(bar, baz), FooBarData(barbaz, bazbaz))
end

However I cannot say whether this is generally useful outside my use case.

[Bart Janssens]

Looking at this example, it seems mighty tempting to have the ability to subtype a concrete type. Are the exact problems with that documented somewhere? I am aware of the following section in the docs:

"One particularly distinctive feature of Julia’s type system is that concrete types may not subtype each other: all concrete types are final and may only have abstract types as their supertypes. While this might at first seem unduly restrictive, it has many beneficial consequences with surprisingly few drawbacks. It turns out that being able to inherit behavior is much more important than being able to inherit structure, and inheriting both causes significant difficulties in traditional object-oriented languages."

I'm just wondering what the "significant difficulties" are, not advocating changing this behaviour.

[Chris Rackauckas]

https://en.wikipedia.org/wiki/Composition_over_inheritance

http://programmers.stackexchange.com/questions/134097/why-should-i-prefer-composition-over-inheritance

https://www.thoughtworks.com/insights/blog/composition-vs-inheritance-how-choose

That's just the start. Overtime, people realized inheritance can be quite fragile, so many style guidelines simply forbid you from doing it.

[Stefan Karpinski]

The biggest practical issue is that if you can subtype a concrete type then you can't store values inline in an array, even if the values are immutable – since a subtype can be bigger than the supertype. This leads to having things like "final" classes, etc. Fundamentally, this is really an issue of failing to separate the concrete type – which is complete and can be instantiated – from the abstract type, which is incomplete and can be subtyped.

[Chris Rackauckas]

Ahh, that makes a lot of sense as well. I can see how that would make everything a lot harder to optimize. Thanks for the explanation!

[Tom Breloff]

I think #2 is the right solution, but I also wish there was a nicer syntax to do it.  Here's how I'd probably tackle it... if I get around to it soon I'll post the implementation:

@abstract type AbstractFoo

    bar::Int

end

@extend AbstractFoo type Foo

end

@extend AbstractFoo type Foobar

    baz::Float64

end

# then:

Foo <: AbstractFoo

Foobar <: AbstractFoo

and both Foo and Foobar have a field bar.

I suspect this will be dirt-simple to implement for non-parametrics, but might be a little tricky otherwise.  It's just a matter of injecting fields (and parameters) into the proper spot of the type definition.

[Chris Stook]

I use a macro to avoid retyping common fields.  

abstract AbstractFoo

macro commonfields()

  return :(

type Foo <: AbstractFoo
  bar
  baz
end

type Foobar <: AbstractFoo

[Chris Stook]

Last post was incomplete.

abstract AbstractFoo

macro commonfields()

  return :(

    bar

    foo

  )

end

type Foo <: AbstractFoo
  @commonfields()
end

type Foobar <: AbstractFoo
  @commonfields()
  barbaz
  bazbaz
end




Chris

[Chris Stook]

[Michael Borregaard]

Thanks for the enlightening discussion. The emerging consensus is to use example #2, but perhaps use macros to make the syntax easier to read and maintain. Alternatively, it looks like my idea with having a FoobarData object as a field would do the job (but would require foobar.foobardata.bazbaz syntax for accessing fields, of course).

It is also interesting to see that there are divergent views. It seems to me, for example, that Tom Breloff's macro syntax would subvert the inheritance design decision that Stefan Karpinski described, by combining the abstract type with the concrete type?

[Tom Breloff]

To be clear...I'm not trying to subvert anything! Just to make it easier and more natural to choose the best option. :)  if it wasn't clear from my example, AbstractFoo would NOT be a concrete type, and couldn't be constructed. 

[Michael Krabbe Borregaard]

First, apologies, I didn't mean that you wanted to 'subvert' (http://www.merriam-webster.com/dictionary/subvert) julia. As a non-native speaker the finer nuances of English sometimes slip. The word I was looking for was perhaps 'sidestep'.

Also I see your point that as long as the Abstract type cannot be instantiated, the problem with putting supertypes in an array should not be relevant. So perhaps this is actually a nice general solution to the issue.

[Tom Breloff]

I stole an hour this morning and implemented this: https://github.com/tbreloff/ConcreteAbstractions.jl

It's pretty faithful to my earlier description, except that the macros names are `@base` and `@extend`.  Comments/criticisms welcome!

[Michael Krabbe Borregaard]

"An hour", pfhh, I only wish I was that efficient. I really like it, also looking very much forward to reading the comments people will make on this.

[Bart Janssens]

Thanks for the replies, very informative. I like the FooData solution best, it separates "code reuse through composition" from "shared interface through inheritance", which is what is recommended in the thoughtworks link Chris posted.

[Greg Plowman]

Bart,

Which one is the FooData solution?

Is this Example 1,2 or 3? Or another solution.

[Greg Plowman]

Another variation on Chris's @commonfields and Tom's @base & @extend:

This is somewhere between example 2 and 3,

Avoids copy and paste of example 2

Avoids delegating to Foo of example 3 

abstract AbstractFoo

type Foo <: AbstractFoo
    bar
    baz
end

type Foobar <: AbstractFoo
    @splatfields Foo
    barbaz
    bazbaz
end

Also allows for "multiple" composition/inheritance.

 

[Tom Breloff]

I like splatfields but I think it's hard to make work with type parameters (which is normally a deal breaker for me)

[Jeffrey Sarnoff]

Tom, the coarseness encountered when trying splatfields with type parameters: is it endemic and likely to persist or does this grow better in v0.6+?

[Bart Janssens]

It's the one posted by Michael later. In the terminology of https://www.thoughtworks.com/insights/blog/composition-vs-inheritance-how-choose FooData and FoobarData would be implementation types, that could also have methods operate on them as part of the "implementation interface", while the AbstractFoo hierarchy is part of the problem domain and cleanly separated here, but using the implementation types by composition: