Constructor Weirdness?

[John Myles White]

Hi all,

I'm demoing out a revision of the DataFrames package. I am completely stumped by one problem with constructing DataVec's.

Specifically, I now define the DataVec type as follows:

type DataVec{T} <: AbstractDataVec{T}
data::Vector{T}
na::BitVector

# Sanity check that new data values and missingness metadata match
function DataVec{T}(new_data::Vector{T}, is_missing::BitVector)
if length(new_data) != length(is_missing)
error("data and missingness vectors not the same length!")
end
new(new_data, is_missing)
end
end

After using this kind of definition, the following fails:

DataVec([1, 2, 3], bitpack([false, false, false]))

It fails with this error message:

no method DataVec(Array{Int64,1},BitArray{Bool,1})
in method_missing at base.jl:81

I feel like I must deeply misunderstand the type system for this to have eluded me now for a few hours. What's the reason our extended constructor isn't being defined for that type signature?

-- John

[Patrick O'Leary]

Stefan answered this one for me a while back. Quoting:

...inside the type declaration you don't put the {T} on the method declarations. I'm not sure what this does, but the inner {T} might shadow the outer {T}. What you want is this:

julia> type Doh{T}

         a::T

         Doh() = new()

       end

julia> doh = Doh{Int64}();

So inside of the type declaration scope, Doh is like Doh{T} outside because T is already bound as a universal quantifier by the type block.

https://groups.google.com/d/msg/julia-dev/zpqfqWCGLy8/4vIxcAke1poJ

[Stefan Karpinski]

The other part of it is that you need an outer constructor too:

DataVec{T}(data::Vector{T}, na::BitVector) = DataVec{T}(data, na)

This is what gets called when someone does

DataVec([1, 2, 3], bitpack([false, false, false]))

Of course, you also might want to make the outer constructor a little less specific too.

--
 
 

[John Myles White]

Sorry for being dense, but I'm still confused.

Patrick's solution is really useful to know about, but doesn't fix the issue in this case, although I don't understand why it doesn't.

Stefan, I don't understand what that outer constructor does. Isn't just defining a new outer constructor that's indistinguishable from the constructor that's automatically defined when the type itself is defined?

 -- John

--
 
 

[Stefan Karpinski]

I will answer in greater detail later. Gotta run now. Sorry for the confusion! I thought that the manual chapter was very clear on this, but perhaps I was not as clear as I believed I was :-\

On Wed, Dec 5, 2012 at 6:51 PM, John Myles White <johnmyl...@gmail.com> wrote:

This may be of no use to others, but the thing I find absolutely bewildering is that removing our custom inner constructor doesn't fix my problem, but it does mean that the following starts to work:

DataVec([1, 2, 3], [false, false, false])

 -- John

On Dec 5, 2012, at 6:25 PM, Stefan Karpinski <ste...@karpinski.org> wrote:

--
 
 

--
 
 

[Stefan Karpinski]

In short, the outer constructor calls the inner constructor. If there are any inner constructors defined, you don't get a default inner or outer constructor and have to roll your own:

julia> type Foo1{T}

         x::T

       end

julia> Foo1(1)

Foo1{Int64}(1)

julia> type Foo2{T}

         x::T

         Foo2(x) = new(x)

       end

julia> Foo2(1)

no method Foo2(Int64,)

 in method_missing at base.jl:81

Because you have a custom inner constructor, you have to supply an outer constructor too.

--
 
 

[John Myles White]

This may be of no use to others, but the thing I find absolutely bewildering is that removing our custom inner constructor doesn't fix my problem, but it does mean that the following starts to work:

DataVec([1, 2, 3], [false, false, false])

 -- John

On Dec 5, 2012, at 6:25 PM, Stefan Karpinski <ste...@karpinski.org> wrote:

--
 
 

[Westley Hennigh]

Just out of curiosity, is there a way to call the inner constructor by explicitly specifying a type?

What I mean is (and I may be misunderstanding the problem), could I say something like:

Foo2{Int64}(1)

? I tried that syntax but couldn't quite get it to work...

[Westley Hennigh]

Wait, I'm retarded, that syntax does work.

@John: Does that make more sense? I'm definitely not the person capable of explaining this best, but as I see it:

Foo2(1) does not automatically result in a call to Foo2{T}(1), instead you need an "outer constructor" that -sort of- forwards the parameters and specifies the types.

[John Myles White]

Westley,

Thanks for the input. Things are still totally broken on my end. I need to chat with someone in real time at some point about this, since the current Julia build segfaults on one of the examples that worked yesterday. I'll still totally bewildered and worry that my specific use case is not a fine point about about constructors, but some weird edge case.

Sorry for continuing to be confused.

 -- John

--
 
 

[John Myles White]

Just to follow up on this one: my problems were actually caused by the way in which load() breaks under the new module system. My code was creating a secondary BitArray type, which is why none of my various constructors worked.

But I did learn a bunch of useful things about inner constructors along the way. Thanks to all for their help!

 -- John

On Dec 5, 2012, at 8:22 PM, Westley Hennigh <westley...@gmail.com> wrote:

--
 
 

[Andrei de Araújo Formiga]

What's curious is that with a non-parametric type the inner constructor alone is enough:

type Foo1

    x::Int

    Foo1(someval::Int) = new(someval)

end

type Foo2{T}

    x::T

    Foo2(someval::T) = new(someval)

end

julia> load("foo.jl")

julia> f1 = Foo1(3)

Foo1(3)

julia> f2 = Foo2(3)

no method Foo2(Int64,)

Why is that?

2012/12/5 Stefan Karpinski <ste...@karpinski.org>

--
 
 

[Patrick O'Leary]

Foo2 is a family of types, not a single type. Which Foo2 constructor do you want?

f2 = Foo2{Int}(3)

This is part of the reason it's common to have an outer constructor so you can tell the compiler that the type parameter is in fact the type of the first argument to the constructor in this case.

[John Myles White]

>
> Foo2 is a family of types, not a single type. Which Foo2 constructor do you want?
>
> f2 = Foo2{Int}(3)

This statement needs to be in bold somewhere in the manual.

-- John

[Patrick O'Leary]

It does seem to be an FAQ at this point.

[Andrei de Araújo Formiga]

Makes sense, it's just that I'm more used to languages where a polymorphic type variable like that can be inferred from the argument, for example in ML languages. But ML has a static type system and global inference, so it's quite different from Julia. 

I misread what Stefan said and thought it was an issue with constructors and defining a custom inner constructor, so it should be the same with polymorphic and monomorphic types. 

2012/12/20 Patrick O'Leary <patrick...@gmail.com>

[Kevin Squire]

Is there any reason that the type couldn't (or shouldn't) be inferred from the argument to create and call the correct constructor?

Kevin

[Tim Holy]

On Thursday, December 20, 2012 02:54:56 PM Andrei de Araújo Formiga wrote:
> Makes sense, it's just that I'm more used to languages where a polymorphic
> type variable like that can be inferred from the argument, for example in
> ML languages. But ML has a static type system and global inference, so it's
> quite different from Julia.

I didn't design any of this, but I suspect what "can" be done is not the
issue. After all, you can tell it to do that inference using a single line of
code. But if you don't like that behavior (say, you want everything to default
to Float64, no matter what variable type is supplied), you can do that too.
More flexible this way.

--Tim

[Stefan Karpinski]

Exactly. This design lets you have any behavior you want, defaulting to something sane and reasonable, but allowing a line or two to implement all sorts of different behaviors. There's no single rule that works well for everything.

--Tim

--

[Christopher Rinderspacher]

So could you state clearly in the manual that parameterized inner constructors _have_ to be redefined externally for type inference to work properly?

Foo2{T}(x::T) = Foo2{T}(x)

This was not clear to me from the manual and has created quite some confusion.

[Stefan Karpinski]

That's not really the case... ?

[Christopher Rinderspacher]

So this is what I get:

type Foo1
x::Real
Foo1(x::Real) = new(x)
end

Foo1(1.1)
Foo1(1.1)

type Foo2{T<:Real}
x::T
Foo2(n::T) = new(n)
end

Foo2(2)
no method Foo2(Int64,)

Foo2{Int64}(2)
Foo2{Int64}(2)

Foo2{T<:Real}(x::T) = Foo2{T}(x)
Foo2(2)
Foo2{Int64}(2)

So what I'm taking away here is that, if I want to access the internal constructor of a parameterized type and have the type inferred, I need to write an external constructor to access the internal constructor. If this is wrong could you provide a counterexample and why the behavior above is exhibited?

[Stefan Karpinski]

The syntax is a little confusing, but it's the best we've come up with so far. You have this:

type Foo2{T<:Real}

  x::T

  Foo2(n::T) = new(n)

end

First, because you provide your own inner constructor, no inner or outer constructors are provided for you – you must provide your own. Second, the definition `Foo2(n::T) = new(n)` defines a method for each specific instance of Foo2{T}; the umbrella type Foo2 has no methods:

julia> methods(Foo2{Int})

# methods for generic function Foo2

Foo2(n::Int64) at none:3

julia> methods(Foo2)

# methods for generic function Foo2

So when you apply Foo2 as a function, you will, of course, get a no method error – because Foo2 doesn't have any methods. If you want Foo2(2) to do something, then you need to define a method of Foo2 that will get called. Namely, you can do

Foo2{T<:Real}(x::T) = Foo2{T}(x)

After this, Foo2 will have a method:

julia> methods(Foo2)

# methods for generic function Foo2

Foo2{T<:Real}(x::T<:Real) at none:1

You can call that method as Foo2(2), which in turn calls Foo2{Int}(2) to construct a Foo2{Int} object. This has nothing to do with type inference – there is no type inference in Julia as a language, it's all method dispatch. (Type inference in Julia is just an implementation detail, not part of the language.)

[Christopher Rinderspacher]

That clarifies the behavior quite a lot, in particular that the inner constructor is like a field declaration. I didn't guess this solution from the manual though. Thanks!

[John Myles White]

Unfortunately I think the only way to make the manual sufficiently clear on this matter is to make the text a little pedantic and walk through some example where surprises are likely to occur for newcomers.

 -- John

[Stefan Karpinski]

Probably so. I'll take a pass at that soon enough.