Composite Type Array

[maxe...@gmail.com]

Hi

I was working on processing large data sets & historically I've used structs in C++ & other languages for this type of task. I attempted to use a Composite Type in Julia & preallocate a large array before filling it w/values as my algo processes the data.

My example was:

type ExampleEvent

        fld1::ASCIIString
        fld2::Int16
        fld3::Int64
        fld4::Int64
        fld5::Int64
        fld6::Int64
        fld7::Int64

end

I googled around & from what I found, & all the docs examples I tried out, there isn't an obvious way to declare an array of composite type without having to do some work arounds.

I liked the language in several other respects but it seems to be missing helpful tools to make the programmer's life easy. Am I missing something? If not, why is a data structure like this not easily available?

thanks in advance

best,
A

[Stefan Karpinski]

It's a little unclear what you want to do that you can't figure out how to accomplish. You can allocate an uninitialized vector of ExampleEvent objects:

julia> type ExampleEvent

               fld1::ASCIIString

               fld2::Int16

               fld3::Int64

               fld4::Int64

               fld5::Int64

               fld6::Int64

               fld7::Int64

       end

julia> events = Vector{ExampleEvent}(1000)

1000-element Array{ExampleEvent,1}:

 #undef

 #undef

 #undef

   ⋮

 #undef

 #undef

 #undef

[maxe...@gmail.com]

Thanks Stefan!

I thought it would be like declaring a struct array in c/c++ i.e. something like ExampleEvent events[2][1000]; Then set each field in the events array as I encounter the required value in my algo: e.g. events[1][1].fld1 = "ABC";  events[1][1].fld2 = 123;   etc

How do I access elements in the events Vector since ndims(events) = 1? I don't see anything in the docs about indexing into a preallocated Vector data structure. I see methods like push! splice! etc but not anything that will let me use the elements of my preallocation on the fly.

[Cameron McBride]

Your ExampleEvent is completely analagous to a C-like struct. With Stefan's bit, you now have a single dimension array of ExampleEvent type (hence ndims(events) is 1). 

You can do this:

julia> events[1] = ExampleEvent("asdf",2,3,4,5,6,7)

ExampleEvent("asdf",2,3,4,5,6,7)

And if you have more, you can do this: 

julia> push!(events, ExampleEvent("ghjk", 2,3,4,5,6,7))

1001-element Array{ExampleEvent,1}:

    ExampleEvent("asdf",2,3,4,5,6,7)

 #undef

   ⋮

 #undef

    ExampleEvent("ghjk",2,3,4,5,6,7)

Cameron

[maxe...@gmail.com]

Thanks Cameron.

What I wanted to do was assign values to fields but the example you posted requires me to pass values to all fields in my type.

If I do: events[1] = ExampleEvent("asdf",123), julia throws an error since I need to pass values to ALL my type's fields. I don't always have to use all fields & sometimes do not have data for all fields, thus I don't want to be obligated to pass even null values to them since it is verbose/excessive. Nor do I want to rewrite constructors for all fields since I already have named fields for sake of programming ease.

See what I mean?

Moreover, let's say I've filled 10 elements of my 1000 element array & a few milliseconds later I get a value for element 5, fld3. How do I index into element 5 of the events Vector & ONLY set fld3?

Yes I've seen push! & looks good but I'm pretty religious about precallocating in any lang even if it is much larger than needed since processing these datasets require as much efficiency as possible.

I think more intuitive Composite Type Arrays for morons like me would make Juila much easier to work with since I've seen a fair amount of uncertainty/confusion from people having similar conceptual use cases. I know I'm thinking from a perspective of other language(s) but some language constructs are there because they work & make a programmer's life easier. Is there an easy way to set field value(s) whenever I get them?

[Jared Crean]

I think what you are looking for is a constructor for your new type that uses incomplete initialization

type ExampleEvent

        fld1::ASCIIString
        fld2::Int16
        fld3::Int64
        fld4::Int64
        fld5::Int64
        fld6::Int64
        fld7::Int64

        function ExampleEvent(fld1, fld2)
          obj = new()  # create new object, without assigning values to fields
          obj.fld1 = fld1
          obj.fld2 = fld2
       
          return obj
        end
end

  Now you can use this constructor when assigning elements to the array

   events = Vector{ExampleEvent}(1000)

   for i=1:10
      events[i] = ExampleEvent("abc", i)
   end
   events[5].fld3 = 7  # assign to previously undefined field

   Note that you will get an error if you try to read a value from a field that hasn't been set yet:

   event = ExampleEvent("abc", 2)
   y = event.fld3  # error, accessing undefined reference

  Best,
    Jared Crean

[Cameron McBride]

A, 

There are a number of ways to do this, and the examples are just based on what you suggested as an array of structs.

The array of uninitialized structs would be roughly equivalent to the C version of allocating an array of pointers to structs of ExampleEvent (without allocating the struct itself). The struct is uninitialized -- so you can't set a value of a named field until it's allocated.

As far as I see it, this is just a syntax issue, as the concept is the same in Julia and C.

One simple solution to continue on this example would be to define a function to give you the event you care about (by index) that includes initialization. You can then process events for fields as you see fit and out of order. 

julia> function get_event(v::Vector{ExampleEvent}, i)

         n = length(v)

         if i > n

           for j in n:i

               push!(v, ExampleEvent("",0,0,0,0,0,0)) # default values

           end

         end

         v[i]

       end

Then you can initialize a zero length vector and use it as such:

julia> events =  Vector{ExampleEvent}()

0-element Array{ExampleEvent,1}

julia> e = get_event(events, 3); e.fld1 = "asdf"; e.fld3 = 3; e.fld7 = 7; events

4-element Array{ExampleEvent,1}:

 ExampleEvent("",0,0,0,0,0,0)

 ExampleEvent("",0,0,0,0,0,0)

 ExampleEvent("asdf",0,3,0,0,0,7)

 ExampleEvent("",0,0,0,0,0,0)

Perhaps that helps.

Cameron

[Cameron McBride]

On Fri, Jul 22, 2016 at 3:09 PM, Cameron McBride <cameron...@gmail.com> wrote:

julia> function get_event(v::Vector{ExampleEvent}, i)

         n = length(v)

         if i > n

           for j in n:i

               push!(v, ExampleEvent("",0,0,0,0,0,0)) # default values

           end

         end

         v[i]

       end

haha, that creates one extra element (ExampleEvent row) on the first pass. Whoops. Here is a quick fix. Likely better ways to do all this, just kicking it further.

julia> function get_event(v::Vector{ExampleEvent}, i)

         n = length(v)

         if i > n

           i1 = n > 0 ? n : 1

           for x in i1:i

               push!(v, ExampleEvent("",0,0,0,0,0,0)) # default values

           end

         end

         v[i]

       end

Cameron

[maxe...@gmail.com]

Thanks alot Cameron & Jared.

This makes much more sense now & the composite type seems more usable. I had erroneous & conflicting expectations of how I would interact with the events data structure after preallocating it.

I'm going to try it out & see how it fits into a larger scale project.