scanLeft and Arity

[Russell Carden]

Hello,
I am new to scalding and scala.
Based on a scanLeft example, I wrote some code that looks like

.insert('tempfield,0.0)
.groupBy('field1,'field2)
{group => group.sortBy('field3)
   .scanLeft(('field4,'field5','temp) -> ('Copyfield4, 'Copyfield5,'desiredfield))
    .... More code

}
The code does what I want, but I am wondering if it is possible to do away with the Copies of field 4 and field 5 as well as the temp field. and have something that looks like


.groupBy('field1,'field2)
{group => group.sortBy('field3)
   .scanLeft(('field4,'field5') -> ('desiredfield))
    .... More code

}

Would this be possible?  Any pointers would be much appreciated.  Is  it a matter of providing an implicit converter for the outField, if so how and where can I specify that?

Thank you

[Oscar Boykin]

I *think* that will just work (if you put the correct type annotations in, I believe). Have you tried?

To be honest, we have been so focused on the typed API that pretty much solves all these issues with clear types, that I don't recall the answer to your question.

The original fields API is not going to be removed, but we haven't put any development work into it in years. If you can, I recommend you use `TypedPipe` and friends instead.

--
You received this message because you are subscribed to the Google Groups "Scalding Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalding-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Russell Carden]

Since you said it was possible,  I kept reading over the scalding's scanleft.   I didn't find any examples on the web.  This was a good scalding exercise. 
Point 1 in Moving Window Calculation (via Scalding) helped me realize the accumulator didn't have to be tied to the output.  With that in mind and the assumption of how
the converters and setters should be used,   the rest was just matching up the tuple types.    I even figured out how to do the setter and converter with anonymous classes.
The code is below.  Is this what you had in mind?

.groupBy('field1,'field2)
{group => group.sortBy('field3)

   .scanLeft(('field4,'field5') -> ('desiredfield))(0.0,0.0,0.0)(
    function that takes in tuple with two doubles from tuple and updates accumulator which is a tuple with three doubles.
    )(new TupleSetter[(Double, Double, Double)]
       {
        override def apply(arg: (Double, Double, Double)): Tuple = { val t = new Tuple(); t.addDouble(arg._3); t }
        override def arity: Int = 1
       }
      , new TupleConverter[(Double, Double)]
       {
        override def apply(te: TupleEntry): (Double, Double) =
         {
          val r =te.getTuple()
          (r.getDouble(0),r.getDouble(1))
         }
        override def arity: Int = 1
       }
     )