How to "parameterize" poly function?

[Mathias]

Miles,

a quick question:

How can I "parameterize" a poly function?

E.g.

scala> import shapeless._

import shapeless._

scala> object double extends Poly1 {

     |   implicit def caseInt = at[Int](x => x + x)

     | }

defined module double

scala> class add(other: Int) extends Poly1 {

     |   implicit def caseInt = at[Int](x => x + other)

     | }

defined class add

scala> (1 :: HNil).map(double)

res0: shapeless.::[Int,shapeless.HNil] = 2 :: HNil

scala> (1 :: HNil).map(new add(42))

<console>:12: error: could not find implicit value for parameter mapper: shapeless.Mapper[add,shapeless.::[Int,shapeless.HNil]]

              (1 :: HNil).map(new add(42))

                             ^

Is there a way to get a "parameter" into a poly function or are they inherently restricted to their own context?

Thanks and cheers,

Mathias

[Miles Sabin]

You need to create a stable identifier for your parametrized Poly1,

scala> object add42 extends add(42)
defined module add42

scala> (1 :: HNil).map(add42)
res0: shapeless.::[Int,shapeless.HNil] = 43 :: HNil

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
skype: milessabin
gtalk: mi...@milessabin.com
g+: http://www.milessabin.com
http://twitter.com/milessabin

[Mathias]

Miles,

that's a bummer, because apparently it's not possible to parameterize a Poly function _dynamically_.

Currently I'm using a DynamicVariable to get a value thats only known at runtime into my Poly function, but this seems like somewhat of a hack to me.

Cheers,

Mathias

On Sunday, July 22, 2012 1:32:25 PM UTC+2, Miles Sabin wrote:

gtalk: miles@....com

[Miles Sabin]

On Mon, Jul 30, 2012 at 8:51 AM, Mathias <mat...@decodified.com> wrote:
> that's a bummer, because apparently it's not possible to parameterize a Poly
> function _dynamically_.

Can you give me a bit more context? What are you trying to do that
isn't covered by the technique that I mentioned in my last mail?

> Currently I'm using a DynamicVariable to get a value thats only known at
> runtime into my Poly function, but this seems like somewhat of a hack to me.

Yes, that really doesn't sound very nice ;-)

Cheers,


Miles

--
Miles Sabin
tel: +44 7813 944 528
skype: milessabin

gtalk: mi...@milessabin.com

[Mathias]

My use case looks similar to this one:

case class Context[L <: List](list: L, env: Env)

trait FixList {

 def fixList[L <: HList](ctx: Context[L])(implicit m: Mapper[?, L]): m.Out = {

  val f = fix(ctx.env)

list.map(f)

 }

}

class fix(env: Env) extends Poly1 {

 ... // transforms some HList elements in the context of the env

}

Basically the problem is that I need to parameterize the poly function _inside_ of another function.

But I need a stable identifier _outside_ of the 'fixList' function, in order to be able to pull in the right Mapper.

The only way I found to solve this problem is via a DynamicVariable.

Cheers,

Mathias

[Miles Sabin]

On Mon, Jul 30, 2012 at 11:12 AM, Mathias <mat...@decodified.com> wrote:
> My use case looks similar to this one:
>
> case class Context[L <: List](list: L, env: Env)
>
> trait FixList {
> def fixList[L <: HList](ctx: Context[L])(implicit m: Mapper[?, L]): m.Out =
> {
> val f = fix(ctx.env)
> list.map(f)
> }
> }
>
> class fix(env: Env) extends Poly1 {
> ... // transforms some HList elements in the context of the env
> }
>
> Basically the problem is that I need to parameterize the poly function
> _inside_ of another function.
> But I need a stable identifier _outside_ of the 'fixList' function, in order
> to be able to pull in the right Mapper.
> The only way I found to solve this problem is via a DynamicVariable.

Have you tried using fold (with a Poly2) instead of map?

[Mathias]

I did consider using a fold but initially shied away from it because my transformation was semantically a map and I didn't want to further increase the already quite high code complexity. However, after a larger refactoring I can now use a nice, clean and simple fold and the problem has disappeared.

Thanks for your support, Miles, and for making shapeless available!

It's a pleasure to work with.

Cheers,

Mathias

[Miles Sabin]

On Mon, Jul 30, 2012 at 9:29 PM, Mathias <mat...@decodified.com> wrote:
> I did consider using a fold but initially shied away from it because my
> transformation was semantically a map and I didn't want to further increase
> the already quite high code complexity. However, after a larger refactoring
> I can now use a nice, clean and simple fold and the problem has disappeared.

Excellent stuff! Do you have a pointer to the use of an HList fold in Spray?

> Thanks for your support, Miles, and for making shapeless available!
> It's a pleasure to work with.

:-)

[Mathias]

This is the fold I was talking about before:

https://github.com/spray/spray/blob/feature/1.0-M3/spray-routing/src/main/scala/cc/spray/routing/directives/ParameterDirectives.scala#L138

This file shows one of the more complex examples I have of something I recently started calling "the Magnet Pattern".

The idea is that you remove method overloading and implicit parameters on a method (which is often a DSL element) by having the method accept only a single parameter (the "magnet"), which only serves as a target for a (potentially complex) structure of implicit conversions, which provide for the actual variety of arguments that can be passed to the original DSL method.

Even multiple arguments can be supported by supplying implicit conversions from tuples to the magnet.

This technique plays really nicely with shapeless, since you only need to support HLists as an argument, similar to this example here:

    val list = 1 :: "2" :: 3.0 :: HNil

    fixture.myDSLmethod(list)

and then, by adding support for tuples with minimal boilerplate (as shown in the example file above), you can then also do:

    fixture.myDSLmethod(1, "2", 3.0)

I'll soon write a blog post about the Magnet Pattern and it playing together with shapeless.

Thanks and cheers,

Mathias

[Miles Sabin]

On Tue, Jul 31, 2012 at 9:32 AM, Mathias <mat...@decodified.com> wrote:
> This is the fold I was talking about before:
> https://github.com/spray/spray/blob/feature/1.0-M3/spray-routing/src/main/scala/cc/spray/routing/directives/ParameterDirectives.scala#L138

Awesome! :-)

> This file shows one of the more complex examples I have of something I
> recently started calling "the Magnet Pattern".
> The idea is that you remove method overloading and implicit parameters on a
> method (which is often a DSL element) by having the method accept only a
> single parameter (the "magnet"), which only serves as a target for a
> (potentially complex) structure of implicit conversions, which provide for
> the actual variety of arguments that can be passed to the original DSL
> method.
> Even multiple arguments can be supported by supplying implicit conversions
> from tuples to the magnet.
>
> This technique plays really nicely with shapeless, since you only need to
> support HLists as an argument, similar to this example here:
>
> val list = 1 :: "2" :: 3.0 :: HNil
> fixture.myDSLmethod(list)
>
> and then, by adding support for tuples with minimal boilerplate (as shown in
> the example file above), you can then also do:
>
> fixture.myDSLmethod(1, "2", 3.0)
>
> I'll soon write a blog post about the Magnet Pattern and it playing together
> with shapeless.

Yes, that technique is used throughout shapeless (see, for example,
the methods on HListOps or Zipper and their corresponding type classes
and instances), and it's nice to give it a name and some context to
encourage it's use more widely. I think "magnet" is a pretty good name
for it ... I've also thought in terms of "anchor", "focus" etc.

Given that this idiom is cranking so heavily on type classes, it
probably means that the "pattern" is exploited in Haskell as well ...
can anyone point to any examples?

[Mathias]

Yes, that technique is used throughout shapeless (see, for example, 
the methods on HListOps or Zipper and their corresponding type classes 
and instances), and it's nice to give it a name and some context to 
encourage it's use more widely. I think "magnet" is a pretty good name 
for it ...

Glad, that you like it!

Btw: By no means I'd like to imply that I'm the first one to use this technique.

It's great to see that it's valid approach that has already proven itself elsewhere (e.g. in shapeless).

I've also thought in terms of "anchor", "focus" etc.

Yes, "anchor" and "focus" where also on my list, just like "receptacle", "sink" and "eye".

However, I like "magnet" most. 

Given that this idiom is cranking so heavily on type classes, it 
probably means that the "pattern" is exploited in Haskell as well ... 
can anyone point to any examples?

Unfortunately I am a complete Haskell Noob and therefore not much help in this regard...

Cheers,

Mathias

On Tuesday, July 31, 2012 11:03:06 AM UTC+2, Miles Sabin wrote:

...

[Eli Bishop]

I'm just getting started with shapeless and am pretty new to typeclasses in general, and I'm having trouble with this concept.  It would be really helpful to me if you could explain a bit more of what you're doing there with ParamDefMagnetAux.  Not knowing what your code looked like before when you were trying to use a more familiar map-based approach, it's hard for me to see how you've translated it and apply a similar translation to my own use case.

Basically my problem sounds like the same one you originally raised:  I need to map over an HList with a dynamically created function that takes some parameter from outside.  This was pretty easy with the scalaz KList, so I was surprised and saddened to find that shapeless can't do it— in every other regard so far, the shapeless HList has been better for my purposes.  (And I would've thought that this kind of thing would be a very common use case; I can't imagine Scala programming without the ability to use curried functions or closures.)  I've tried to understand how fold could possibly take the place of map for this kind of thing, but I'm drawing a blank.

Eli

[Brian McKenna]

Hi Eli,

Good to see you on the Shapeless mailing list :)

I added a `map` function to NatTRel a couple of weeks ago. It allows
you to apply a natural transformation over a HList with a common type
constructor:

https://github.com/milessabin/shapeless/blob/master/core/src/main/scala/shapeless/hlist.scala#L543

Its use is something like so:

import shapeless._
object setToList extends (Set ~> List) {
def apply[A](a: Set[A]): List[A] = a.toList
}

def mapSetToList[In <: HList, Out <: HList](xs: In)(implicit
nattrel: NatTRel[In, Set, Out, List]): Out =
nattrel.map(setToList, xs)

val x = Set(1, 2, 3) :: Set("Hello", "World") :: Set(true) :: HNil
mapSetToList(x)

Now, since you're saying that it takes an outside parameter, I'm not
sure if that's going to make sense with a natural transformation... Is
this similar to what you're trying to do with a KList?

Thanks,
Brian

> --
> You received this message because you are subscribed to the Google Groups
> "Shapeless Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to shapeless-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Eli Bishop]

hi Brian,

Yeah, I think unfortunately what I need to do isn't possible with your transformation, because your function there is still a singleton object that can't have any new data injected into it. Basically what I have is a KList of things that are in effect a function of A => X, where A is the same for every element in the list (but not the same every time this code is used) and X varies for each element; and the transformation I want to apply is to pass some value of type A into each of them and build a list of the resulting X's. So, in order for the transformation to have whatever the appropriate A value is each time, I think it can't be declared as object.

[Miles Sabin]

Hi Eli,

> Basically what I have is a KList of things that are in effect a function of A => X, where A is the same for every element in the list (but not the same every time this code is used) and X varies for each element; and the transformation I want to apply is to pass some value of type A into each of them and build a list of the resulting X's. So, in order for the transformation to have whatever the appropriate A value is each time, I think it can't be declared as object.

100% confident that anything that can be done with a KList can be done
with a shapeless HList.

I'm not 100% confident that I know exactly what you're aiming for
though ... could you show us an example of what you'd like to be able
to write, or at least an example of the input, the function to be
applied and the expected output?

[Travis Brown]

Hi Eli,

Here's a quick example of a couple of approaches that may work for the kind of thing you're describing:

https://gist.github.com/travisbrown/5124684

Both let you do something like the following:

    val fs = ((i: Int) => "a" * i) :: ((i: Int) => i + 42) :: HNil

    val x = 3

    applyAllTo(x)(fs) // "aaa" :: 45 :: HNil

The first uses a new type class designed particularly for this purpose, and the second uses the existing ZipApply type class (but also needs some extra machinery in the form of an HList.fill method).

Travis

[Travis Brown]

Actually I just realized that of course there's a much simpler way to do this with the type classes in Shapeless now:

    fs zipApply (fs mapConst x)

No need for HList.fill.

Travis

[Gabriele Petronella]

Sorry for being late at the party, but I ran into the same issue (dynamically parametrize a Poly1 for a mapping purposes) and I stumbled upon this thread.
Mathias' solution of using the so-called Magnet Pattern looks incredibly elegant, but also way beyond my humble comprehension.
On the other hand, Miles' suggestion of using a fold and a Poly2 was enlightening for me, so I'd like to share my solution, hoping it will be beneficiary to future readers

Here's the relevant Q/A on StackOverflow: http://stackoverflow.com/questions/25288806/dynamically-parametrize-poly1-function-in-shapeless/25316124#25316124

The general overview is to pass the external parameter along with the accumulator (an `HNil`) and so have it available within the fold function.
Then we simply map over the element (using the parameter as input) and append the value to the accumulator.
The final result is a tuple, with the mapped hlist as second element, so we take it and we're done.

While it's semantically not excellent (a map would be definitely more natural) it doesn't even look to terribly complex.

I hope this will save some time to the next guy landing here!

--
Gabriele