patternmatching and string interpolation

[Adriaan Moors]

Not sure if this has been documented before. I think it's pretty cool.

Loving string interpolation (and the pattern matcher) more every day :-)

$ scala 

Welcome to Scala 2.10.1 yadayada

implicit class RContext(sc: StringContext) {

  def rx = 

    new util.matching.Regex(

         sc.parts.mkString(""), 

         sc.parts.tail.map(_ => "x"): _*)

}

object I { def unapply(x: String): Option[Int] = scala.util.Try { x.toInt } toOption }

scala> "one2" match { case rx"""one(\d*)${I(d)}""" => println(d + 1) }

3

Came up while working on my talk for tomorrow at www.meetup.com/SF-Scala/events/111419142/.  Real use case: https://github.com/adriaanm/replhtml/blob/master/src/main/scala/ReplMain.scala#L39

--

See you at Scala Days (June 10–12, NYC)!

[Eugene Burmako]

Doesn't look like this is documented: http://docs.scala-lang.org/overviews/core/string-interpolation.html.

We have previously used pattern matching with quasiquoting interpolators. There's currently a problem with UnApply nodes, which prevents the full power of macros from being used in pattern matching, though the patch we applied to fix that is tiny. Denys (in /cc) knows the details.

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jan Niehusmann]

Hi Adriaan,

On 04/17/13 08:00, Adriaan Moors wrote:
> Real use case:
> https://github.com/adriaanm/replhtml/blob/master/src/main/scala/ReplMain.scala#L39

Are you serious or am I missing the joke? The change doesn't look like
an improvement to me. The original code was quite readable, the changed
code looks more like some perl accident...

Regards,
Jan

[Adriaan Moors]

Hi Jan,

I was being serious. Isn't it expected that regexes look like perl accidents?

The interesting bit I thought was defining the regex extractor inline, complete with nested subpatterns. A testament to the tight integration of string interpolation into the language. An example of Scala the Unifier. 

cheers

adriaan

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-internals+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Eugene Burmako]

Btw why rx and not jusr r?

To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.

[Adriaan Moors]

Hi Eugene,

On Tue, Apr 16, 2013 at 11:04 PM, Eugene Burmako <xen...@gmail.com> wrote:

Doesn't look like this is documented: http://docs.scala-lang.org/overviews/core/string-interpolation.html.

I think it's covered by the SIP: http://docs.scala-lang.org/sips/pending/string-interpolation.html (I noticed there's a typo: it's missing the dollar signs before the curlies.).

I meant the concrete example of regexps with subgroups bound to subpatterns. 

The example could be improved on further using macros similar to the printf interpolator.

We have previously used pattern matching with quasiquoting interpolators.

Ah yes, a fine use case!

 

There's currently a problem with UnApply nodes, which prevents the full power of macros from being used in pattern matching, though the patch we applied to fix that is tiny. Denys (in /cc) knows the details.

Right, I guess you're talking about parametric extractors. I'd be surprised if that can be fixed by a small patch. I've looked into it in the past and it required quite a bit of refactoring in the type checker. Some of that has been done already, but you're right that we're not there yet. It would also be a language change. One I'd welcome, though.

cheers

adriaan

[Adriaan Moors]

ah, good point -- I thought we already had the r interpolator

it's getting late here

[Jan Niehusmann]

Hi Adriaan,

On 04/17/13 09:28, Adriaan Moors wrote:
> I was being serious. Isn't it expected that regexes look like perl
> accidents?

Well, probably :-)

But still, I like

val Complete = """complete@(\d*)""".r

more than

rx"""complete@(\d*)${I(pos)}"""


Of course, that may be just because the former is more familiar.

> The interesting bit I thought was defining the regex extractor inline,
> complete with nested subpatterns. A testament to the tight integration of
> string interpolation into the language. An example of Scala the Unifier.

Yes, I fully agree with that. However, I think it's a feature that has
the potential of obfuscating real code. And, IMHO, that happens in the
example you cited as 'Real use case'.

Bye,
Jan

[Martin Grigorov]

Hi,

On Wed, Apr 17, 2013 at 11:38 AM, Jan Niehusmann <j...@gondor.com> wrote:

Hi Adriaan,

On 04/17/13 09:28, Adriaan Moors wrote:

I was being serious. Isn't it expected that regexes look like perl
accidents?

Well, probably :-)

But still, I like

val Complete = """complete@(\d*)""".r

more than

rx"""complete@(\d*)${I(pos)}"""

It is also confusing because dollar sign is a special in regular expressions. 

When I looked at the example I asked myself: Is $ the end of the regex or it is part of the string interpolation ? 

The same for "(pos)" - is it a regex group or a parameter to I.unapply ?

 

Of course, that may be just because the former is more familiar.

The interesting bit I thought was defining the regex extractor inline,
complete with nested subpatterns. A testament to the tight integration of
string interpolation into the language. An example of Scala the Unifier.

Yes, I fully agree with that. However, I think it's a feature that has the potential of obfuscating real code. And, IMHO, that happens in the example you cited as 'Real use case'.

Bye,
Jan

[Adriaan Moors]

Yes, you're right. This is a nice example of how Scala's power should be used judiciously.

However, I think that's always more about defining guidelines of what's appropriate for your project/team, rather than restricting the whole language. 

At its core, string interpolation is simple. The translation is simply: drop $-prefixed holes from the string, pack the remaining parts in a call to StringContext, and tack on a call to the interpolator named in the prefix, with the holes as arguments. Since this is unified with the other concepts in the language, it becomes quite powerful. You can use macros to make a typesafe printf, use them as patterns, implement a quasi quoting library, replace xml matching (perhaps?).

The power is there for you, just like electricity is. We all know not to put our fingers in the socket. That challenge is more subtle in Scala and we (as a community) are still learning how to avoid a wakeup call once in a while, I suppose :-)

I'd like to defend my example a little bit more, even though I agree the syntax looks funny. That wasn't the main point. It's not about regular expression matching. It's about the tight integration of string interpolation and pattern matching.

Eugene gave a much better example: quasi quoting. I'm sure macro author / reflection user will agree it's much nicer to write the tree you're looking for directly rather than putting every single one in a variable first.

So, the context for my example: we're hacking on a 50-line script to expose the repl to the browser via websockets. It should also fit on a slide or two :-)

We should be comparing

val Complete = """complete@(\d*)""".r

... match {
  case Array(Complete(I(pos)), source) =>

}

to

... match {
  case Array(r"""complete@(\d*)${I(pos)}""", source) =>
}

I'm assuming the r interpolator is in a library, or at least is amortized over repeated patterns that check for different commands. If you're used to string interpolation, the ${} should be familiar as the "hole in the string", and since we're in a pattern, the hole is described by another pattern, I(pos).

All this means is that string interpolation is truly first class. You can write it directly in a pattern, or encapsulate it in a variable first. Up to you.

I hope we all agree the comparison to perl (there's oo ways to do it) ends at the regex syntax.

Jan

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-internals+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Tony Sloane]

On Thursday, April 18, 2013 2:57:19 AM UTC+10, Adriaan Moors wrote:

Yes, you're right. This is a nice example of how Scala's power should be used judiciously.

However, I think that's always more about defining guidelines of what's appropriate for your project/team, rather than restricting the whole language. 

At its core, string interpolation is simple. The translation is simply: drop $-prefixed holes from the string, pack the remaining parts in a call to StringContext, and tack on a call to the interpolator named in the prefix, with the holes as arguments. Since this is unified with the other concepts in the language, it becomes quite powerful. You can use macros to make a typesafe printf, use them as patterns, implement a quasi quoting library, replace xml matching (perhaps?).

We've been experimenting along these lines and it's all very promising. One write-up of a simple example can be found here:

    http://hootenannylas.blogspot.com.au/2013/02/pattern-matching-with-string.html 

As the end of that post says, we're interested in more complex pattern matching where the syntax is that of a programming language for example (sometimes called "concrete object syntax" in the literature). We have some examples but I haven't had time to write them up in the blog yet. Some example code is in these talk slides:

   http://wiki.kiama.googlecode.com/hg/talks/scalasyd13.pdf

cheers,

Tony

[Johannes Rudolph]

On Wed, Apr 17, 2013 at 11:35 PM, Tony Sloane <inky...@gmail.com> wrote:
> We've been experimenting along these lines and it's all very promising. One
> write-up of a simple example can be found here:

This looks much more idiomatic IMO because positioning, format info
and extraction are neatly separated and abstracted. Now put that into
a macro so that the regexp parts are fused at compile time and Regexp
starts to become usable.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

[Tony Sloane]

On 18/04/2013, at 7:36 PM, Johannes Rudolph <johannes...@googlemail.com> wrote:

> On Wed, Apr 17, 2013 at 11:35 PM, Tony Sloane <inky...@gmail.com> wrote:
>> We've been experimenting along these lines and it's all very promising. One
>> write-up of a simple example can be found here:
>
> This looks much more idiomatic IMO because positioning, format info
> and extraction are neatly separated and abstracted. Now put that into
> a macro so that the regexp parts are fused at compile time and Regexp
> starts to become usable.

Yeah, we are also pretty excited about the full concrete object syntax
version (interpolation defined by parsers not regexps). It will be very
useful for defining pattern matching and construction for program
analyses and transformations, for example.

cheers,
Tony

[Paolo G. Giarrusso]

Dear Tony,

I'm also pretty interested in using concrete object syntax for pattern matching, in combination with Lightweight Modular Staging. Right now, one can use concrete object syntax only to express the replacement code (the "right-hand side" of a rewrite rule), not the pattern (the left-hand side of a rewrite rule), at least I'd be happy to reuse a solution for having concrete object syntax in matching.

See for instance this example (from http://arxiv.org/abs/1210.6284v1, page 6, also published at AOSD'13):

val mergeFilters = ExpTransformer {

  case Sym(Filter(Sym(Filter(collection, pred2)), pred1)) ⇒

    collection.filter(x ⇒ pred2(x) && pred1(x))

}

The pattern here, in concrete object syntax, is something like "collection.filter(pred2).filter(pred1)".

Best,
Paolo

[Tony Sloane]

On 20/04/2013, at 12:45 AM, Paolo G. Giarrusso <p.gia...@gmail.com> wrote:

I'm also pretty interested in using concrete object syntax for pattern matching, in combination with Lightweight Modular Staging. Right now, one can use concrete object syntax only to express the replacement code (the "right-hand side" of a rewrite rule), not the pattern (the left-hand side of a rewrite rule), at least I'd be happy to reuse a solution for having concrete object syntax in matching.

See for instance this example (from http://arxiv.org/abs/1210.6284v1, page 6, also published at AOSD'13):

val mergeFilters = ExpTransformer {

  case Sym(Filter(Sym(Filter(collection, pred2)), pred1)) ⇒

    collection.filter(x ⇒ pred2(x) && pred1(x))

}

The pattern here, in concrete object syntax, is something like "collection.filter(pred2).filter(pred1)".

Yes, this is a perfect application for it. We are certainly aiming for a general solution, but the current approach we are using requires a bit of boilerplate. We're a bit busy with other things at the moment, but I hope to get back to it soon.

cheers,

Tony

[Johannes Rudolph]

On Wed, Apr 17, 2013 at 8:04 AM, Eugene Burmako <xen...@gmail.com> wrote:
> We have previously used pattern matching with quasiquoting interpolators.
> There's currently a problem with UnApply nodes, which prevents the full
> power of macros from being used in pattern matching, though the patch we
> applied to fix that is tiny. Denys (in /cc) knows the details.

Is this related to https://issues.scala-lang.org/browse/SI-5903 ?
What's the status on this?

[Johannes Rudolph]

On Wed, Apr 17, 2013 at 8:04 AM, Eugene Burmako <xen...@gmail.com> wrote:

> We have previously used pattern matching with quasiquoting interpolators.
> There's currently a problem with UnApply nodes, which prevents the full
> power of macros from being used in pattern matching, though the patch we
> applied to fix that is tiny. Denys (in /cc) knows the details.

Was it this commit:

https://github.com/scalamacros/kepler/commit/f037d97ddb488b7426eb1bc9b37b885fefa62070

Can we have this for the next 2.10.x as well?

[Eugene Burmako]

Yes, that's the one, but it's not only up for me to decide whether new features such as untyped macros can be added to Scala. /cc Adriaan & Jason

[Johannes Rudolph]

On Mon, Apr 29, 2013 at 1:51 PM, Johannes Rudolph
<johannes...@googlemail.com> wrote:
> Was it this commit:
>
> https://github.com/scalamacros/kepler/commit/f037d97ddb488b7426eb1bc9b37b885fefa62070
>
> Can we have this for the next 2.10.x as well?

Answering myself: Maybe not, because a pattern matching macro
application doesn't work on expressions so that it doesn't really fit
into the 2.10 typed macro scheme. I didn't want to imply to add
untyped macros to 2.10.

Is there a workaround in 2.10? The prefix you get passed into an
unapply macro with 2.10 is "new
CtxExtension(ctx).unapply(<unapply-selector>)", is there anything to
return from the macro which doesn't crash the compiler or fails with
an error?

[Eugene Burmako]

Well you can create a call to another unapply method passing that dummy <unapply-selector> as an argument.

Or, as Adriaan told me last week, you could even try returning a Block that would incorporate your custom logic inline (with the same expressive power as in the case if you could replace the entire pattern, albeit a bit less effective because of boxing/unboxing via options and tuples). That's what I tried to do right now, but failed: https://gist.github.com/xeno-by/5481456. Adriaan, could you please comment?

[Adriaan Moors]

the NPE seems to indicate the block with the unapply logic wasn't typed

I would love to see this work, but I don't have the bandwidth for this right now

[Eugene Burmako]

The printout shows the block has been typed, no?

[Adriaan Moors]

you're right -- the problem is that the code expects a selection of an unapply[Seq] method that has a symbol

a block's symbol is null

the symbol is used to determine whether the call is an unapply or an unapplySeq, so not sure how to do that for a block

[Paul Phillips]

On Fri, May 3, 2013 at 2:36 AM, Adriaan Moors <adriaa...@typesafe.com> wrote:

you're right -- the problem is that the code expects a selection of an unapply[Seq] method that has a symbol

a block's symbol is null

the symbol is used to determine whether the call is an unapply or an unapplySeq, so not sure how to do that for a block

That reminds me of https://issues.scala-lang.org/browse/SI-3353 where lukas says

"OK, I see.. I'm not sure what to do.

Using default arguments in patterns is currently not supported, the fact that it's not disallowed for this extractor is rather a missed case, maybe we should disallow it for now.

There's a whole space to explore on the interaction of named/default arguments and pattern matching. If we start doing this, we will end up with this kind of blocks during pattern matching (and btw, blocks don't need to have symbols)."