Lift Weekly Recap 2: Compatibility breaks and Markdown support

[Antonio Salazar Cardozo]

Back again with another one of those block rockin' beats! It's time for the second

in this hopefully-to-be-long-running series of weekly posts regarding the activity

that's happened in Lift in the past week. Still haven't had time to find a more

permanent home for this, so for now it's still a mailing list post.

As an ongoing rule, if you have any ideas for things that might be mentioned in

something like this, drop me a line at savedfastcool AT gmail.com !

As mentioned last week, the first few weeks will be recaps of stuff that's happened

over the past months in Lift 2.6 and Lift 3, as a way to get us to a point where

everyone's all caught up and we can start covering forward development.

Before we get to compatibility breakage, there was one actor-related improvement

that landed in both Lift 2.6 and Lift 3: MockLiftActor. MockLiftActor is designed

precisely to allow mocking in test scenarios. It provides an implementation of the

message-sending method, !, that appends the given message to an internal list

of received messages. The actor makes no attempt to process the message or

the received message list, the method is thread-safe, and there are methods

available to query whether a message made its way to the actor, and to query

how many messages the actor received. You can use it something like this:

val myActor = new MockLiftActor
 
myActor ! LogFbUserIn("savedfastcool", "oauth-token")
myActor ! RequestPermissions(FbProfilePermission, FbAwesomePermission)
 
myActor.hasReceivedMessage_?(LogFbUserIn("savedfastcool", "oauth-token")) must beTrue
myActor.hasReceivedMessage_?(RequestPermissions(FbProfilePermission, FbAwesomePermission)) must beTrue
myActor.messageCount must_== 2

Now, on to some breaking changes in Lift 3 specifically.

First off, Lift 3 is only supported on Scala 2.10. Depending on when Lift 3 is

released, it may ultimately end up being Scala 2.11-only, but for now it's

2.10-only.

Additionally, Lift 3 should only be run on versions of Java 6 > u23, or Java 7.

This is because Lift 3 no longer bundles its own double-parsing routine for

use in Helpers.asDouble and lift-json. Lift has included this fix since version

2.3-RC1, and will continue to include it in the 2.6 series. However, Lift 3

will no longer include it, and is therefore not safe on versions of

Java 6 <= u23. More about the double parsing bug, which had been filed

against the Sun JVM since 2001 and caused an infinite loop when trying

to parse a very specific double value from a String, can be found in

this blog post. The bug was fixed in the JVM on February 8, 2011 with

release u24 of Java 6.

Lift 3 should also finally remove open_! and openTheBox. These have been

deprecated since Lift 2.5-M1, in favor of openOrThrowException.

openOrThrowException is intentionally long-winded, both as a way to

discourage using it in the first place, and as a way to ensure that, when

used, a proper justification is provided.[1]

Both in Lift 2.6 and 3, JValue now has a withFilter method. This is mostly

done to silence warnings from the compiler in Scala 2.10; however, as

specified by the withFilter contract, withFilter is lazy. You should rarely

use this directly, but it will be used automatically by Scala when

converting for comprehensions into method calls.

Also both in 2.6 and 3, JValue's .hashCode method now returns the same

value in cases where its .equals method returns true, properly meeting

the typical JVM contract between equals and hashCode. This is thanks to

a kind contribution from Kenji Yoshida.

Starting in 2.6-M1, and also in Lift 3, thanks to a contribution from kluyg,

lift-json properly parses JSON doubles that include e+ or E+ in their

representation. lift-json previously failed to parse these doubles altogether.

Now, for the big one. Starting in Lift 2.6 (M1, specifically) and therefore

likewise in Lift 3, we have the lift-markdown module, which is the new

home of the Actuarius markdown parser from Christoph Henkelmann.

lift-markdown allows you to parse Markdown, mostly sticking to the

original specification from John Gruber, which you can find on

Daring Fireball's Markdown syntax page. You can use the Transformer

trait or one of its implementations, SingleThreadedTransformer and

ThreadLocalTransformer, to parse markdown into Strings:

import net.liftweb.markdown._
 
val myString = "# This is markdown!"
 
val transformer = new SingleThreadedTransformer
transformer(myString) // => "<h1>This is markdown!</h1>"
 
val threadLocalTransformer = new ThreadLocalTransformer
threadLocalTransformer(myString) // => "<h1>This is markdown!</h1>"

It's probably best to use ThreadLocalTransformer whenever dealing with

snippets or anything of the sort, since lift-markdown uses Scala parser

generators, which are not thread-safe. Indeed, it's a good idea to avoid

SingleThreadedTransformer in a Lift application unless you're very very

certain that you know that's what you need.

You can also mix in the Transformer trait and use its apply function to

parse things; you can, for example, do this in a snippet:

class MySnippet extends Transformer {
  def outputMarkdown = {
    XML.loadString(apply(myString))
  }
}

Notice that to get a NodeSeq out of the markdown parser, you have to use

XML.loadString! Markdown by default supports inline XML, which is a

first-class way to have attack code injected into your HTML, so be careful

who you allow to put Markdown into your application! One good way of

dealing with this issue is to pre-escape Markdown before you try to parse

it. You can do this like so:

val myString = "# This is markdown! <span>and some HTML</span>"
val escaped = <escaper>{myString}</escaper>.child(0).toString
 
threadLocalTransformer(escaped) // => <h1>This is markdown! &amp;lt;span&amp;gt;and some HTML&amp;lt;/span&amp;gt;</h1>

This lets Scala's XML escaping routines do their thing.[2]

Ok, that's it for this week! Next week we'll look at some sweet niceties

added to Lift's [Request|Session|Container|*]Vars, as well as some

tweaks to Boolean String parsing and changes in the CSS selectors.

[1] - Unfortunately, a commit that removed these in Lift 3 was reverted. Not

sure why, but rest assured the methods will be gone by the time Lift 3

is released.

[2] - There seems to be a way to get the markdown parser not to allow

inline XML, but I didn't have time to figure out exactly how to get this

working. Don't be surprised if one of these weeks the weekly recap includes

a note that we disabled inline XML by default ;)

[Tyler Weir]

Awesome stuff again Antonio!

[Andrew Goodnough]

I appreciate these summaries too. Keep 'em coming.

Andy

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Antonio Salazar Cardozo]

Thanks, will do :)

Antonio

[Peter Petersson]

On 01/15/2014 10:07 PM, Tyler Weir wrote:

Awesome stuff again Antonio!

+1 on that :)

--

[Vasya Novikov]

Great!

A question about markdown secureness. Can the list of "markdown
transformations" be limited? For example, I would want users to be able
to write bold text. But I would not like to allow them posting images
(to prevent browser headers sneaking, for example).

> will no longer include it, and is therefore /not safe /on versions of

> Java 6 <= u23. More about the double parsing bug, which had been filed
> against the Sun JVM since 2001 and caused an infinite loop when trying
> to parse a very specific double value from a String, can be found in
> this blog post

> <http://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/>.

> The bug was fixed in the JVM on February 8, 2011

> <https://blogs.oracle.com/security/entry/security_alert_for_cve-2010-44>

> with
> release u24 of Java 6.
>
> Lift 3 should also finally remove open_! and openTheBox. These have been
> deprecated since Lift 2.5-M1, in favor of openOrThrowException.
> openOrThrowException is intentionally long-winded, both as a way to
> discourage using it in the first place, and as a way to ensure that, when
> used, a proper justification is provided.[1]
>
> Both in Lift 2.6 and 3, JValue now has a withFilter method. This is mostly
> done to silence warnings from the compiler in Scala 2.10; however, as
> specified by the withFilter contract, withFilter is lazy. You should
> rarely
> use this directly, but it will be used automatically by Scala when
> converting for comprehensions into method calls.
>
> Also both in 2.6 and 3, JValue's .hashCode method now returns the same
> value in cases where its .equals method returns true, properly meeting
> the typical JVM contract between equals and hashCode. This is thanks to

> a kind contribution from Kenji Yoshida <https://github.com/xuwei-k>.

>
> Starting in 2.6-M1, and also in Lift 3, thanks to a contribution from

> kluyg <https://github.com/kluyg>,

> lift-json properly parses JSON doubles that include e+ or E+ in their
> representation. lift-json previously failed to parse these doubles
> altogether.
>
> Now, for the big one. Starting in Lift 2.6 (M1, specifically) and
> therefore
> likewise in Lift 3, we have the lift-markdown module, which is the new
> home of the Actuarius markdown parser from Christoph Henkelmann

> <https://github.com/chenkelmann>.

> lift-markdown allows you to parse Markdown, mostly sticking to the
> original specification from John Gruber, which you can find on
> Daring Fireball's Markdown syntax

> <http://daringfireball.net/projects/markdown/syntax><http://daringfireball.net/projects/markdown/syntax>page
> <http://daringfireball.net/projects/markdown/syntax>. You can use the

[Matt Farmer]

Always fun seeing code I wrote get mentioned. :-)

On Wednesday, January 15, 2014 12:19:32 AM UTC-5, Antonio Salazar Cardozo wrote:

[Diego Medina]

Awesome summary Antonio!

--

--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Diego Medina
Lift/Scala consultant
di...@fmpwizard.com
http://fmpwizard.telegr.am

[Antonio Salazar Cardozo]

On Sunday, February 2, 2014 3:34:37 PM UTC-5, Vasya Novikov wrote:

Great!

A question about markdown secureness. Can the list of "markdown
transformations" be limited? For example, I would want users to be able
to write bold text. But I would not like to allow them posting images
(to prevent browser headers sneaking, for example).

 Yes, the way to do this is the same way that you would disable inline XML, which is

using a decorator on the transformer. I haven't experimented with it extensively, but

theoretically something like this:

class MyThreadLocalTransformer extends Transformer {

    private[this] val threadLocalTransformer = new ThreadLocal[SingleThreadedTransformer] {
        override def initialValue = new SingleThreadedTransformer {
          val deco = new Decorator {
            def allowVerbatimXml = false
            def decorateImg(alt:String, src:String, title:Option[String]) = src
          }
        }
    }

    override def apply(s: String) = threadLocalTransformer.get()(s)
}

This will set up a transformer that will simply emit the image URL in cases where it runs into an
image, and will disable verbatim XML. You can customize differently, as well. See the Decorator
documentation at http://liftweb.net/api/26/api/#net.liftweb.markdown.Decorator for other things
you can override.
Thanks,
Antonio