proof-of-concept Scala.js REPL

[Haoyi Li]

I have a basic proof-of-concept REPL working with Scala.js. Here's a 2 minute video showing it working:

https://vimeo.com/114640774

My implementation so far is pasted below. It does not support import-commands, and it does not echo the resulting values back to the user, but these are small details and trivial to add after-the-fact. The main REPL things work: defining variables and methods follow the same pattern as the normal REPL (wrapped in a object and imported), re-adding previously-DCE-ed fields/methods/super-classes to things uses the same technique as my live updating demo. And lastly it uses the normal workbench way of telling the browser to eval some arbitrary javascript blobs.

The size of the compilation input grows (together with compile times) as you keep using the thing, and in theory eventually you'll OOM, just like the normal Scala REPL =P This probably can be improved somewhat with more twiddling to the source/class/ir-handling code, and can be made at least as efficient as the normal Scala REPL, albeit with the addition cost of fastOptJS that needs to be paid every command. I could also probably get rid of the `sjs` prefix I needed before each command, given even more time to twiddle with the icky compiler APIs.

It probably isn't strictly sound, for example I think newly-referencing previously-DCEd `var`s and `val`s will cause it to blow up. On the other hand, I think it may be useful enough to provide value despite its shortcomings; only time will tell.

val sjs = inputKey[Unit]("lalala")
var replHistory = ""
var replCount = 0
def munge(s0: String) = {
  var s = s0
  s = s.replace("\nvar ScalaJS = ", "\nvar ScalaJS = ScalaJS || ")
  s = s.replaceAll(
    "\n(ScalaJS\\.c\\.[a-zA-Z_$0-9]+\\.prototype) = (.*?\n)",
    """
      |$1 = $1 || {}
      |(function(){
      |  var newProto = $2
      |  for (var attrname in newProto) { $1[attrname] = newProto[attrname]; }
      |})()
      |""".stripMargin
  )
  for(char <- Seq("d", "c", "h", "i", "n", "m")){
    s = s.replaceAll("\n(ScalaJS\\." + char + "\\.[a-zA-Z_$0-9]+) = ", "\n$1 = $1 || ")
  }
  s
}


sjs <<= sjs.dependsOn(packageJSDependencies, packageLauncher, preLinkClasspath in fastOptJS, products in fastOptJS),
sjs := {
  import sbt.complete.Parsers._

  val s = streams.value
  val output = WritableMemVirtualJSFile("repl.js")
  val taskCache = WritableMemVirtualJSFile("(memory)")

  val jCtx = new JavaContext()
  lazy val settings = new scala.tools.nsc.Settings
  val vd = new VirtualDirectory("(memory)", None)
  settings.outputDirs.setSingleOutput(vd)

  val compiledIr = products.value.flatMap(_ ** "*.sjsir" get)

  val bootFiles = for {
    prop <- Seq("java.class.path", "sun.boot.class.path")
    path <- System.getProperty(prop).split(System.getProperty("path.separator"))
    vfile = scala.reflect.io.File(path)
    if vfile.exists && !vfile.isDirectory
  } yield path

  for(file <- fullClasspath.value.map(_.data.getAbsolutePath) ++ bootFiles){
    settings.classpath.append(file)
    settings.bootclasspath.append(file)
  }


  val compiler = new nsc.Global(settings, new ConsoleReporter(settings)){ g =>
    override lazy val plugins = List[scala.tools.nsc.plugins.Plugin](
      new scala.scalajs.compiler.ScalaJSPlugin(this)
    )
  }

  /**
   * Converts a bunch of bytes into Scalac's weird VirtualFile class
   */
  def makeFile(src: Array[Byte]) = {
    val singleFile = new scala.tools.nsc.io.VirtualFile("Main.scala")
    val output = singleFile.output
    output.write(src)
    output.close()
    singleFile
  }

  val run = new compiler.Run()
  val str = sbt.complete.Parsers.any.*.parsed.mkString
  replHistory += "\n\n" + s"@scalajs.js.annotation.JSExport object O$replCount{ $str }; import O$replCount._"

  run.compileFiles(List(makeFile(replHistory.getBytes)))
  val inputCp = for{
    x <- vd.iterator.to[collection.immutable.Traversable]
    if x.name.endsWith(".sjsir")
  } yield {
    val m = new MemVirtualSerializedScalaJSIRFile(x.canonicalPath)
    m.content = x.toByteArray
    m
  }
  val relSourceMapBase = None
  val oldCp = (preLinkClasspath in fastOptJS).value
  val newCp = new CompleteIRClasspath(
    oldCp.jsLibs,
    oldCp.scalaJSIR ++ compiledIr.map(new FileVirtualScalaJSIRFile(_)) ++ inputCp,
    oldCp.requiresDOM,
    oldCp.version
  )
  import ScalaJSOptimizer._
  (scalaJSOptimizer in fastOptJS).value.optimizeCP(
    Inputs(input = newCp),
    OutputConfig(
      output = output,
      cache = Some(taskCache),
      wantSourceMap = false,
      relativizeSourceMapBase = relSourceMapBase,
      checkIR = (checkScalaJSIR in fastOptJS).value,
      disableInliner = (inliningMode in fastOptJS).value.disabled,
      batchInline = (inliningMode in fastOptJS).value.batch
    ),
    s.log
  )

  val fileOutput = new java.io.File("repl.js")

  IO.write(fileOutput, munge(s"${output.content}\n\nO$replCount()" ).getBytes())

  replCount += 1
  server.value.Wire[Api].run("http://localhost:12345/repl.js", None).call()
}

[Justin du coeur]

On Tue, Dec 16, 2014 at 3:39 AM, Haoyi Li <haoy...@gmail.com> wrote:

I have a basic proof-of-concept REPL working with Scala.js. Here's a 2 minute video showing it working:

... wow, that's quite neat.  Bit by bit, the pieces are starting to look like elements of a serious IDE... 

[Haoyi Li]

I've merged this into trunk at https://github.com/lihaoyi/workbench. Will probably publish a new release when 0.6.0-FINAL comes out.

Hopefully the Scala.js internals that this relies on (mostly in the naive javascript-translation) won't change too much from 0.5.x and it'll be a straightforward forward port.