How to tell SBT to use a local version of Scala libraries?

1,645 views
Skip to first unread message

Simon Ochsenreither

unread,
Aug 7, 2013, 3:12:51 PM8/7/13
to simple-b...@googlegroups.com
Hi,

I compiled my own version of Scala with a few changes and I have a SBT project which requires these changes.

I tried http://www.scala-sbt.org/0.13.0/docs/Howto/scala.html#local already, but I wasn't successsful. I also tried placing the libraries into a lib/ directory and/or setting autoScalaLibrary := false.

My SBT build definition looks like this:

import sbt._
import Keys._

object BuildSettings {
  val buildSettings = Defaults.defaultSettings ++ Seq(
    organization := "com.github.aloiscochard.enum-paradise",
    version := "0.2-SNAPSHOT",
    scalacOptions ++= Seq("-deprecation", "-feature", "-Xlint"),
    scalaVersion := "2.10.3-20130807-192455-04dbfe9680",
    // autoScalaLibrary := false, // Didn't work!
    // scalaHome := Some(file("/home/soc/Entwicklung/lib/scala-enums-2.10.3-20130807-192455-04dbfe9680/")),  // Didn't work either!
    resolvers += Resolver.sonatypeRepo("snapshots"),
    addCompilerPlugin("org.scala-lang.plugins" % "macro-paradise_2.10.2" % "2.0.0-SNAPSHOT")
  )
}

object MyBuild extends Build {
  import BuildSettings._

  lazy val root: Project = Project(
    "root",
    file("."),
    settings = buildSettings ++ Seq(
      run <<= run in Compile in core
    )
  ) aggregate(macros, core)

  lazy val macros: Project = Project(
    "macros",
    file("macros"),
    settings = buildSettings ++ Seq(
      libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-reflect" % _),
      libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-compiler" % _))
  )

  lazy val core: Project = Project(
    "core",
    file("core"),
    settings = buildSettings
  ) dependsOn(macros)
}

The error messages look like this:

> reload
[info] Loading global plugins from /home/soc/.sbt/0.13/plugins
[info] Loading project definition from /home/soc/Entwicklung/enum-paradise/project
[info] Compiling 1 Scala source to /home/soc/Entwicklung/enum-paradise/project/target/scala-2.10/sbt-0.13/classes...
[info] Set current project to root (in build file:/home/soc/Entwicklung/enum-paradise/)
> compile
[info] Updating {file:/home/soc/Entwicklung/enum-paradise/}macros...
[info] Updating {file:/home/soc/Entwicklung/enum-paradise/}root...
[info] Resolving org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-reflect;2.10.2 ...
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     :: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[info] Resolving org.scala-lang#scala-reflect;2.10.2 ...
[info] Done updating.
[trace] Stack trace suppressed: run last macros/*:update for the full output.
[error] (macros/*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[error] Total time: 3 s, completed 07.08.2013 21:10:35

Can someone help me a bit with this?

Paul Phillips

unread,
Aug 7, 2013, 3:22:18 PM8/7/13
to simple-b...@googlegroups.com
I can't tell you what the build definition should look like, but I can say I've had good results using ++ with 0.13.0. For instance.

% sbt -sbt-version 0.13.0-RC4 "++ /scala/inst/3" about
[info] Set current project to tmp (in build file:/private/tmp/)
[info] Setting Scala home to /scala/inst/3 with actual version 2.11.0-20130806-080317-4c62f7db6f
[info] and using 2.11.0-20130806-080317-4c62f7db6f for resolving dependencies.
[info] Set current project to tmp (in build file:/private/tmp/)
[info] This is sbt 0.13.0-RC4
[info] The current project is {file:/private/tmp/}tmp 0.1-SNAPSHOT
[info] The current project is built against local Scala version 2.11.0-20130806-080317-4c62f7db6f at /scala/inst/3
[info] 
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.2

Simon Ochsenreither

unread,
Aug 7, 2013, 3:48:35 PM8/7/13
to simple-b...@googlegroups.com

I can't tell you what the build definition should look like, but I can say I've had good results using ++ with 0.13.0. For instance.

Thanks, it looks like this improved things a bit:

> reload
[info] Loading global plugins from /home/soc/.sbt/0.13/plugins
[info] Loading project definition from /home/soc/Entwicklung/enum-paradise/project
[info] Compiling 1 Scala source to /home/soc/Entwicklung/enum-paradise/project/target/scala-2.10/sbt-0.13/classes...
[info] Set current project to root (in build file:/home/soc/Entwicklung/enum-paradise/)
> ++ /home/soc/Entwicklung/lib/scala-enums-2.10.3-20130807-192455-04dbfe9680/
[info] Setting Scala home to /home/soc/Entwicklung/lib/scala-enums-2.10.3-20130807-192455-04dbfe9680 with actual version 2.10.3-20130807-192455-04dbfe9680
[info]     and using 2.10.3-20130807-192455-04dbfe9680 for resolving dependencies.

[info] Set current project to root (in build file:/home/soc/Entwicklung/enum-paradise/)
> compile
[info] Updating {file:/home/soc/Entwicklung/enum-paradise/}macros...
[info] Resolving org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-reflect;2.10.2 ...
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     :: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last macros/*:update for the full output.
[error] (macros/*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[error] Total time: 2 s, completed 07.08.2013 21:44:46

It seems like the issue is caused by the dependencies defined in the macro subproject:


    settings = buildSettings ++ Seq(
      libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-reflect" % _),
      libraryDependencies <+= (scalaVersion)("org.scala-lang" % "scala-compiler" % _))

Somehow they don't realize that we have explicitly set the scala home directory and dependency resolution...

Any ideas?

Thanks,

Simon

Paul Phillips

unread,
Aug 7, 2013, 4:11:27 PM8/7/13
to simple-b...@googlegroups.com

On Wed, Aug 7, 2013 at 12:48 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:
Somehow they don't realize that we have explicitly set the scala home directory and dependency resolution...

I think you need to sprinkle "in ThisBuild" around, like

scalaVersion in ThisBuild := "2.10.3-20130807-192455-04dbfe9680"


Simon Ochsenreither

unread,
Aug 7, 2013, 5:06:34 PM8/7/13
to simple-b...@googlegroups.com

I think you need to sprinkle "in ThisBuild" around, like

scalaVersion in ThisBuild := "2.10.3-20130807-192455-04dbfe9680"

No, didn't work:

../sbt

[info] Loading global plugins from /home/soc/.sbt/0.13/plugins
[info] Loading project definition from /home/soc/Entwicklung/enum-paradise/project
[info] Compiling 1 Scala source to /home/soc/Entwicklung/enum-paradise/project/target/scala-2.10/sbt-0.13/classes...
[info] Set current project to root (in build file:/home/soc/Entwicklung/enum-paradise/)
> compile
[info] Updating {file:/home/soc/Entwicklung/enum-paradise/}macros...
[info] Updating {file:/home/soc/Entwicklung/enum-paradise/}root...

[info] Resolving org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/scala-library-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/scala-library-2.10.3-20130807-192455-04dbfe9680.pom

[info] Resolving org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-reflect/2.10.3-20130807-192455-04dbfe9680/scala-reflect-2.10.3-20130807-192455-04dbfe9680.pom
[info] Resolving org.scala-lang#scala-reflect;2.10.2 ...
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     :: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680: not found

[warn]     :: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[info] Resolving org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/scala-library-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-library/2.10.3-20130807-192455-04dbfe9680/scala-library-2.10.3-20130807-192455-04dbfe9680.pom

[info] Resolving org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680 ...
[warn]     module not found: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680
[warn] ==== local: tried
[warn]   /home/soc/.ivy2/local/org.scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-lang/scala-compiler/2.10.3-20130807-192455-04dbfe9680/scala-compiler-2.10.3-20130807-192455-04dbfe9680.pom
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680: not found

[warn]     :: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last root/*:update for the full output.

[trace] Stack trace suppressed: run last macros/*:update for the full output.
[error] (root/*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680: not found

[error] unresolved dependency: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[error] (macros/*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-compiler;2.10.3-20130807-192455-04dbfe9680: not found
[error] unresolved dependency: org.scala-lang#scala-library;2.10.3-20130807-192455-04dbfe9680: not found
[error] unresolved dependency: org.scala-lang#scala-reflect;2.10.3-20130807-192455-04dbfe9680: not found
[error] Total time: 19 s, completed 07.08.2013 23:01:52
 

Chris Hodapp

unread,
Aug 7, 2013, 5:37:55 PM8/7/13
to simple-b...@googlegroups.com
In my experience, you want to set scalaHome to the "pack" directory of a (built) Scala compiler checkout.
That's <checkout-root>/build/pack.

If you are pulling in libraries, you may also want to set scalaVersion (to something like 2.10.2), so
that your %%'s do the right thing, but it seems not to affect what sbt does for the actual compiler
(at least with 0.13.0).

You shouldn't have to (and probably simply shouldn't) set autoScalaLibrary to false.

Simon Ochsenreither

unread,
Aug 7, 2013, 6:14:22 PM8/7/13
to simple-b...@googlegroups.com
Hi Chris!


In my experience, you want to set scalaHome to the "pack" directory of a (built) Scala compiler checkout.
That's <checkout-root>/build/pack.

Yes, that's basically what I'm doing. I just copied the jars to a different directory so they aren't gone if I do a ant clean (for whatever reason).


If you are pulling in libraries, you may also want to set scalaVersion (to something like 2.10.2), so
that your %%'s do the right thing, but it seems not to affect what sbt does for the actual compiler
(at least with 0.13.0).

Ok, I'll check that.

Thanks!

Simon

Chris Hodapp

unread,
Aug 7, 2013, 6:32:57 PM8/7/13
to simple-b...@googlegroups.com
Do note that a "pack" directory looks something like:
/Users/clhodapp/gsoc/git/scala/build/pack
clhodapp %> tree
.
├── META-INF
│   └── MANIFEST.MF
├── bin
│   ├── fsc
│   ├── fsc.bat
│   ├── scala
│   ├── scala.bat
│   ├── scalac
│   ├── scalac.bat
│   ├── scaladoc
│   ├── scaladoc.bat
│   ├── scalap
│   └── scalap.bat
├── bin.complete
├── lib
│   ├── diffutils.jar
│   ├── jline.jar
│   ├── scala-actors.jar
│   ├── scala-compiler.jar
│   ├── scala-library.jar
│   ├── scala-parser-combinators.jar
│   ├── scala-partest-javaagent.jar
│   ├── scala-partest.jar
│   ├── scala-reflect.jar
│   ├── scala-swing.jar
│   ├── scala-xml.jar
│   ├── scalacheck.jar
│   ├── scalap.jar
│   └── test-interface.jar
└── misc
    └── scala-devel
        └── plugins
            └── continuations.jar
So, it's not just the jars in a directory. Perhaps you knew this, but I just wanted to draw attention to it.

Chris Hodapp

unread,
Aug 7, 2013, 7:15:33 PM8/7/13
to simple-b...@googlegroups.com
A build.sbt with the contents:

lazy val buildSettings = Defaults.defaultSettings ++ Seq(
    organization := "com.github.aloiscochard.enum-paradise",
    version := "0.2-SNAPSHOT",
    scalacOptions ++= Seq("-deprecation", "-feature", "-Xlint"),
    scalaVersion := "2.10.2",
    resolvers += Resolver.sonatypeRepo("snapshots"),
    addCompilerPlugin(
"org.scala-lang.plugins" % "macro-paradise_2.10.2" % "2.0.0-SNAPSHOT"
),
scalaHome := Some(file("/Users/clhodapp/gsoc/git/scala-2.10/build/pack"))
)

lazy val rootSettings = buildSettings ++ Seq(
run <<= run in Compile in core
)

lazy val root = Project("root", file(".")).settings(
rootSettings: _*
).aggregate(macros, core)

lazy val macroSettings = buildSettings ++ Seq(
libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value,
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
)

lazy val macros = Project("macros", file("macros")).settings(
macroSettings: _*
)

lazy val core = Project("core", file("core")).settings(
buildSettings: _*
).dependsOn(macros)

seems to work for me.
Reply all
Reply to author
Forward
Message has been deleted
0 new messages