Newb question: scalala and scalanlp build

[Brendan O'Connor]

Hi folks, just a newb question for getting started.

What's the recommended way to (1) build scalala, then (2) build scalanlp using it?  (I'm currently trying to use HEAD checkouts from github.)

I'm trying (1) ./sbt then "package" then "publish-local" for scalala, then (2) "mvn compile" for scalanlp.  Step #1 seems to succeed, but it Step #2 keeps complaining it can't find the jar.  Upon further inspection, it seems that step #1 published the jar into my local ~/.ivy2, instead of my local ~/.m2, where everything downloaded from the web seems to be going to.

(Originally I was trying to directly build scalanlp, but it wanted an "RC2" version of Scalala which didn't exist in the public repository; I was guessing that means I had to build scalala from source.)

Any suggestions?

Thanks,

Brendan

--

brenocon.com

[Brendan O'Connor]

Hi, I read the recent email thread between Steven and David and I guess the current versions of things just don't sync correctly yet.  A few stories about how far I got.  I don't understand the Maven/Ivy/SBT java+scala build system at all so I'm sure this is bad practice, but...

(1) for Scalala, ./sbt publish-local.  This makes the jar in ~/.ivy2

(2) Now transfer the ~/.ivy2 package into ~/.m2 via

% cd ~/.ivy2/local/org.scalala/scalala_2.8.1/1.0.0.RC2-SNAPSHOT/poms

% ln -s scalala_2.8.1.pom pom.xml

% mvn install

[.. lots of crap ..]

[INFO] Installing /Users/brendano/.ivy2/local/org.scalala/scalala_2.8.1/1.0.0.RC2-SNAPSHOT/poms/target/scalala_2.8.1-1.0.0.RC2-SNAPSHOT.jar to /Users/brendano/.m2/repository/org/scalala/scalala_2.8.1/1.0.0.RC2-SNAPSHOT/scalala_2.8.1-1.0.0.RC2-SNAPSHOT.jar

... actually that was a horrible hack and it only made a fake empty jar file. (The only good thing about that step was it pushed around all these pom.xml metainformation files.)  I copied the jar directly like so:

% cp /Users/brendano/.ivy2/local/org.scalala/scalala_2.8.1/1.0.0.RC2-SNAPSHOT/jars/scalala_2.8.1.jar  /Users/brendano/.m2/repository/org/scalala/scalala_2.8.1/1.0.0.RC2-SNAPSHOT/scalala_2.8.1-1.0.0.RC2-SNAPSHOT.jar

(3) Note: the name is now inside org.scalala not org.scalanlp !  I guess this is what Maven calls the "groupId".  Now to build scalala-core/data, it needs to be updated, or else it will give an error it can't find scalala.  I did this:

~/sw/scala/scalanlp-core/data % git diff

diff --git a/data/pom.xml b/data/pom.xml

index 10a03c9..355b0f4 100644

--- a/data/pom.xml

+++ b/data/pom.xml

@@ -81,7 +81,7 @@

       <version>070108</version>

     </dependency>

     <dependency>

-      <groupId>org.scalanlp</groupId>

+      <groupId>org.scalala</groupId>

       <artifactId>scalala_${scala.version}</artifactId>

       <version>1.0.0.RC2-SNAPSHOT</version>

     </dependency>

(4) inside scalala-core/data,  I ran "mvn compile".  It seems to find the dependencies then hits errors when compiling:

https://gist.github.com/1071131

These might be real errors, not build configuration errors, at this point.

Brendan

--
You received this message because you are subscribed to the Google Groups "ScalaNLP" group.
To view this discussion on the web visit https://groups.google.com/d/msg/scalanlp/-/LgRTZsEoQiwJ.
To post to this group, send email to scal...@googlegroups.com.
To unsubscribe from this group, send email to scalanlp+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scalanlp?hl=en.

[David Hall]

Your sir are a champ.

Sorry I didn't reply sooner; I was a the first meeting of the Bay Area
NLP Meetup, of all things.

I, err, guess I've been a bad steward. (Namely, pushing to my own
github branch and not communicating at all.) Here's the deal:

1) Scalala now builds with SBT. Because I didn't want to deal with the
machinations that you went through, I gave up and ported ScalaNLP's
build to SBT along with the port to Scala 2.9 and new Scalala. So much
porting. (I actually think SBT's reliance on reflection is really
hacky and gross, but that's another story.)

2) sbt update publish-local should just work.

3) I just pushed to the scalanlp account.

4) In addition to the port, there are many new things!
-- I cleaned up the optimize package quite a bit. Everything is a lot
more unified in terms of interface. (FirstOrder methods support an
iterator for each iteration and such.)
-- I added an LP solver (InteriorPoint) that seems to be pretty ok,
though I haven't tested it on anything large scale.
-- Along with the LP solver is a new DSL for solving LPs:
* val lp = new LP;
* import lp._;
* val x = new Positive("x");
* val y = new Positive("y");
*
* val result = maximize ( (3 * x+ 4 * y)
* subjectTo( x <= 3, y <= 1))
*
* result.valueOf(x) // 3
-- The classify package is way more sensible than it used to be. It
also now includes unit tests, a Logit/MaxEnt trainer that makes sense,
SMO and Pegasos for training SVM's, Naive Bayes and Perceptron. They
all seem to perform ok.
-- The sampling package is now called distributions, and includes
methods for finding the MLE for 7 or 8 exponential families. Also
includes sampling routines, as before.
-- I added a little more to the Tour section on the wiki (today)
-- Probably other changes, but I don't think they're important.

Ok, I think that's it.

-- David

[Brendan O'Connor]

mwahaha i have coaxed documentation out of you! great, the optimization stuff was what i was most interested in atm

i like having only one build system excellent

[Brendan O'Connor]

Great!  I even got it to work now :)

Just in case anyone else tries to do this, here's my blow-by-blow:

(1) get 'master' checkouts of  github.com/dlwh/Scalala and github.com/dlwh/scalanlp

(2) rm -rf ~/.ivy2/local/*  ... just in case

(3) edit Scalala/project/build.properties to force scala 2.9:

~/sw/scala/Scalala_dlwh % git diff

diff --git a/project/build.properties b/project/build.properties

index ab15e23..e4fb611 100644

--- a/project/build.properties

+++ b/project/build.properties

@@ -5,5 +5,5 @@ project.name=scalala

 sbt.version=0.7.4

 project.version=1.0.0.RC2-SNAPSHOT

 def.scala.version=2.7.7

-build.scala.versions=2.8.1 2.9.0

+build.scala.versions=2.9.0

 project.initialize=false

(4) build scalala.

~/sw/scala/Scalala_dlwh % ./sbt

> update

> publish-local

(5) build scalaNLP.  Use same 'sbt' version as scalala.

~/sw/scala/scalanlp-core % ../Scalala_main/sbt

> update

> publish-local

Everything eventually goes into ~/.ivy2/local , though i guess for many purposes the "compile" or "package" SBT actions may suffice.

-Brendan

--

brenocon.com

[David Hall]

On Fri, Jul 8, 2011 at 11:12 AM, Brendan O'Connor <bren...@gmail.com> wrote:
> Great!  I even got it to work now :)
> Just in case anyone else tries to do this, here's my blow-by-blow:
> (1) get 'master' checkouts of  github.com/dlwh/Scalala and
> github.com/dlwh/scalanlp
> (2) rm -rf ~/.ivy2/local/*  ... just in case
> (3) edit Scalala/project/build.properties to force scala 2.9:
> ~/sw/scala/Scalala_dlwh % git diff
> diff --git a/project/build.properties b/project/build.properties
> index ab15e23..e4fb611 100644
> --- a/project/build.properties
> +++ b/project/build.properties
> @@ -5,5 +5,5 @@ project.name=scalala
>  sbt.version=0.7.4
>  project.version=1.0.0.RC2-SNAPSHOT
>  def.scala.version=2.7.7
> -build.scala.versions=2.8.1 2.9.0
> +build.scala.versions=2.9.0
>  project.initialize=false

fwiw: sbt +update +publish-local should work without patches.

-- David

[Anthony Di Franco]

Neither of these is working for me right now.

I am eager to sidestep this by asking:

Which, if any, of the versions of scalala in your maven repository works with Scala 2.9.0-1?

[David Hall]

Um, unfortunately I think nothing. Dan Ramage is the one who publishes
Scalala and he's been busy with his wedding/honey moon for the past n
months. (Also graduating, defending, and writing his dissertation...)
I don't have the keys to the kingdom for Scalala any more, and I don't
want to be publishing dlwh-scalala unless it's really necessary.

This works for me:

git clone git://github.com/dlwh/Scalala.git
cd scalala
sbt +update +publish-local
cd ..
git clone git://github.com/dlwh/scalanlp-core.git
cd scalanlp-core/
sbt update publish-local

If that doesn't work, I'll fix errors as you send them to me. I'll be
traveling starting Monday for two weeks or so, however. Also, I plan
on getting Jenkins up and running again for my projects. I'll talk to
Dan if he wants to start automating deploys of Scalala.

-- David

>
> --
> You received this message because you are subscribed to the Google Groups
> "ScalaNLP" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/scalanlp/-/eUhr9fJIAUAJ.

[Anthony Di Franco]

After nuking my ~/.m2/repository/* in addition to ~/.ivy2/cache/* I have progressed to compilation errors within a properly-running sbt console.

Perhaps I will figure this out after all.

[Anthony Di Franco]

Yes, in the end after doing that this worked nicely to publish to my local nexus instance.

I probably could have gone in the direction of making the repository externally configurable instead of making the credentials hard-coded, but it will do.

diff --git a/project/build.properties b/project/build.properties
index ab15e23..e4fb611 100644
--- a/project/build.properties
+++ b/project/build.properties
@@ -5,5 +5,5 @@ project.name=scalala
 sbt.version=0.7.4
 project.version=1.0.0.RC2-SNAPSHOT
 def.scala.version=2.7.7
-build.scala.versions=2.8.1 2.9.0
+build.scala.versions=2.9.0
 project.initialize=false

diff --git a/project/build/Project.scala b/project/build/Project.scala
index 5c337e2..9823d78 100644
--- a/project/build/Project.scala
+++ b/project/build/Project.scala
@@ -93,19 +93,15 @@ class Project(info: ProjectInfo) extends DefaultProject(info
   //  
   val publishToRepoName = "Sonatype Nexus Repository Manager"
   val publishTo = {
-    val repoUrl = "http://nexus.scala-tools.org/content/repositories/" +
+    val repoUrl = "http://localhost/nexus/content/repositories/" +
       (if (version.toString.endsWith("-SNAPSHOT")) "snapshots" else "releases")
     publishToRepoName at repoUrl
   }
 
-  lazy val publishUser = system[String]("build.publish.user")
-  lazy val publishPassword = system[String]("build.publish.password")
+  lazy val publishUser = "admin"
+  lazy val publishPassword = "admin123"
+
+  Credentials.add(publishToRepoName, "localhost", publishUser, publishPassword)
 
-  (publishUser.get, publishPassword.get) match {
-    case (Some(u), Some(p)) =>
-      Credentials.add(publishToRepoName, "nexus.scala-tools.org", u, p)
-    case _ =>
-      Credentials(Path.userHome / ".ivy2" / ".credentials", log)
-  }
 }

[David Hall]

Isn't this only necessary if you want to sbt publish instead of sbt
publish-local?

> --
> You received this message because you are subscribed to the Google Groups
> "ScalaNLP" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/scalanlp/-/hYiM6ng5Kx0J.

[Anthony Di Franco]

I think so, but I keep an instance of nexus going for myself so that I only have to build stuff / configure 3rd party repositories in one place while working from several machines and on different projects with somewhat overlapping sets of obscure dependencies.

And since you're not publishing your snapshots to your repo and I can't this was the next most convenient thing for me to do.

[Daniel Duckworth]

Forgive me if this is a silly question, but am I to understand that
ScalaNLP-core is to be built exclusively with Scala v2.9.0 from this
point onwards?

[David Hall]

That's right, unless someone has a compelling need for 2.8.x

I'm traveling right now, but i'll get to your bug report soon.

-- David

>
> --
> You received this message because you are subscribed to the Google Groups "ScalaNLP" group.

[Daniel Duckworth]

Hi David,

I've written my own fix (which simply performs the algorithm on the transpose of the matrix, then converts the results back) and set up a pull request.  I wasn't able to actually build ScalaNLP though, so even though I've written and run the Unit Test in my own project, I haven't done the same inside ScalaNLP.  

Daniel Duckworth
Electrical Engineering and Computer Science
University of California, Berkeley