Problem with SCP service
Phil Hagelberg
I've created a Vagrant file to make it easy to get set up with a local
copy of Clojars for hacking:
https://github.com/technomancy/clojars-web/tree/vagrant
The web app works great, but there's an issue with the scp service:
Exception in thread "NGSession 3: 127.0.0.1: clojars.scp" java.lang.ClassCastException: java.io.PrintStream cannot be cast to com.martiansoftware.nailgun.ThreadLocalPrintStream
at com.martiansoftware.nailgun.NGSession.run(Unknown Source)
I did a few tweaks including removing (set! *warn-on-reflection* true)
and adding some logging in. Now I don't get that exception message, but
scp just hangs. According to the logging, the call to .read on line 138
of clojars.scp never returns.
Any ideas what's going on here? I know that with Vagrant, ssh
connections take noticeably longer to open, so maybe that's causing issues?
-Phil
Alex Osborne
> I've created a Vagrant file to make it easy to get set up with a local
> copy of Clojars for hacking:
>
> https://github.com/technomancy/clojars-web/tree/vagrant
That's pretty nice! :-)
> The web app works great, but there's an issue with the scp service:
>
> Exception in thread "NGSession 3: 127.0.0.1: clojars.scp" java.lang.ClassCastException: java.io.PrintStream cannot be cast to com.martiansoftware.nailgun.ThreadLocalPrintStream
> at com.martiansoftware.nailgun.NGSession.run(Unknown Source)
That exception is due to a race condition in redirecting output. It
happens in production sometimes too. Probably the real fix is to
upgrade to Maven 3 which has a nice interface for capturing error
output so we don't have to do any redirection.
> I did a few tweaks including removing (set! *warn-on-reflection* true)
> and adding some logging in. Now I don't get that exception message, but
> scp just hangs. According to the logging, the call to .read on line 138
> of clojars.scp never returns.
>
> Any ideas what's going on here? I know that with Vagrant, ssh
> connections take noticeably longer to open, so maybe that's causing issues?
I think you're probably using an incompatible version of the ng client.
Clojars uses the last stable release 0.7.1. Nailgun trunk has changed
the protocol.
In production I'm using nailgun client 0.7.1 plus this patch:
The nailgun package in Ubuntu's repository should be compatible as well
but it suffers from the busy loop problem that patch fixes.
Cheers,
Alex
Phil Hagelberg
> I think you're probably using an incompatible version of the ng client.
> Clojars uses the last stable release 0.7.1. Nailgun trunk has changed
> the protocol.
>
> In production I'm using nailgun client 0.7.1 plus this patch:
>
> http://mail.martiansoftware.com/pipermail/nailgun-users_martiansoftware.com/2009-November/000009.html
This helps, but I still can't deploy over SSH:
phil@enigma ~/src/clojure-http-client $ scp pom.xml
clojure-http-client-1.1.0-SNAPSHOT.jar localclojars:
Welcome to Clojars, technomancy!
pom.xml
100% 2806 2.7KB/s 00:00
clojure-http-client-1.1.0-SNAPSHOT.jar
100% 5120 5.0KB/s 00:00
[INFO] Retrieving previous build number from clojars
Error: org.apache.maven.wagon.providers.file.FileWagon.openConnection()V
java.lang.AbstractMethodError:
org.apache.maven.wagon.providers.file.FileWagon.openConnection()V
at org.apache.maven.wagon.AbstractWagon.connect(AbstractWagon.java:143)
at org.apache.maven.artifact.manager.DefaultWagonManager.getRemoteFile(DefaultWagonManager.java:440)
at org.apache.maven.artifact.manager.DefaultWagonManager.getArtifactMetadataFromDeploymentRepository(DefaultWagonManager.java:380)
at org.apache.maven.artifact.repository.metadata.DefaultRepositoryMetadataManager.getArtifactMetadataFromDeploymentRepository(DefaultRepositoryMetadataManager.java:380)
at org.apache.maven.artifact.repository.metadata.DefaultRepositoryMetadataManager.resolveAlways(DefaultRepositoryMetadataManager.java:348)
at org.apache.maven.artifact.transform.SnapshotTransformation.resolveLatestSnapshotBuildNumber(SnapshotTransformation.java:161)
at org.apache.maven.artifact.transform.SnapshotTransformation.transformForDeployment(SnapshotTransformation.java:100)
at org.apache.maven.artifact.transform.DefaultArtifactTransformationManager.transformForDeployment(DefaultArtifactTransformationManager.java:78)
at org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:90)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
at clojars.maven$deploy_model.invoke(maven.clj:98)
at clojars.scp$finish_deploy.invoke(scp.clj:109)
at clojars.scp$nail.invoke(scp.clj:141)
at clojars.scp$_nailMain.invoke(scp.clj:160)
at clojars.scp.nailMain(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.martiansoftware.nailgun.NGSession.run(Unknown Source)
Any ideas?
thanks,
Phil
Phil Hagelberg
-Phil
Alex Osborne
> Any idea what's going on here?
>>
>> 100% 5120 5.0KB/s 00:00
>> [INFO] Retrieving previous build number from clojars
>> Error: org.apache.maven.wagon.providers.file.FileWagon.openConnection()V
>> java.lang.AbstractMethodError:
I was initially completely stumped, but I've just taken another look at
it.
Okay so they have the exact same jars in lib/ (checked the md5sums). The
exact same code in src/. But...
clojars@natty:~$ lein version
Leiningen 1.5.2 on Java 1.6.0_22 OpenJDK Client VM
clojars@clojars:~$ lein version
Leiningen 1.3.1 on Java 1.6.0_20 OpenJDK Client VM
No it couldn't possibly be...
*quiet sobbing*
ato@clojars:/home/clojars/prod/lib$ for x in *.jar; do echo $x; unzip -l $x | grep wagon-file; done
[...]
maven-ant-tasks-2.0.10.jar
META-INF/maven/org.apache.maven.wagon/wagon-file/
META-INF/maven/org.apache.maven.wagon/wagon-file/pom.xml
META-INF/maven/org.apache.maven.wagon/wagon-file/pom.properties
[...]
wagon-file-1.0-beta-6.jar
Archive: wagon-file-1.0-beta-6.jar
META-INF/maven/org.apache.maven.wagon/wagon-file/
META-INF/maven/org.apache.maven.wagon/wagon-file/pom.xml
META-INF/maven/org.apache.maven.wagon/wagon-file/pom.properties
Try this:
https://github.com/ato/clojars-web/commit/a89a696698e6364066ab8d5ed6990a3206b1bf91
I'm tempted to suggest we make lein uberjar give a warning if there's a
file conflict but there might be a lot of harmless things like
project.clj and README files.
Phil Hagelberg
> Okay so they have the exact same jars in lib/ (checked the md5sums). The
> exact same code in src/. But...
>
> clojars@natty:~$ lein version
> Leiningen 1.5.2 on Java 1.6.0_22 OpenJDK Client VM
>
> clojars@clojars:~$ lein version
> Leiningen 1.3.1 on Java 1.6.0_20 OpenJDK Client VM
I tried rebuilding with your change as well as simply rolling back to
lein 1.3.1 on my VM, but I can't get uploads to work. I did restart
the VM just to be sure the new uberjar was being used. Were you able
to get it working in your VM by getting rid of the explicit wagon-file
jar?
-Phil
Phil Hagelberg
> I tried rebuilding with your change as well as simply rolling back to
> lein 1.3.1 on my VM, but I can't get uploads to work. I did restart
> the VM just to be sure the new uberjar was being used. Were you able
> to get it working in your VM by getting rid of the explicit wagon-file
> jar?
Going off your wagon-file version mismatch, I went through and looked
for other places in which the versions inside maven-ant-task
mismatched what ended up in lib. There were a few:
;; version mismatches
;; plexus-interpolation 1.0 vs 1.11
;; maven-artifact-manager 2.0.10 vs 2.2.1
;; maven-repository-metadata 2.0.10 vs 2.2.1
;; maven-project 2.0.10 vs 2.2.1
;; maven-profile 2.0.10 vs 2.2.1
;; maven-plugin-registry 2.0.10 vs 2.2.1
Removing maven-artifact-manager, maven-project, and maven-model from
project.clj took care of all but the first. Oddly enough, the 1.11
dependency of plexus-interpolation comes from maven-ant-tasks, which
bundles 1.0 inside itself. Adding an explicit dependency on 1.0
doesn't help though. Just a shot in the dark, but I thought I'd
mention it to save others from going on a wild goose chase.
-Phil
Alex Osborne
> I tried rebuilding with your change as well as simply rolling back to
> lein 1.3.1 on my VM, but I can't get uploads to work. I did restart
> the VM just to be sure the new uberjar was being used. Were you able
> to get it working in your VM by getting rid of the explicit wagon-file
> jar?
Blast. Oh how I hate this mess.
When it's working for me I'm seeing it pull in wagon-file-1.0-beta-2.jar
rather than beta-6 (which causes it to break).
Maybe there's still an ordering issue though. I'm including the output
from when I make a working uberjar below. Maybe check if it's the same
order as what's happening on your box.
clojars@natty:~/prod$ git fetch technomancy
clojars@natty:~/prod$ git diff technomancy/vagrant
diff --git a/project.clj b/project.clj
index bbf19e5..be9d667 100644
--- a/project.clj
+++ b/project.clj
@@ -6,7 +6,7 @@
[org.apache.maven/maven-artifact-manager "2.2.1"]
[org.apache.maven/maven-model "2.2.1"]
[org.apache.maven/maven-project "2.2.1"]
- [org.apache.maven.wagon/wagon-file "1.0-beta-6"]
+;; [org.apache.maven.wagon/wagon-file "1.0-beta-6"]
[compojure "0.5.2"]
[ring/ring-jetty-adapter "0.3.1"]
[hiccup "0.3.0"]
clojars@natty:~/prod$ lein uberjar
Cleaning up.
Copying 40 files to /home/clojars/prod/lib
Reflection warning, clojars/scp.clj:105 - call to println can't be resolved.
Reflection warning, clojars/scp.clj:105 - call to println can't be resolved.
Reflection warning, clojars/scp.clj:105 - call to println can't be resolved.
Reflection warning, clojars/scp.clj:105 - call to println can't be resolved.
Reflection warning, clojars/scp.clj:112 - call to println can't be resolved.
Copying 40 files to /home/clojars/prod/lib
Created /home/clojars/prod/clojars-web-0.6.2.jar
Including clojars-web-0.6.2.jar
Including nestedvm-1.0.jar
Including maven-ant-tasks-2.0.10.jar
Including jtidy-4aug2000r7-dev.jar
Including plexus-container-default-1.0-alpha-9-stable-1.jar
Including jetty-util-6.1.14.jar
Including wagon-http-shared-1.0-beta-2.jar
Including maven-profile-2.2.1.jar
Including commons-fileupload-1.2.1.jar
Including maven-artifact-2.0.10.jar
Including compojure-0.5.2.jar
Including ant-1.6.2.jar
Including plexus-interpolation-1.11.jar
Including maven-repository-metadata-2.2.1.jar
Including maven-artifact-manager-2.2.1.jar
Including wagon-http-lightweight-1.0-beta-2.jar
Including maven-model-2.2.1.jar
Including maven-project-2.2.1.jar
Including servlet-api-2.5-6.1.14.jar
Including backport-util-concurrent-3.1.jar
Including classworlds-1.1-alpha-2.jar
Including maven-plugin-registry-2.2.1.jar
Including commons-codec-1.4.jar
Including nailgun-0.7.1.jar
Including commons-io-1.4.jar
Including wagon-provider-api-1.0-beta-2.jar
Including maven-settings-2.0.10.jar
Including clojure-contrib-1.2.0.jar
Including clout-0.3.1.jar
Including xml-apis-1.0.b2.jar
Including hiccup-0.3.0.jar
Including jetty-6.1.14.jar
Including wagon-file-1.0-beta-2.jar
Including ring-jetty-adapter-0.3.1.jar
Including ring-servlet-0.3.1.jar
Including clojure-1.2.0.jar
Including ring-core-0.3.1.jar
Including servlet-api-2.5.jar
Including plexus-utils-1.5.5.jar
Including sqlite-jdbc-3.6.17.jar
Including maven-error-diagnostics-2.0.10.jar
Created /home/clojars/prod/clojars-web-0.6.2-standalone.jar
clojars@natty:~/prod$ java -jar clojars-web-0.6.2-standalone.jar 8001 7601
clojars-web: starting jetty on port 8001
2011-08-01 04:41:31.558::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2011-08-01 04:41:31.563::INFO: jetty-6.1.x
2011-08-01 04:41:31.580::INFO: Started SocketC...@0.0.0.0:8001
clojars-web: starting nailgun on 127.0.0.1 port 7601
% scp -P 2222 pom.xml lein-clojars-0.7.0.jar clojars@localhost:
Welcome to Clojars, ato!
pom.xml
lein-clojars-0.7.0.jar
[INFO] Retrieving previous metadata from clojars
[INFO] Uploading repository metadata for: 'artifact lein-clojars:lein-clojars'
[INFO] Uploading project information for lein-clojars 0.7.0
%
Phil Hagelberg
> Phil Hagelberg <ph...@hagelb.org> writes:
>
>> I tried rebuilding with your change as well as simply rolling back to
>> lein 1.3.1 on my VM, but I can't get uploads to work. I did restart
>> the VM just to be sure the new uberjar was being used. Were you able
>> to get it working in your VM by getting rid of the explicit wagon-file
>> jar?
Aw geez; my bad. I rebuild the copy in /vagrant instead of the one in
/home/clojars/prod. Works great now; thanks!
-Phil
J. Pablo Fernández
I experience the hung and then
java.lang.ClassCastException: java.io.PrintStream cannot be cast to
com.martiansoftware.nailgun.ThreadLocalPrintStream
at com.martiansoftware.nailgun.NGSession.run(Unknown Source)
com.martiansoftware.nailgun.ThreadLocalPrintStream
at com.martiansoftware.nailgun.NGSession.run(Unknown Source)
$ scp -P 2222 foobar-1.0.0-SNAPSHOT.jar pom.xml clojars@localhost:
Welcome to Clojars, pupeno!
foobar-1.0.0-
SNAPSHOT.jar
100% 1401 1.4KB/s 00:00
pom.xml
100% 1287 1.3KB/s 00:00
Error:
org.apache.maven.wagon.providers.file.FileWagon.openConnection()V
java.lang.AbstractMethodError:
Phil, should we rebase from Alex's master branch? Or is the vagrant
branch supposed to work out of the box at the moment?