Re: Uploading job to azkaban2 via command line

Richard

unread,

Apr 3, 2013, 7:45:31 PM4/3/13

to azkab...@googlegroups.com

This looks like an issue we fixed in trunk. We'll be peeling off a release soon.

Jetty is unfortunately embedded so we don't use jetty.xml. You can either get trunk, or wait for the packages.

On Wednesday, April 3, 2013 4:10:24 PM UTC-7, S V wrote:

Does anyone have uploading a job to Azkaban2 via command line working?

Based on an earlier post, I'm trying something like:

curl -X POST --insecure --data-binary "@large_file.zip" --data "session.id=$session_id&action=upload&project=test_project" https://server:8443/manager

Not sure this is the right curl, I'm getting a 500 because I'm uploading 37 megs, (and there's a magic 10M jetty limit which I can't find anywhere). Upload via the web interface works though. How do I change this jetty limit without a jetty.xml?

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 500 Form too large37015332>10485760</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /manager. Reason:
<pre> Form too large37015332>10485760</pre></p><h3>Caused by:</h3><pre>java.lang.IllegalStateException: Form too large37015332>10485760
at org.mortbay.jetty.Request.extractParameters(Request.java:1561)
at org.mortbay.jetty.Request.getParameter(Request.java:859)
at azkaban.webapp.servlet.AbstractAzkabanServlet.hasParam(AbstractAzkabanServlet.java:109)
at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.getSessionFromRequest(LoginAbstractAzkabanServlet.java:99)
at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doPost(LoginAbstractAzkabanServlet.java:134)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:713)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
</pre>
<hr /><i><small>Powered by Jetty://</small></i><br/>

Richard

unread,

Apr 4, 2013, 8:46:25 PM4/4/13

to azkab...@googlegroups.com

Barring any unexpected changes, ETA is end of next week.

And, yes, we've gotten commandline to work for uploads. However, we could be cleaner in our API though.

We curl multiform, but you'll need to authenticate first... either by passing username/password with your upload, or by pre-authenticating and using an expiring session id.

We're in the process of writing a lot of docs at the moment. Our new package release should coincide with more how-to documentation.

On Wednesday, April 3, 2013 9:37:23 PM UTC-7, S V wrote:

Thanks - is there an ETA for the release?
And, will there be an official API for job upload, or do you have a curl command that you know works?

Dan Streit

unread,

Apr 16, 2013, 11:47:45 AM4/16/13

to azkab...@googlegroups.com

Richard, I noticed the 2.1 branch was merged into master. Are we close to getting a new release?

Dan

Richard Park

unread,

Apr 16, 2013, 12:44:59 PM4/16/13

to azkab...@googlegroups.com

Yes, we're running it now at LinkedIn. There were a few kinks we had to work out, mainly due to a pretty large refactoring of the execution code. It looks nearly ready.

We are running a bit behind on documentation. We're not finding the Github wiki solution ideal so we're considering putting up our own Azkaban site for docs.

--
You received this message because you are subscribed to the Google Groups "azkaban" group.
To unsubscribe from this group and stop receiving emails from it, send an email to azkaban-dev...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Reply all

Reply to author

Forward