Troubleshooting SOAP 11.x Access

143 views
Skip to first unread message

Leonardo Canessa

unread,
Jul 7, 2016, 8:54:30 AM7/7/16
to sakai-dev
All,

I am attempting to add a bunch of users using a Python SOAP script onto our 11.x dev server.

The script works fine on our 2.9 instance (with the appropriate URLs), but I get the following error when using it on 11.x:
lcanessa1 ~/repos/scripts/webservice_scripts (master) $ python createUsers.py
Login_url: https://isidore-dev.udayton.edu/sakai-ws/soap/login?wsdl
Traceback (most recent call last):
File "createUsers.py", line 29, in <module>
sessionId = loginProxy.login(username, password)
File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 470, in __call__
return self.__r_call(*args, **kw)
File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 492, in __r_call
self.__hd, self.__ma)
File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 363, in __call
config = self.config)
File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 263, in call
raise HTTPError(code, msg)
SOAPpy.Errors.HTTPError: <HTTPError 301 Moved Permanently>

Here's the code snippet (the error occurs on the last line):
login_url = server_url + "/sakai-ws/soap/login?wsdl"
script_url = server_url + "/sakai-ws/soap/sakai?wsdl"
 
loginProxy = WSDL.SOAPProxy(login_url)
scriptProxy = WSDL.SOAPProxy(script_url)
 
if debug:
print "Login_url: {}".format(login_url)
 
sessionId = loginProxy.login(username, password)

The full script is available here, http://apaste.info/Fc9 , (will expire in 30 days).

Here is the webservices sakai.property section:
webservices.allowlogin = true
webservices.allow = *
webservices.log-allowed = true
webservices.log-denied = true

I believe there is some sort of configuration issue that is missing. When you browse to 

The only thing I've tried so far is set webservices.allow to the glorious *.

Any other suggestions?

Thank you,
Leonardo Canessa
Web Developer - E-Learning

Leonardo Canessa

unread,
Jul 8, 2016, 11:20:23 AM7/8/16
to sakai-dev
Resolution:
webservices.allow = .+

Leonardo Canessa
Web Developer - E-Learning

Leonardo Canessa

unread,
Jul 8, 2016, 12:38:09 PM7/8/16
to sakai-dev
False alarm. I still receive the HTTP 301 error from the python script. Though, am now able to browse to /sakai-ws/soap/login?wsdl.

Leonardo Canessa
Web Developer - E-Learning

Sam Ottenhoff

unread,
Jul 8, 2016, 12:45:19 PM7/8/16
to Leonardo Canessa, sakai-dev
Your Tomcat isn't aware that there is an HTTPS load balancer in front of it:


Use X-Forwarded-Proto to let Tomcat know that HTTPS is the correct protocol.

On Thu, Jul 7, 2016 at 8:54 AM, Leonardo Canessa <lcan...@udayton.edu> wrote:

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sakai-dev+...@apereo.org.
To post to this group, send email to saka...@apereo.org.
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

Leonardo Canessa

unread,
Jul 8, 2016, 2:24:25 PM7/8/16
to Sam Ottenhoff, sakai-dev
Specifying the server directly results in the following error:
Login_url: http://sakai11-dev.<host>.edu:8080/sakai-ws/soap/login?wsdl
Traceback (most recent call last):
  File "createUsers.py", line 29, in <module>
    sessionId = loginProxy.login(username, password)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 470, in __call__
    return self.__r_call(*args, **kw)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 492, in __r_call
    self.__hd, self.__ma)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 406, in __call
    raise p
SOAPpy.Types.faultType: <Fault soap:Client: Message part login was not recognized.  (Does it exist in service WSDL?)>


Leonardo Canessa
Web Developer - E-Learning

Leonardo Canessa

unread,
Jul 12, 2016, 2:09:18 PM7/12/16
to Sam Ottenhoff, sakai-dev, Sakai Production
The setup is an haproxy in front of various tomcat instances. The haproxy config is as follows:
frontend https-backup
  bind            X.Y.Z.212:443 ssl crt /etc/haproxy/certs/ no-sslv3
  reqadd          X-Forwarded-Proto:\ https
  capture         request header X-Forwarded-For len 64
  capture         request header User-agent len 256
  capture         request header Cookie len 64
  capture         request header Accept-Language len 64
  use_backend     isidore-dev if { ssl_fc_sni isidore-dev.<host>.edu }
  default_backend catch-all

backend isidore-dev
  http-request set-header X-Forwarded-Port %[dst_port]
  http-request add-header X-Forwarded-Proto https if { ssl_fc }
  option       httpchk HEAD /portal HTTP/1.1\r\nHost:localhost
  cookie       SERVERID insert indirect nocache
  balance      leastconn
   server sakai11-dev.<host>.edu <IP>:8080 cookie 1 check inter 5000

The following was added to the Tomcat config (within the <Engine> tag of conf/server.xml):
 <Valve className="org.apache.catalina.valves.RemoteIpValve"
               internalProxies="X.Y.Z/24"
               remoteIpHeader="x-forwarded-for"
               remoteIpProxiesHeader="x-forwarded-by"
               protocolHeader="x-forwarded-proto" />

An older format of the internalProxies was also attempted> X\.Y\.Z\.

With these configuration's the HTTP 301 error was still received . Bypassing haproxy and connecting directly to the server results in the "part login not recognized error" (as shown in my previous email).

I attempted to run the same script against the nightly Master server and received the same error, part login not recognized:

lcanessa1 ~/repos/scripts/webservice_scripts (master) $ python createUsers.py 
Traceback (most recent call last):
  File "createUsers.py", line 33, in <module>
    sessionId = loginProxy.login(username, password)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 470, in __call__
    return self.__r_call(*args, **kw)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 492, in __r_call
    self.__hd, self.__ma)
  File "/usr/lib/pymodules/python2.7/SOAPpy/Client.py", line 406, in __call
    raise p
SOAPpy.Types.faultType: <Fault soap:Client: Message part login was not recognized.  (Does it exist in service WSDL?)>

Is this a bug in SOAPpy? Or is this a bug in Sakai?


Leonardo Canessa
Web Developer - E-Learning

Leonardo Canessa

unread,
Jul 15, 2016, 12:37:30 PM7/15/16
to Sam Ottenhoff, sakai-dev, Sakai Production
The "Message part login was not recognized" error was resolved by switching to the zeep python library.

Leonardo Canessa
Web Developer - E-Learning

Earle Nietzel

unread,
Jul 15, 2016, 2:59:34 PM7/15/16
to Leonardo Canessa, Sam Ottenhoff, sakai-dev, Sakai Production
In case your interested here is a sample script using scala!

Leonardo Canessa

unread,
Jul 15, 2016, 3:11:48 PM7/15/16
to Earle Nietzel, Sam Ottenhoff, sakai-dev, Sakai Production
Thanks for sharing!

Here's the python3 script I made and used:

Leonardo Canessa
Web Developer - E-Learning

Neal Caidin

unread,
Jul 15, 2016, 4:09:11 PM7/15/16
to Leonardo Canessa, Earle Nietzel, Sam Ottenhoff, sakai-dev, Sakai Production
Cool. What's zeep?

Python SOAP client?

Leonardo Canessa

unread,
Jul 15, 2016, 4:21:35 PM7/15/16
to Neal Caidin, Earle Nietzel, Sam Ottenhoff, sakai-dev, Sakai Production
Yep! That's right!

Pip is a "package manager" for python libraries.

Here are instructions on how to install pip on windows:

If you are on any decent Linux, it should all ready be installed.

Once pip is installed, you can use it to install zeep.
pip install zeep

Note: That you will need to be running the python3 version of pip (typically pip3) to run the createUsers.py script I posted earlier.

Google is your friend.

Leonardo Canessa
Web Developer - E-Learning

Reply all
Reply to author
Forward
0 new messages