SOAP frustrations
Andrew Dalke
I've been working with one of my clients trying to get a SOAP
interface going for a project. I've had a rather frustrating time
trying to get it to work, and am using this as a chance to vent.
The first problem was choosing a SOAP client&server library. The
choices were:
- /F's soaplib
- ZSI
- SOAP.py
(I didn't look at SOAPy because we aren't using WSDL.)
We excluded soaplib because we wanted Python/Perl/Java interoperability
and http://www.pythonware.com/products/soap/profile.htm suggests
problems with that, with unicode, with namespaces, and a few more.
Various people recommended ZSI. I tried the most recent distribution
(ZSI1.3RC2 from May 14, 2002) and the tests failed, since there's a
bug in ZSI/TCapache.py line 44. The distribution has
print >>sw, "<%s%s%s>" % n, attrtext or '', tstr
and should be
print >>sw, "<%s%s%s>" % (n, attrtext or '', tstr)
That makes me skittish about the rest of the code.
(NOTE TO ALL -- if you have self-tests, please make sure they work!)
In addition, I'm rather confused about terms like "actor" which
appear in the documentation, and the coding style is kinda unusual
(method names starting with a capital letter throws me off, as does
indentation like
def __init__(self, out,
envelope=1, encoding=SOAP.ENC, header=None, nsdict=None, **kw):
instead of
def __init__(self, out,
envelope=1, encoding=SOAP.ENC, header=None,
nsdict=None, **kw):
or
def __init__(self, out, envelope=1, encoding=SOAP.ENC, header=None,
nsdict=None, **kw):
and lack of indentation like
if not self.envelope: return
)
Nothing serious, but it made it hard to figure out the code.
We ended up using SOAP.py, since it was easy to set up and get
working for our experimental project. OTOH, I just spent most this
day trying to make it work as a .cgi instead of running as its own
server. (We already have Apache running, so having two servers seems
like overkill.) As far as I can tell, you need to roll your own
for that, so I did. By comparison, the docs for ZSI say it can be
done with
from ZSI import dispatch
dispatch.AsCGI()
and in the SOAP book by Snell/Tidwell/Kulchenko, published by O'Reilly,
the SOAP::Lite equivalent is also simple.
use SOAP::Transport::HTTP;
SOAP::Transport::HTTP::CGI ->
dispatch_to('Hello::(?:sayHello)') -> handle;
In addition, SOAP.py has similar coding style pecularities, and there's
some bug that comes up in Python 2.2 because of changes in Python's
type/class system. (We're using Python 2.1 so it isn't yet a problem.
I also haven't tested ZSI to see if this is a problem.)
Finally, it seems that neither SOAP.py nor ZSI are actively developed.
There hasn't been a recent release of either package, and the CVS logs
show little change.
All in all, it's rather discouraging for me to work with SOAP in
Python, and that's given a rather non-trivial amount of experience
in doing Python. I would think others have worse problems.
It's such that the more CS-y people at my client's site believe that
web services in Python are still very immature and want to switch the
project over to Java and WebLogic(!). (I did point out that developing
the library I did was still less time than setting that up, but there
are other factors in the mix as well.)
What I don't get is, I figure there's enough people doing web services/
SOAP with Python that there shouldn't be these problems. Why then am
I so frustrated with it?
Andrew Dalke
da...@dalkescientific.com
Christopher Browne
> What I don't get is, I figure there's enough people doing web services/
> SOAP with Python that there shouldn't be these problems. Why then am
> I so frustrated with it?
The problem is that every single one of the implementations was
sponsored by a company that has since gone out of business. (Not
because their implementation was bad, rather the vagaries of the tech
market these days...)
ZSI looked promising to become the implementation "of choice," but
then Zolera went out of business. SOAP.py is probably the best option
around, and the fact that it's not necessarily ideal is, well,
unfortunate, but nonetheless the case...
--
(concatenate 'string "cbbrowne" "@ntlug.org")
http://www3.sympatico.ca/cbbrowne/soap.html
There are no "civil aviation for dummies" books out there and most of
you would probably be scared and spend a lot of your time looking up
if there was one. :-) -- Jordan Hubbard in c.u.b.f.m
Martin v. Loewis
> What I don't get is, I figure there's enough people doing web services/
> SOAP with Python that there shouldn't be these problems.
Why do you think this is the case?
Regards,
Martin
Culley Angus
and I have also had several frustrating experiances with it.
But for me it was the only logical choice, once we made the decision
that python was the language of choice (for many other reasons than
it's SOAP support).
Although it is no longer supported by a commercial concern, the state
of the SOAP.py code is far from immature.
I am currently running it in a python client application, and it works
almost seamlessly with the latest release version of the Apache Axis
framework, which we are running on the server.
We havn't had to change any major functional area of it at all (we did
have to update it a little for python 2.2.1).
It did take a while to figure out how it worked, and we are using a
seperate wrapper module to access it, but it seems to work just fine.
All the best.
A.M. Kuchling
> What I don't get is, I figure there's enough people doing web services/
> SOAP with Python that there shouldn't be these problems. Why then am
> I so frustrated with it?
Maybe most people are using XML-RPC instead of SOAP. SOAP seems so
complicated, and entangled with yet more complicated standards such as
WSDL and UDDI, that it doesn't seem worth the trouble to me; XML-RPC
for the simpler stuff, and REST-style HTTP for the more elaborate
stuff, seems a more practical bet to me. (Admittedly, that doesn't
help you if the rest of the project has committed to SOAP for whatever
reason.)
--amk
Graham Dumpleton
>
> All in all, it's rather discouraging for me to work with SOAP in
> Python, and that's given a rather non-trivial amount of experience
> in doing Python. I would think others have worse problems.
>
> It's such that the more CS-y people at my client's site believe that
> web services in Python are still very immature and want to switch the
> project over to Java and WebLogic(!). (I did point out that developing
> the library I did was still less time than setting that up, but there
> are other factors in the mix as well.)
>
> What I don't get is, I figure there's enough people doing web services/
> SOAP with Python that there shouldn't be these problems. Why then am
> I so frustrated with it?
A few strange questions which you might answer for me. Would be interested
in feedback from others as well.
What specifically do you believe you gain from using Python as opposed to
another language for implementing a web service where SOAP is specifically
used to interface to it? Are you specifically after a solution which is
pure Python or would you accept a system which allows you to use Python
to write your web services but where the bulk of the system is actually
C++? Do you want a system whereby you are dealing directly with SOAP
concepts, or is a more generic service API which might be callable from
XML-RPC as well as SOAP sound better?
I ask for the purposes of trying to determine how people feel about hybrid
systems using Python and C++. Yes they may not be as simple to set up in
the first place because of the extra step of having to compile C++ code,
but if the end result is still just as easy to use for those who want to
write only in Python, is the reliance on C++ a deterent?
The problem I have at the moment is that I have a system which is C++ at
its core and provides a means of building standalone or distributed systems
whereby services within the system communicate using an internal messaging
system. Those same services however, can also be accessed using XML-RPC
using a RPC over HTTP gateway that plugs easily and transparently into
the system.
With the way the system works, you can use C++ exclusively, Python exclusively
or a mix whereby some processes use Python and some use C++. In all cases
where Python is used, it is still not pure Python but a wrapper around the
C++ core.
At the moment, there is actually a SOAP gateway as well, but it relies on
using either ZSI or SOAP.py. From my experience in using those packages I
can relate to your frustrations as they don't always work how they should.
For those reasons I don't specifically promote there use and suggest people
use the XML-RPC interface if they need remote access in that way.
Note that whether XML-RPC or SOAP is used, in writing your service, you
are quite divorced from those interfaces and you write to a more generic
messaging system API. This obviously means that you can't explicitly
manipulate SOAP messages and you may not have access to all SOAP features.
It does however possibly mean that the whole thing is much easier to use
as something else worries about all the hard parts for you.
What I have been thinking about lately is using an existing C++ library
for SOAP, rather than a Python one, to write a RPC over HTTP gateway into
this system. I know this will most likely be more complicated to get
working in the first instance than when Python was used, but does achieve
the same end. That is, you can still write your services in Python, it is
just that the bit handling the SOAP requests is written in C++ and in
between the underlying messaging system handles the translation.
Such a system obviously means a bigger commitment than wacking together a
quick once off SOAP CGI script for one task. Such a system has the potential
to dramatically alter how you might otherwise implement a system. For a
real simple system it might be viewed as overkill, but for a large system,
it certainly has the range of functionality to make the task a lot easier.
Enough of the ramble. My final question and one for which I wouldn't be
surprised if I got no answers, is are hybrid frameworks for building more
general purpose service based systems, where web access is a useful extra,
of interest to Python programmers?
As much as it might have to offer in certain areas, so far, it seems to be
a hard slog in getting Python programmers interested.
Derek Thomson
> In article <3DAC6F04...@mindspring.com>, Andrew Dalke wrote:
>
>>What I don't get is, I figure there's enough people doing web services/
>>SOAP with Python that there shouldn't be these problems. Why then am
>>I so frustrated with it?
>
>
> Maybe most people are using XML-RPC instead of SOAP.
Yes, I found XML-RPC to be quite easy and sensible.
If you want to try something a little odd, check out the xml-rpc branch
of the Fnorb CVS tree on http://sourceforge.net/projects/fnorb.
You can expose your Python objects as CORBA objects, or XML-RPC objects,
or even both at the same time, without changing one line of code. You
just have to register them with differentloy configured object adaptors.
If enough people say, "this is nice", I may merge it into the trunk one
day. I only really did it to prove a point to myself, and I'm not sure
how popular it would be.
--
D.
Derek Thomson
>
> It's such that the more CS-y people at my client's site believe that
> web services in Python are still very immature and want to switch the
> project over to Java and WebLogic(!). (I did point out that developing
> the library I did was still less time than setting that up, but there
> are other factors in the mix as well.)
I think most CS-y people believe that web-services themselves are very
immature, and are likely to remain so for a while yet.
--
D.
Graham Dumpleton
As far as the Internet goes I would probably agree on that point. After all,
there aren't exactly many (any?) web services of any great significance which
are generally available over the Internet through only a SOAP or XML-RPC
interface. Where such SOAP or XML-RPC interfaces exist, they in general
merely add something on top of what is an existing browser based service.
The problem in the Internet arena is that you are asking too much in putting
up a web service on the Internet, publishing your WSDL and expecting people
to write their own clients. Yes it may work for very simple stuff like Google
searching or stock quote services, but not for anything too much more
complicated than that.
Where web services make more sense at this point is on company Intranets
where the application developers can also build and deploy the clients which
might make use of such systems in a meaningful way. Even in this setting I
still don't see services relying purely on XML-RPC or SOAP type protocols
for communications. What is a better mix is to use more traditional frameworks
for interprocess communications and/or messaging and have these web based
protocols more as a convenience for accessing a restricted set of interfaces
of a system as a whole.
In such an approach, having such interfaces can be especially good as providing
a way of monitoring or controlling the application. Even using such interfaces
merely for the task of getting information out of an application during
debugging and testing or even after it is deployed, can be enormously useful
and save a lot of time and effort.
The trick of course in all of this, is having a framework where providing
such interfaces isn't an undue burden. Preferably it shouldn't require any
extra work at all over the minimum you have to do to get your work done in
the first place. :-)
Derek Thomson
Graham Dumpleton wrote:
> Derek Thomson <de...@wedgetail.com> wrote in message news:<tS7r9.452$i84....@news.optus.net.au>...
>
>>Andrew Dalke wrote:
>>
>>>It's such that the more CS-y people at my client's site believe that
>>>web services in Python are still very immature and want to switch the
>>>project over to Java and WebLogic(!). (I did point out that developing
>>>the library I did was still less time than setting that up, but there
>>>are other factors in the mix as well.)
>>
>>I think most CS-y people believe that web-services themselves are very
>>immature, and are likely to remain so for a while yet.
>
>
> As far as the Internet goes I would probably agree on that point. After all,
> there aren't exactly many (any?) web services of any great significance which
> are generally available over the Internet through only a SOAP or XML-RPC
> interface. Where such SOAP or XML-RPC interfaces exist, they in general
> merely add something on top of what is an existing browser based service.
Yes, we've been doing this kind of thing via "web-scraping" for years.
>
> The problem in the Internet arena is that you are asking too much in putting
> up a web service on the Internet, publishing your WSDL and expecting people
> to write their own clients.
Why can't people just distribute their own clients for their services?
Ah, because then you wouldn't need "web services" at all ;)
> Yes it may work for very simple stuff like Google
> searching or stock quote services, but not for anything too much more
> complicated than that.
There are other issues. How do you do distributed transactions between
objects on different servers? Or asynchronous events? My basic problem
is that SOAP is just re-inventing the wheel, because DCOM failed to
scale (and now seems to have been swept under the carpet). I already
have CORBA for all this stuff. Why do I need to wait for SOAP to catch up?
I also have a problem with the basic architecture. This whole "no object
references" thing. How then, is it OO in any way? What does the 'O' in
SOAP stand for? How can I implement a method that creates a new object
on the server, and then returns a pointer/reference to that new object?
How do I pass object references between clients and/or servers?
I got around this with XML-RPC on Fnorb by implementing my own object
reference structure analogous to CORBA object references on top. Which
is fine as XML-RPC makes no claim to OO-ness. SOAP has the letter 'O' in
it, and yet it not only doesn't have object references, but by my
reading explicity forbids using them in an way.
How then, can SOAP ever be used to create a non-trivial application,
where objects are being created and passed about all the time?
And, I don't see how distributed transactions can ever work, given that
you need to be able to say "here's a bunch of objects that are part of
this transaction". If you can't refer to objects, just "services", it
can't be done, as far as I can see.
>
> Where web services make more sense at this point is on company Intranets
> where the application developers can also build and deploy the clients which
> might make use of such systems in a meaningful way. Even in this setting I
> still don't see services relying purely on XML-RPC or SOAP type protocols
> for communications. What is a better mix is to use more traditional frameworks
> for interprocess communications and/or messaging and have these web based
> protocols more as a convenience for accessing a restricted set of interfaces
> of a system as a whole.
Yes, perhaps. I can only see a non-hype niche for it where people, for
political reasons, want a non-binary protocol flowing across the
network. And when they are prepared to pay the bandwidth cost, and the
large processing cost at each end. (every integer must be converted to a
string and back again, for example).
I think that would be misguided, but I can see people just liking it
regardless.
>
> In such an approach, having such interfaces can be especially good as providing
> a way of monitoring or controlling the application. Even using such interfaces
> merely for the task of getting information out of an application during
> debugging and testing or even after it is deployed, can be enormously useful
> and save a lot of time and effort.
Really? I don't find XML to be particularly readable, especially with
SOAP, where the tag density is very high. Try looking at some XML for a
complex structure that is correct, and then one that is bad. Can you
really tell? You'll need some sort of dumper/visual tool to help, and I
can easily create something like that for any binary protocol.
Anyway, as my XML-RPC experiment with Fnorb neatly demonstrates, I *can*
run CORBA over XML, if I don't mind the overhead. And I get a decent
object model :)
>
> The trick of course in all of this, is having a framework where providing
> such interfaces isn't an undue burden. Preferably it shouldn't require any
> extra work at all over the minimum you have to do to get your work done in
> the first place. :-)
Yes.
--
D.
Gustavo Niemeyer
> In addition, SOAP.py has similar coding style pecularities, and there's
> some bug that comes up in Python 2.2 because of changes in Python's
> type/class system. (We're using Python 2.1 so it isn't yet a problem.
> I also haven't tested ZSI to see if this is a problem.)
>
> Finally, it seems that neither SOAP.py nor ZSI are actively developed.
> There hasn't been a recent release of either package, and the CVS logs
> show little change.
I've had the same problems when looking for a library some time ago. I
ended up forking ZSI and fixing whatever I found interesting for my
usage (free software is beautiful, isn't it?). The result is at
http://moin.conectiva.com.br/NINZ, and is actually being used for some
internal projects in the company.
> It's such that the more CS-y people at my client's site believe that
> web services in Python are still very immature and
[...]
They're right.
--
Gustavo Niemeyer
[ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]
Christopher Browne
> Derek Thomson <de...@wedgetail.com> wrote in message news:<tS7r9.452$i84....@news.optus.net.au>...
>> Andrew Dalke wrote:
>> >
>> > It's such that the more CS-y people at my client's site believe that
>> > web services in Python are still very immature and want to switch the
>> > project over to Java and WebLogic(!). (I did point out that developing
>> > the library I did was still less time than setting that up, but there
>> > are other factors in the mix as well.)
>>
>> I think most CS-y people believe that web-services themselves are very
>> immature, and are likely to remain so for a while yet.
>
> As far as the Internet goes I would probably agree on that point. After all,
> there aren't exactly many (any?) web services of any great significance which
> are generally available over the Internet through only a SOAP or XML-RPC
> interface. Where such SOAP or XML-RPC interfaces exist, they in general
> merely add something on top of what is an existing browser based service.
The /eventual/ intent is for SOAP to do something much more ambitious
than XML-RPC, namely to have a single interface that provides /two/
forms of functionality:
a) An RPC scheme (not unlike XML-RPC or CORBA), and
b) A way of transferring substantial XML documents around in a
'verifiable' way.
The latter is pointed to by the hordes of 'layered standards' that
take over where WSDL leaves off, where you can do things like
attaching digital signatures and encrypting portions of documents.
> The problem in the Internet arena is that you are asking too much in
> putting up a web service on the Internet, publishing your WSDL and
> expecting people to write their own clients. Yes it may work for
> very simple stuff like Google searching or stock quote services, but
> not for anything too much more complicated than that.
Indeed. There's a lot that people don't "get" about it...
> Where web services make more sense at this point is on company
> Intranets where the application developers can also build and deploy
> the clients which might make use of such systems in a meaningful
> way. Even in this setting I still don't see services relying purely
> on XML-RPC or SOAP type protocols for communications. What is a
> better mix is to use more traditional frameworks for interprocess
> communications and/or messaging and have these web based protocols
> more as a convenience for accessing a restricted set of interfaces
> of a system as a whole.
The place where it's liable to "come into its own" is when BizTalk
interfaces start popping up; they'll be transferring around big XML
documents to describe accounting transactions, and you might as well
use SOAP to transfer around something that is a big XML document. The
only alternative with any merit would be a message queueing system
like MQSeries (or M$FT's clone, called MSMQ)...
The interesting thing is that at /that/ point, you would no longer
want to treat SOAP as an RPC scheme, where the use of XML is being
hidden under the table. You want to have the contents of the message
provided as an XML message; perhaps as a 'raw' stream of bytes, or as
a parsed XML document object that you can "walk" through.
Therein lies a /critical/ problem in the design of the existing Python
SOAP implementations; they take the approach that SOAP is 'just RPC,'
so that the XML is trash to be thrown away. Nothing could be further
from the truth, as in many cases, if you throw away /any/ of the data
in the message, you may have just trashed the message.
Perl's SOAP::Lite takes, to my mind, the /right/ approach to this; it
creates what the implementors call a "SOM" object which is essentially
a 'parse tree' of the XML message.
$foo = some_soap_call(arguments);
print $foo->result(), "\n"; # Prints the result
print $foo, "\n"; # $foo is an object containing the
# parse tree...
The problem with many of the SOAP implementations (including those for
Python) is that the call to the result() method is implicit, and you
/never/ get a chance to intercept the XML so as to be able to do more
sophisticated things with it, such as looking for encrypted portions
and decrypting them...
--
(reverse (concatenate 'string "moc.enworbbc@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/sap.html
Rules of the Evil Overlord #126. "Rather than having only one secret
escape pod, which the hero can easily spot and follow, I'll
simultaneously launch a few dozen decoys to throw him off track."
<http://www.eviloverlord.com/>
Graham Dumpleton
> Hi Graham,
>
>
> > The problem in the Internet arena is that you are asking too much in putting
> > up a web service on the Internet, publishing your WSDL and expecting people
> > to write their own clients.
>
> Why can't people just distribute their own clients for their services?
> Ah, because then you wouldn't need "web services" at all ;)
Agree. In distributing your own clients, you get back to the problem of having
to identify what platform the client has to run on. At least in an Intranet
the client desktop is usually mostly standardised and it is possible to get
away with one or perhaps two client implementations. On the Internet though
you are always going to dissappoint someone because your specialised client
doesn't work for them. Okay, you could use Java as you client, but at that
point you may as well use something like RMI.
> I also have a problem with the basic architecture. This whole "no object
> references" thing. How then, is it OO in any way? What does the 'O' in
> SOAP stand for? How can I implement a method that creates a new object
> on the server, and then returns a pointer/reference to that new object?
> How do I pass object references between clients and/or servers?
>
> I got around this with XML-RPC on Fnorb by implementing my own object
> reference structure analogous to CORBA object references on top. Which
> is fine as XML-RPC makes no claim to OO-ness. SOAP has the letter 'O' in
> it, and yet it not only doesn't have object references, but by my
> reading explicity forbids using them in an way.
I agree that the object references thing is an issue. In the main though
I would suggest this is a deficiency of the bulk of web service toolkits
available. The problem with many toolkits is that they only support the
concept of a single fixed URL as the gateway into the application. To
dynamically define new URLs which can be accessed is nigh on impossible.
I identify this as a problem, because a simple way of having an object
reference is for each to have a unique URL. Some people get around it
by passing an object reference as a parameter to every method they
call which relates to an object, but I see this as a clumsy way of doing
it. First off you need the magic parameter and second, if you aren't
careful, your method namespace potentially gets polluted with large numbers
of unrelated methods for different object types.
Thus what I do is when you want to create a new server side object from a
remote client, you call a create() method which returns a new URL. The client
then uses this new URL to talk to the object. When the client no longer
wants the object, it can explicitly call a destroy() method on it, or just
leave it, and an idle timer on the server side will reclaim it after a period
of not being used.
How does this compare with what you do with Fnorb? Do you use a separate
URL or do you pass an object reference as a parameter? Or do you do it some
other way?
> How then, can SOAP ever be used to create a non-trivial application,
> where objects are being created and passed about all the time?
>
> And, I don't see how distributed transactions can ever work, given that
> you need to be able to say "here's a bunch of objects that are part of
> this transaction". If you can't refer to objects, just "services", it
> can't be done, as far as I can see.
There is also the problem of reliability and actually detecting when calls
fail and where, whether you have to undo the transaction and all the
associated mechanisms. Such things would all have to be built from scratch,
even if you had a way of referring to objects.
To my mind another feature which is lacking in a pure XML-RPC or SOAP
system is the idea of presence. In a complicated system it can be extremely
useful to subscribe to the existance of a service and be able to be notified
when such a service comes into existance or dissappears. Another useful
feature is being able to subscribe to values published by a service. In
both cases it requires an ability for a service to call back to a client
at any time. The idea that a client should run its own little web service
to allow this to be done just isn't practical.
> > In such an approach, having such interfaces can be especially good as providing
> > a way of monitoring or controlling the application. Even using such interfaces
> > merely for the task of getting information out of an application during
> > debugging and testing or even after it is deployed, can be enormously useful
> > and save a lot of time and effort.
>
> Really? I don't find XML to be particularly readable, especially with
> SOAP, where the tag density is very high. Try looking at some XML for a
> complex structure that is correct, and then one that is bad. Can you
> really tell? You'll need some sort of dumper/visual tool to help, and I
> can easily create something like that for any binary protocol.
I am not suggesting one would look at the XML directly. You definitely
need to use something to present the data in a more palatable form. The
easiest way of doing this is to construct a web page which uses a Perl
or Python SOAP or XML-RPC client to call into the backend service to get
the data and then render it as HTML to the browser.
Yes you could also have some specialised GUI client to achieve the same
thing, but you then get back to having to deploy it to all desktops. At
least if such things are rendered as HTML pages delivered up by a web server
you only have to deploy it to one spot and any browser can be used. By
using XML-RPC or SOAP in this case, you also still get a clean seperation
between backend model and presentation.
Back tracking a bit, I also don't think this is the answer for everything.
All my own low level debugging tools are stanadlone GUIs which talk the
messaging system protocol. For what they are doing this is preferred as
they get access to the presence functionality along with ability to
subscribe to values published by services.
> > The trick of course in all of this, is having a framework where providing
> > such interfaces isn't an undue burden. Preferably it shouldn't require any
> > extra work at all over the minimum you have to do to get your work done in
> > the first place. :-)
>
> Yes.
And it is unfortunately where many toolkits are very lacking.
Martin v. Loewis
> Okay, you could use Java as you client, but at that point you may as
> well use something like RMI.
Or you use Python.
Regards,
Martin
Graham Dumpleton
The issue is that you don't want to have to make a user explicitly download
some package to get the client to work. This would therefore rule out the use
of CPython.
As for JPython, what is involved in having Python run inside a Java applet
to use it in that way? Is this a trivial exercise in that required stuff
will be downloaded automatically, or does it still rely on a user having to
download stuff explicitly? Or is it simply not possible because of the
structures of Java applets?
What is required is a solution which people can use without thinking. Except
for Microsofts latest attempts to sabotage Java on the desktop, one could at
least be partly comfortable with the notion that the browser has Java support
and thus why for maximum audience that would have been the way to go if you
aren't going to use a HTML based browser application.
Graham Dumpleton
Oh, forgot to mention one thing.
If one were to use JPython in a Java applet, we in part get back to the
same issue as this post was originally about. That is, the inadequacy of
SOAP implementations.
This is because if you were to use Python, you still have to talk to the
back end service somehow. Your choices would be to roll your own system
with your own protocol or using Python pickling to transfer data around,
or use an XML-RPC or SOAP implementation.
Thus we haven't really got anywhere. At least RMI and associated protocols
in Java present a more complete and robust base to start with.
The argument thus still stands that Python doesn't provide a really mature
way of implementing web services, which was where the original poster was
getting frustrated.
Paul Boddie
>
> Therein lies a /critical/ problem in the design of the existing Python
> SOAP implementations; they take the approach that SOAP is 'just RPC,'
> so that the XML is trash to be thrown away. Nothing could be further
> from the truth, as in many cases, if you throw away /any/ of the data
> in the message, you may have just trashed the message.
I agree. One of the biggest issues is that people think, "SOAP? Ah,
that's just RPC over the Internet where it doesn't get more
complicated than fetching stock quotes." [*] And they don't give it
any further thought. Yet in the situations where SOAP is most
interesting to use, it's as an XML document that one wants to see the
information.
I'd go further than that and state that in many situations, "end to
end" XML is extremely interesting, and that I can't understand why
people have to invent their own object model for every application
they write, especially when the object model they've invented doesn't
provide any more features than the DOM does, doesn't expose itself to
querying in the way that various XML technologies permit when used in
conjunction with the DOM, and especially when they're only packaging
data up as objects for purely obsessive purposes: "I'm not using
thousands of objects in my application - something must be wrong!"
> Perl's SOAP::Lite takes, to my mind, the /right/ approach to this; it
> creates what the implementors call a "SOM" object which is essentially
> a 'parse tree' of the XML message.
>
> $foo = some_soap_call(arguments);
> print $foo->result(), "\n"; # Prints the result
> print $foo, "\n"; # $foo is an object containing the
> # parse tree...
Well, unless "SOM" is interoperable with the DOM, I don't think it's
entirely the right approach. I've been using various Java APIs to do
SOAP stuff, and someone at Sun clearly decided that instead of
exposing a normal DOM interface to SOAP messages, they would invent
another API which is very similar but incompatible. If it weren't for
Jaxen (the XPath package for Java) stepping in to save the day, I'd be
at an even higher point of irritation with JAXM than I already am.
> The problem with many of the SOAP implementations (including those for
> Python) is that the call to the result() method is implicit, and you
> /never/ get a chance to intercept the XML so as to be able to do more
> sophisticated things with it, such as looking for encrypted portions
> and decrypting them...
My unfashionable opinion is that if you want to do RPC, especially if
other languages are involved, then CORBA is a mature enough technology
that probably won't be as frustrating as SOAP to use or to deploy.
Paul
[*] If a technology's usefulness can only be justified with "stock
quote" applications, then such a technology clearly isn't useful
enough to be widely adopted.
Andrew Dalke
>What I don't get is, I figure there's enough people doing web services/
>SOAP with Python that there shouldn't be these problems.
Martin v. Loewis wrote:
> Why do you think this is the case?
Because I did a project with SOAP and Python over 2 years ago,
using /F's soaplib. (Python to Python SOAP.)
Because there are at least 4 different SOAP projects, meaning
there's enough interest for several different groups to not
only develop a SOAP library but to make it public.
Because 1) SOAP has been a buzzword for years, 2) Python projects
are easy to develop, and 3) more obscure projects get developed
with Python, so I figured the likelihood was high.
Yes, I know, volunteer efforts, open source, could "scratch my
own itch", etc.
Andrew
da...@dalkescientific.com
Andrew Dalke
> A few strange questions which you might answer for me. Would be interested
> in feedback from others as well.
Sure. Some background first.
The project I'm working on uses Python to integrate about 30 other
programs into one consistent interface. On top of that is a web
application to interface to that Python code. This is all intranet
access only.
People would like to use the service to compute various properties,
and use it as a RPC from whichever client. Requests are conceptually
stateless and RPC-like, but the existing CGI interface is more
complicated than that in order to improve usability. (Eg, the results
page provides alternative ways to view and download the data that
a client wouldn't need.)
So not only would screen scraping be bad because it makes it hard
for use to change the CGI-based systems, it's more complicated because
the client would need to turn a stateful server into a stateless one.
Hence we decided to work on a new way to access the service.
The clients can be from Python, Perl, VB, C++, and perhaps a couple
other languages, so we want something where libraries already exist.
We are already running a web server and would prefer not running
another one. And yes, we are not immune to buzzwords, so SOAP comes
before XML-RPC on the list of mechanisms to consider.
> What specifically do you believe you gain from using Python as
> opposed to another language for implementing a web service where
> SOAP is specifically used to interface to it? Are you specifically
> after a solution which is pure Python or would you accept a system
> which allows you to use Python to write your web services but where
> the bulk of the system is actually C++? Do you want a system whereby
> you are dealing directly with SOAP concepts, or is a more generic
> service API which might be callable from XML-RPC as well as SOAP
> sound better?
Most of the project is written in Python (others in Perl, C, and in
domain-specific languages for each of the different components). So
a big gain is reduction of mental effort when switching between
different parts of the system.
We could also use Perl, but I am a Python advocate so have my own
biases. We don't have any existing Java code, so using that would
be rather more work.
I personally prefer pure Python because it's easier to install and
maintain. (Eg, some of the machines don't have a C compiler, but
it's easy enough to go to a machine that does during installation.)
We already use extensions written in C so this isn't a real sticking
point.
We still prefer a way to use the web services under a .cgi model,
spawned off from the web server, although integration with Apache
is not out of the questions.
I know little about "SOAP concepts." I want to do RPC. I want
a list of arguments and I want to return a simple data structure.
I asked about XML-RPC. No one said they wanted to run an XML-RPC
client if SOAP is functionally equivalent. No one mentioned any
other API besides SOAP they wanted to use. Yes, I could also
consider CORBA and DCOM, but I don't know how to layer those on
top of a web server, so setting up a new server would call for more
complicated administration.
> I ask for the purposes of trying to determine how people feel
> about hybrid systems using Python and C++.
We've already got C extensions for Python. I believe the C++
compiler is also installed on their systems, so this wouldn't be
a serious consideration.
> Enough of the ramble. My final question and one for which I wouldn't be
> surprised if I got no answers, is are hybrid frameworks for building more
> general purpose service based systems, where web access is a useful extra,
> of interest to Python programmers?
There is a Python project called Twisted, which is a general purpose
service based system. I looked at Twisted since I figured it might
have SOAP support, but it does not. Zope also fits, since it has
(as I recall) multiple ways to access its services.
> As much as it might have to offer in certain areas, so far, it
> seems to be a hard slog in getting Python programmers interested.
It may be that my needs are rather simple. I don't need a larger
framework -- I don't need multiple protocols nor real object
support. So I prefer not having to learn a complicated solution
to make things work. Thus it would be hard to convince me to
use your system.
Andrew
da...@dalkescientific.com
Martin v. Löwis
> Me:
> >What I don't get is, I figure there's enough people doing web services/
> >SOAP with Python that there shouldn't be these problems.
>
> Martin v. Loewis wrote:
> > Why do you think this is the case?
>
> Because I did a project with SOAP and Python over 2 years ago,
> using /F's soaplib. (Python to Python SOAP.)
Looking at your complaints, this is no reason to expect what you
expected:
- ZSI does not work: Why would the state of soaplib have any effect
on the state of ZSI?
- SOAP.py is written in unreadable code: Same question.
> Because there are at least 4 different SOAP projects, meaning
> there's enough interest for several different groups to not
> only develop a SOAP library but to make it public.
Maybe the reason for those people to develop a new library is that
they could not understand the old ones? Maybe the reason to publish
them was in the hope that users would contribute corrections and
improvements? Maybe neither corrections nor improvements materialized,
because nobody uses SOAP?
> Because 1) SOAP has been a buzzword for years, 2) Python projects
> are easy to develop, and 3) more obscure projects get developed
> with Python, so I figured the likelihood was high.
1) Being a buzzword means that many people experiment with it. If it
is just a buzzword (instead of being truly useful), they leave off
at the point where they realize that it does not do any good. In
turn, the code base they leave behing will be hardly working and
completely unmaintained. So something being a buzzword sounds like
a guarantee to observe what you have observed.
2) So it is easier to start your own project than to use somebody
else's code base. People get tricked into believing that things
lose their inherent complexity by using Python. For something as
complex as the Simple Object Access Protocol, Python can only help
to manage the inherent complexity, not to avoid it. Or, if you
avoid the inherent complexity, your code will be suitable just for
your specific application. You seem to have observed that as well.
3) People working on more obscure stuff often do so because of a real
need, instead just because of buzzword compliance. Their libraries
are then often more reliable, since they have been used in real
applications.
Regards,
Martin
Martin v. Loewis
> This is because if you were to use Python, you still have to talk to the
> back end service somehow. Your choices would be to roll your own system
> with your own protocol or using Python pickling to transfer data around,
> or use an XML-RPC or SOAP implementation.
Or else use CORBA. This will interwork nicely between many systems,
can be accessed in Python both through Java/Jython, and natively, and
works with many other languages as well.
Regards,
Martin
Christopher Browne
> Christopher Browne <cbbr...@acm.org> wrote in message news:<aojsqj$ms9tp$1...@ID-125932.news.dfncis.de>...
>> creates what the implementors call a "SOM" object which is essentially
>> a 'parse tree' of the XML message.
>>
>> $foo = some_soap_call(arguments);
>> print $foo->result(), "\n"; # Prints the result
>> print $foo, "\n"; # $foo is an object containing the
>> # parse tree...
>
> Well, unless "SOM" is interoperable with the DOM, I don't think it's
> entirely the right approach. I've been using various Java APIs to do
> SOAP stuff, and someone at Sun clearly decided that instead of
> exposing a normal DOM interface to SOAP messages, they would invent
> another API which is very similar but incompatible. If it weren't
> for Jaxen (the XPath package for Java) stepping in to save the day,
> I'd be at an even higher point of irritation with JAXM than I
> already am.
I'm not sure if "SOM" is interoperable with anything at all, and I
frankly don't care. It's easy enough to work with, if we consider the
evils of doing recursive tree walks in Perl to be a "stipulated
given."
The point is that it involves keeping the XML parse tree, which is the
Right Thing, as opposed to parsing it and throwing the XML data away,
which is the Wrong Thing that naive SOAP implementations do.
>> The problem with many of the SOAP implementations (including those for
>> Python) is that the call to the result() method is implicit, and you
>> /never/ get a chance to intercept the XML so as to be able to do more
>> sophisticated things with it, such as looking for encrypted portions
>> and decrypting them...
> My unfashionable opinion is that if you want to do RPC, especially if
> other languages are involved, then CORBA is a mature enough technology
> that probably won't be as frustrating as SOAP to use or to deploy.
There's one problem with CORBA: It's nastier to get it through
firewalls, because there are lots of reasonably interoperable HTTP
proxies, but IIOP proxies are much rarer.
Of course, SOAP has /no/ "call-back" approach, so its functionality
"bottoms out" rather quickly...
Note that I've books in print that demonstrate the use of both CORBA
and SOAP using Python; while "future royalties" commend the idea of my
commending interest in SOAP, I certainly prefer CORBA...
--
(concatenate 'string "aa454" "@freenet.carleton.ca")
http://cbbrowne.com/info/publications.html
"The nice thing about standards is that you have so many to choose
from. Furthermore, if you do not like any of them, you can just wait
for next year's model." -- Tanenbaum, "Computer Networks"
Christopher Browne
> I know little about "SOAP concepts." I want to do RPC. I want
> a list of arguments and I want to return a simple data structure.
> I asked about XML-RPC. No one said they wanted to run an XML-RPC
> client if SOAP is functionally equivalent. No one mentioned any
> other API besides SOAP they wanted to use. Yes, I could also
> consider CORBA and DCOM, but I don't know how to layer those on
> top of a web server, so setting up a new server would call for more
> complicated administration.
If you want to do RPC, I'd suggest that XML-RPC probably /is/ a better
option than SOAP.
- XML-RPC is much simpler than SOAP.
- You can "stack" RPC requests together into one request, with the
result that, with a little planning, it can be /fabulously/ more
efficient than SOAP.
- There are much more mature and complete implementations of XML-RPC
for Python than there are SOAP implementations.
- There more than likely never will be a "mature and complete" SOAP
implementation for Python because the express intent of SOAP is
to provide a framework for great gobs of "layered standards," many
of which likely require pretty proprietary stuff that likely won't
get provided in Python. (Think: VeriSign crypto tools that you
/must/ get from VeriSign that they only deploy in Java.)
There are decent XML-RPC implementations everywhere; they're merely
"less buzzword compliant" because the big marketing organizations
(IBM, Microsoft, VeriSign, Sun, ...) are much more interested in
getting you to pay $20K a pop for the software licenses required for
their full scale SOAP environments... (I'm making up the $20K figure.
So sue me.)
--
(concatenate 'string "aa454" "@freenet.carleton.ca")
http://cbbrowne.com/info/
You should talk to the DOCTOR.
Robert Oschler
<snip>
>
> Note that I've books in print that demonstrate the use of both CORBA
> and SOAP using Python; while "future royalties" commend the idea of my
> commending interest in SOAP, I certainly prefer CORBA...
> --
What is the best book you have out for an advanced level programmer (for
SOAP)?
thx
Paul Boddie
>
> I'm not sure if "SOM" is interoperable with anything at all, and I
> frankly don't care. It's easy enough to work with, if we consider the
> evils of doing recursive tree walks in Perl to be a "stipulated
> given."
Well, in Java, as opposed to Python (and Perl, if the object
orientation is anywhere near Python's), the interoperability part is
pretty important. Somehow, JAXM exposes its not-exactly-standard Dom4j
roots, and justice is eventually served through other packages such as
Jaxen. I'd hate to not be able to use XPath on anything DOM-like
straight out of the box, especially if the reason for such
technological obstruction were because the representation concerned
wasn't "DOM enough".
> The point is that it involves keeping the XML parse tree, which is the
> Right Thing, as opposed to parsing it and throwing the XML data away,
> which is the Wrong Thing that naive SOAP implementations do.
Agreed.
Paul
John Keeling
mar...@v.loewis.de (Martin v. Loewis) wrote in message news:<m3elapq...@mira.informatik.hu-berlin.de>...
Over the internet, you really want to use Web Services rather than
CORBA, because CORBA/IIOP uses multiple ports that have to be
explicitly allowed on the firewall ( on both the client and server
side). This is an admin and security issue. (If you are not going
through firewalls then CORBA is a great choice. CORBA can go through
firewalls using IIOP proxies .. however support is incomplete and not
easy.)
Long-term I think SOAP is the way to go, because of the industry
momentum in that direction. An option that you might consider is
linking JPython to IONA Technologies' XMLBus product. Previous
versions of XMLBus were free previews, and current versions have a low
developer cost (I think) Check XMLBus out at
http://www.iona.com/downloads/
AFAIK this should work well.
Cheers,
John
Martin v. Löwis
> Over the internet, you really want to use Web Services rather than
> CORBA, because CORBA/IIOP uses multiple ports that have to be
> explicitly allowed on the firewall ( on both the client and server
> side).
That is not true. CORBA can be easily restricted to a single port. If
you really want to, CORBA can run on port 80.
> Long-term I think SOAP is the way to go, because of the industry
> momentum in that direction.
Just because you can misuse port 80?
Regards,
Martin
Derek Thomson
John Keeling wrote:
>
> Over the internet, you really want to use Web Services rather than
> CORBA, because CORBA/IIOP uses multiple ports that have to be
> explicitly allowed on the firewall ( on both the client and server
> side). This is an admin and security issue.
Yes. It *is* a security issue. By misusing port 80 for remote procedure
calls, you are violating the contract between you and your sysadmin, and
quite likely violating your organization's (our your client's) security
policy. That port was opened to allow web pages to be served up, not to
run bits of arbitrary code on request.
If I were a sysadmin, I'd be blocking anything that looks like SOAP at
the firewall. Sysadmins I've spoken to about SOAP have reacted with
horror when I explain the nature of SOAP, and have started writing
little filters already ;) Expect a few "anti-SOAP" scripts on
sourceforge any time now.
The problem with this whole firewall issue is that it's a political and
organizational one. It *cannot* be solved technically. If you want RPC
you have to allow it through the firewall. Many developers/deployers
don't even think to find out what the admins think at the deployment
site, or even what the security policy is, and certainly don't do what
they *should* do - which is to involve the admins who are looking after
the target system.
Instead, they get all upset when the admin, who has a professional
responsibility for the security of his users' data, refuses to just open
up random ports without notice. Then the deployer views the admin as the
problem that must be "worked around".
Enter SOAP and misusing port 80 to get RPC requests throught without
needing a anyone's authority. All that will happen is the sysadmins will
be forced to filter it, and you are back to square one.
The lesson is - you can't hack around a problem like this. You have to
negotiate, and get all the people who will be impacted by your
development involved from day one. Sure, they might say, "this is
against our policy, it will never be allowed", but isn't it better to
find that out *before* starting? Then, if you want, you can work to
change the policy. Screwing around with port 80 is going to cause a lot
of ill-will. The first time a SOAP request comes through that runs some
random bit of code that causes some damage, you can fully expect to be
hung out to dry for it.
Organizations have security policies for a reason. Hacking around them
will *never* end well.
--
D.
Derek Thomson
>
>
>>>The problem with many of the SOAP implementations (including those for
>>>Python) is that the call to the result() method is implicit, and you
>>>/never/ get a chance to intercept the XML so as to be able to do more
>>>sophisticated things with it, such as looking for encrypted portions
>>>and decrypting them...
>>
>
>>My unfashionable opinion is that if you want to do RPC, especially if
>>other languages are involved, then CORBA is a mature enough technology
>>that probably won't be as frustrating as SOAP to use or to deploy.
>
>
> There's one problem with CORBA: It's nastier to get it through
> firewalls, because there are lots of reasonably interoperable HTTP
> proxies, but IIOP proxies are much rarer.
I predict it won't be easier to run SOAP through firewalls for long.
Wait until sysadmins realize you are running RPC calls ever port 80, and
filter incoming SOAP requests out.
I know forwarding thinking sysadmins already starting to do this, as
port 80 was *not* opened for RPC, and any attempt to use it as such
would violate their organization's security policies, and requires
explicit permission.
See my previous post here on why this problem cannot be solved technically.
--
D.
Christopher Browne
I certainly don't disagree with you on the notion that there's a
pretty security issue there.
But that does not contradict the notion that SOAP is somewhat easier
to work with from a "firewalling perspective" since it uses a protocol
(HTTP) for which huge numbers proxies are widely available and widely
deployed, whereas the same is /not/ true for IIOP.
(Note that I wouldn't blindly call that difference /totally/
beneficial; there's lots about HTTP that "sucks," as IIOP is
substantially more expressive and offers much more sophisticated
semantics.)
But the fact remains: You can probably get a CD from AOL that contains
an HTTP proxy, if you microwave it long enough. In contrast, the
equivalents for IIOP tend to be proprietary and may only interoperate
with a pretty lowest-common-denominator of functionality, as high end
ORB vendors are likely to offer interesting extensions. (My tongue is
in my cheek, here, but I'm not /totally/ unserious.)
--
(concatenate 'string "cbbrowne" "@ntlug.org")
http://cbbrowne.com/info/corba.html
Referring to undocumented private communications allows one to claim
virtually anything: "we discussed this idea in our working group last
year, and concluded that it was totally brain-damaged".
-- from the Symbolics Guidelines for Sending Mail
Nicolas Chauvat
> change the policy. Screwing around with port 80 is going to cause a lot
> of ill-will. The first time a SOAP request comes through that runs some
> random bit of code that causes some damage, you can fully expect to be
> hung out to dry for it.
>
> Organizations have security policies for a reason. Hacking around them
> will *never* end well.
[applause]
More reading:
http://www.prescod.net/rest/security.html
http://lists.xml.org/archives/xml-dev/200202/msg01057.html
http://mail.python.org/pipermail/xml-sig/2002-February/007183.html
http://www.prescod.net/ (see HTTP and REST)
http://www.eekim.com/distcomp/dialogmaps/webservices.html
--
Nicolas Chauvat
http://www.logilab.com - "Mais oů est donc Ornicar ?" - LOGILAB, Paris (France)
Christopher Browne
> "Christopher Browne" <cbbr...@acm.org> wrote in message
> news:aomqcm$o4u41$3...@ID-125932.news.dfncis.de...
>> and SOAP using Python; while "future royalties" commend the idea of my
>> commending interest in SOAP, I certainly prefer CORBA...
>
> (for SOAP)?
/I/ have only one; see the URL below.
I have looked at many of them, and there is /considerable/ variation
in the nature of the material they cover.
My book has the most detailed presentation (plug! plug! I am not
unbiased :-)) that I am aware of about the use of SOAP::Lite, so if
you're into Perl, which is almost certainly the best "SOAP for
scripting languages" implementation, I would /strongly/ commend it.
(Irrespective of my bias as author.) It has decent coverage of Axis
(the Apache project's Java-based imp), and says more about Python
SOAPs than any other book I am aware of. Unfortunately, in that
regard, it's arguably a little obsolete already, as it has
considerable discussion of ZSI, which is no longer being worked on.
A lot of the books on the market on the subject have been rushed into
place, and spend a whole lot more time discussing XML standardsthan
they do getting into the "meat" of how to do sophisticated things with
SOAP. As such, /they suck/, and you'll probably find little point in
buying them.
In the "MSFT/Java-frameworked" approaches, WSDL (analagous to CORBA
IDL) is a /totally vital/ piece, whereas in implementations of SOAP
for Python, Perl, Ruby, and such, WSDL is virtually
irrelevant/useless. That's almost certainly the most vital thing that
determines which "direction" you'd want to take:
- /My/ operating (and authoring) assumption is normally that WSDL is
useless, because SOAP::Lite and SOAP.py and SOAP4r make minimal if
any use of it.
- Someone talking about .NET-based SOAP will make the exact opposite
assumption, that WSDL is pervasive, ubiquitous, and mandatory in order
to make the slightest use of SOAP.
Those are totally divergent (though, in their respective contexts,
legitimate) perspectives, and may be helpful in determining what
"literary" direction to take. Most books take the "WSDL-is-pervasive"
perspective, though not usually consciously.
If what you want is to do "sophisticated SOAP stuff" in Python, I'm
not sure what your best choice is. The implementations suck fairly
badly, and supporting companies have gone out of business, so that
there's little code worth documenting. My book probably has the most
relevant Python coverage, but there's some risk that you'll feel
teased, as it may not get as 'advanced' as you want. On the other
hand, since the implementations aren't likely to be as 'advanced as
you want,' there is no hope of there being a /better/ option.
--
(concatenate 'string "chris" "@cbbrowne.com")
http://cbbrowne.com/info/soap.html
Rules of the Evil Overlord #169. "If I have massive computer systems,
I will take at least as many precautions as a small business and
include things such as virus-scans and firewalls."
<http://www.eviloverlord.com/>
Robert Oschler
>
> If what you want is to do "sophisticated SOAP stuff" in Python, I'm
> not sure what your best choice is. The implementations suck fairly
> badly, and supporting companies have gone out of business, so that
> there's little code worth documenting. My book probably has the most
> relevant Python coverage, but there's some risk that you'll feel
> teased, as it may not get as 'advanced' as you want. On the other
> hand, since the implementations aren't likely to be as 'advanced as
> you want,' there is no hope of there being a /better/ option.
> --
Thanks Chris, I'll check out the URL. What about NINZ, is that defunct too?
thx
Cameron Laird
Martin v. Löwis <loe...@informatik.hu-berlin.de> wrote:
.
.
.
>> Long-term I think SOAP is the way to go, because of the industry
>> momentum in that direction.
>
>Just because you can misuse port 80?
.
.
Yup. That's a more solid base than other, even
more momentous, engineering decisions have had.
OK, it's not just the firewall chicanery; it's
also because IBM and Microsoft (and Intel, inci-
dentally) agree WS is a Good Thing. Why not
CORBA? Prob'ly because they simply don't want
to support something managers associate with
Unix.
--
Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html
John Keeling
loe...@informatik.hu-berlin.de (Martin v. Löwis wrote in message news:<j4hefk7...@informatik.hu-berlin.de>...
> johnfk...@yahoo.com (John Keeling) writes:
>
> > Over the internet, you really want to use Web Services rather than
> > CORBA, because CORBA/IIOP uses multiple ports that have to be
> > explicitly allowed on the firewall ( on both the client and server
> > side).
>
> That is not true. CORBA can be easily restricted to a single port. If
> you really want to, CORBA can run on port 80.
It is true that a single CORBA server can be restricted to a single
port.
Most CORBA systems use multiple servers ... e.g. Naming Server &
multiple business servers, and hence require multiple ports.
From my experience, it would be unusual (but possible) to implement
all remote services into one single CORBA server. More common is to
use a proxy to put IIOP traffic through for the multiple servers. As
before, these proxies have not always been available as the CORBA spec
has evolved.
Martin, maybe you know a better way to approach this?
>
> > Long-term I think SOAP is the way to go, because of the industry
> > momentum in that direction.
>
> Just because you can misuse port 80?
It's just an opinion, I don't know for sure.
As above, one typically needs a port for each CORBA server (per host).
It is a major security and admin issue. Why do you think that
CORBA/IIOP hasn't taken over the world in cross-internet services? It
certainly has had enough time to do so. SOAP couldn't have got off the
ground if CORBA/IIOP was ideal in cross-internet environment.
Let me know if I am missing something.
Thanks,
John
John Keeling
what you are saying is that when the organizational politics is
sorted, SOAP/HTTP will offer better a compromise to all parties than
CORBA/IIOP, in general. This is due to the issue that each CORBA
server on a host must have its own port, and the typical
implementation of a CORBA system uses multiple servers per host and
hence multiple ports. SOAP systems are typically designed with much
more conservative use of ports.
Cheers,
John
Derek Thomson <de...@wedgetail.com> wrote in message news:<3db01b05$0$18872$afc3...@news.optusnet.com.au>...
> Hi John,
>
> John Keeling wrote:
> >
> > Over the internet, you really want to use Web Services rather than
> > CORBA, because CORBA/IIOP uses multiple ports that have to be
> > explicitly allowed on the firewall ( on both the client and server
> > side). This is an admin and security issue.
>
> Yes. It *is* a security issue. By misusing port 80 for remote procedure
> calls, you are violating the contract between you and your sysadmin, and
> quite likely violating your organization's (our your client's) security
> policy. That port was opened to allow web pages to be served up, not to
> run bits of arbitrary code on request.
> ....
Martin v. Loewis
> It is true that a single CORBA server can be restricted to a single
> port.
> Most CORBA systems use multiple servers ... e.g. Naming Server &
> multiple business servers, and hence require multiple ports.
> From my experience, it would be unusual (but possible) to implement
> all remote services into one single CORBA server. More common is to
> use a proxy to put IIOP traffic through for the multiple servers. As
> before, these proxies have not always been available as the CORBA spec
> has evolved.
>
> Martin, maybe you know a better way to approach this?
It depends on how you design your service. If you want to use CORBA to
do what you are doing with SOAP/XML-RPC (just easier and faster), you
would normally not expose the naming service to your eventual clients.
Instead, you would use either a fixed object reference for all
clients, or some easily-computed object reference to distinguish
clients (like a session key). You could also place the current IOR on
the web server and have clients do an http download of the current
IOR. In all these cases, you don't need a name service.
You might find that you also don't need distributed (to various
machines) objects on the server side for a service accessible in a
WAN. Instead, you expose only a single object reference, and have the
implementation delegate to the calls to the various individual objects
if you need those.
With that, you can use a single host/port combination per
service. This improves the security on the firewall, as it restores
the traditional TCP/Unix convention of one-port-is-one-service: the
firewall administrator can now readily block individual services,
perhaps distinguishing legitimit users from illegitime ones.
Regards,
Martin
Andrew Dalke
> I predict it won't be easier to run SOAP through firewalls for long.
> Wait until sysadmins realize you are running RPC calls ever port 80, and
> filter incoming SOAP requests out.
>
> I know forwarding thinking sysadmins already starting to do this, as
> port 80 was *not* opened for RPC, and any attempt to use it as such
> would violate their organization's security policies, and requires
> explicit permission.
Yeah? And when they find out I'm running CGI scripts accessible
through port 80 then they'll shut down those RPCs too?
Shouldn't the filtering be done on the URI for the SOAP request?
That is, only requests to public/published requests should go through
and others are blocked?
It's besides the point in that the service I'm helping provide (I'm
the OP for this thread) is all behind the firewall and only for internal
users. Ditto for the CGI scripts. Though I don't know if they have
internal firewalls.
Andrew
da...@dalkescientific.com
Dave Kuhlman
>> Yes. It *is* a security issue. By misusing port 80 for remote procedure
>> ... ^^^^^^^^
>> change the policy. Screwing around with port 80 is going to cause a lot
>> of ill-will. The first time a SOAP request comes through that runs some
>> random bit of code that causes some damage, you can fully expect to be
>> hung out to dry for it.
>>
>> Organizations have security policies for a reason. Hacking around them
>> will *never* end well.
>
> [applause]
I been waiting for someone else to ask this question so that I would not
have to show my ignorance. No such luck.
What about CGI? Uses port 80. Can run "random" bits of code. Powerful
enough to destroy servers with a single script. How is CGI any less of a
security threat? It doesn't take much of a wizard to write a Perl or
Python CGI script that trashes a system, or ruins a database, or ... Isn't
SOAP just another way of invoking a script. And, ditto for XML-RPC. With
SOAP (and XML-RPC) I put some request information in the body of the
request that I would have tacked onto the URL in CGI. How does that make
CGI any less dangerous.
Basically, hire any computer science student as an intern to come in and
write either CGI scripts or SOAP Web service scripts and you are asking for
"excitement".
And, isn't the point that any of these (CGI, SOAP, XML-RPC) are no more
dangerous than the scripts on the back-end that implement them, which puts
control on the back-end where we want it.
- Dave
[snip]
--
Dave Kuhlman
dkuh...@rexx.com
http://www.rexx.com/~dkuhlman
Martin v. Loewis
> Yeah? And when they find out I'm running CGI scripts accessible
> through port 80 then they'll shut down those RPCs too?
Depends on your organization's policy. If the rationale for having the
firewall is to protect proprietary information from accidentally
leaking the organization, through unreviewed services, then: perhaps.
The administrator may either decide shut down all scripts that have
not been reviewed for correctness and safety, or he may decide to
disconnect the server running the CGI from any critical information.
> It's besides the point in that the service I'm helping provide (I'm
> the OP for this thread) is all behind the firewall and only for
> internal users. Ditto for the CGI scripts. Though I don't know if
> they have internal firewalls.
In this case, the firewall argument of the SOAP-vs-CORBA discussion
goes away.
Regards,
Martin
Derek Thomson
> Derek Thomson wrote:
>
>> I predict it won't be easier to run SOAP through firewalls for long.
>> Wait until sysadmins realize you are running RPC calls ever port 80,
>> and filter incoming SOAP requests out.
>>
>> I know forwarding thinking sysadmins already starting to do this, as
>> port 80 was *not* opened for RPC, and any attempt to use it as such
>> would violate their organization's security policies, and requires
>> explicit permission.
>
>
> Yeah? And when they find out I'm running CGI scripts accessible
> through port 80 then they'll shut down those RPCs too?
In most environments, CGI scripts have to be carefully vetted and approved.
That's the point. You can't, and shouldn't, "hack around" the
organisation's security. You *have* to get authorization for this stuff.
Therefore SOAP hasn't solved anything, and you are back to square one -
needing authorization to run this stuff over the firewall.
This is my point - it's not any "easier" to go over a firewall using
SOAP, as the problem is non-technical in nature.
--
D.
Derek Thomson
> Nicolas Chauvat wrote:
>
>
>>>Yes. It *is* a security issue. By misusing port 80 for remote procedure
>>>... ^^^^^^^^
>>>change the policy. Screwing around with port 80 is going to cause a lot
>>>of ill-will. The first time a SOAP request comes through that runs some
>>>random bit of code that causes some damage, you can fully expect to be
>>>hung out to dry for it.
>>>
>>>Organizations have security policies for a reason. Hacking around them
>>>will *never* end well.
>>
>>[applause]
>
>
> I been waiting for someone else to ask this question so that I would not
> have to show my ignorance. No such luck.
>
> What about CGI? Uses port 80. Can run "random" bits of code. Powerful
> enough to destroy servers with a single script. How is CGI any less of a
> security threat?
CGI scripts must be negotiated and vetted before they are installed.
That is, you must follow the organization's security procedures to get
one installed. And I know quite a few sysadmins who *do not allow* CGI
scripts, for obvious reasons. Others allow CGI scripts, but they're
blocked from contacting any other servers within the firewall. Very wise.
And this is the point. SOAP solves exactly nothing WRT firewalls. You
will still need to get the proper authorization in a professional manner.
--
D.
Cayce Ullman
Andrew Dalke <ada...@mindspring.com> wrote in message news:<3DAC6F04...@mindspring.com>...
> Hey all,
>
> I've been working with one of my clients trying to get a SOAP
> interface going for a project. I've had a rather frustrating time
> trying to get it to work, and am using this as a chance to vent.
>
snip
> In addition, SOAP.py has similar coding style pecularities, and there's
> some bug that comes up in Python 2.2 because of changes in Python's
> type/class system. (We're using Python 2.1 so it isn't yet a problem.
> I also haven't tested ZSI to see if this is a problem.)
>
I apologize for the insanity that is SOAP.py's code. One of our
issues is we decided to use SAX over DOM for perfomance reasons, that
made the XML bits of SOAP.py mind bending. On the plus side, you can
generally parse and fire a SOAP message with SOAP.py faster than DOM
can even validate the XML.
> Finally, it seems that neither SOAP.py nor ZSI are actively developed.
> There hasn't been a recent release of either package, and the CVS logs
> show little change.
>
> All in all, it's rather discouraging for me to work with SOAP in
> Python, and that's given a rather non-trivial amount of experience
> in doing Python. I would think others have worse problems.
>
> It's such that the more CS-y people at my client's site believe that
> web services in Python are still very immature and want to switch the
> project over to Java and WebLogic(!). (I did point out that developing
> the library I did was still less time than setting that up, but there
> are other factors in the mix as well.)
>
>
> What I don't get is, I figure there's enough people doing web services/
> SOAP with Python that there shouldn't be these problems. Why then am
> I so frustrated with it?
>
> Andrew Dalke
> da...@dalkescientific.com
Andrew, I appreciate your frustrations, and I do still get the
occasional email complaining about some feature/bug of SOAP.py (I
guess you get what you pay for :). Unfortunately, my current job has
little to do with SOAP, so I can no longer afford to maintain it.
We gave it a BSD-style license for a reason. Feel free to fix it and
fork it. For that matter, let this serve as an open call to anyone
who would like to take over as the maintainer of the "official"
SOAP.py.
Andrew Dalke
>It's besides the point in that the service I'm helping provide (I'm
>the OP for this thread) is all behind the firewall and only for
>internal users. Ditto for the CGI scripts. Though I don't know if
>they have internal firewalls.
Martin v. Loewis wrote:
> In this case, the firewall argument of the SOAP-vs-CORBA discussion
> goes away.
It does?
My client has over 50,000 employees in installations around the world.
Everything is behind a firewall -- I have to visit their site to use
their machines. However, different sites may have internal firewalls.
(That's a conjecture, but given that I can't access my Hotmail account
when I'm on-site, because it's blocked, I'll assume they have strong
defense in depth.) We already have permission from the internal
systems people to provide services via CGI scripts spawned off by a
server they maintain.
We want to provide a simple stateless service to internal users.
It's easier for us to work behind Apache rather than have the systems
people maintain yet another server, and have us merge the different
logging facilities, and learn techniques of debugging both systems.
In addition, SOAP clients seem simpler. For example, SOAP.py works
through a single file. For CORBA it seems like a much more
complicated install, from the times I've built omniORB (not saying
it's hard, just that a single text file is easy to install). The
people writing the clients may be in one of several dozen sites so
the simpliest solution seems preferred.
I am not saying that CORBA is a better or worse solution. All I'm
suggesting is that even though we are behind a firewall, there are
other reasons which may make SOAP/XML-RPC/simple HTTP requests prefered
over CORBA.
Andrew
da...@dalkescientific.com