Does anyone know more?
Thanks,
Thomas
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden
starship.python.net was compromised. It looked like a rootkit may have been
installed. The volunteer admins are in the process of reinstalling the OS
and rebuilding the system. That process will probably take a few days at
least.
---Tom
Does anyone know more?
What about the integrity of the python packages hosted there?
When was the site compromised?
I just installed the python 2.5 pywin module last week.
Should I be concerned?
Is this related to the Python security problem recently announced?
Did you even read about the vulnerability?
Robert
Yes. Do you have any answers, or do you just enjoy posting irrevelant
responses?
I guess his response implied that what's irrelevant here is the
vulnerability, and accordingly your worries about it.
Thanks for the info. I appreciate the work that the admins are doing.
Thanks,
Thomas
Then perhaps he should have said that, in which case I would
have explained why he did not understand what he read. Let me
try again...
1. A site which hosts (I think, hence the questions) a number
of high profile, popular python projects was compomised.
2. It was compromised with a root kit which by their nature,
often go undetected for a long time.
3. It is common for miscreants to attempt to introduce
backdoors into software that will be widely distributed.
4. Anyone downloading and installing such trojaned software
will also be compromised.
5. Verifying that such a thing has not happened can be very
difficult, particularly if the date and other details of the
compromise cannot be accurately determined.
6. Many organisations give image and pr a higher priority
than the safety of their customers/users and wave off security
breechs with "don't worry, everything is fine. We're sure
nothing has been touched" when in fact they have no idea.
7. I have seen no public statements or information about
this leading me to wonder about the stuation and how it's
being handled, hence my seeking of further information.
That's what I am concerned about, ok?
I don't really care how the site was compromised and my
question about the python security vunerability was curiosity.
But, I am still completely at a loss why you, he, or anyone,
based on the information presented so far,.would conclude
that the python security problem is unrelated.
Care to enlighten me?
But more inmportantly, how about addressing my original
questions which are, even if you do not think so, pretty
important for anyone who has recently downloaded software
from or built there.
Well, let's have some answers then.
> 1. A site which hosts (I think, hence the questions) a number
> of high profile, popular python projects was compomised.
Yes. However, it doesn't *seem* as if the machine was deliberately
targeted, and I think it's unlikely the attackers were interested in
trojanning software. But of course the machine was rooted, so it's
pretty hard to be sure of these things.
> 2. It was compromised with a root kit which by their nature,
> often go undetected for a long time.
As far as I can tell, the machine was compromised on 2006-09-02.
Irritatingly we didn't find out until just after logrotate had deleted
the logs for around the time of the attack.
It wasn't a very subtle rootkit -- installing a version of netstat with
different command line options, for example...
> 5. Verifying that such a thing has not happened can be very
> difficult, particularly if the date and other details of the
> compromise cannot be accurately determined.
I guess you should find out from the author of whatever you downloaded
what the checksums should have been for what you downloaded and check
that against what you downloaded.
If you don't still have the downloaded files, I can tell you what the
md5's of the files in the back up are.
> 6. Many organisations give image and pr a higher priority
> than the safety of their customers/users and wave off security
> breechs with "don't worry, everything is fine. We're sure
> nothing has been touched" when in fact they have no idea.
There is no organization behind python.net.
I am a volunteer. I help run python.net in my spare time.
> 7. I have seen no public statements or information about
> this leading me to wonder about the stuation and how it's
> being handled, hence my seeking of further information.
I'm sorry, I'm busy trying to get the server going again.
> But, I am still completely at a loss why you, he, or anyone,
> based on the information presented so far,.would conclude
> that the python security problem is unrelated.
Why would it be? For all it's position in the community, there aren't
actually many python web apps running on python.net, certainly not as
root...
Cheers,
mwh
--snip--
> As far as I can tell, the machine was compromised on 2006-09-02.
So it was compromised for over a month.
> Irritatingly we didn't find out until just after logrotate had deleted
> the logs for around the time of the attack.
Murphy strikes again. :-(
> It wasn't a very subtle rootkit -- installing a version of netstat with
> different command line options, for example...
>
> > 5. Verifying that such a thing has not happened can be very
> > difficult, particularly if the date and other details of the
> > compromise cannot be accurately determined.
>
> I guess you should find out from the author of whatever you downloaded
> what the checksums should have been for what you downloaded and check
> that against what you downloaded.
>
> If you don't still have the downloaded files, I can tell you what the
> md5's of the files in the back up are.
I don't think that would help in the case of Pywin32 since the
Sourceforge dates for build 210 are 9/22.
I emailed Mark Hammond but have not heard anything back yet.
> > 6. Many organisations give image and pr a higher priority
> > than the safety of their customers/users and wave off security
> > breechs with "don't worry, everything is fine. We're sure
> > nothing has been touched" when in fact they have no idea.
>
> There is no organization behind python.net.
>
> I am a volunteer. I help run python.net in my spare time.
Organizations do not have to be formal or official to exhibit
similar behavior.
> > 7. I have seen no public statements or information about
> > this leading me to wonder about the stuation and how it's
> > being handled, hence my seeking of further information.
>
> I'm sorry, I'm busy trying to get the server going again.
I understand, and appreciate your (and the other people
involved) efforts. I know it must be a royal pain in the
ass. But I am still responsible for the code I (and my
clients) run so I had to ask.
> > But, I am still completely at a loss why you, he, or anyone,
> > based on the information presented so far,.would conclude
> > that the python security problem is unrelated.
>
> Why would it be? For all it's position in the community, there aren't
> actually many python web apps running on python.net, certainly not as
> root...
That's what one would hope but to assume that without better
information (such as you just provided) would be foolish.
Thanks again for taking the time to answer my questions.
> > > 5. Verifying that such a thing has not happened can be very
> > > difficult, particularly if the date and other details of the
> > > compromise cannot be accurately determined.
> >
> > I guess you should find out from the author of whatever you downloaded
> > what the checksums should have been for what you downloaded and check
> > that against what you downloaded.
> >
> > If you don't still have the downloaded files, I can tell you what the
> > md5's of the files in the back up are.
>
> I don't think that would help in the case of Pywin32 since the
> Sourceforge dates for build 210 are 9/22.
> I emailed Mark Hammond but have not heard anything back yet.
In the case of pywin32, are you at all sure that you actually
downloaded anything from starship.python.net? AFAICT all the files are
now hosted on sf, and there doesn't seem to be any vaguely new files in
the backup of /home/www.
Cheers,
mwh
> But, I am still completely at a loss why you, he, or anyone,
> based on the information presented so far,.would conclude
> that the python security problem is unrelated.
Because he's read the security advisory, perhaps, and understands what
it says?
</F>
The files I downloaded were from sourceforge. I don't know if
starship.python.net hosts the source files or plays any role in
building the disrtribution package. It may be that is all done
elsewhere. But given starship.python.net's historical association
with Pywin32, I am not going to just assume that.
Then perhaps you or he could explain it to us less intelligent
people in very simple terms?
> Then perhaps you or he could explain it to us less intelligent
> people in very simple terms?
the security advisory explains that the cause of the problem is a bug
in the source code used to implement repr() for 32-bit Unicode strings,
on all Python versions from 2.2 and onwards.
Python 2.2 was released in 2001.
</F>
I admit I am totally flmmexed by your answer.
What does when the bug was introduced have to do with
anything? It is present in contemporary versions of Python.
It "can lead to execution of arbitrary code". It is important
enough to drive an "emergency" (my term) bug fix python
release.
It seems to have been disscussed publically starting around
Oct 6 or 7 (I didn't do a though search so this may be wrong.)
It was fixed in Python 2.5 so either it was treated as a
ordinary bug with unrecognised security implications,
or the developers were aware of the security issues and
sat on them.
Regardless, I don't see anything in the advisory that either
makes it an unimportant issue, or makes clearly unrelated
to the starship.python.net compromise.
So could you please try to explain again in even simpler
terms?
So, are we to infer that Starship was running Python 2.1 or earlier at
the time the server was compromised? Otherwise I missed your point, sorry.
The vulnerability described by PSF-2006-001 could easily lead to server
compromises. AFAIK, most Linux distributions enable UCS-4 by default,
and they have done so for years. To compromise a server using the
PSF-2006-001 vulnerability, an intruder just needs to find a Python CGI
script running on that server that converts some bad input to unicode,
then cause that script to raise an error while processing the request
containing the bad input. There's a good chance the script will log an
error with the repr() of the bad input, allowing the intruder to mess
with the stack. If the server is running a distribution-supplied build
of Python, the intruder may be able to inject arbitrary code.
I don't know if this concern applies to Starship specifically, but it
seems to apply to thousands of web sites running Python CGIs and Python
web servers.
Shane
It was fixed in a checkin on August 21 (rev 51450). While it's possible in
theory that this was the root of the compromise, the fact that none of the
security memos floating around suggested that it had been exploited gives me
a fairly warm feeling that it wasn't the cause of the starship breakin.
Also, the fact that it has been around, apparently unexploited, since 2001
suggests that it was sufficiently obscure that either a) nobody who knew
about it found a way to take advantage of it, or b) it was only recently
discovered back in August shortly before the problem was fixed in the source
code.
Skip
> I don't know if this concern applies to Starship specifically, but it
> seems to apply to thousands of web sites running Python CGIs and
> Python web servers.
so are we seeing thousands of web sites running Python CGIs and web
servers being attacked right now?
</F>
No, but it often takes a long time for servers to get patched, so the
window for intruders is going to be open for a while. I'm trying to
understand:
a) how urgent and/or exploitable this is,
b) how I can check whether a given Python installation (running on a
server) has been patched, and
c) whether the security advisory downplays the risk more than it should,
since it appears that many Zope/Plone web servers are vulnerable.
Shane
Shane> a) how urgent and/or exploitable this is,
Perhaps not very. As I indicated in an earlier post, the exploit has been
available since 2001, so it is probably fairly hard to exploit.
Shane> b) how I can check whether a given Python installation (running
Shane> on a server) has been patched, and
If it's running 2.4.4 or 2.5 it should be okay. If it's running some
earlier version a lot will depend on whether Python was installed by a Linux
distributor (in which case check their version numbers and their release
notes) or installed locally from source.
Shane> c) whether the security advisory downplays the risk more than it
Shane> should, since it appears that many Zope/Plone web servers are
Shane> vulnerable.
I can't pretend to divine the true meaning behind all the wording of the
various security advisories. You'd have to ask each one of the security
organizations.
Here's one example:
http://secunia.com/advisories/22276/
The application has to work with Unicode on a UCS-4-compiled version of
Python and use the repr() function on such Unicode strings. Furthermore,
the black hat would have to figure out how to get a suitably crafted Unicode
string into the repr() function at just the right place.
I'm not saying it can't be done, but I think it would be a fairly
challenging undertaking.
Skip
> I admit I am totally flmmexed by your answer.
> What does when the bug was introduced have to do with
> anything?
oh, I thought your main concern was whether the packages available had
been compromised, and that you asked if that was the reason an advisory
was released last week.
if someone has developed an exploit for the vulnerability, chances are
that they'd attack more than just a single obscure and mostly abandoned
server.
</F>
It might well be difficult to exploit to run arbitrary code because
your exploit code needs to have no unprintable bytes in it; repr()
turns unprintable characters into \xNN, after all, and isn't doing a
straightforward string copy. (But hackers can be very clever...)
--amk
In email, Mark Hammond said that that the Pywin32 code is not
hosted at starship.python.net, nor are the distributions built there.
(I assume this does not apply to the very old win32all stuff that
*is* at python.net but I doubt anyone uses those anymore.)
Yes.
> and that you asked if that was the reason an advisory
> was released last week.
No, I asked if there was any relationship.
http://groups.google.com/group/comp.lang.python/msg/f1974d9b5a42639e?hl=en&
> if someone has developed an exploit for the vulnerability, chances are
> that they'd attack more than just a single obscure and mostly abandoned
> server.
If someone's goal was to compromise machines by compromising
software that was likely to be installed by many people, they would
be wise to minimize the chance of detection by attacking as few
machines as possible. But given what mwh wrote earlier about the
incident, and what you say about starship.python.net's lack
of prominence, obviously it was unlikely their goal.
>It might well be difficult to exploit to run arbitrary code because
>your exploit code needs to have no unprintable bytes in it; repr()
>turns unprintable characters into \xNN, after all, and isn't doing a
>straightforward string copy. (But hackers can be very clever...)
Someone made years ago an UUDecode executable program consisting
entirely of printable ASCII characteres (.COM, for DOS, might still
work on XP...)
--
Gabriel Genellina
Softlab SRL
__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas