Is Python a commercial proposition ?

[lipska the kat]

Pythoners

Firstly, thanks to those on the tutor list who answered my questions.

I'm trying to understand where Python fits into the set of commonly
available, commercially used languages of the moment.

My most recent experience is with Java. The last project I was involved
with included 6775 java source files containing 1,145,785 lines of code.
How do I know this? because I managed to cobble together a python script
that walks the source tree and counts the lines of code. It ignores
block and line comments and whitespace lines so I'm fairly confident
it's an accurate total. It doesn't include web interface files (mainly
.jsp and HTML) or configuration files (XML, properties files and what
have you). In fact it was remarkably easy to do this in python which got
me thinking about how I could use the language in a commercial environment.

I was first attracted to python by it's apparent 'Object Orientedness' I
soon realised however that by looking at it in terms of the language I
know best I wasn't comparing like with like. Once I had 'rebooted the
bioware' I tried to approach python with an open mind and I have to say
it's growing on me.

The questions I have are ...

How is python used in the real world.
What sized projects are people involved with
Are applications generally written entirely in python or is it more
often used for a subset of functionality.

I hope this is an acceptable question for this group

Many thanks

Lipska

--
Lipska the Kat: Troll hunter, sandbox destroyer
and farscape dreamer of Aeryn Sun

[Michael Hrivnak]

http://www.djangosites.org/

Instagram, Pinterest, Washington Post, and The Onion all use djangoto
run their websites.

http://stackoverflow.com/questions/1906795/what-are-some-famous-websites-built-in-django

Django is of course a very highly-regarded web framework written in
python, but there are other good python frameworks out there with
strong user bases.

Python is used frequently on the server side of web applications for
sites of all sizes, with the UI generally being done in javascript.
It's also used heavily for administrative purposes such as:

- Operating system installer: http://fedoraproject.org/wiki/Anaconda
- Software repository management: http://pulpproject.org/
- Software package installation:
http://en.wikipedia.org/wiki/Ubuntu_Software_Center
- Cloud computing: http://en.wikipedia.org/wiki/OpenStack

You can write web applications that scale automatically on hosting
services like:
- Google App Engine: https://developers.google.com/appengine/
- OpenShift: http://openshift.redhat.com/
- Heroku: http://www.heroku.com/

In sum, python is used widely for a variety of purposes by some of the
largest enterprises down to very small projects.

Michael

> --
> http://mail.python.org/mailman/listinfo/python-list

[Mark Lawrence]

You are hard pushed to find anything here that's unacceptable, that's
why I like reading this list so much.

>
> Many thanks
>
> Lipska
>

There's a list of companies who use python on www.python.org top right
of the page. You may have heard of one or two of them.

--
Cheers.

Mark Lawrence.

[Stefan Behnel]

lipska the kat, 29.07.2012 18:01:

> My most recent experience is with Java. The last project I was involved
> with included 6775 java source files containing 1,145,785 lines of code.
> How do I know this? because I managed to cobble together a python script
> that walks the source tree and counts the lines of code. It ignores block
> and line comments and whitespace lines so I'm fairly confident it's an

> accurate total. It doesn't include web interface files (mainly ..jsp and

> HTML) or configuration files (XML, properties files and what have you). In
> fact it was remarkably easy to do this in python

Just a comment on this part. An even easier way to count source lines is by
using the tool "SLOCCount". It also works with various languages.

Stefan

[Brandon Schaffer]

Another common use is to create automated regression testing
frameworks, and other automation tools.
I see posting for python developers for this type of thing all the
time on stack overflow careers.

On Sun, Jul 29, 2012 at 11:43 AM, Mark Lawrence <bream...@yahoo.co.uk> wrote:
> On 29/07/2012 17:01, lipska the kat wrote:
>>

> You are hard pushed to find anything here that's unacceptable, that's why I
> like reading this list so much.
>
>>
>> Many thanks
>>
>> Lipska
>>
>
> There's a list of companies who use python on www.python.org top right of
> the page. You may have heard of one or two of them.
>
> --
> Cheers.
>
> Mark Lawrence.
>

> --
> http://mail.python.org/mailman/listinfo/python-list

[Tim Chase]

On 07/29/12 12:13, Michael Hrivnak wrote:
> - Operating system installer: http://fedoraproject.org/wiki/Anaconda
> - Software repository management: http://pulpproject.org/
> - Software package installation:
> http://en.wikipedia.org/wiki/Ubuntu_Software_Center
> - Cloud computing: http://en.wikipedia.org/wiki/OpenStack

I'll include both the Bazaar and Mercurial DVCS tools which are
mostly Python (I understand some inner loops drop to C, but both
have the option to fall back to a pure Python implementation).

-tkc

[Andrew Cooper]

On 29/07/2012 17:01, lipska the kat wrote:

> Pythoners
>
> Firstly, thanks to those on the tutor list who answered my questions.
>
> I'm trying to understand where Python fits into the set of commonly
> available, commercially used languages of the moment.
>
> My most recent experience is with Java. The last project I was involved
> with included 6775 java source files containing 1,145,785 lines of code.
> How do I know this? because I managed to cobble together a python script
> that walks the source tree and counts the lines of code. It ignores
> block and line comments and whitespace lines so I'm fairly confident
> it's an accurate total. It doesn't include web interface files (mainly
> .jsp and HTML) or configuration files (XML, properties files and what
> have you). In fact it was remarkably easy to do this in python which got
> me thinking about how I could use the language in a commercial environment.
>
> I was first attracted to python by it's apparent 'Object Orientedness' I
> soon realised however that by looking at it in terms of the language I
> know best I wasn't comparing like with like. Once I had 'rebooted the
> bioware' I tried to approach python with an open mind and I have to say
> it's growing on me.
>
> The questions I have are ...
>
> How is python used in the real world.
> What sized projects are people involved with
> Are applications generally written entirely in python or is it more
> often used for a subset of functionality.

Python is used extensively in XenServer, although arguably more of a
glue between components. The installer is entirely python, whereas the
running product uses it more for scripts and toolstack plugins.

Behind the scenes, our regression test framework is entirely python,
which performs several thousand machine hours of tests a night, from
simple tests such as "verify the installer runs correctly and verify
settings are preserved across upgrade from an older version" to "set up
a pool of 4 servers, kill the master server (by removing its power), and
verify that the remaining 3 fail over and one gets promoted to master"

~Andrew

[Stefan Behnel]

Tim Chase, 29.07.2012 20:28:

I find it perfectly reasonable to use C code (and other kinds of low-level
code) in Python tools and applications. In fact, easy interfacing with
low-level code is one of (C)Python's major selling points.

Stefan

[Terry Reedy]

On 7/29/2012 12:01 PM, lipska the kat wrote:
> I'm trying to understand where Python fits into the set of commonly
> available, commercially used languages of the moment.

Ever heard of a little startup called Google? It was built with C, Java,
... and Python. I believe Youtube is scripted in Python.

> Are applications generally written entirely in python
> or is it more often used for a subset of functionality.

Even if the direct application is in Python, if it runs on CPython, it
probably uses modules coded or re-coded in C, and it certainly used
builtin functions and classes coded in C.

The first 'killer app' for Python, in the 1990s, was its use as a glue
language for interactive and batch scientific/numerical computation
mostly done in compiled Fortran.

--
Terry Jan Reedy

[Bernd Waterkamp]

Michael Hrivnak schrieb:

> Python is used frequently on the server side of web applications for
> sites of all sizes, with the UI generally being done in javascript.

Two large companies with lots of python code are dropbox and youtube:

http://highscalability.com/blog/2011/3/14/6-lessons-from-dropbox-one-million-files-saved-every-15-minu.html
http://highscalability.com/blog/2012/3/26/7-years-of-youtube-scalability-lessons-in-30-minutes.html

> It's also used heavily for administrative purposes such as:
>
> - Operating system installer: http://fedoraproject.org/wiki/Anaconda
> - Software repository management: http://pulpproject.org/
> - Software package installation:
> http://en.wikipedia.org/wiki/Ubuntu_Software_Center
> - Cloud computing: http://en.wikipedia.org/wiki/OpenStack

- Frameworks/tools like func, fabric or ipython are used in medium and
large networks/"clouds".
- Python is also used a lot for admin tasks instead of shell scripts.
- I know IBM WebSphere is not the favorite choice as an application server
for most Java programmers *g*, but it uses Jython for the admin CLI.

python and python based tools are used for engineering and scientific
computing - some random examples: numpy, Sage, matplotlib, NetworkX.

Regards,
Bernd

[Aled Evans]

If you check Google's job listings:

http://www.google.com/about/jobs/teams/engineering/

On the software side of things you'll see very few listings that don't ask for coding in Python as a requirement or, at the very least, an advantageous skill to have.

I'd say slowly, but surely, many people are coming around the fact that Python is not only as powerful as Java and other high-level languages, but it's also easier to read and write. That, and people (that I've spoken to, at least), find it far more fun to code in Python!

[Paul van der Linden]

Scripting is one of the strong sides of python. I use it al the time to quickly write a script to analyze something or automate. That is probably the reason it is used to glue (script) things together and is embedded in some programs (like Maya and such).

At the company we're using python and django for websites, from small micro sites till big enterprise sites.

Currently I'm working on a highly configurable application to control a maya render cluster.

You can also use it for testing and even frontend testing.

I think you can use python for almost everything, tough I think it's less suitable for non-web gui applications.

lipska the kat <lip...@yahoo.co.uk>schreef:

--
http://mail.python.org/mailman/listinfo/python-list

[Thomas 'PointedEars' Lahn]

Michael Hrivnak wrote:

> Python is used frequently on the server side of web applications for
> sites of all sizes, with the UI generally being done in javascript.

There is no javascript.

--
PointedEars

Please do not Cc: me. / Bitte keine Kopien per E-Mail.

[Rodrick Brown]

Sent from my iPhone

On Jul 29, 2012, at 12:07 PM, lipska the kat <lip...@yahoo.co.uk> wrote:

> Pythoners
>
> Firstly, thanks to those on the tutor list who answered my questions.
>
> I'm trying to understand where Python fits into the set of commonly available, commercially used languages of the moment.

Python is a glue language much like Perl was 10 years ago. Until the
GIL is fixed I doubt anyone will seriously look at Python as an option
for large enterprise standalone application development.

I work in financials and the majority of our apps are developed in C++
and Java yet all the tools that startup, deploy and conduct rigorous
unit testing are implemented in Python or Shell scripts that wrap
Python scripts.

Python definitely has its place in the enterprise however not so much
for serious stand alone app development.

I'm starting to see Python used along side many statistical and
analytical tools like R, SPlus, and Mathlab for back testing and
prototype work, in a lot of cases I've seen quants and traders
implement models in Python to back test and if successful converted to
Java or C++.

[Andrew Berg]

On 7/29/2012 7:12 PM, Rodrick Brown wrote:
> Python is a glue language much like Perl was 10 years ago. Until the
> GIL is fixed I doubt anyone will seriously look at Python as an option
> for large enterprise standalone application development.

The GIL is neither a bug to be fixed nor an inherent part of the
language. It is a design choice for CPython. There are reasons the
CPython devs have no intention of removing the GIL (at least in the near
future). A recent outline of these reasons (written by one of the
CPython devs) is here:

http://python-notes.boredomandlaziness.org/en/latest/python3/questions_and_answers.html#but-but-surely-fixing-the-gil-is-more-important-than-fixing-unicode
--
CPython 3.3.0b1 | Windows NT 6.1.7601.17803

[Rodrick Brown]

Hence the reason why no one will seriously look at Python for none
glue work or simple web apps. When it comes to designing complex
applications that need to exploit large multicore systems Python just
isn't an option.

Its still not possible to be a pure Python developer and find gainful
employment today.

> --
> CPython 3.3.0b1 | Windows NT 6.1.7601.17803

> --
> http://mail.python.org/mailman/listinfo/python-list

[Paul Rubin]

Rodrick Brown <rodric...@gmail.com> writes:
> Hence the reason why no one will seriously look at Python for none
> glue work or simple web apps. When it comes to designing complex
> applications that need to exploit large multicore systems Python just
> isn't an option.

That's wrong, I've run multicore apps in Python, by just using separate
processes. There was no GIL issue, just separate processes for each
core. The cpython interpreter is so slow that the GIL is usually not
the bottleneck anyway. For lots of applications this just doesn't
matter since the app is either not cpu-intensive or (in my case) all the
work is done in native libraries.

> Its still not possible to be a pure Python developer and find gainful
> employment today.

Certainly any serious programmer should be good in multiple languages,
and in fact I got to write a little bit of C code at work a few months
ago, but it wasn't really needed. The program is all Python.

[Stefan Behnel]

Rodrick Brown, 30.07.2012 02:12:

> On Jul 29, 2012, at 12:07 PM, lipska the kat wrote:
>> I'm trying to understand where Python fits into the set of commonly available, commercially used languages of the moment.
>

> Python is a glue language much like Perl was 10 years ago. Until the
> GIL is fixed

Enough people have commented on this piece of FUD already.

> I doubt anyone will seriously look at Python as an option
> for large enterprise standalone application development.

I know enough examples to recognise this as nonsense. You mentioned working
in "financials" and even there I know at least one not-so-small bank that's
been developing their internal (EAI and business process) code in Python
for almost a decade now. And their developers are quite happy with it,
certainly happier than many of the Java developers I've met in other banks.

Still, you may still get away with the above statement by providing a
sufficiently narrow definition of "standalone". By my definition, there
isn't much "standalone" code out there. Most code I know interfaces with a
couple of external tools, libraries or backends, usually written in
languages I don't have to care about because they provide a language
independent interface.

Stefan

[Chris Angelico]

On Mon, Jul 30, 2012 at 4:07 PM, Stefan Behnel <stef...@behnel.de> wrote:
> Still, you may still get away with the above statement by providing a
> sufficiently narrow definition of "standalone". By my definition, there
> isn't much "standalone" code out there. Most code I know interfaces with a
> couple of external tools, libraries or backends, usually written in
> languages I don't have to care about because they provide a language
> independent interface.

Agreed, and the flip-side of that is that there aren't many
mono-language developers either. Sure, it'd be possible to make a
career of nothing but Objective-C, writing apps for Apple to make all
the money off, but even then you'll probably benefit from knowing some
glue languages.

Python's an excellent glue language, but it's also fine for huge
applications. Yes, it can't multithread across cores if you use
CPython and are CPU-bound. That's actually a pretty specific
limitation, and taking out any component of that eliminates the GIL as
a serious problem.

ChrisA

[Tim Chase]

On 07/29/12 21:31, Rodrick Brown wrote:
> Its still not possible to be a pure Python developer and find
> gainful employment today.

I'm not sure where you get your facts, but unless you define "pure"
in a super-narrow way, it's just flat-out wrong. I've been employed
doing primarily Python for the past 7+ years. Yes, there's been
some SQL involved; yes, I've done code-reviews for somebody else's
C# (the nice thing about C-like languages is they all read mostly
the same); yes, some of the web apps have required knowing
ECMAScript, HTML, XML, CSS, etc. But the day to day is mostly
coding in Python. And the several recruiters that have contacted me
in the past week or two about additional Python positions seem to
think there are pure Python jobs available.

Maybe you intended to write "not possible to be a poor Python
developer and find gainful employment today" which could surely be
the case, as I've met LOTS of programmers (Python and otherwise)
that I'd never consider hiring because of their poor
skills/understanding of their tools.

-tkc

[Roy Smith]

In article <mailman.2717.1343634...@python.org>,

Chris Angelico <ros...@gmail.com> wrote:

> Python's an excellent glue language, but it's also fine for huge
> applications. Yes, it can't multithread across cores if you use
> CPython and are CPU-bound. That's actually a pretty specific
> limitation, and taking out any component of that eliminates the GIL as
> a serious problem.

These days, I'm working on a fairly large web application (songza.com).
The business/application logic is written entirely in Python (mostly as
two django apps). That's what we spend 80% of our developer time
writing.

As for scale, we're currently running on 80 cores worth of AWS servers
for the front end. Another 50 or so cores for the database and other
backend functions. Yesterday (Sunday, so a slow day), we served 27
million HTTP requests; we're not facebook-sized, but it's not some
little toy application either.

Every time we look at performance, we can't hardly measure the time it
takes to run the Python code. Overall, we spend (way) more time waiting
on network I/O than anything else. Other than I/O, our biggest
performance issue is slow database queries, and making more queries than
we really need to.

The engineering work to improve performance involves restructuring our
data representation in the database, caching (at multiple levels), or
eliminating marginal features which cost more than they're worth. None
of this would be any different if we used C++, except that we'd spend so
much time writing and debugging code that we'd have no time left to
think about the really important stuff.

As far as the GIL is concerned, it's just not an issue for us. We run
lots of server processes. Perhaps not as elegant as running fewer
multi-threaded processes, but it works just fine, is easy to implement,
and we never have to worry about all the horrors of getting memory
management right in a multi-threaded C++ application.

[lipska the kat]

On 30/07/12 14:06, Roy Smith wrote:
> In article<mailman.2717.1343634...@python.org>,
> Chris Angelico<ros...@gmail.com> wrote:
>
>> Python's an excellent glue language, but it's also fine for huge
>> applications. Yes, it can't multithread across cores if you use
>> CPython and are CPU-bound. That's actually a pretty specific
>> limitation, and taking out any component of that eliminates the GIL as
>> a serious problem.
>
> These days, I'm working on a fairly large web application (songza.com).
> The business/application logic is written entirely in Python (mostly as
> two django apps). That's what we spend 80% of our developer time
> writing.
>

snip

"We are very sorry to say that due to licensing constraints we cannot
allow access to Songza for listeners located outside of the United States."

Arse :-(

[Grant Edwards]

On 2012-07-30, Stefan Behnel <stef...@behnel.de> wrote:

> Still, you may still get away with the above statement by providing a
> sufficiently narrow definition of "standalone". By my definition, there
> isn't much "standalone" code out there. Most code I know interfaces with a
> couple of external tools, libraries or backends, usually written in
> languages I don't have to care about because they provide a language
> independent interface.

It's not really relevent to this discussion, but there is _lots_ of
stand-alone code out there. It runs in sub-one-dollar microcontrollers
that are programmed in assembly language or in C without external
libraries (sometimes not even the "libc" that's included in the C
language definition). Those microcontrollers are everywhere in toys,
appliances, and all sorts of other "non-computer" things.

--
Grant Edwards grant.b.edwards Yow! Mr and Mrs PED, can I
at borrow 26.7% of the RAYON
gmail.com TEXTILE production of the
INDONESIAN archipelago?

[rusi]

On Jul 29, 9:01 pm, lipska the kat <lip...@yahoo.co.uk> wrote:
> Pythoners
>
> Firstly, thanks to those on the tutor list who answered my questions.
>
> I'm trying to understand where Python fits into the set of commonly
> available, commercially used languages of the moment.
>
> My most recent experience is with Java. The last project I was involved
> with included 6775 java source files containing 1,145,785 lines of code.
> How do I know this? because I managed to cobble together a python script
> that walks the source tree and counts the lines of code. It ignores
> block and line comments and whitespace lines so I'm fairly confident
> it's an accurate total. It doesn't include web interface files (mainly
> .jsp and HTML) or configuration files (XML, properties files and what
> have you). In fact it was remarkably easy to do this in python which got
> me thinking about how I could use the language in a commercial environment.
>
> I was first attracted to python by it's apparent 'Object Orientedness' I
> soon realised however that by looking at it in terms of the language I
> know best I wasn't comparing like with like. Once I had 'rebooted the
> bioware' I tried to approach python with an open mind and I have to say
> it's growing on me.
>
> The questions I have are ...
>
> How is python used in the real world.
> What sized projects are people involved with
> Are applications generally written entirely in python or is it more
> often used for a subset of functionality.

I think when people talk of scripting this area tends to get missed:
(Or if someone mentioned it, I missed it :-) )
http://wiki.python.org/moin/AppsWithPythonScripting

[Emile van Sebille]

On 7/29/2012 5:12 PM Rodrick Brown said...
> Until the
> GIL is fixed I doubt anyone will seriously look at Python as an option

> for large enterprise standalone application development.

See openERP -- http://www.openerp.com/ -- they've been actively
converting SAP accounts and have recently absorbed a couple SAP resellers.

Emile

[Prasad, Ramit]

> I work in financials and the majority of our apps are developed in C++
> and Java yet all the tools that startup, deploy and conduct rigorous
> unit testing are implemented in Python or Shell scripts that wrap
> Python scripts.
>
> Python definitely has its place in the enterprise however not so much
> for serious stand alone app development.
>
> I'm starting to see Python used along side many statistical and
> analytical tools like R, SPlus, and Mathlab for back testing and
> prototype work, in a lot of cases I've seen quants and traders
> implement models in Python to back test and if successful converted to
> Java or C++.

Maybe this is true in *your* experience but *my* experience is very
different. I have seen Python modules become modules that end up
rewritten in C for performance reasons but I consider that part of the
power of Python and an argument *for* Python.

Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.

[alex23]

On Jul 30, 12:31 pm, Rodrick Brown <rodrick.br...@gmail.com> wrote:
> Its still not possible to be a pure Python developer and find gainful
> employment today.

I have been working as a "pure" Python developer for six+ years now
(in that the bulk of my coding is done in Python, with some interface
behaviour in JS). On average, every two months I'm contacted by a
recruiter or an employer wanting me for my Python experience. I've
worked for government, education and private industry, and the only
time I didn't get paid was my last week working for a start-up.

So I'm pretty confident that I'm "gainfully" employed.

[Steven D'Aprano]

On Mon, 30 Jul 2012 14:09:38 +0000, Grant Edwards wrote:

> On 2012-07-30, Stefan Behnel <stef...@behnel.de> wrote:
>
>> Still, you may still get away with the above statement by providing a
>> sufficiently narrow definition of "standalone". By my definition, there
>> isn't much "standalone" code out there. Most code I know interfaces
>> with a couple of external tools, libraries or backends, usually written
>> in languages I don't have to care about because they provide a language
>> independent interface.
>
> It's not really relevent to this discussion, but there is _lots_ of
> stand-alone code out there. It runs in sub-one-dollar microcontrollers
> that are programmed in assembly language or in C without external
> libraries (sometimes not even the "libc" that's included in the C
> language definition). Those microcontrollers are everywhere in toys,
> appliances, and all sorts of other "non-computer" things.

And at that level, you aren't going to write your app in Python anyway,
and not because of the GIL. (These microcontrollers are unlikely to have
multiple cores -- why the hell does your microwave oven need two cores?)

It seems to me that those who claim that the GIL is a serious barrier to
Python's use in the enterprise are mostly cargo-cult programmers,
parroting what they've heard from other cargo-cultists. It really is
astonishing the amount of misinformation and outright wrong-headed
ignorance that counts as accepted wisdom in the enterprise.



--
Steven

[Paul Rubin]

Steven D'Aprano <steve+comp....@pearwood.info> writes:
> And at that level, you aren't going to write your app in Python anyway,
> and not because of the GIL. (These microcontrollers are unlikely to have
> multiple cores -- why the hell does your microwave oven need two cores?)

http://greenarrays.com ;-)

> It seems to me that those who claim that the GIL is a serious barrier to
> Python's use in the enterprise are mostly cargo-cult programmers,

I would say, it puts a crimp into Python's versatility but there are
still lots of areas where it's not a serious issue. A real compiler
(PyPy) will help Python performance far more than multi-core currently
can.

[Stefan Behnel]

Paul Rubin, 31.07.2012 06:45:

> A real compiler (PyPy) will help Python performance far more than
> multi-core currently can.

That's too general a statement to be meaningful.

Stefan

[Steven D'Aprano]

On Mon, 30 Jul 2012 21:45:51 -0700, Paul Rubin wrote:

> Steven D'Aprano <steve+comp....@pearwood.info> writes:
>> And at that level, you aren't going to write your app in Python anyway,
>> and not because of the GIL. (These microcontrollers are unlikely to
>> have multiple cores -- why the hell does your microwave oven need two
>> cores?)
>
> http://greenarrays.com ;-)
>
>> It seems to me that those who claim that the GIL is a serious barrier
>> to Python's use in the enterprise are mostly cargo-cult programmers,
>
> I would say, it puts a crimp into Python's versatility but there are
> still lots of areas where it's not a serious issue.

Of course it's a crimp. Nobody likes the GIL for its own sake, and nobody
likes the fact that it does slow CPython down under some circumstances.
It's not like the Python devs scheme behind closed doors on how to make
Python slower. If somebody came up with a way to remove the GIL without
the harmful side-effects, or volunteered to do the enormous amount of
work needed, the devs would be as enthusiastic for it as anyone else.

Recognising that there are *some* applications where Python isn't
suitable (for whatever reason, not just because of the GIL) is simply
common sense. There are a whole lot of factors leading to the choice of a
compiler. "Can use all available cores on a CPU" is only one of many.

[rant]
But to hear some people talk, CPU-bound multi-threaded apps are the only
"serious" enterprise apps, and further more, there is no possible way to
make a program fast enough *except* threads, hence no possible way that
Python is suitable except by removing the GIL.

Do they consider that perhaps there are alternatives to threads? Or that
there already are Python implementations that don't include the GIL,
running on big enterprise-friendly platforms like Java and .NET? No they
do not. If Python didn't have the GIL, they'd find some other excuse to
dismiss it for "serious" work.

And that is why I consider that anyone repeating without nuance the
canard that the GIL makes Python unsuitable for serious enterprise work
is a cargo-cultist.
[/rant]

And now I have to go yell at some kids who are on my lawn.



--
Steven

[Roy Smith]

In article <50177b4d$0$29867$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> Do they consider that perhaps there are alternatives to threads?

There's basically two reasons people use threads.

First is because it's a convenient way to multiplex short-lived tasks on
a single processor. What people tend to forget is that we managed to do
that before there were threads. Event loops still work perfectly fine.
For an interesting example, http://www.gevent.org/.

Second is because we have multiple long-lived, compute-bound tasks and
want to be able to take advantage of multiple processors. For that,
multiprocessing works just fine most of the time. And multiprocess has
the advantage over multithreading in that the processes can run on
different machines, so you're not limited by the number of cores you can
get on a single CPU.

What multiprocessing doesn't give you is fine-grain task switching. So
we're left with the set of jobs which consist of many (but not more than
the number of cores on one CPU) compute-bound tasks, that we know how to
parallelize, and require fine-grain task switching. Sure, that's an
important set of jobs, but there's a whole huge world of computing that
doesn't fit into that box.

[David]

On 30/07/2012, lipska the kat <lip...@yahoo.co.uk> wrote:
> On 30/07/12 14:06, Roy Smith wrote:
>>
>> These days, I'm working on a fairly large web application (songza.com).
>

> "We are very sorry to say that due to licensing constraints we cannot
> allow access to Songza for listeners located outside of the United States."
>
> Arse :-(

A free[1] US proxy could bypass[2] that page ... eg something like
http://www.airproxy.ca/

[1] as in beer
[2] for research purposes

[lipska the kat]

Oooooh that's a bit good isn't it, just for research purposes of course.

There's one (as in 1 above) in the pump for you.

lipska

[David]

On 01/08/2012, lipska the kat <lip...@yahoo.co.uk> wrote:
> On 31/07/12 14:52, David wrote:
>>
>> [1] as in beer
>> [2] for research purposes
>

> There's one (as in 1 above) in the pump for you.

Great, more beer => better research => \o/\o/\o/
But, "pump" sounds a bit extreme .. I usually sip contentedly from a glass :p

[Mark Lawrence]

On 01/08/2012 00:31, David wrote:

> On 01/08/2012, lipska the kat <lip...@yahoo.co.uk> wrote:
>> On 31/07/12 14:52, David wrote:
>>>
>>> [1] as in beer
>>> [2] for research purposes
>>

>> There's one (as in 1 above) in the pump for you.
>

> Great, more beer => better research => \o/\o/\o/
> But, "pump" sounds a bit extreme .. I usually sip contentedly from a glass :p
>

You complete ignoramus, if it gets poured in advance that's no good to
anybody as it'll go flat. Has to stay in the pump until you're ready to
drink it from the glass. Don't you know anything about the importance
of process and timing? :)

--
Cheers.

Mark Lawrence.

[lipska the kat]

Heh heh, obviously never got drunk ... er I mean served behind the bar
at uni/college/pub %-}

[David]

On 01/08/2012, lipska the kat <lip...@yahoo.co.uk> wrote:
> On 01/08/12 09:06, Mark Lawrence wrote:
>>
>> You complete ignoramus, if it gets poured in advance that's no good to
>> anybody as it'll go flat. Has to stay in the pump until you're ready to
>> drink it from the glass. Don't you know anything about the importance of
>> process and timing? :)
>
> Heh heh, obviously never got drunk ... er I mean served behind the bar
> at uni/college/pub %-}

Nah, obviously *is* drunk ;p

[Stefan Behnel]

David, 01.08.2012 13:59:

> On 01/08/2012, lipska the kat wrote:
>> On 01/08/12 09:06, Mark Lawrence wrote:
>>>
>>> You complete ignoramus, if it gets poured in advance that's no good to
>>> anybody as it'll go flat. Has to stay in the pump until you're ready to
>>> drink it from the glass. Don't you know anything about the importance of
>>> process and timing? :)
>>
>> Heh heh, obviously never got drunk ... er I mean served behind the bar
>> at uni/college/pub %-}
>

> Nah, obviously *is* drunk ;p

Would you mind taking this slightly off-topic discussion off the list?

Thanks.

Stefan

[David]

On 01/08/2012, Stefan Behnel <stef...@behnel.de> wrote:
>
> Would you mind taking this slightly off-topic discussion off the list?

I always strive to stay on-topic. In fact immediately this thread went
off topic, 4 messages back, I did try to go off list, but got this
result from the OP:

Delivery to the following recipient failed permanently:
lip...@yahoo.co.uk
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the
recipient domain. We recommend contacting the other email provider for
further information about the cause of this error. The error that the
other server returned was: 554 554 delivery error: dd This user
doesn't have a yahoo.co.uk account (lip...@yahoo.co.uk) [-5] -
mta1050.mail.ukl.yahoo.com (state 17).
Date: Wed, 1 Aug 2012 09:31:43 +1000
Subject: Re: Is Python a commercial proposition ?
From: David <bounci...@gmail.com>
To: lipska the kat <lip...@yahoo.co.uk>

Then, if someone is going to call me an ignoramus on a public list,
they will receive a response in the same forum.

So, I apologise to the list, but please note the unusual circumstances. Thanks.

[Mark Lawrence]

I'm in stuck record mode here, but one of the things I really enjoy
about reading here is the way things do go off topic. IMHO makes for a
far more interesting experience. YMMV.

--
Cheers.

Mark Lawrence.

[lipska the kat]

On 02/08/12 04:10, David wrote:
> On 01/08/2012, Stefan Behnel<stef...@behnel.de> wrote:
>>
>> Would you mind taking this slightly off-topic discussion off the list?
>
> I always strive to stay on-topic. In fact immediately this thread went
> off topic, 4 messages back, I did try to go off list, but got this
> result from the OP:
>
> Delivery to the following recipient failed permanently:
> lip...@yahoo.co.uk

snip

This is my fault, I set the reply to address incorrectly. You HAVE
corresponded successfully with me in the past however... I apologise for
the inconvenience.

JFTR I did not call you an ignoramus (it's a funny word though isn't it,
makes me smile anyway).

[Prasad, Ramit]

> I'm in stuck record mode here, but one of the things I really enjoy
> about reading here is the way things do go off topic. IMHO makes for a
> far more interesting experience. YMMV.

+1

[Stefan Behnel]

Prasad, Ramit, 03.08.2012 08:51:

Huh? Who's still trying to sell viruses these days? I thought they came for
free?

Stefan

[Chris Withers]

On 30/07/2012 03:31, Rodrick Brown wrote:
> Hence the reason why no one will seriously look at Python for none
> glue work or simple web apps. When it comes to designing complex
> applications that need to exploit large multicore systems Python just
> isn't an option.

>
> Its still not possible to be a pure Python developer and find gainful
> employment today.

Oh come on, are you *actually* serious?

Pretty much everything you've said is nothing but inept bullshit.
Please stop.

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk

[lipska the kat]

On 23/08/12 22:46, Chris Withers wrote:
> On 30/07/2012 03:31, Rodrick Brown wrote:
>> Hence the reason why no one will seriously look at Python for none
>> glue work or simple web apps. When it comes to designing complex
>> applications that need to exploit large multicore systems Python just
>> isn't an option.
>>
>> Its still not possible to be a pure Python developer and find gainful
>> employment today.
>
> Oh come on, are you *actually* serious?
>
> Pretty much everything you've said is nothing but inept bullshit.
> Please stop.

OK, well excuse me for butting in but it's obvious from your website
that you have some experience with Python in the 'real world' I'd be
most interested to hear if you have experience of or have heard of
Python being used in any of the following circumstances.

Critical Systems:

wikipedia does a better job than I on the definition of a critical
system although I'm not suggesting that everything in this article
refers to a potential software system but it's a good illustration.

http://en.wikipedia.org/wiki/Life-critical_system

Real time systems:

An example of a real time system in this context is
an odds arbitrage back end publishing to the WWW via a
collection of web services, of course this is just one possible
interface, it could publish to another computer system or another HCI
This requirement of course implies a complete decoupling of view and
implementation. For more on odds arbitrage

http://en.wikipedia.org/wiki/Arbitrage_betting

It's 'real time' because publishing incorrect odds could cost someone a
great deal of money, the odds need to be available for manipulation by
the arbitrage engine as soon as they appear on the relevant bookies
website (in fact I've had them appearing _before_ this as I managed to
process them before the bookies web site did :-).

http://www.python.org/about/success/#real-time has a few examples
but I'd be interested to hear 'from the horses mouth'

Again, wikipedia is your friend
http://en.wikipedia.org/wiki/Real-time_computing

What is the largest Python project you have experience of, you can use
any metric you want, a simple KLOC, function point or cost analysis will
be fine.

This is a genuine enquiry and not designed to 'diss' Python in any way.


Many thanks

[Roy Smith]

On Sunday, July 29, 2012 12:01:00 PM UTC-4, lipska the kat wrote:
> How is python used in the real world.

songza.com is pretty close to 100% python. The only significant non-python code on the server side are mongodb, haproxy, and nginx.

> What sized projects are people involved with

We've got 102 KLOC in the application (i.e. code we wrote), plus another 104 KLOC in django and 337 KLOC of other installed packages (i.e. everything in our virtualenv's site-packages directory).

Those numbers are just the raw output of doing "find ... -name *.py | xargs wc -l", so they include blank lines and comments. So, all told, roughly a half million lines of python code.