Re: When will Python go mainstream like Java?
Krister Svanlund
> That will be superb
>
> --
> Passion is my style
And when will <insert random band> be as famous as the beatles?
Phlip
wrote:
> And when will <insert random band> be as famous as the Beatles?
And when will <insert random non-schmaltzoid singer) be as famous as
Phil Collins?
Edward A. Falk
--
-Ed Falk, fa...@despams.r.us.com
http://thespamdiaries.blogspot.com/
Shawn Milochik
Shawn
Jonathan Gardner
It already has.
--
Jonathan Gardner
jgar...@jonathangardner.net
Chris Rebert
<jgar...@jonathangardner.net> wrote:
> On Mon, Feb 22, 2010 at 1:56 PM, AON LAZIO <aonl...@gmail.com> wrote:
>> That will be superb
>>
> It already has.
Indeed. Python is at position 7, just behind C#, in the TIOBE Index:
http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
Although for Java-level mainstreamness, you'd probably need to be in
the top 3 or 4.
Cheers,
Chris
--
The TIOBE Index is by no means perfect though.
http://blog.rebertia.com
Steve Howell
> On Mon, Feb 22, 2010 at 8:36 PM, Jonathan Gardner
>
> > On Mon, Feb 22, 2010 at 1:56 PM, AON LAZIO <aonla...@gmail.com> wrote:
> >> That will be superb
>
> > It already has.
>
> Indeed. Python is at position 7, just behind C#, in the TIOBE Index:http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
>
> Although for Java-level mainstreamness, you'd probably need to be in
> the top 3 or 4.
>
> Cheers,
> Chris
> --
> The TIOBE Index is by no means perfect though.http://blog.rebertia.com
I am sure Python could rise to number six with some squigglies! ;)
Krister Svanlund
Javas popularity was very much a product of its time. It was something
new and exciting and people got a bit too excited maybe, Python just
does the same thing but better really, therefor it will not become as
popular.
Ishwor Gurung
Java - Monstrous language that was Sun's flagship language. Now, it's Oracles.
Python - Hobby-ish hacking language that we all love so much (that we
wish everything was written using Python).
Java - The JVM code been hacked to death by Sun engineers (optimised)
Python - The PVM code has seen speed-ups in Unladen or via Pyrex..
ad-infinitum but nowhere as near to JVM
I like both Python and Java but given the amount of resources put into
JVM and Java (JEE is _huge_ in Enterprise if you didn't know that
already and there are universities that speak Java fluently), it's
kind of sad that Python till the day hasn't seen speedup in mainline
releases.
I see Python more as a hacker's language which will gradually evolve
and support SMEs and alike in the long run than Java (and of course we
write our weekend-only hacking projects in it :-) but for a
market-uptake like Java requires universities, colleges and students
to learn this wonderful little language and requests energetic hackers
to fix lock-contention issues and the like in the core implementation.
Perhaps I see a light, perhaps I see nothing.. but I feel the day is
coming nearer when Python would run as fast as Java/C. Only time can
tell - I hope the time is right about this.
--
Regards
Ishwor Gurung
Key id:0xa98db35e
Key fingerprint:FBEF 0D69 6DE1 C72B A5A8 35FE 5A9B F3BB 4E5E 17B5
Richard Lamboj
Am Tuesday 23 February 2010 09:07:43 schrieb Krister Svanlund:
> On Tue, Feb 23, 2010 at 1:01 AM, Edward A. Falk <fa...@mauve.rahul.net>
wrote:
> new and exciting and people got a bit too excited maybe, Python just
> does the same thing but better really, therefor it will not become as
> popular.
Good morning,
i don't like Java/JSP, the synthax is blown up and the programs are damn slow.
For ecllipse you should buy a cluster. There is C/C++/D, Python, Ruby,
Gambas, TCL, PHP, SmallTalk and some other nice Programming Languages, so i
don't understand why people use Java. "Java is the one an only OOP Language,
the best one" - Yeah and whats with multiple inheritance? I'am in love with
Python ;-)
Kind Regards
hackingKK
> Am Tuesday 23 February 2010 09:07:43 schrieb Krister Svanlund:
>
>> On Tue, Feb 23, 2010 at 1:01 AM, Edward A. Falk<fa...@mauve.rahul.net>
>>
> wrote:
>
>> new and exciting and people got a bit too excited maybe, Python just
>> does the same thing but better really, therefor it will not become as
>> popular.
>>
> Good morning,
>
> i don't like Java/JSP, the synthax is blown up and the programs are damn slow.
> For ecllipse you should buy a cluster. There is C/C++/D, Python, Ruby,
> Gambas, TCL, PHP, SmallTalk and some other nice Programming Languages, so i
> don't understand why people use Java. "Java is the one an only OOP Language,
> the best one" - Yeah and whats with multiple inheritance? I'am in love with
> Python ;-)
>
There are a few reasons why we don't see python as a "buz word". Java
was well marketed and the time when it came out with libraries like
swing, there was no popularly known alternative.
As a matter of fact I don't really go by popularity with technologies,
specially when it comes to programming languages.
Just show me 2 or 3 big apps or web sites which are scalable and take
multiple requests. show me just 2 instances where heavy number
crunching is done efficiently and I am convinced.
I don't care how many apps are developed using java as long as they
remain heavy and sloooooooow.
google runs on python and so do many other big applications.
marketing is more about exaggeration, which Sun did for Java.
Python was always in the hands of programmers who wanted their work done
and wanted scalable apps.
So the conclusion is that "all that is popular need not be good for
every thing ".
Happy hacking.
Krishnakant.
Stefan Behnel
> Indeed. Python is at position 7, just behind C#, in the TIOBE Index:
> http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
That index is clearly flawed. A language like PHP (whatever that is
supposed to be comparable with) can't possibly be on the rise, can it?
Stefan
Bruno Desthuilliers
(snip)
> I don't care how many apps are developed using java as long as they
> remain heavy and sloooooooow.
> google runs on python
Please get your facts right.
Python is one of the languages used internally at Google, true, but so
is Java.
And google-the-search-engine does not "run on python".
Roald de Vries
> That will be superb
I guess static typing will have to be added, so that tools like
eclipse can inspect (and autocomplete) your programs [better].
Bruno Desthuilliers
Yet another troll...
sste...@gmail.com
On Feb 23, 2010, at 10:10 AM, Martin P. Hellwig wrote:
> Actually I am still waiting for Java to be mainstream :-)
> You could say it is popular, which it is without doubt but in my opinion after C handed over it's pseudo de facto standard (mostly because a lot of OS'es are written in it) nobody else has had enough momenta to reach for that crown.
>
> Actually I quite like the soup of languages these days, what amuses me though, that Python seems to emerge as the de facto glue language to bind them all :-)
I'm sure there's a Tolkien 1-liner in there somewhere ;-).
S
mk
> That will be superb
Well I for one wouldn't want Python to go exactly Java way, see this:
http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=java&l=uk
This is the percentage of job offers in UK where the keyword "Java" appears.
Same for C#, it looks like C# is eating Java's lunch now:
http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=csharp&l=uk
What worries me somewhat (although not much) is that after long period
of solid growth the market can't decide about Python:
http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=python&l=uk
I learned painfully that in corporate setting merits of a programming
language do not matter much, it's more like "whatever catches the
groupthink" at the moment. "Java is good because big ones select Java",
"static typing is good because compiler catches programmer's errors"
(this one is particularly appealing to managers I found), etc.
Although all my "internal use" tools are written in Python, there's no
way I could convince managers to use Python as the main application
devel language.
This, however, is not of itself a problem: as long as language is lively
and has at least a few percent of programmers using it -- which is
important for existence of libraries, not much more -- there's no
problem for people who want to get ahead of competition / waste less
time by using advanced programming langauges. Frankly, I have yet to
encounter a problem for which either a sizable Python extension or
bindings to a popular library wouldn't exist. This in itself is a
hallmark of a language being "enough of mainstream to actually matter in
practice".
This I find quite insightful: http://www.paulgraham.com/avg.html
Regards,
mk
mk
Well it looks like it is at least stabilized:
http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=php&l=uk
I find job offers to be rather good index of the extent to which the
language is actually used, and this is what this index is based on
(percentage of job offers with the keyword "php" in them).
Regards,
mk
Martin P. Hellwig
You could say it is popular, which it is without doubt but in my opinion
after C handed over it's pseudo de facto standard (mostly because a lot
of OS'es are written in it) nobody else has had enough momenta to reach
for that crown.
Actually I quite like the soup of languages these days, what amuses me
though, that Python seems to emerge as the de facto glue language to
bind them all :-)
--
mph
Ben Finney
Why not? What do you think the TIOBE measures, and why would PHP not be
rising by that measure?
--
\ “To save the world requires faith and courage: faith in reason, |
`\ and courage to proclaim what reason shows to be true.” |
_o__) —Bertrand Russell |
Ben Finney
Lawrence D'Oliveiro
wrote:
> Actually I am still waiting for Java to be mainstream :-)
Too late, I think. Sun dilly-dallied over making it open source for too
long, until it practically didn’t matter any more.
Lawrence D'Oliveiro
Gurung wrote:
> Java - The JVM code been hacked to death by Sun engineers (optimised)
> Python - The PVM code has seen speed-ups in Unladen or via Pyrex..
> ad-infinitum but nowhere as near to JVM
Python is still faster, though. I think a key reason is that its VM supports
reference-counting, which the Java folks never quite got the grasp of.
So you see, it’s all about working smarter, not harder.
Nobody
>> Java - The JVM code been hacked to death by Sun engineers (optimised)
>> Python - The PVM code has seen speed-ups in Unladen or via Pyrex..
>> ad-infinitum but nowhere as near to JVM
>
> Python is still faster, though. I think a key reason is that its VM supports
> reference-counting, which the Java folks never quite got the grasp of.
So how does Python handle circular references?
Lie Ryan
There is an optional garbage collector built into the interpreter. Look
at here on how it works: http://python.ca/nas/python/gc/
George Sakkis
> Well I for one wouldn't want Python to go exactly Java way, see this:
>
> http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=jav...
>
> This is the percentage of job offers in UK where the keyword "Java" appears.
>
> Same for C#, it looks like C# is eating Java's lunch now:
>
> http://www.itjobswatch.co.uk/charts/permanent-demand-trend.aspx?s=csh...
This seems to be a UK-specific trend; in the US (and most other
countries I know of) Java is still going strong, e.g.
http://www.indeed.com/jobtrends?q=java%2C+c%23&l=
George
Steve Holden
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS: http://holdenweb.eventbrite.com/
Steve Holden
> Chris Rebert, 23.02.2010 06:45:
>> Indeed. Python is at position 7, just behind C#, in the TIOBE Index:
>> http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
>
> That index is clearly flawed. A language like PHP (whatever that is
> supposed to be comparable with) can't possibly be on the rise, can it?
>
established survey. All indexes are flawed, they are merely flawed in
different ways is all. I track TIOBE because it shows trends. Python's
absolute position on the list doesn't interest me so much.
Steve Holden
Peter Parker
> At 12.34 pm on November 13, 2011
>
At December 21, 2012 at 11:11 am (according to the Maya calendar)
Martin P. Hellwig
On August 29, 1997, Java became mainstream. In a panic, Microsoft tried
to embrace, extend and exterminate the system, prompting Sun to
retaliate with a lawsuit, knowing that Microsoft's counterattack would
eliminate all its main competitors in the U.S. This initiates an
indeterminately long period of new language development culminating in a
battle against corporate monopoly, which gained ever-increasing
capabilities of FUD.
--
mph
mk
Interesting, and I was thinking that UK sample was big enough for such
things not to matter.
Regards,
mk
Wanja Gayk
<l...@geek-central.gen.new_zealand>:
>> Java - The JVM code been hacked to death by Sun engineers (optimised)
>> Python - The PVM code has seen speed-ups in Unladen or via Pyrex..
>> ad-infinitum but nowhere as near to JVM
>
> Python is still faster, though.
In synthetic microbenchmarks without any practical importance - possibly.
> I think a key reason is that its VM supports
> reference-counting, which the Java folks never quite got the grasp of.
Reference counting is about the worst technique for garbage collection.
Modern Java VM won't count references. They will just trace the active
references from the rootand mark all objects it finds as active and save
these. The remaining ones are garbage. The reason why this is faster is
that there are usually less live objects than dead ones and there are less
refereces to look at.
kind regards
W
--
Erstellt mit Operas revolutionärem E-Mail-Modul: http://www.opera.com/mail/
--- news://freenews.netfront.net/ - complaints: ne...@netfront.net ---
Lawrence D'Oliveiro
> Reference counting is about the worst technique for garbage collection.
It avoids the need for garbage collection. It means I can write things like
contents = open(filename, "r").read()
and know the file object will be immediately closed after its contents are
returned.
It also means I don’t have to predefine how much memory my program will use.
A garbage collector will only kick in when the app runs low on memory. On a
system with dynamic memory allocation, that will not happen until the entire
system runs low on memory, unless you limit the app to some fixed amount.
Which is an antiquated way of doing things.
And then there’s caching. Modern CPUs owe most of their speed to assumptions
that programs will obey locality of reference. Pointer-chasing is a cache-
hostile activity. Garbage collection involves a lot of pointer-chasing,
particularly of dead objects that have long since been flushed from the
cache, compared with reference counting of recently-accessed objects.
Therefore garbage collection loses performance.
Ben Finney
> [Reference counting] avoids the need for garbage collection. It means
> I can write things like
>
> contents = open(filename, "r").read()
>
> and know the file object will be immediately closed after its contents
> are returned.
Not quite; it means you know that the object *becomes a candidate* for
cleanup immediately after its contents are returned.
When, or whether, that cleanup actually happens is not something that is
necessarily promised by a reference-counting implementation.
--
\ “I tell you the truth: some standing here will not taste death |
`\ before they see the Son of Man coming in his kingdom.” —Jesus |
_o__) Christ, c. 30 CE, as quoted in Matthew 16:28 |
Ben Finney
sjde...@yahoo.com
central.gen.new_zealand> wrote:
> In message <op.u8nfpex8y5e8ok@laptopwanja>, Wanja Gayk wrote:
>
> > Reference counting is about the worst technique for garbage collection.
>
> It avoids the need for garbage collection.
That's like saying that driving a VW Beetle avoids the need for an
automobile. Reference counting is a form of garbage collection (like
mark-sweep, copy-collect, and others), not a way of avoiding it.
You're right that ref counting in many implementations is more
deterministic than other common forms of garbage collection; IMO,
Python would be well-served by making the ref-counting semantics it
currently has a guaranteed part of the language spec--or at least
guaranteeing that when a function returns, any otherwise unreferenced
locals are immediately collected.
I could be convinced otherwise, but I _think_ that that change would
offer an alternative to all of the interesting cases of where the
"with" statement is "useful".
Paul Rubin
> IMO, Python would be well-served by making the ref-counting semantics it
> I could be convinced otherwise, but I _think_ that that change would
> offer an alternative to all of the interesting cases of where the
> "with" statement is "useful".
The whole point of introducing the "with" statement was help cleanly get
rid of the ugly and unsound reliance on refcounting semantics found in
so much python code. Your proposal is aimed in the wrong direction.
Paul Rubin
> involves a lot of pointer-chasing, particularly of dead objects that
> have long since been flushed from the cache, compared with reference
> counting of recently-accessed objects. Therefore garbage collection
> loses performance.
Serious gc's these days only touch live data, not garbage; and they work
on small enough regions of memory most of the time (generational
scavenging) to not have too many cache misses. Also, hyperthreading
seems to be coming back, allowing cpu cores to multitask in the presence
of cache misses.
Gregory Ewing
> And then there’s caching. Modern CPUs owe most of their speed to assumptions
> that programs will obey locality of reference. Pointer-chasing is a cache-
> hostile activity.
Another thing to consider is the rate at which garbage is
created. Java's fundamental types (ints, floats, etc.) are
unboxed, and objects are only used for relatively heavyweight
things. So you can do quite a lot of useful computation in
Java without creating any objects or leaving behind any
garbage.
In Python, on the other hand, you can't even do arithmetic
without creating and destroying intermediate objects at
every step. Collecting those objects promptly means that
the same areas of memory tend to get re-used over and
over, resulting in better cacheing behaviour.
So I don't think you can assume that just because Java's
garbage collection scheme works well for Java that it would
also necessarily work well for Python.
--
Greg
Alf P. Steinbach
Well, relying on ref-counted garbage collection for cleanup actions (other than
reclaiming memory) is pretty brittle in a language based on general garbage
collection. As soon as a reference to X sneaks away somewhere, X doesn't clean
up at scope's end, which is not a catastrophe in itself but means there's no
telling when it happens or indeed whether it happens at all.
I'm not sure but as I recall the D language has a solution to this, sort of
declaring X so that no references to it can be retained anywhere, but it's a
very different language. And Eiffel has the "expanded type" thing for "no
references", but does it use that to support deterministic cleanup? I don't know
whether the Eiffel 'dispose' is called for an expanded object.
C# has a 'using' like Python 'with'. It works, sort of, but adds a lot of
complexity. C++ has the notion of optional garbage collection (formalized in
C++0x) with cleanup actions not tied to memory reclamation but to
deterministically invoked destructors and I think that as an /idea/ that is
clean and good, but in practice almost no-one uses garbage collection with C++,
and reportedly the least unpopular implementation (the Boehm collector) imposes
some very stiff requirements on the code.
So I think there's no really good solution: the price for simplicity in one
dimension is some complexity in another dimension, here deterministic cleanup
and the problem of zombie objects (garbage collection simplifies a lot of things
while zombie objects, objects that have had explicit cleanup performed and thus
are invalid in a sense, compensate by adding back in a lot of complexity).
Cheers,
- Alf
Paul Rubin
> So I think there's no really good solution: the price for simplicity
> in one dimension is some complexity in another dimension,
You could look at the way Cyclone (http://cyclone.thelanguage.org) does
region inference, and maybe combine that with a tracing gc. There are
some functional language compilers that also use region inference.
Chris Rebert
You're forgetting global context objects, such as those for Decimal:
http://docs.python.org/library/decimal.html#decimal.localcontext
Cheers,
Chris
--
http://blog.rebertia.com
Stefan Behnel
> things not to matter.
Lies, damn lies, and statistics ...
Stefan
Lie Ryan
> Am 24.02.2010, 00:22 Uhr, schrieb Lawrence D'Oliveiro
> <l...@geek-central.gen.new_zealand>:
>
>>> Java - The JVM code been hacked to death by Sun engineers (optimised)
>>> Python - The PVM code has seen speed-ups in Unladen or via Pyrex..
>>> ad-infinitum but nowhere as near to JVM
>>
>> Python is still faster, though.
>
>
>> I think a key reason is that its VM supports
>> reference-counting, which the Java folks never quite got the grasp of.
>
> Reference counting is about the worst technique for garbage collection.
I disagree IMO, assuming there is no circular reference, reference
counting is the simple, lightweight, and best garbage collection
technique. Reference counting can free memory (or return it to cache)
right after the refcount turns zero; other traditional garbage
management techniques runs periodically and there's a delay between
getting to zero reference to actual memory release. The other weakness
of traditional garbage managers is the symptomatic "stopping the time"
whenever the GC is running, which makes it unsuitable for time-sensitive
program.
The only weakness of reference counting is its inability to detect
circular reference, which is why CPython have an additional garbage
collector.
> Modern Java VM won't count references. They will just trace the active
> references from the rootand mark all objects it finds as active and save
> these. The remaining ones are garbage. The reason why this is faster is
> that there are usually less live objects than dead ones and there are
> less refereces to look at.
Reference counting is even faster since there is only one object to be
looked at: the object that is being dereferenced. There is no need to
look at other live objects, there is no need to look other dead objects,
just the current object.
Silvio
"Lawrence D'Oliveiro" <l...@geek-central.gen.new_zealand> wrote in message
news:hm4icr$q4o$1...@lust.ihug.co.nz...
Reference counting is a technique that is only applicable for a subset of
scenarios GC can handle. RC fails when circular dependencies exist. In an
ancient past I have worked with RC-based C++ frameworks and they where
either very restrictive to prevent circular dependencies or ignored them
resulting in memory leaks.
In addition, for fast create/destroy scenario's (for examples loops that
allocate objects with single cycle lifetimes) a generational GC is MUCH
faster than manual malloc/free new/delete code because it enables a
completely different heap management logic. Reference counting only adds to
the slowness in this scenario.
You are also incorrect in stating that a GC will only kick in when memory
becomes scarce. Partial GCs will happen very frequently (thereby
invalidating your point about caching) and are extremely fast. Only full GCs
are infrequent and take more time but they will typically run in a separate
thread.
No general statement about the relative speed of generational GC systems
versus manual heap management (or the slower variant RC) can be made since
they have completely different best, general and worse case behavior and
scenarios.
In response to your original post: please provide links to examples of
quality benchmark results. There are numerous language speed comparison
benchmarks that usually put Python in the tail region with Ruby and the
likes for many different applications. If you have opposing examples then
you should provide them.
Silvio
Mel
> You're right that ref counting in many implementations is more
> deterministic than other common forms of garbage collection; IMO,
> Python would be well-served by making the ref-counting semantics it
> currently has a guaranteed part of the language spec--or at least
> guaranteeing that when a function returns, any otherwise unreferenced
> locals are immediately collected.
That would be nice. The argument against it is that it would make
implementations like Jython very difficult to make, if it didn't rule them
out altogether.
Mel.
Wanja Gayk
<l...@geek-central.gen.new_zealand>:
> In message <op.u8nfpex8y5e8ok@laptopwanja>, Wanja Gayk wrote:
>
>> Reference counting is about the worst technique for garbage collection.
>
> It avoids the need for garbage collection. It means I can write things
> like
>
> contents = open(filename, "r").read()
>
> and know the file object will be immediately closed after its contents
> are returned.
Which has nothing to do with reference counting in general.
This is what JVMs can do aswell.
Just have a look at the behaviour of Weak- and SoftReferences to get an
idea what goes on behind the scenes.
For example:
int x = new Point(10,20).getX();
The point object will be collectable as soon as getX() returned - even
better: todays JIT compilers will inline the call and not even allocate
the object, if it's allocation is free of side effects.
What you write above is something entirely different: it's just a
mechanism to release file handles, syntactic sugar for what you have in
java when you close a stream in a finally-block.
If you like to know a litle more about reference counting, feel free to
study the following paper on a very sophistcated reference counting GC for
the JVM by Levanoni and Petrank:
http://reference.kfupm.edu.sa/content/e/o/e___an_on_the_fly_reference_counting_gar_61636.pdf
The paper talks about some ofthe problems that occur with reference
counting.
Their algorithm uses some nice techniques to avoid thread contention (the
reference-counter updates must be atomic, thus require that all "mutators"
are stopped) and other problems connected to "referece counting" in
general, such as slow writes (-> cache coherence, etc. read-only access is
faster).
What the paper doesn't tell is how they deal with fragmented heap, so I
wouldn't trust that this Refcount-GC will have a stable performance over a
long period of time (we're talking apps like Ebay, which run 24/7 for
months if not years).
Their algorithm may have a "comparable" speed to the (meanwhile) old
implementation the JVM's GC, but it still needs the original JVM clean up
what it leaves behind(it forgets some references) - maybe that's also how
they deal with a fragmeted heap, as java uses a compacting collector.
So, yes, it can be fast, or let's say responsive, but in general it's not.
And when it is up to speed you get stability issues at least I have not
heard anything different yet.
To have some numbers:
Benchmark Original RC
Total 2582.2 2676.0
compress 720.8 723.3
db 374.0 383.7
jack 264.6 299.7
javac 225.0 235.2
jess 181.7 209.7
mpegaudio 607.1 610.6
As you see: SUN's GC is faster. Even though: what those folks have done is
impressive (but buggy).
> It also means I don’t have to predefine how much memory my program will
> use.
Which again has nothing to do with GC. This has been a design decision.
In fact todays JVM dynamically allocate memory and gro their heaps.
A quick look at the command line options will make that clear:
java -X
[...]
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
Todays GC run as folows, they initialize the heap size at what you giove
them in your Xmx-Setting (or use the default).
When the heap is filled up to that initial size, they do a full GC and if
that won't free enough memory, they will allocate some more.
This is done until the upper limit from -Xmx is hit, when the program
requires more than that, it will fail with an OutOfMemoryError.
As you can imagine those limits may influence the GC, but they are no
requirement for a GC.
Those limits exists for the security of the host system (which may not be
a system that allows setting own itself, like Unix/Linux).
> A garbage collector will only kick in when the app runs low on memory.
Wrong. Please inform yourself. Current generational GCs have minor and
major collections. The JLS only requires that before the program runs out
of memory the garbage has to be collected, which does not mean it cannot
do that earlier. In fact you may also request a full GC by calling
"System.gc()" when you know there's time for it. But it's not recommended
in general. With JDK 7 we will get a new "Garbage First" Collector, which
works differently than the ones we have today. It will promise lower
response times and a more predictable behaviour with about the same speed.
Starting with Java6 update 14 you may try that one (if you like, I can
give ou the command line switch to activate it). Basically it is also a
copying collector and in some way it's also generational, but it's
"promotion" strategy is different.
Anyway, it also won't only clean up when memory is low, but when it
detects that there is a segment which is hardly live.
> On a
> system with dynamic memory allocation, that will not happen until the
> entire
> system runs low on memory, unless you limit the app to some fixed amount.
> Which is an antiquated way of doing things.
Which is wrong, see above.
Today eve the C++ folks start using Garbage Collectors, because
allocating/deallocating space using free-lists has proven to be slower
than copying live objects.
> And then there’s caching. Modern CPUs owe most of their speed to
> assumptions that programs will obey locality of reference.
Well, and what's the point? Stack allocation is just not necessary in a
copy-colllector, bevause such a collector knows no deallocation (it just
saves the survovors), so there is no gain from stack allocation. Stack
allocation is faster in traditional C++ programs (without GCs), because
the free-lists have to be managed.
Todays HotSpot-Compilers take memory-locality into account aswell, just
read some papers about it, have a look at "escape analysis" for example.
Some further reading:
http://www.ibm.com/developerworks/java/library/j-jtp09275.html
http://www.ibm.com/developerworks/java/library/j-jtp04223.html
> Pointer-chasing is a cache- hostile activity. Garbage collection
> involves a lot of pointer-chasing,
> particularly of dead objects that have long since been flushed from the
> cache, compared with reference counting of recently-accessed objects.
> Therefore garbage collection loses performance.
You haven't read my previous posts, have you?
Dead objects are not considered by today's Garbage collectors.
Those trace only "reachable" objects and copy those. They won't even look
at "dead" objects.
Since most objects die young, this is where you gain a lot of performance.
http://www.ibm.com/developerworks/library/j-jtp01274.html
Reference counters, on the other hand, must detect when the reference
count falls to zero. So they must look at each object, no matter if they
are about to die, just to find out if they will die.
Regards
-Wanja-
Chris Gray
> In message <op.u8nfpex8y5e8ok@laptopwanja>, Wanja Gayk wrote:
>
> > Reference counting is about the worst technique for garbage collection.
>
> It avoids the need for garbage collection. It means I can write things like
I'm by no means an expert, but how does reference counting deal with
arbitrary long cycles of references (btw I've *written* a simple
reference counter for a programming language)?
When I asked someone whose knowlege of Java I trust, he said that modern
Java's do both reference counting and garbage collection. That was 2 or 3
years ago. I would have guessed that Python was the same.
--
Experience should guide us, not rule us.
Chris Gray
Alf P. Steinbach
> Lawrence D'Oliveiro <l...@geek-central.gen.new_zealand> writes:
>
>> In message <op.u8nfpex8y5e8ok@laptopwanja>, Wanja Gayk wrote:
>>
>>> Reference counting is about the worst technique for garbage collection.
>> It avoids the need for garbage collection. It means I can write things like
>
> I'm by no means an expert, but how does reference counting deal with
> arbitrary long cycles of references (btw I've *written* a simple
> reference counter for a programming language)?
Generally it doesn't. Some solutions have however been proposed for
std::shared_ptr in C++. None have made it to any noteworthy status, but they
demonstrate that there are more or less practical solutions for the most common
cases.
Currently the upshot is that if you need the kind of self-referential spaghetti
structure (no offense) created by mathematical expressions that refer to each
other, then you've passed beyond what can be handled practically by reference
counting alone or even in combination with cycle handling techniques.
For that you're better off with some functional language, or Python... ;-)
> When I asked someone whose knowlege of Java I trust, he said that modern
> Java's do both reference counting and garbage collection. That was 2 or 3
> years ago. I would have guessed that Python was the same.
Yes, Python adds general garbage collection to deal with cycles.
Essentially the reference counting deals efficiently and immediately with
objects created by expression evaluation, while the general garbage collection
deals with cyclic structures.
But there's no magic: in particular in Java, Python and like languages you must
still remember to remove references installed in singletons and globals (e.g.
for event notifications), otherwise objects will still be referenced and thus
not garbage collected, causing a memory leak.
Cheers & hth.,
- Alf
Steven D'Aprano
> I'm by no means an expert, but how does reference counting deal with
> arbitrary long cycles of references (btw I've *written* a simple
> reference counter for a programming language)?
It doesn't, or at least the CPython one doesn't.
In CPython, even a short reference cycle defeats it, which is why CPython
also uses a second garbage collector. If we turn that off, we can see how
even a one-element cycle defeats reference counting:
>>> import gc
>>> gc.disable()
>>>
>>> class K:
... def __del__(self):
... print "Bye now"
...
>>> k = K()
>>> del k # Test it works as expected.
Bye now
>>>
>>> k = K() # Now try with a cycle.
>>> k.attr = k
>>> del k
>>>
>>>
Normally, you would clear the ref cycle this way:
>>> gc.collect()
2
But in this case it doesn't work because the object in the cycle has a
__del__ method, and Python can't predict a safe order to run the __del__
method. (I'm surprised that this is an issue with a single object, but
what do I know?) So we have to inspect the list of garbage, manually
break the cycle, clear new references we've created, and then it will be
garbage collected:
>>> gc.garbage[0].attr = None # Break the cycle
>>> del gc.garbage[:]
Bye now
This is one reason why __del__ methods are not recommended.
> When I asked someone whose knowlege of Java I trust, he said that modern
> Java's do both reference counting and garbage collection. That was 2 or
> 3 years ago. I would have guessed that Python was the same.
CPython has both now. Jython uses whatever the Java implementation uses,
and IronPython uses .Net's memory management scheme. Other Pythons do
whatever they like :)
Some interesting information here:
http://www.digi.com/wiki/developer/index.php/Python_Garbage_Collection
--
Steven
Mensanator
Is that really considered a leak, a single instance
of an uncollected global as opposed to a function
that is called millions of times, and each time
it's called creates uncollected temporary variables?
Steven D'Aprano
> Is that really considered a leak, a single instance of an uncollected
> global as opposed to a function that is called millions of times, and
> each time it's called creates uncollected temporary variables?
I think it's generally called "garbage". When the amount of garbage
grows, or *could* grow, it is called a leak. But nobody is going to slap
you for calling a single instance of an uncollectable object a leak.
--
Steven
John Nagle
> Lawrence D'Oliveiro wrote:
>> And then there’s caching. Modern CPUs owe most of their speed to
>> assumptions that programs will obey locality of reference.
>> Pointer-chasing is a cache-
>> hostile activity.
>
> Another thing to consider is the rate at which garbage is
> created. Java's fundamental types (ints, floats, etc.) are
> unboxed, and objects are only used for relatively heavyweight
> things. So you can do quite a lot of useful computation in
> Java without creating any objects or leaving behind any
> garbage.
>
> In Python, on the other hand, you can't even do arithmetic
> without creating and destroying intermediate objects at
> every step.
This is really a CPython implementation problem. Face it,
CPython is a "naive interpreter". It pretty much does the obvious.
In other words, the code for the worst case is used for all cases.
It doesn't optimize out reference count updates, do type inference
to avoid unnecessary boxing, or figure out at compile time which
objects could be "slotted" and don't need dictionary lookups for
fields. Plus, of course, there's the GIL problem.
Reference counts aren't inherently slow. About 90% of reference
count updates could be optimized out. The compiler needs to understand
what's a temporary object and what isn't. See
The language isn't inherently slow, but CPython is.
The Shed Skin developer has shown what's possible. He
doesn't have enough resources to finish a full implementation,
but he's on the right track. There's grumbling about the
restrictions in Shed Skin, but only some of Shed Skin's
restrictions are inherently necessary. The one Shed Skin
developer has accomplished far more than the PyPy army of ants.
John Nagle