Non biased performance comparison between Java and Objective-C on GNUStep

[Carl Jokl]

This is not any kind of flame bait thread.

I am hoping to get some unbiased information on the relative
performance difference between Java and Objective-C.

More specifically using up to date versions of the Java 6 JRE and
using an up to date GNUStep implementation of Objective-C both running
on Linux.

I can find quite a few articles with people comparing Java vs C++ but
I suppose Objective-C is more niche and tends not to be used in
comparisons.

I know that the opinion still prevails in many circles that Java is
slower and/or significantly slower than code written in C/C++. There
have been articles more recently stating that Java was capable of
outperforming C++ these days.

I am cautious with benchmarks because obviously if someone is trying
to prove a point, that person can be selective about using benchmarks
which support their case.

I know for example that Java has performance limits doing lots of
heavy floating point operations or trigonometry due to not using
native acceleration where it would cause the values to be less
accurate than required by the specifications (Though I believe this
was being addressed).

That used case may only apply to applications that use a lot of
floating point calculations. Many business web based or enterprise
applications may do very little floating point number crunching.

For the sake of context I am in a company that has a system where Java
is used with Servlets to provide a web front end to a system the core
of which is written in Objective-C running on GNUStep.

Considering the dates in some of the Java source I know the system
must have existed at least as long ago as the year 2000.
Back then the versions of Java would have been a lot slower. Java 6
included a big performance increase. It would have made sense back
then to argue that the Objective-C code was going to probably perform
better than Java. Now I don't think the difference would be as big. It
would be within my margin of error that Java may outperform Objective-
C due to compile time optimisation that can be done in Java that
cannot be done in Objective-C that is more geared towards dynamic,
runtime behaviour. In that case I would expect C++ to be faster than
Objective-C generally speaking.

I am hoping to get some feedback from the community to sanitise my
assumptions here.

I don't want to end up looking like an idiot if Objective-C
significantly outperforms Java.

The JNI overhead of having Java talk to Objective-C could cancel out
or more than cancel out any performance gains in Objective-C if the
difference is only slight. This depends also whether the cross
communication comprises of many small JNI calls vs few calls that do a
lot of processing before the call returns.

Any performance information / experience would be helpful.

[Kevin Wright]

Actually, one of the main performance benefits Java has is pre-allocating memory.

C/C++/etc will often grab their (non-stack) memory on demand via malloc/free, which is what default invocations of new and delete will actually do behind the scenes.

Java grabs a big lump up front, "allocation" is then reduced to moving a pointer up and down this pre-allocated heap space.

malloc is slow, sometimes very slow, it all depends on your OS, virtual memory, paging, etc, etc.

Moving a pointer is very very fast.

Other JVM optimisations aside, this one difference alone can make Java programs significantly faster that C/C++/Obj-C equivalents.

[Carl Jokl]

If you were in my situation, would you feel comfortable putting your
professional credibility out on a limb to say that Java could be
faster than Objective-C to the technical manager?

In theory and from what I know I think it could well be faster. That
is an unbiased view and the unbiased view also hold that it is within
the margin of error for it to be a bit slower too. What I don't have
is any actual performance benchmarks which would give me empirical
evidence for comparison.

I wonder also if it could end up being anecdotal as I really don't
know if the suggestion of porting the Objective-C part of the system
to Java would be dismissed as totally impractical. However it is not a
huge company and the system isn't massive. It is big enough that it
could take significant time but not hopeless.

The other pain with the setup is that the Java code cannot be compiled
on its own. It uses a tool called Jigs to allow Java code to call
Objective-C by generating wrappers to make the objects look like Java
ones. I suppose this might be a GNUStep answer to SWIG with C++.

The compilation uses GNU makefiles because the extra dependency
resolution is being handled by GNU Step extensions to GNU make.

The practicality of this means every new Java file I create is having
to be added to a GNU make file. Each new package requires a GNU make
file in it and the parent package make file has to be updated to add
the new sub directory. This all feels like a bunch of extra tedious
work that I never had to do before when working with Ant. It is an
annoyance but liveable. When illegal argument guys argue the merits of
Ant vs Ivy vs Maven maybe they should spare a thought for the poor guy
using GNU make.

[Ricky Clarkson]

I imagine that implementations of malloc/free exist or could be written that preallocate. Thus this advantage can be achieved in C.

---

From: Kevin Wright <kev.lee...@gmail.com>

Sender: java...@googlegroups.com

Date: Thu, 1 Dec 2011 11:46:34 +0000

To: <java...@googlegroups.com>

ReplyTo: java...@googlegroups.com

Subject: Re: [The Java Posse] Non biased performance comparison between Java and Objective-C on GNUStep

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Carl Jokl]

I don't know if GNUStep preallocates or not. I might have to drudge
through the documentation and/or user groups to find out.

[Kevin Wright]

Sure you can.  The so-called "placement new"[1] allows you exact control over where you stick your objects.

There are also various implementations available, such as pool[2], from the Boost libraries

There's a limit to how far you can take this though.  C++ still forces you to make a lot of it far more explicit, so there's a cognitive overhead to how far you can take it in an ever-growing system.

It also makes for some real fun and games when debugging & profiling

[1] http://stackoverflow.com/questions/222557/cs-placement-new

[2] http://www.boost.org/doc/libs/1_48_0/libs/pool/doc/html/boost_pool/pool/introduction.html

[Fabrizio Giudici]

On Thu, 01 Dec 2011 13:11:18 +0100, Carl Jokl <carl...@gmail.com> wrote:

> I don't know if GNUStep preallocates or not. I might have to drudge
> through the documentation and/or user groups to find out.

Carl, I've faced and I often face with this kind of situation (no ObjC,
though). I'm personally sure that in most cases C/C++/ObjC can be replaced
by Java with no performance loss, but when I face with a new customer I'm
pretty cautionary in speaking. The problem is that I know that people not
knowing Java (and VMs in general) are reluctant to believe. The good
approach is a practical one: try to find the time/resources to write a
prototype and measure.

--
Fabrizio Giudici - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
fabrizio...@tidalwave.it
http://tidalwave.it - http://fabriziogiudici.it

[Carl Jokl]

I was thinking that most likely I should try to prototype. It may be
bit tricky though. I may only be able to work on a prototype out of
hours. The company is very security concious. I don't know I would be
allowed to checkout the source on my own computer to work on out of
hours. I have a desktop in the office. I might be able to put in some
extra time at the office but there is currently a problem with all the
users being used up on the office alarm such that newer employees are
not yet able to lockup or open up the office.

It remains to be seen what the political fallout would be to suggest
replacing the Objective-C code particularly from the Objective-C code
developers. Even if the company accepted moving to pure Java I think
the Objective-C developers are hardly going to thank me for making
their role redundant though I believe they also know Java so I would
hope it would not put their job at risk. At the end of the day I am
talking about 'porting' code and not writing new code from scratch so
I would not think the existing Objective-C work is wasted.

[Jon Kiparsky]

Now, I don't spend a lot of time thinking about performance of one language versus another, so I might be off base here, but wouldn't the important first question be "is there enough of a difference to make it a consideration for this application?" A variant form of the question might be "will the user notice the difference between the two?"

Assume that one language is significantly slower than another, and the product is written in that language. If the user is not reporting problems with application responsiveness, I would say the answer to both questions is "no". In other words, in that case you'd be contemplating a major Knuth violation to change languages for the sake of performance.
Of course, there are many cases you could bring up where this would be invalid, for example if you're contemplating an increase in the program's capabilities or if there are other advantages to conversion, but in this simple case it seems that conversion to the faster language would be added risk for no gain.

The larger point I'm getting at is that it seems to me that the question of raw performance of a language, even if there is a single comprehensive way to gather that up, is so seldom interesting to the outcomes given today's hardware and the demands of most applications that "Language L is faster than Language Q" is just not an interesting question for most applications, and for most developers there will be much more important factors to consider.

Of course, I could be completely wrong on this, and of course if I am I'd like very much to know about it so I can stop being wrong....

[Steel City Phantom]

raw speed doesn't have to be your only argument.  

finding a "realistic to your environment" benchmark between a compiled to machine code language and a JVM language is going to be tough, if even possible,  due to the fact they are different styles of coding with very different libraries available.  but it really depends on what the objectve c code is doing.  if your c code is doing a lot of floating point calculations, your going to find that the jvm and c are pretty much on par with each other.  where the performance is usually gained and lost is the design of the code that handles your loops, choosing the best type of arrays or collections for the situations, having an efficient data model with well indexed queries, etc.

take the system im working on now.  there was an import process that previously took 4 hours to complete.  i looked at the code and rewrote about 60 lines of code in a search/math routine and it now runs in 20 seconds.  and the only reason it takes that long is im not allowed to reindex the database tables yet.

hell years ago i increased the performance of a bank transaction system 5 fold by simply putting a new network card in it.  everyone at the company (programmers included) were so focused on blaming java and "we should have never written this in java" for the system being slow that no one noticed that the network card was an old 100 meg card that had been pegged at 100% for as long as the log files went back.  put 2 1 gigabit cards in it and that system screamed.  and you don't want to know how many times ive increased the performance of a system by simply putting a new patch cord on the server.

if you are looking for something to hang your credibility on, i would not hang it on the number of cpu cycles a specific language takes to add 2 + 2, instead, your skills as a software designer and programmer to take this existing system and replace it with a more efficient, easer, cleaner, and cheaper system to maintain

and when your proposing it, remember, the check writers love the words FASTER, and CHEAPER so if you can come up with a proposal that simplifies the system into one enterprise java system that allows you to make changes to it 50% faster (therefor 50% cheaper) because you dont have to deal with C, well that will perk up some ears.

--
You want it fast, cheap, or right.  Pick two!!

[Carl Jokl]

For me personally, it is not just about the performance but having a
cleaner and more manageable system.

Setting up a new server could be as simple as installing Linux,
installing Java, installing the database and a Servlet container
container then dropping in the War and the system is running.

The existing system requires a bunch of extra work for the benefit of
installing GNU Step and also deploying the Java in such a way that GNU
Step and Java can reference each other.

Performance seems a focus of the technical manager. I have made
changes in the code that were aimed at making the system cleaner and
better architecturally structured but the question I get asked seems
to just be "will this be faster?"

I am approaching the problem on the terms I think my development
manager cares about. If it was a question of clean architecture I
don't know whether GNU Step talking to a Java web layer would have
been chosen. I believe my development manager was involved in the GNU
Step project and so I believed used it because he was a fan of the
technology rather than because it was necessarily a good fit. At least
that is the impression I get. I have not been with the company very
long.

The Java code in the system is probably some of the worsted I have
worked on. Hashtables and Vectors everywhere. Coding to
implementations including all the collections. The concrete
implementations of collections i.e. Vectors and Hashtables are exposed
in the class APIs. The code seems to involve lots of bloated Servlets
frequently with multi page long methods. Many standard Java
contentions can be ignored like using upper case package names not
following JavaBeans style but that is of lesser concern to me than the
coding to implementations and massive complex methods that should have
been broken down into smaller ones.

This system also rolls its own web framework and its own logging
framework.

I would have my work cut out for me just cleaning up the Java. The
Objective-C would be more liveable if the two systems were black boxes
with a well defined boundary interface for communicating back and
fourth but I think the use of wrappers means that Objective-C classes
will just be sprinkled liberally through the Java code.

All the Java code I have seen speaks to me of a lack of thought about
architecture or design patterns / principles. This might be explained
if the top priority is 'performance' or at least perceived
performance.

[Ricky Clarkson]

> the question I get asked seems
> to just be "will this be faster?"

If you're new, you might be missing some context from previous
projects. If there is a focus on performance, presumably there were
problems regarding performance before. At least, it's an option worth
entertaining and might be more favourable than assuming the manager is
insane.

Are you sure you could not take further steps to automate this build?
For reference, Maven has some support for building non-JVM languages,
and you can call maven from other scripts or scripts from maven. In a
previous job, we had a JNI library wrapping some of ffmpeg's libs,
involving using swig to generate code, and all we had to do that was
out of the ordinary was to start mvn from msys instead of cmd (in
Windows, elsewhere there was no difference at all).

> I believe my development manager was involved in the GNU
> Step project and so I believed used it because he was a fan of the
> technology rather than because it was necessarily a good fit. At least
> that is the impression I get.

Just ask him. It sounds like you're communicating more with us than
with your team. :)

> The Java code in the system is probably some of the worsted I have
> worked on. Hashtables and Vectors everywhere.

That probably indicates its age or when the coders last deliberately
learned anything, rather than quality directly.

> Coding to implementations including all the collections.

That isn't a big problem and can be refactored out fairly automatically.

I'd suggest code reviews for most of your remaining points.

> This system also rolls its own web framework and its own logging
> framework.

Choose a logging framework, rename your logging framework's methods to
match that, then delete the jar and go through fixing the imports. :)

> I would have my work cut out for me just cleaning up the Java. The
> Objective-C would be more liveable if the two systems were black boxes
> with a well defined boundary interface for communicating back and
> fourth but I think the use of wrappers means that Objective-C classes
> will just be sprinkled liberally through the Java code.

This is probably your bigger problem. I'd be looking to create
islands of pure Java and islands of pure Objective-C, and to stretch
an analogy, form continents of each connected by satellite link.

> All the Java code I have seen speaks to me of a lack of thought about
> architecture or design patterns / principles.

So introduce them as you see fit but measure the performance so that
you are armed for when you are questioned.

[Steel City Phantom]

if thats the case, then i would start with the java code.  propose a small, say, 3 week RnD project where you branch the java code and swap out a well chosen part of the java code with well designed code.  then run performance tests on the old version and your version and give those results to your manager.  this gives you leverage to say ok boss, lets break this java code out into 3 or 4 or however many necessary sections and attach each one at a time and bring them up to par.  keep those sections very contained, like say, step one, we will pull out the custom logging system and replace it with log4j.  then release.  step two, we will pull out hibernate and replace it with jdbc because raw jdbc is better suited for this type of app.  step three, we will go thru all of our arrays and switch them over to a more appropriate data type for the task they are trying to accomplish, etc, etc, etc.  get the existing java to really scream and perform to its full potential.  just keep the scope of each section to 1 or 2 month release cycles, something that doesn't drag on endlessly from a managers point of view.

if you take that approach, then the question of which language is faster becomes moot.  (removing the hardware factor from the following statement) well designed code will run faster than bad code every time.  this will give you leverage in the future on new features to say well, it will run faster if we bypass the whole c thing and just do it directly in java, you've got the benchmarks to backup your statements.

from your point of view your basically implementing a planned obsolescence, but you don't have to tell the boss thats what your doing  ;)

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Carl Jokl]

I intend to talk to my technical manager about my impressions. I am
still figuring out the best way of approaching it. I want to be
tactful about it and not upset him with anything I say. I am not
always the most tactful and I am still in the probationary period
where I know contractually my employment could be terminated at a
weeks notice. Therefore I would want to really want to think through
what I want to say and how I plan to say it before I discuss matters.

[Steven Huwig]

You could point out that Apple made this exact transition back in 2000 or so. WebObjects used to be in Objective-C, now it's Java, and it's still used everywhere at Apple. Maybe Joe can say more about it (but maybe not).

[phil swenson]

perf is not the reason to switch.  maintainability/tools is much more compelling justification.

[opinali]

In the general case, in C/C++ you cannot have the same allocation efficiency of Java or other modern garbage-collected systems - these include not just "new is just ptr++", but other important things like multithreaded allocation without synchronization, or automatic NUMA optimizations.  The problem is that all these nice tricks rely on the ability to eventually move objects to other addresses, e.g. in order to defragment the heap (to make bump-point allocation possible). And you cannot do that in a non-GC-safe language.

Sure you can program special C++ allocators that can be used in very restricted scenarios, but it's never the same thing, and adds even more to the complexity of memory management.

On the other hand, it's also way too simplistic to declare Java superior because of that. Allocation is faster, but deallocation (GC) is the problem. Some programs can exhibit extremely superior performance in deallocation too - basically, programs that make generational collectors happy - but this is not always possible, and there are other problems like cache and paging efficiency, and also other traits of Java (limited typesystem without structs and other "lightweight" constructs) that cause disadvantages but are completely unrelated to manual vs. automatic memory management.

But in the average, Java is faster simply because 90% of the responsibility for memory management is on the shoulders of the runtime; so a mediocre programmer (or just a good programmer who has too much stuff to do to waste time in optimization) will easily write a program that has good memory management performance.  This is harder in the case of C/C++, where, for memory-intensive systems, you can easily beat Java's performance but you need either a quite good programmer, or lots of time to burn in optimization - preferably both. 

A+

Osvaldo

[Kirk]

We can banter about all day on the cost of memory management in Java vs. C++. At the end of the day there is no practical way to measure it and it's not why we moved to garbage collection anyways. That said, object allocation in generational space based heaps has proven to be a huge performance win. You talk about cheap allocation. The reason we have cheap allocation is that generational spaces makes fast-path allocations possible. And for the same reasons, generational spaces makes reclamation  almost free. In the spirit of "there is no free lunch", the cost of generational spaces comes when you need to move things from one memory pool to another. But again, this cost is generally smallish in most systems because most data is discarded after a few machine cycles. One or two Java specific instructions on the processor would allow the JVM to ensure that many of the dead objects that are still in the processor's cache lines wouldn't never see the light of RAM. This optimization is *only* possible in runtimes that rely on garbage collection. Ok, i could be possible in C++ also but only at a huge complexity cost.

But in the average, Java is faster simply because 90% of the responsibility for memory management is on the shoulders of the runtime; so a mediocre programmer (or just a good programmer who has too much stuff to do to waste time in optimization) will easily write a program that has good memory management performance.  This is harder in the case of C/C++, where, for memory-intensive systems, you can easily beat Java's performance but you need either a quite good programmer, or lots of time to burn in optimization - preferably both. 

You know, I was in an interview with a company that was proto=typing in C++ because they believed in some of the arguments put forth here.and I think that demegrating Java developers is just down right wrong. I used to work in Smalltalk, C, C++ and various forms of assembler when needed. I spent today looking at some C++ code that I need to port to OSX so that I might get a better view of chip level counters. I wasn't the only Java developer looking at this stuff. Why are all these C++ guys working with Java? It's because I want to solve interesting problems and while mucking about with pointers is a nice recreational programming sport for some, it's no longer interesting.for me. The C++ bigotry that existed in the company that interviewed me also brought that company down. Their clever C++ guys could have easily as been working in Java and they might have made it to market before their funding ran out. (I declined the offer).

You're statement that one can easily beat a Java programmer isn't so true in a general sense. The JVM can optimize to instruction sets that are specific to single rev's of a specific chip. GCC isn't going to do that for you unless you build a different binary for every different version of every single chip out there. It's impractical and so one builds to the common instruction set leaving some nice performance optimizations on o the table. NUMA is one example where your application doesn't need to be recompiled to take advantage of different chip architectures.  Some of the optimizations provided my the JVM produce non-obvious native code that I doubt that even the best optimizing C++ compilers could provide. Try looking at the assembler dumped from a fast debug version of the JVM and ask the question, could I have hand rolled that? Ok, in some cases the answer will be yes but there will be just as many where the answer was no.

Regards,

Kirk

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/5XaYGBOmjx4J.

[opinali]

On Friday, December 2, 2011 8:59:50 PM UTC-5, kirk wrote:

That said, object allocation in generational space based heaps has proven to be a huge performance win. You talk about cheap allocation. The reason we have cheap allocation is that generational spaces makes fast-path allocations possible. And for the same reasons, generational spaces makes reclamation  almost free. In the spirit of "there is no free lunch", the cost of generational spaces comes when you need to move things from one memory pool to another. But again, this cost is generally smallish in most systems because most data is discarded after a few machine cycles.

Unfortunately this is only true if, when the GC is triggered, most objects in the young-gen are dead. But GC will rarely coincide with these ideal times, unless you call System.gc() in the right places, which is viable only for single-threaded apps.  If you're unlucky and GC happens when you're in the middle of some massive task and there's a ton of live short-lived objects, then GC has a relatively high cost. And that's still best-case wrt the old-gen; the real perf problem comes to apps that produce a continuous (even if small) stream of medium-lived objects, which are promoted to the old-gen but will eventually die, requiring some full-GCs. This is extremely common, so there's a ton of effort to make full-GC faster with incremental and concurrent collectors etc. Of course, microbenchmarks that apparently shows that GC is faster than manual allocation, never have these problems, they're usually programs with excellent generational behavior and completely stable old-space. In summary, YMMV a lot with app behavior.

 

You know, I was in an interview with a company that was proto=typing in C++ because they believed in some of the arguments put forth here.and I think that demegrating Java developers is just down right wrong. I used to work in Smalltalk, C, C++ and various forms of assembler when needed. I spent today looking

Notice that I'm a strong Java advocate too :) so I'm not denigrating anything, but also I don't think that exaggerating Java's real-world performance serves any good purpose either. I also worked with everything from ASM/C/C++ to Java, so I'm well informed about what is possible to do in lower-level languages. You are correct that these languages will lose in aspects like time-to-market, and also, the convenience of JIT compilation is great for apps that have to be installed in many different systems... on the "C/C++ can easily beat Java", I was discussing CPU+memory-intensive applications, not apps that spend most time waiting for a database or some other I/O. Or have many concurrent users (heavy multithreading means that CPU caches are wiped out all the time due to task switching - so the much superior memory locality of native apps with manual memory management is less win.) But these two sentences describe >90% of all software projects these days, so yeah a native language will not be "easily faster" and this certainly contributes to the success of Java.

A+

Osvaldo

 

at some C++ code that I need to port to OSX so that I might get a better view of chip level counters. I wasn't the only Java developer looking at this stuff. Why are all these C++ guys working with Java? It's because I want to solve interesting problems and while mucking about with pointers is a nice recreational programming sport for some, it's no longer interesting.for me. The C++ bigotry that existed in the company that interviewed me also brought that company down. Their clever C++ guys could have easily as been working in Java and they might have made it to market before their funding ran out. (I declined the offer).

You're statement that one can easily beat a Java programmer isn't so true in a general sense. The JVM can optimize to instruction sets that are specific to single rev's of a specific chip. GCC isn't going to do that for you unless you build a different binary for every different version of every single chip out there. It's impractical and so one builds to the common instruction set leaving some nice performance optimizations on o the table. NUMA is one example where your application doesn't need to be recompiled to take advantage of different chip architectures.  Some of the optimizations provided my the JVM produce non-obvious native code that I doubt that even the best optimizing C++ compilers could provide. Try looking at the assembler dumped from a fast debug version of the JVM and ask the question, could I have hand rolled that? Ok, in some cases the answer will be yes but there will be just as many where the answer was no.

Regards,

Kirk

A+

Osvaldo

On Thursday, December 1, 2011 7:36:04 AM UTC-5, KWright wrote:

Sure you can.  The so-called "placement new"[1] allows you exact control over where you stick your objects.

There are also various implementations available, such as pool[2], from the Boost libraries

There's a limit to how far you can take this though.  C++ still forces you to make a lot of it far more explicit, so there's a cognitive overhead to how far you can take it in an ever-growing system.

It also makes for some real fun and games when debugging & profiling

[1] http://stackoverflow.com/questions/222557/cs-placement-new

[2] http://www.boost.org/doc/libs/1_48_0/libs/pool/doc/html/boost_pool/pool/introduction.html

On 1 December 2011 11:04, Ricky Clarkson <ricky...@gmail.com> wrote:

I imagine that implementations of malloc/free exist or could be written that preallocate. Thus this advantage can be achieved in C.

---

From: Kevin Wright <kev.l...@gmail.com>

Sender: jav...@googlegroups.com

Date: Thu, 1 Dec 2011 11:46:34 +0000

To: <jav...@googlegroups.com>

ReplyTo: jav...@googlegroups.com

[Carl Jokl]

GAAAAH!!! Today I came across a method that is nearly one thousand
lines long! So help me I am going to start writing a report on all the
issues with the code after working hours today.