Datastore Performance Test: Java runs 5 times SLOWER than Python ?

[风笑雪]

I did a datastore performance test with both Java (using JDO) and Python tonight, and find Python is much faster than Java.

I defined a simple model which contains only 1 string property, and you can find the code in the attachment.

There are two links for you to test:

http://java.latest.gaejava.appspot.com

http://python.latest.gaejava.appspot.com

I found the cpu time and api cpu time are almost the same between Java and Python, the only difference is response time.

I don't know whether it's a defect of Java version or problem of my code (it's the first time I write app in Java), can anybody check it?

This is my test result:

Java:

Add:

Time of save 1 entity:       54ms

Time of create 100 entities: 0ms

Time of save 100 entities:   7765ms

Time of save 1 entity:       52ms

Time of create 100 entities: 0ms

Time of save 100 entities:   7059ms

Time of save 1 entity:       61ms

Time of create 100 entities: 0ms

Time of save 100 entities:   7384ms

Update:

Time of fetch 1 entity:      222ms

Time of update 1 entity:     25ms

Time of fetch 100 entities:  40ms

Time of update 100 entities: 7098ms

Time of fetch 1 entity:      33ms

Time of update 1 entity:     25ms

Time of fetch 100 entities:  35ms

Time of update 100 entities: 8038ms

Time of fetch 1 entity:      14ms

Time of update 1 entity:     24ms

Time of fetch 100 entities:  34ms

Time of update 100 entities: 8425ms

Delete:

Time of fetch 1 entity:      16ms

Time of delete 1 entity:     82ms

Time of fetch 100 entities:  36ms

Time of delete 100 entities: 7655ms

Time of fetch 1 entity:      93ms

Time of delete 1 entity:     104ms

Time of fetch 100 entities:  45ms

Time of delete 100 entities: 9825ms

Time of fetch 1 entity:      176ms

Time of delete 1 entity:     98ms

Time of fetch 100 entities:  60ms

Time of delete 100 entities: 7784ms

Log:

06-17 07:34AM 51.575 /del 200 8145ms 7970cpu_ms 6823api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:34AM 36.326 /del 200 10094ms 8083cpu_ms 6823api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:34AM 21.466 /del 200 7819ms 7942cpu_ms 6823api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:34AM 02.319 /update 200 8519ms 13189cpu_ms 11929api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:33AM 48.282 /update 200 8159ms 13105cpu_ms 11929api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:33AM 35.099 /update 200 7426ms 13664cpu_ms 11929api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:33AM 20.908 /add 200 7476ms 10684cpu_ms 9453api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:33AM 07.749 /add 200 7146ms 10684cpu_ms 9453api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:32AM 49.846 /add 200 7862ms 10825cpu_ms 9453api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

Python:

Add:

Time of save 1 entity:       129.576921ms

Time of create 100 entities: 2.582073ms

Time of save 100 entities:   1210.234880ms

Time of save 1 entity:       54.707050ms

Time of create 100 entities: 2.454042ms

Time of save 100 entities:   1481.308937ms

Time of save 1 entity:       47.955990ms

Time of create 100 entities: 2.021074ms

Time of save 100 entities:   1310.760021ms

Update:

Time of fetch 1 entity:      388.538837ms

Time of update 1 entity:     53.938150ms

Time of fetch 100 entities:  96.820831ms

Time of update 100 entities: 1242.298126ms

Time of fetch 1 entity:      409.255028ms

Time of update 1 entity:     43.366909ms

Time of fetch 100 entities:  92.492104ms

Time of update 100 entities: 1119.529963ms

Time of fetch 1 entity:      431.589842ms

Time of update 1 entity:     37.049055ms

Time of fetch 100 entities:  76.496124ms

Time of update 100 entities: 2046.729803ms

Delete:

Time of fetch 1 entity:      417.876959ms

Time of delete 1 entity:     62.140942ms

Time of fetch 100 entities:  105.495930ms

Time of delete 100 entities: 1161.437988ms

Time of fetch 1 entity:      614.541054ms

Time of delete 1 entity:     49.677849ms

Time of fetch 100 entities:  90.868950ms

Time of delete 100 entities: 1129.801989ms

Time of fetch 1 entity:      563.830853ms

Time of delete 1 entity:     66.990137ms

Time of fetch 100 entities:  188.508034ms

Time of delete 100 entities: 1076.126814ms

Log:

06-17 07:36AM 51.238 /del 200 1979ms 7102cpu_ms 6823api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:36AM 42.483 /del 200 1896ms 6942cpu_ms 6823api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:36AM 33.591 /del 200 1778ms 6962cpu_ms 6823api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:36AM 16.895 /update 200 2605ms 12258cpu_ms 12034api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:36AM 00.586 /update 200 1679ms 12258cpu_ms 12034api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:35AM 51.585 /update 200 1789ms 12230cpu_ms 12034api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:35AM 41.851 /add 200 1367ms 9584cpu_ms 9453api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:35AM 34.596 /add 200 1549ms 9621cpu_ms 9453api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

06-17 07:35AM 26.595 /add 200 1458ms 9649cpu_ms 9453api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

[Nick Johnson (Google)]

Hi keakon,

Where do you get your figure of '5 times slower' from? The statistics you posted demonstrate Python being slightly slower than Java, not the other way around (and certainly not by a factor of 5).

-Nick Johnson

--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[Alkis Evlogimenos ('Αλκης Ευλογημένος)]

All the puts() are slower in Java it seems.

2009/6/17 Nick Johnson (Google) <nick.j...@google.com>

--

Alkis

[风笑雪]

You can see the first compare result of adding:

Java:

Time of save 100 entities:   7765ms

06-17 07:32AM 49.846 /add 200 7862ms 10825cpu_ms 9453api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

Python:

Time of save 100 entities:   1210.234880ms

06-17 07:35AM 26.595 /add 200 1458ms 9649cpu_ms 9453api_cpu_ms 0kb python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1 Safari/531.0,gzip(gfe)

2009/6/18 Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlog...@gmail.com>

[Peter Recore]

I think the problem might be related to the fact that you're using JDO
on the java side and the low level datastore api on the python side.

The JDO implementation does not efficiently batch multiple updates/
inserts. I would guess that you will see much closer performance if
the low level api is used, since your results for inserting one entity
at a time seem to be comparable.

http://groups.google.com/group/google-appengine-java/browse_thread/thread/e717f7ba37749ea4/ab034e0d30653f47?lnk=gst&q=batch+insert#ab034e0d30653f47

On Jun 17, 11:10 am, 风笑雪 <kea...@gmail.com> wrote:
> You can see the first compare result of adding:
>
> Java:

> Time of save 100 entities:   7765ms06-17 07:32AM 49.846 /add 200 7862ms

> 10825cpu_ms 9453api_cpu_ms 0kb java.latest.gaejava.appspot.com Mozilla/5.0
> (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko)
> Chrome/3.0.187.1 Safari/531.0,gzip(gfe)
>
> Python:
> Time of save 100 entities:   1210.234880ms
> 06-17 07:35AM 26.595 /add 200 1458ms 9649cpu_ms 9453api_cpu_ms 0kb
> python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT 5.1;
> en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1
>  Safari/531.0,gzip(gfe)
>

> 2009/6/18 Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlogime...@gmail.com>

>
> > All the puts() are slower in Java it seems.
>

> > 2009/6/17 Nick Johnson (Google) <nick.john...@google.com>

> >>>>  06-17 07:32AM 49.846 /add 200 7862ms 10825cpu_ms 9453api_cpu_ms 0kb
> >>>>> java.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U; Windows NT
> >>>>> 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.187.1Safari/531.0,gzip(gfe)
>

> >>>>  06-17 07:35AM 26.595 /add 200 1458ms 9649cpu_ms 9453api_cpu_ms 0kb
> >>>>> python.latest.gaejava.appspot.com Mozilla/5.0 (Windows; U;
>

> ...
>
> read more »

[风笑雪]

I don't know how to use low level api because the document is too short to understand.

But as Ross says, JDO doesn't support batch operation at this moment, so I'll try to also use JPA and low level api to do this test, but I don't think anybody would like use low level api to build his app.

BTW, there is only one datastore api in Python, and I don't think it's low level since its rather easier to use than JDO. However, I really like the Google Plugin for Eclipse, it's a wonderful tool.

2009/6/18 Peter Recore <peter...@gmail.com>

[风笑雪]

I have updated it to support low level api.

http://gaejava.appspot.com/

To avoid the start delay, I test it for 3 times and this is the 3rd test result :

Language / API	add 1 entity	add 10 entities	add 100 entities	fetch 1 entity	fetch 10 entities	fetch 100 entities	update 1 entity	update 10 entities	update 100 entities	delete 1 entity	delete 10 entities	delete 100 entities
Python	37ms	117ms	936ms	712ms	232ms	97ms	757ms	340ms	1191ms	894ms	545ms	1007ms
JDO	79ms	534ms	5684ms	1315ms	477ms	343ms	980ms	995ms	10470ms	1184ms	1008ms	6345ms
Low-level API(Java)	48ms	126ms	916ms	842ms	332ms	114ms	2892ms	1043ms	2005ms	1436ms	450ms	1271ms

2009/6/18 风笑雪 <kea...@gmail.com>