Speed question

[Bruce Sherwood]

On June 18, Pierre said, "Ok, there is the performance issue, but things are getting better; the release I'm currently working on (3.2.7) runs the pystone test 2.5 faster than 3.2.6. It's still 15 times slower than CPython, but in the early days it was thousands of times slower." 

However, in March 2015 in his blog article 

https://brythonista.wordpress.com/2015/03/28/comparing-the-speed-of-cpython-brython-skulpt-and-pypy-js/ 

he showed pystone benchmarks in which Brython was much faster than CPython for many of the individual tests, and in only one test was Brython nearly 15 tims slower (create_function.py, a factor of 14.38).

I don't understand how to reconcile these two comparisons.

Bruce Sherwood

P.S. I've been using RapydScript to compile to JavaScript in the browser at glowscript.org. RapydScript produces pretty efficient JavaScript, at the cost of not attempting to produce bit-perfect true Python behavior. It's close enough for the purposes to which GlowScript VPython is typically put. I'm in the process of updating to rapydscript-ng, which comes closer to producing true Python behavior.

[Pierre Quentel]

Le lundi 19 décembre 2016 16:25:40 UTC+1, Bruce Sherwood a écrit :

On June 18, Pierre said, "Ok, there is the performance issue, but things are getting better; the release I'm currently working on (3.2.7) runs the pystone test 2.5 faster than 3.2.6. It's still 15 times slower than CPython, but in the early days it was thousands of times slower." 

However, in March 2015 in his blog article 

https://brythonista.wordpress.com/2015/03/28/comparing-the-speed-of-cpython-brython-skulpt-and-pypy-js/ 

he showed pystone benchmarks in which Brython was much faster than CPython for many of the individual tests, and in only one test was Brython nearly 15 tims slower (create_function.py, a factor of 14.38).

I don't understand how to reconcile these two comparisons.

The comparison didn't cover other tests such as creating instances of a class, calling a function with default parameters, etc. There are more tests in directory /speed of the repository. These operations are also slower in Brython than CPython, and they are used intensively in the Pystone benchmark ; this is why the overall result is about 15 times slower.

[Bruce Sherwood]

Thanks for the explanation.

[Bruce Sherwood]

I ran pystones using the rapydscript-ng transpiler, and it benchmarked 5 times the speed of CPython on my Windows 10 computer, 600000 pystones/sec vs. 125000 pystones/sec. I hasten to say that while speed is important for the computational uses to which VPython is put, speed may not be the important issue for other Python-to-JavaScript usage patterns.

[Bruce Sherwood]

The factor of about 5 times CPython speed for rapydscript-ng increases to about 7 times CPython if I turn off operator overloading that is used in GlowScript VPython, in which for example a+b is converted to a["+"](b); this is done to permit easy manipulation of 3D vectors.

[kiko(gmail)]

On 22/12/16 01:50, Bruce Sherwood wrote:

The factor of about 5 times CPython speed for rapydscript-ng increases to about 7 times CPython if I turn off operator overloading that is used in GlowScript VPython, in which for example a+b is converted to a["+"](b); this is done to permit easy manipulation of 3D vectors.

Benchmarkings, in general, are not very useful as what they compare, in most situations, is not the same. And JS is a language specially complicated (MS, Google, Mozilla,,..., i.e., SpiderMonkey, V8, Chackra,...) to benchmark.

The goal of rapydscript ("pythonic javascript"^TM) is not the same than the Brython one. There are a lot of special cases on the syntax so it is not Python. Of course, this is not bad and it has its uses.

If you need better performance go for rapydscript or even go for plain js. If you need Python3 compatibility then you will be more lucky with Brython.

[Bruce Sherwood]

Agreed about the limitations of benchmarking. The most useful benchmark is the particular program of interest, not a general program such as pystones. As I too said, the goal of a Python-to-JavaScript transpiler may be different for different usage patterns. GlowScript VPython is aimed at making it possible even for novice programmers to write computational programs that generate navigable real-time 3D animations as a side effect of the computations, with syntax that is as close as possible to true Python for the basic aspects of Python. Divergence from true Python is tolerable in this situation for elements of Python unlikely to be used by novice programmers. One can write programs at glowscript.org in JavaScript, but most users choose to write in Python.

As I perceive the situation, Alex Tsepkov's goal for rapydscript is to provide a thin wrapper around JavaScript, to facilitate the work of web programmers, with easy intermixing of (for example) jquery and JavaScript, whereas Kovid Goyal's goal for rapydscript-ng is to come closer to true Python syntax while still preserving as much as possible rapydscript's speed. Since GlowScript is not aimed at web programmers, I'm in the process of changing from rapydscript to rapydscript-ng. I've tried to track some of the developments of other transpilers, and recently I saw a post somewhere that suggested that Brython's speed had increased substantially, which is why I raised my questions.

[Kiko]

2016-12-22 15:18 GMT+01:00 Bruce Sherwood <bruce.s...@gmail.com>:

Agreed about the limitations of benchmarking. The most useful benchmark is the particular program of interest, not a general program such as pystones. As I too said, the goal of a Python-to-JavaScript transpiler may be different for different usage patterns. GlowScript VPython is aimed at making it possible even for novice programmers to write computational programs that generate navigable real-time 3D animations as a side effect of the computations, with syntax that is as close as possible to true Python for the basic aspects of Python. Divergence from true Python is tolerable in this situation for elements of Python unlikely to be used by novice programmers. One can write programs at glowscript.org in JavaScript, but most users choose to write in Python.

Great!!
 

As I perceive the situation, Alex Tsepkov's goal for rapydscript is to provide a thin wrapper around JavaScript, to facilitate the work of web programmers, with easy intermixing of (for example) jquery and JavaScript, whereas Kovid Goyal's goal for rapydscript-ng is to come closer to true Python syntax while still preserving as much as possible rapydscript's speed. Since GlowScript is not aimed at web programmers, I'm in the process of changing from rapydscript to rapydscript-ng. I've tried to track some of the developments of other transpilers, and recently I saw a post somewhere that suggested that Brython's speed had increased substantially, which is why I raised my questions.

It seems rapydscript development is stagnated and the team behind rapydscript-ng is small. Maybe this could become an issue. There are other options like transcrypt (http://transcrypt.org/).

 

--
You received this message because you are subscribed to the Google Groups "brython" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brython+unsubscribe@googlegroups.com.
To post to this group, send email to bry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brython/c6ed106a-4017-45f2-8d81-cdae61b70387%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruce Sherwood]

Thanks for the reference to transcrypt, which I hadn't heard of. At the moment it's not an option for glowscript.org, which uses Google App Engine whose Python server component currently is restricted to Python 2.7, whereas transcript compiling is done using Python 3.5 or 3.6.

It is true that rapydscript development had stalled, but three months ago Tsepkov started work on it again. There's no difference in the manpower for rapydscript (all work done by Tsepkov) and rapydscript-ng (all work done by Goyal). I've been impressed by Goyal's remarkably quick responses to questions and bug reports.

[Bruce Sherwood]

On second thought, transcrypt is useless for my purposes. With transcrypt you need to run a Python program to compile Python to JavaScript. The GlowScript environment is quite different: transpiling using rapydscript is done in the browser, not in the server. An important aspect is that glowscript.org stores user programs in the cloud and serves the graphics library to a user's browser, but all compilation and execution is done in the browser. As a result, despite there being more than 15,000 user accounts at glowscript.org, the charges for Google App Engine services are about $2 per month. It all scales extremely well.

[Bruce Sherwood]

Here's an interesting recent blog article by Alex Tsepkov about his intentions:

http://blog.pyjeon.com/2016/08/30/state-of-rapydscript/

As you can see, he's aiming at a kind of hybrid of Python and JavaScript, whereas Kovid Goyal is aiming at more closely approximating true Python syntax.

[Pierre Quentel]

Le mercredi 21 décembre 2016 18:52:41 UTC+1, Bruce Sherwood a écrit :

I ran pystones using the rapydscript-ng transpiler, and it benchmarked 5 times the speed of CPython on my Windows 10 computer, 600000 pystones/sec vs. 125000 pystones/sec. I hasten to say that while speed is important for the computational uses to which VPython is put, speed may not be the important issue for other Python-to-JavaScript usage patterns.

The figures are impressive ! I am trying to reproduce them, with no success so far, rapydscript can't compile the version of pystones included in the Brython distribution (various error messages : the time module doesn't exist, the form "lambda x:x[:]" is not supported, an unknow name "global" is inserted in the Javascript code, apparently for the variables defined with the Python global keyword - I gave up there).

Can you give a pointer to the version you used for your test ?
 

[Bruce Sherwood]

Attached is the version I used, not with rapydscript in fact but with rapydscript-ng, because the original rapydscript doesn't have some Python features, such as chr(), as far as I know. I eliminated the minor use of lambda for setting up some arrays. I didn't notice an issue with "global".

I think the key speed issue is that rapydscript (and rapydscript-ng) are in some sense very thin wrappers around JavaScript, which is not what everyone needs or wants, since it can't match true Python behavior.

I should say that I have no previous experience in using pystones.py, so at least theoretically it could be that when I run it with rapydscript-ng some of the tests are no-ops, and I wouldn't realize that. However, my general experience with GlowScript VPython is that VPython programs run faster in the browser than in CPython, which is consistent with the pystones measurement. 

In particular, loop overhead is low in JavaScript and high in CPython, so computational programs tend to benefit quite a bit. In fact, CPython situations that require invoking numpy just to eliminate loops often can be written simply as ordinary loops in RapydScript/JavaScript. Take a look at the "Example programs" at glowscript.org, where you'll see pretty peppy behavior in computational programs.

I'm making progress in moving from rapydscript to rapydscript-ng, but I haven't quite finished updating other libraries used by GlowScript, those associated with operator overloading and synchronous -> asynchronous (Streamline). So glowscript.org is still based on the older rapydscript.

Bruce

[Pierre Quentel]

I must be doing soemthing wrong, but I can't compile this version of pystones, here is what I get :

c:\brython\20161207\www\speed\benchmarks>rapydscript --output pystones_bruce.js
pystones_bruce.py
pystones_bruce.py:Failed Import: 'time' module doesn't exist in any of the impor
t directories: C:\Users\ypqa5222\AppData\Roaming\npm\node_modules\rapydscript-ng
\src\lib:. (line: 40, col: 10, pos: 1327)

Error
    at ImportError.__init__ (release\compiler.js:3917:27)
    at ImportError.__init__ (release\compiler.js:3963:78)
    at new ImportError (release\compiler.js:3959:44)
    at import_error (release\compiler.js:9203:23)
    at do_import (release\compiler.js:9281:21)
    at import_ (release\compiler.js:9432:21)
    at statement (release\compiler.js:8907:32)
    at with_embedded_tokens (release\compiler.js:8544:28)
    at run_parser (release\compiler.js:11203:31)
    at Object.parse (release\compiler.js:11390:146)
 

[Bruce Sherwood]

My apologies; I gave you the wrong file. Replace from "time import clock" with

def clock():

    return 0.001*performance.now()

or

def clock():

    return 0.001*(new Date().getTime())

(A feature of RapydScript is that one can reference JavaScript elements directly.)

Bruce

[Pierre Quentel]

Thanks, I tried with this version and rapydscript-ng could compile the script.

I tried to embed it in an HTML file :

<html>
<head>
<meta charset="utf-8">
<title>Test pystone rapydscript</title>
</head>
<body>
<script src="pystones_bruce.js"></script>
</body>
</html>

When I load this file here is the message I get in the browser console, both in Firefox and Chrome :

ReferenceError: global is not defined
at line :  pystones_bruce.js:3611:13

The line 3611 is the one that I marked with --> below :

        function Proc0() {
            var loops = (arguments[0] === undefined || ( 0 === arguments.length-1 && arguments[arguments.length-1] !== null && typeof arguments[arguments.length-1] === "object" && arguments[arguments.length-1] [ρσ_kwargs_symbol] === true)) ? Proc0.__defaults__.loops : arguments[0];
            var ρσ_kwargs_obj = arguments[arguments.length-1];
            if (ρσ_kwargs_obj === null || typeof ρσ_kwargs_obj !== "object" || ρσ_kwargs_obj [ρσ_kwargs_symbol] !== true) ρσ_kwargs_obj = {};
            if (Object.prototype.hasOwnProperty.call(ρσ_kwargs_obj, "loops")){
                loops = ρσ_kwargs_obj.loops;
            }
            var starttime, i, nulltime, PtrGlbNext, PtrGlb, String1Loc, IntLoc1, IntLoc2, String2Loc, EnumLoc, BoolGlob, IntLoc3, CharIndex, benchtime, loopsPerBenchtime;
-->         global;
            IntGlob;
            global;

Don't you have the same error message ?

 

[Bruce Sherwood]

Brief statement: Again, my mistake, or rather my confusion, or maybe even some confusion in both rapydscript and rapydscript-ng. Change the occurrences of "global" to "nonlocal", and the program should run okay.

Non-brief statement: The documentation for both flavors of rapydscript advocate the use of nonlocal rather than global, with reference to Python 3, with the comment "There is also global". However, it looks like a bug that rapydscript (including rapydscript-ng) 1) has not implemented "global" and 2) has not restricted "nonlocal" to use in nested methods. At the time of implementing the use of rapydscript in GlowScript, I was unfamiliar with "nonlocal" and in a preprocessing stage simply converted a user program "global" to "nonlocal", which did what was expected (except that in true Python one need not separately declare the variable in global scope, whereas in rapydscript one must do so). I'll ask Goyal about this.

Footnote: I do quite a lot of preprocessing of a user VPython program before passing the preprocessed source to the rapydscript transpiler. This includes for example inserting Streamline markers where necessary to convert synchronous code to asynchronous code, inserting line numbers that enable accurate error reporting despite major upheaval associated with Streamline, etc. So changing "global" to "nonlocal" was a simple matter, but I now see that the implementationg of "nonlocal" is apparently flawed, and "global" isn't implemented at all.

Bruce

[Pierre Quentel]

Thanks again, I replaced all the "global" by "nonlocal" and with another minor change in the code (string formatting doesn't seem to be supported) I could run the pystone test. The results are in the order of what you observed, much better than CPython and thus than Brython.

I will see if I can borrow a few ideas from the code generated by rapydscript-ng to improve Brython performance.

[Bruce Sherwood]

Thanks for enduring my multiple mistakes! I hope you get some useful ideas from the rapydscript work.

I'm new to the ng version of rapydscript, but I thought I'd read that string formatting was supported -- maybe one has to specifically invoke a run-time file. String formatting worked for me because I wrote a JavaScript implementation that is invoked in the glowscript.org context where I was running. The code is at line 360 in

https://github.com/BruceSherwood/glowscript/blob/master/lib/glow/api_misc.js