idea for new operators

[Alexander Tsepkov]

There have been a number of threads on the mailing list asking for ability to overwrite RapydScript operators, to which I have objected due to performance and clarity of generated JavaScript. However, I do understand the use case of developers wanting this functionality. Likewise, there are multiple cases where I could find operator overloading useful myself - one of the simplest cases is == test between arrays, which is a notorious problem with JavaScript (and for which RapydScript currently relies on deep_eq).

I've recently been considering adding new operators (even though I'm against adding unpythonic features) to handle these cases. My idea was using the '.' prefix for the operators to signify that they can be overloaded (the names would match their Python equivalent overload names (https://docs.python.org/2/library/operator.html), or perhaps even use the same name as a method name: i.e. 'Array.== = ' which would compile to 'Array.__eq__').

There are a couple things that appeal to me about the '.' prefix:

- It's unobtrusive and compact, yet alerts the developer that the operator isn't a standard JS operator

- '.' is already used to signify a method call of a class/object, these overloadable operators would compile into method calls, hence this seems like a consistent convention

The idea is that each '.' operator would compile into something like this:

original:

a .== b

compiled:

_$rapyd$__eq__(a, b)

function _$rapyd$__eq__(a, b) {

    if (typeof a.__eq__ === 'function') return a.__eq__(b); // user-defined

    return deep_eq(a, b); // default case, a slower but saner equality test

}

The equivalent method would get generated for all operators used (similar to other baselib logic). This, of course, would still require you to tweak your Python code slightly when porting it to RapydScript, but it wouldn't be a tedious task for the few cases that need it. I'm thinking of allowing these alternatives for all operators:

.=, .==, .!=, .<, .>, .+, ./, .*, .-, .and, .or, as well as binary operators

Additionally, I was thinking of introducing a 'debug' mode for compilation, that would, using logic similar to the logic above compile all operators into something safer yet slower, allowing the developer to catch errors earlier. For example:

original:

string + integer

debug compile:

_$rapyd__debugadd__(string, integer)

function _$rapyd$__debugadd__(a, b) {

    if (typeof a === typeof b) {

        if (typeof a === 'object') throw new TypeError("Concatenation of objects/arrays using + operator will result in implicit conversion to string or NaN, use .+ operator instead"); 

        return a + b;

    } else {

        throw new TypeError(''Cannot add/concatenate "' + typeof a + '" and "' + typeof b + '" objects');

    }

}

As you can see above, debug mode wouldn't rely on the same functions as the '.' prefix, but the structure would be similar. Whereas '.' would check if overloaded method exists, and if not, call the safer/pythonic function, debug would not check overloaded method at all, but would warn the user when the likely intent is to use the safer/pythonic function. The debug mode would run slower but give developer much better sanity checks when the code is executed.

To summarize, there are two ideas here:

- addition of '.' prefix for each operator to signify overloaded/pythonic version of it without forcing the overhead of pythonic version on all cases

- addition of debug mode that would rely on similar logic to temporarily overload basic operators during a debug build for stricter type checking

I wanted to hear the community's yay or nay for each of the above ideas, as well as any gotchas with syntax/logic you think would arise that I may have overlooked.

[Salvatore Di Dio]

Hy Alexander,

From my user perspertive, I really like the 'light' intrusion in the syntax.

Regards

[John P Charlesworth]

HI Alexander,

   I think both your ideas would be useful developments and would help to attract more interest from the developer community. I don't see any disadvantages

While you are thinking about debugging, I'd like to raise a problem that I've run into a few times. Here's a little piece of very simple code.

class MyClass:
    pass
myobj = MyClass()
myobj.propa = 2
myobj propb = 3

which compiles without error to

function MyClass() {
}
myobj = new MyClass();
myobj.propa = 2;
myobj;
propb = 3;

Of course I've left out the qualifying dot in line 5. But errors like that, which would be a syntax error in Python, are easy to make and can be hard to find in 1500 lines of code, because the code runs and then misbehaves but may not produce any errors. If there was a way of either flagging this sort of thing as an error, or even issuing a warning, that would be quite useful.

Regards

[Teemu Y]

Hi Alexander,

please excuse me for going offtopic here...

I have just found RapydScript, and skimmed the website documentation and some of the example code. Thanks for the effort, I hope to be able to use RapydScript in my projects. In a team Python webapp project that I work with, other devs have complained about the need to context-switch between JavaScript and Python. I am evaluating if RapydScript would solve that.

As a first impression of someone who has only written in-browser code in JavaScript until now, conforming to valid Python syntax whenever reasonably possible would be preferable. The more deviations there are, the more you need to focus to writing RapydScript, not Python. I do realize that there are intricacies I am not familiar with in the trade-offs you have to make. I also did see that the flexibility of syntax (allowing JavaScript within RapydScript) is a stated design choice as opposed to other Python-in-browser implementations, which is fine. However, I myself would like the option of being able to write valid Python syntax to the extent it is possible, as I believe that would be what our team would find easiest to adopt.

As for your actual question, I personally would like to have a (compiler) option to take the overhead instead of having to use a non-Python syntax. The stricter debug mode you suggest also sounds like a fantastic idea! I know my colleagues and I would love an "ultra-strict mode" that could be used during development for early detection of errors which otherwise would pass as legal JavaScript.

This might be a FAQ: What bothered me a little about the examples was the non-pythonic syntax used for method definitions within __init__(), ie: "methodname = def()..." . While I realize this closely matches the JavaScript side, I was wondering if those could not be written using Python's nested function syntax; ie. a def within a def block (is function nesting supported in RapydScript?) or would that cause conflicts elsewhere?

Again, sorry for hijacking the thread with my ignorant questions, and kudos for the great work you've done with RapydScript!

Teemu

[Alexander Tsepkov]

Thanks for the feedback.

Teemu, I understand your concerns and I definitely want to minimize context switch - that was one of the main motivators for the project. However, the other motivator was compatibility with native JavaScript libraries out of the box. Remember the whole "batteries included" mantra of Python? Well, the batteries in the browser happen to be different, and RapydScript was built to work with them. Moreover, some design patterns make sense in browser world and Python was not built with them in mind, hence the slight deviations like anonymous functions - we want to make code easy to read/write, not necessarily 100% compatible with a standard planned for a different environment in a different age. I also recommend this blog post I wrote on the topic a while ago: http://blog.pyjeon.com/2014/01/18/what-is-pythonic/

Once the functionality is there, it would be relatively easy for me to add a compile flag that will force regular operators to be treated as dot-operators (.==) as well. However, I can guarantee you that you won't be happy with the results of automatic conversion for large projects. This isn't minor overhead, it will add up fast. This overhead was one of the main reasons for the downfall of Pyjamas, and if you want a preview of what "pure Pythonic" will mean for your project, take the Asteroids example bundled with RapydScript and compile it with --auto-bind flag. This flag will automatically bind functions to the object on object creation, just like Python would. Now run the example in a browser like Firefox (that doesn't have Chrome's aggressive JS optimizations) and see performance go downhill as soon as you press "fire". Problem is that each one of the bullets is created as an object, they're created rapidly, and usually this is not a problem, unless you do a bunch of additional Pythonic logic for binding. Now imagine the same problem extended to all arithmetic. The Paint demo that's also bundled with RapydScript was originally written by me for Pyjamas. Problem is it simply didn't function in Pyjamas without hacks, it is the reason I decided to build my own language - to port my Python/Pyjamas code to better JavaScript. The paint-bucket tool, for example, uses flood-fill algorithm (http://en.wikipedia.org/wiki/Flood_fill). This algorithm would freeze the browser in Pyjamas (because the math was too slow), in native JS and RapydScript it works just fine.

As far as anonymous functions in examples, there is nothing that prevents you from nesting functions inside other functions the way Python would do it. This is a design pattern I adopted to more easily differentiate a function/method that is native to the object (one that would typically be bound) from one that's meant to be assigned or passed to a different object/handler after creation (a callback, if you will).