Map and Each in Oni
Matt Brubeck
combinators. Motivation: I want to use Oni to call an asynchronous
function on each element in an array.
http://github.com/mbrubeck/oni-map/
If these look useful to other Oni developers, you might consider
adding something like them as new primitives to the Oni language. (Or
maybe eventually there will be an "Oni.contrib" module for new
functions that aren't part of the core language.)
As I noted on that page, I needed a way to pass a JavaScript array as
arguments to an Oni operator. I used JavaScript's
Function.prototype.apply, which felt a bit hacky. Maybe there should
be a way to do this in "pure" Oni code?
Alexander Fritze
On Mar 6, 12:54 am, Matt Brubeck <mbrub...@limpet.net> wrote:
> Here's a version of "for each" and "map" that I wrote as Oni
> combinators. Motivation: I want to use Oni to call an asynchronous
> If these look useful to other Oni developers, you might consider
> adding something like them as new primitives to the Oni language. (Or
> maybe eventually there will be an "Oni.contrib" module for new
> functions that aren't part of the core language.)
merge your new primitives into oni.js, pretty much as they are. (The
Oni code is licensed under MIT license - I've just added a header to
that effect to oni.js.)
For ease of use, I'd prefer not to have an "Oni.contrib" module, but
rather split the code along functional lines: Everything that's useful
to have in the core should go into oni.js. Stuff that's relevant only
for web api programming (such as wrappers around XMLHttpRequest)
should go into oni-web.js, and so forth.
> As I noted on that page, I needed a way to pass a JavaScript array as
> arguments to an Oni operator. I used JavaScript's
> Function.prototype.apply, which felt a bit hacky. Maybe there should
> be a way to do this in "pure" Oni code?
BTW, somewhat loosely related to your code: One idea I have been
tossing around for a while is to add an Oni-native lazy list
structure. I was looking at this for expressing producer-consumer
patterns in a structured way. I'm still a bit hazy on the details, but
the idea is to have a bunch of fundamental primitives that operate on
lists and produce lists, e.g.:
'first-executed first-out'
Fexfo([a,b,c]) : executes a,b,c in parallel and produces a lazy array
in the order that the processes a,b,c finish. So e.g. if a,b,c finish
in the order b,c,a, then an array [b,c,a] will be returned.
The interesting bit is that fexfo returns a *lazy* array (actually not
sure lazy is the right word here), so the parts of the array will be
available to consumers as soon as they are there, and conversely,
pending executions of the array will be aborted when the results
become unreachable. With this scheme, 'Alt' could be implemented in
terms of 'Fexfo':
Alt = Defun(args, Car(Fexfo(args)));
Here, all pending executions of the args array will be cancelled as
soon as Alt returns.
I think to make this work will require some significant changes in the
Oni semantics, so this is definitely a more long-term thing.
Cheers,
Alex
Matt Brubeck
> This looks really great, and if you're ok with it I think we should
> merge your new primitives into oni.js, pretty much as they are. (The
> Oni code is licensed under MIT license - I've just added a header to
> that effect to oni.js.)
etc. if you want to. As you can see, my source code on GitHub was
already MIT-licensed.
> BTW, somewhat loosely related to your code: One idea I have been
> tossing around for a while is to add an Oni-native lazy list
> structure. I was looking at this for expressing producer-consumer
> patterns in a structured way.
the patterns from Oliver Steele's "Practical Functional JavaScript"
talk in Oni:
http://osteele.com/archives/2008/09/practical-functional-javascript
Some of them like the "outstanding-count throttle" rely on queues.
I'll have to think about whether lazy lists would help there.
http://osteele.com/talks/ajaxian-2008/samples/throttling-6.js.html
Alexander Fritze
On 6. Mar2009, at 17:31 , Matt Brubeck wrote:
>
> On Mar 6, 7:29 am, Alexander Fritze <alex.fri...@gmail.com> wrote:
>> This looks really great, and if you're ok with it I think we should
>> merge your new primitives into oni.js, pretty much as they are. (The
>> Oni code is licensed under MIT license - I've just added a header to
>> that effect to oni.js.)
>
> Sure, sounds great! Feel free to change the names, implementation,
> etc. if you want to. As you can see, my source code on GitHub was
> already MIT-licensed.
I merged your code into oni.js (or rather oni.js.in, from which oni.js
is generated). Please take a look at the attached patch and let me
know if it looks ok to you (and if you want your email address in the
license header).
Your new operators are the first instances of macro-like higher-order
operators we have in Oni (in that they operate on oni operator *names*
rather than expressions), and I'm still having a bit of a hard time
wrapping my head around them :-)
One thing I've been wondering is if it is possible/useful to write
them in partially curried style (I've tried but haven't been very
successful with this yet):
EachWith(Par|Seq|Alt) -> returns an oni_fexp which expects args
(oni_fexp, js_array)
Then the resulting oni_fexps could be used indirectly, e.g.:
var ParEach = EachWith(Par);
var SeqEach = EachWith(Seq);
Let( { execution_strategy : If(some_condition, ParEach, SeqEach) },
...
Apply(Get('execution_strategy'), some_func, some_data_arr) )
Cheers,
Alex
Alexander Fritze
On 8. Mar2009, at 23:52 , Alexander Fritze wrote:
> [...]
> I merged your code into oni.js (or rather oni.js.in, from which oni.js
> is generated). Please take a look at the attached patch and let me
> know if it looks ok to you (and if you want your email address in the
> license header).
>
Oops... here is the patch:
diff --git a/oni.js.in b/oni.js.in
--- a/oni.js.in
+++ b/oni.js.in
@@ -1,12 +1,13 @@
// Copyright (c) 2008-2009 Oni project contributors
//
// Contributors:
// Alexander Fritze <al...@croczilla.com>
+// Matt Brubeck <>
//
// Permission is hereby granted, free of charge, to any person
// obtaining a copy of this software and associated documentation
// files (the "Software"), to deal in the Software without
// restriction, including without limitation the rights to use, copy,
// modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
@@ -66,17 +67,38 @@ function timeout(cont, duration_ms) {
timer = null;
}
},
duration_ms);
return function() { if (timer) timer.cancel(); };
}
#else
-
+// Array.prototype.map compatibility code from
+// https://developer.mozilla.org/En/Core_JavaScript_1.5_Reference:Objects:Array:map
+if (!Array.prototype.map)
+{
+ Array.prototype.map = function(fun /*, thisp*/)
+ {
+ var len = this.length >>> 0;
+ if (typeof fun != "function")
+ throw new TypeError();
+
+ var res = new Array(len);
+ var thisp = arguments[1];
+ for (var i = 0; i < len; i++)
+ {
+ if (i in this)
+ res[i] = fun.call(thisp, this[i], i, this);
+ }
+
+ return res;
+ };
+}
+
// Binds a function to an object; the function will be executed with
// its 'this' variable set to 'obj':
function bind(f, obj) {
return function() {
return f.apply(obj, arguments);
}
}
@@ -1008,16 +1030,35 @@ function precondition_arg_list(l) {
for (var i=0; i<l.length; ++i) {
if (is_oni_fexp(l[i]))
l[i] = oni_fexp_to_OEN(l[i]);
else if (!isOEN(l[i]))
l[i] = Quote(l[i]);
}
}
+// array helpers:
+function get_at(arr, i) { return arr[i]; }
+function set_at(arr, i, val) { arr[i] = val; }
+
+function GetAt(arr_var, i_var) {
+ return Apply(SLift(get_at), Get(arr_var), Get(i_var));
+}
+function SetAt(arr_var, i_var, val) {
+ return Apply(SLift(set_at), Get(arr_var), Get(i_var), val);
+}
+
+function range(n) {
+ var result = [];
+ for (var i=0; i<n; i++) {
+ result.push(i);
+ }
+ return result;
+};
+
//----------------------------------------------------------------------
// oni_exp constructors:
// Seq : exp* -> oni_exp
function Seq(/* exp1, exp2, ... */) {
precondition_arg_list(arguments);
return new OEN_Seq(arguments);
}
@@ -1130,16 +1171,44 @@ function Break(exp) {
return Throw(loop_exit, exp);
}
function Continue() {
return Throw(loop_continue);
}
EXPORT_SYMBOL(Loop);
EXPORT_SYMBOL(Break);
EXPORT_SYMBOL(Continue);
+
+
//----------------------------------------------------------------------
+// higher-order operators:
+
+// EachWith(Par|Seq|Alt, oni_fexp, java_arr) -> oni_exp
+//
+// EachWith(Par, F, [a,b,...]) -> Par(F(a), F(b), ...)
+// EachWith(Seq, F, [a,b,...]) -> Seq(F(a), F(b), ...)
+function EachWith(Combinator, F, arr) {
+ return Combinator.apply(this, arr.map(F));
+}
+EXPORT_SYMBOL(EachWith);
+
+// MapWith(Par|Seq|Alt, oni_fexp, java_arr) -> oni_exp
+//
+// MapWith(Combinator, F, [a,b,...]).run() -> [<F(a)>, <F(b)>, ...]
+//
+// "Combinator" can be Par (to call F in parallel) or Seq (to call F
sequentially).
+//
+function MapWith(Combinator, F, arr) {
+ return Let({ result: [], arr: arr },
+ Seq(
+ EachWith(Combinator,
+ Lambda(['i'], SetAt('result', 'i', F(GetAt('arr', 'i')))),
+ range(arr.length)),
+ Get('result')));
+}
+EXPORT_SYMBOL(MapWith);
//----------------------------------------------------------------------
// oni_fexp constructors:
var ONI_FEXP_TOKEN = {};
var ONI_FEXP_TO_OEN_TOKEN = {};
function is_oni_fexp(e) {
Matt Brubeck
> I merged your code into oni.js (or rather oni.js.in, from which oni.js
> is generated). Please take a look at the attached patch and let me
> know if it looks ok to you (and if you want your email address in the
> license header).
<mbru...@limpet.net>.
> Your new operators are the first instances of macro-like higher-order
> operators we have in Oni (in that they operate on oni operator *names*
> rather than expressions), and I'm still having a bit of a hard time
> wrapping my head around them :-)
argument is a function, in the native JavaScript sense if not the
oni_fexp sense. "EachWith" just applies the function to its
argument. Apply/Exec will happily work on operator "names" too,
though there's normally no reason to do so: Apply(Par, exp1,
exp2, ...)
> One thing I've been wondering is if it is possible/useful to write
> them in partially curried style (I've tried but haven't been very
> successful with this yet):
>
> EachWith(Par|Seq|Alt) -> returns an oni_fexp which expects args
> (oni_fexp, js_array)
>
> Then the resulting oni_fexps could be used indirectly, e.g.:
>
> var ParEach = EachWith(Par);
> var SeqEach = EachWith(Seq);
http://github.com/mbrubeck/oni-map/commit/97e8dd24c0bd7b265428c463ebd98e274ed73cf9
I used native JavaScript closures for the currying. One thing I'm
still getting used to with Oni is what to do in the host language and
what to do in "pure" Oni.
Alexander Fritze
On 9. Mar2009, at 17:01 , Matt Brubeck wrote:
>
> On Mar 8, 3:52 pm, Alexander Fritze <a...@croczilla.com> wrote:
>> I merged your code into oni.js (or rather oni.js.in, from which
>> oni.js
>> is generated). Please take a look at the attached patch and let me
>> know if it looks ok to you (and if you want your email address in the
>> license header).
>
> The patch looks good. Please do add my email address
> <mbru...@limpet.net>.
>
Done & committed: http://hg.mozilla.org/users/alex_croczilla.com/oni/rev/fabbc62abdac
.
I've also added a couple of samples that exercise the new primitives:
http://hg.mozilla.org/users/alex_croczilla.com/oni/raw-file/fad93da44825/samples/eachwith.html
http://hg.mozilla.org/users/alex_croczilla.com/oni/raw-file/fad93da44825/samples/mapwith.html
>> Your new operators are the first instances of macro-like higher-order
>> operators we have in Oni (in that they operate on oni operator
>> *names*
>> rather than expressions), and I'm still having a bit of a hard time
>> wrapping my head around them :-)
>
> I thought of them as combinators rather than macros - the "Combinator"
> argument is a function, in the native JavaScript sense if not the
> oni_fexp sense. "EachWith" just applies the function to its
> argument. Apply/Exec will happily work on operator "names" too,
> though there's normally no reason to do so: Apply(Par, exp1,
> exp2, ...)
>
Apply(Par, ...) will work, but "Par" is the odd one out in that it is
an oni_fexp. "Seq" & "Alt" are oni_exps and won't work with Apply(...).
Apply(f, arg1, arg2, arg3, ..., argn) evaluates arg1..n before passing
them to f, i.e. it only works for function-like applicative-order
constructs, which in Oni are oni_fexps. oni_exps on the other hand are
constructs that take charge of the evaluation of their arguments
themselves. They are like what in Scheme you would call 'special forms'.
>> One thing I've been wondering is if it is possible/useful to write
>> them in partially curried style (I've tried but haven't been very
>> successful with this yet):
>>
>> EachWith(Par|Seq|Alt) -> returns an oni_fexp which expects args
>> (oni_fexp, js_array)
>>
>> Then the resulting oni_fexps could be used indirectly, e.g.:
>>
>> var ParEach = EachWith(Par);
>> var SeqEach = EachWith(Seq);
>
> Ooh, I like that. I tried this out here, seems to work well:
> http://github.com/mbrubeck/oni-map/commit/97e8dd24c0bd7b265428c463ebd98e274ed73cf9
>
Cool, I'll check it out in a bit.
> I used native JavaScript closures for the currying. One thing I'm
> still getting used to with Oni is what to do in the host language and
> what to do in "pure" Oni.
Yeah, I find also find it quite tricky...
I think in the case of the partially curried functions it makes sense
to use Lambda() (or Defun()), so that the resulting construct can be
used indirectly with Apply().
Matt Brubeck
> Apply(Par, ...) will work, but "Par" is the odd one out in that it is
> an oni_fexp. "Seq" & "Alt" are oni_exps and won't work with Apply(...).
> > I used native JavaScript closures for the currying. One thing I'm
> > still getting used to with Oni is what to do in the host language and
> > what to do in "pure" Oni.
>
> Yeah, I find also find it quite tricky...
> I think in the case of the partially curried functions it makes sense
> to use Lambda() (or Defun()), so that the resulting construct can be
> used indirectly with Apply().
the macro-like approach (passing bare Oni operators to native
functions like "map" and "apply") doesn't work inside an Oni program.
I'm still working on this one...
Alexander Fritze
On 10. Mar2009, at 18:18 , Matt Brubeck wrote:
>> [...]
>> Yeah, I find also find it quite tricky...
>> I think in the case of the partially curried functions it makes sense
>> to use Lambda() (or Defun()), so that the resulting construct can be
>> used indirectly with Apply().
>
> Yes, that makes it much harder to write the curried versions, since
> the macro-like approach (passing bare Oni operators to native
> functions like "map" and "apply") doesn't work inside an Oni program.
> I'm still working on this one...
Actually, thinking about this again, I think we should take your
partially curried versions as they are now. I'm not so sure there is
much value in trying to get EachWith(.)/MapWith(.) to return an Oni
Lambda (or get them to behave like one) just for the purpose of
getting them to work with Apply().
In Scheme parlance, I view EachWith/MapWith as constructs that create
"syntactic keywords" (like Alt, Seq, ...) rather than "procedures".
Syntactic keywords introduce "special forms" rather than strict
function application, so they cannot be used with Apply/Exec. So I
think EachWith(.)/MapWith(.) not working with Apply is consistent.
I have some ideas btw of how to get rid of the "special
form"/"procedure" distinction by making Oni mostly non-strict like
Haskell, but it looks like this is much more complex for a concurrency
language than for a "conventional" sequential language. E.g. while in
a conventional language there are mostly only three evaluation
strategies for arguments (strict, delayed, memoized), in a concurrent
setting it makes sense to add the concept of futures [1]. I'm still
thinking of ways to harmonize all of the different concepts...
[1] http://en.wikipedia.org/wiki/Future_%28programming%29
Alexander Fritze
> [...]
> partially curried versions as they are now. I'm not so sure there is
> much value in trying to get EachWith(.)/MapWith(.) to return an Oni
> Lambda (or get them to behave like one) just for the purpose of
> getting them to work with Apply().
>
http://hg.mozilla.org/users/alex_croczilla.com/oni/rev/88f2e52a28b3