Request for comments, async ISO core standard I/O

[Mostowski Collapse]

Now I got an idea how to asyncify the ISO core standard I/O.
The idea is very simple and I try to solve one problem: How
can we make for example get_code/2 asyncified, without

sacrificing performance? I came up with this sketch:

get_code(S, C) :- get_code_buffered(S, C), !.
get_code(S, C) :- async_read_buffer(S), get_code(S, C).

There are some unresolved issues in the above code, like
how return end of file, i.e. -1 or how to deal with surrogate pairs.
But essentially the idea is that the stream S has a buffer

somewhere and that the fast path is to read from this buffer,
and we only need to yield when we replenish the buffer via
some I/O. This could give a quite fast asyncified get_code/2.

Is there any Prolog system that did already something like this?
I always see that get_code/2 is a primitive and was also folllowing
this approach in my systems so far. In the above I bootstrap it from two

other primitives, a synchronous one and a asynchronous one,
make it itself not anymore primitive. get_code_buffered/2 would
fail if it has reached the end of the buffer.

[Mostowski Collapse]

But there is a lot of work to do, peek_code/2 needs a similar
treatment, and then in my system I have also get_atom/3, which
would need some treatment. Scryer Prolog would go for the

XXX_char predicates first, if they are present in a Prolog system
they would also need some treatment. Also the Python and the
JavaScript target might turn out different. In JavaScript there are

different fetch modes, chunked and non-chunked. In non-chunked
we might get the whole file in one go, a second call to async_read_buffer/1
would not exist. Chunked is related to what SWI-Prolog calls http

streaming. I don’t know the details yet. And then maybe it should
transparently choose something synchronous if asynchronous
behaviour is not demanded from the Prolog interpreter at all.

[Mostowski Collapse]

Concerning Scryer Prolog I find a mention of async here:

socket_server_accept/4 cannot be interrupted with Ctrl-c
https://github.com/mthom/scryer-prolog/issues/485

But this was already a while ago, like 2020.
And does Scryer Prolog have a yield instruction?

[Mostowski Collapse]

'When a clown moves into a palace, he doesn't
become a king. The palace becomes a circus.'

-Turkish proverb

Now we have two ClownPrologs:

ClownProlog Nr. 1: Ciao Prolog:
This here looks like a Prolog terminal in the browser,
but doesn't behave like a Prolog terminal:
https://ciao-lang.org/playground/
Try read/1 etc.. Its documented that this is a limitation.

ClownProlog Nr. 2: SWI-Prolog:
Maybe inspired by Ciao Prolog playground, that they made
quickly a rip off? Its a little better off, one can abort a query.
https://dev.swi-prolog.org/wasm/shell
But try read/1 etc..

My proposal for async ISO core standard I/O could
adress this problem. But interestingly its also not top
priority for Dogelog. I just figured out that I need other

stuff first. But Dogelog does not try to present the illusion
of a terminal, when it does not yet have a terminal. None
of the examples try to provide such a illusion.

[Mostowski Collapse]

Since priorities are different, it will clearly take weeks
or months, for a demonstration. Also there are not
much shoulders to climb, at least I didn't find

a lot of precedents, and who wants to climb giant
morons? ClownProlog Nr. 2 is even not asyncified, they
even don't know what this is. ClownProlog Nr. 1 could

be better off, they seem to have a haze idea.

LoL

[Mostowski Collapse]

Why could be ClownProlog Nr. 1 better of.
They have an async facade, at least their file
says so ciao-async.js. Thats already a good

start, although it has the unholy combination
with a worker. ClownProlog Nr. 2 demonstrates
a solution without a worker. For ClownProlog Nr. 1

I am refering to this here:

Monaco Playground for Ciao Prolog
Madrid, Junio 2022 - Guillermo Garcia Pradales
https://oa.upm.es/71073/1/TFG_GUILLERMO_GARCIA_PRADALES.pdf

Pitty they are so closed. The tutorial application
mentioned in the PDF is nowhere seen. They are
similarly close like when they published this here:

Lightweight compilation of (C)LP to JavaScript
https://arxiv.org/pdf/1210.2864.pdf

It never made it into any Ciao distribution. Which is
anyway now obsolete, since Ciao WASM is faster
than Ciao JS. Ciao WASM is really brilliant!

[Mostowski Collapse]

Rust struggles with Async, from the Chalk blog

Async cancellation: a case study of pub-sub in mini-redis
https://smallcultfollowing.com/babysteps/blog/2022/06/13/async-cancellation-a-case-study-of-pub-sub-in-mini-redis/

I wonder where he gets the nonsense from that
JavaScript does create tasks for async functions,
and that an async helper would even create a further

Task. Sounds pretty much FUD to me, trying to defend some
Rust alternative. Similar to the FUD that an asyncified WASM
has overhead and unfavorable footprint.

Thats just your humanly ape brain acceptance curve:
- Resistance
- Denial
- Exploration
- Commitment

Where are you on this timeline?

LMAO!

[Mostowski Collapse]

Anyway why ask for permission to write enhancement
proposals (EPs), just write them? From the time of Paulo
Moura there are the most EPs. Prolog threads/messages

(October 28, 2008), Prolog global variables, what else? You
see the focus of his endeavours here, everything stalled
in the below forum in 2009, where did it go?

- Core Revision
- Definite Clause Grammars
- Globals and Arrays
- Multi-Threading
- Portable Operating-System Interface
- Unicode
https://www.prolog.logtalk.org/

What PEPs do you want to write? Did his focus match the
requirements of Web Prolog. I do see missing:

- Cooperative Multitasking
- Foreign Functions Calls
- Format and Protocolls
- Prolog Corouting & Attributed Variables
- What else?

Cooperative Multitasking would cover issues that recently
poped up in the WASM and non-WASM integration of
Prolog system into a HTML page, similarly Foreign Functions

Calls, like having a DOM event handler call Prolog, or manipulating
DOM by Prolog. The Format and Protocolls item can mean HTTP,
JSON, etc… The first 3 topics are somehow interconnected, so if you

botch one of the 3 topics, the overall result is not so good.

[Mostowski Collapse]

LoL, I guess somebody didn't update their servers:

[phpBB Debug] PHP Warning: in file [ROOT]/includes/bbcode.php on line 483:
preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead

https://www.prolog.logtalk.org/ is producing nonsense now.

[Mostowski Collapse]

Try the text of a message:
https://www.prolog.logtalk.org/viewtopic.php?f=5&t=24

Empty, could be the bbcode.php error...

[Mostowski Collapse]

Woa! Maybe make it priority a library(lists) standard?

- Lists

Poor LogNonsenseTalk had to implement everything on
its own. Doesn't this lead to some code bloat? If you have
a playground and WASM, you probably don't want this,

everything twice going over the wire:
https://github.com/LogtalkDotOrg/logtalk3/blob/master/library/types/list.lgt

Mostowski Collapse schrieb am Freitag, 12. August 2022 um 15:49:54 UTC+2:

[Mostowski Collapse]

Mostlikely there will never be a Lists PEP, since the
Prolog systems are much to much ossified. A little chance
might have some newcomers, like for example:

- Tau Prolog
- Scryer Prolog
- Trealla Prolog
- Dogelog Player
- Ichiban Prolog
- What else?

But the old whales have possibly no chance to do something,
would require too much refactoring. Because most likely lists and
friends are used internally by the compiler and runtime.

What would have a chance maybe, if there were forks especially
for some playgrounds. Lets say such a fork were Ciao2. Then
there would be maybe a chance if we would see:

- Ciao2 Prolog
- GNU2 Prolog
- SICStus2 Prolog
- What else?

[Mostowski Collapse]

If I only pick substract/3 its already here:

- YAP Prolog
- ECLiPSe Prolog
- What else?

I cannot asses the YAP Prolog systems so easily. For example,
there is maybe a pretty printing problem. I see strange stuff
like this extra memberchk/0 and extra subtract/0:

subtract([], _, []) :- subtract.
subtract([E|T], D, R) :-
memberchk(E, D), memberchk,
subtract(T, D, R).
subtract([H|T], D, [H|R]) :-
subtract(T, D, R).
https://www.dcc.fc.up.pt/~vsc/yap/lists_8yap_source.html

Some broken pretty printer?

[Mostowski Collapse]

Ulrich Neumerkel was invesitigating certain predicates in
the past. Like for example here. This seems be a kind of
updated, I see Ichiban Prolog column, must be newer:

https://www.complang.tuwien.ac.at/ulrich/iso-prolog/length

If you append .html,v, you see the versions of this file:

1.23
date 2022.07.16.13.24.22

Whether somebody can jump into this role of Ulrich Neumerkel,
especially from academia, I doubt. They don’t have time
for that. They have other jobs to do. Maybe a better

automatization would help. I wrote a new version of a test
runner for Dogelog player. There are various test runners
included in various Prolog systems. Even Logtalk has

a test runner. For a continous integration / continous
deployment pipeline, that would do a kind of PEP assesment
across Prolog systems, that is visible together with the PEP,

somesuch could be a useful tool.

[Mostowski Collapse]

Unfortunately the test campaigns of Ulrich Neumerkel
don't have much merit. On the other hand he recently
raised a ticket about:

?- setof(t,(L=2;L=1),L).
uncaught exception: error(type_error(list,1),sort/2), unexpected.

slightly implying he wants not a type error? On the other
hand his latest length/2 test cases, are like 75% type
error and domain error tests of some output argument,

which in my opinion can be easily dropped from a certain
interpretation of the ISO core standard. It only causes
slowdown of the Prolog system. Among the Prolog systems

that pass these nonsense tests are also mostly Prolog
systems that are not the fastest. See for yourself:

https://www.complang.tuwien.ac.at/ulrich/iso-prolog/length

[Mostowski Collapse]

In length/2 its easy to nevertheless support these errors,
since one can use a var/1 test in the second argument instead
of a ground/1 test in the first argument, when deciding the mode.

In as far I can offer this behaviour:

?- length([a,b,c],x).
error(type_error(integer, x), [user:1])
?- length([a,b,c],-1).
fail.
?- length(X,-1).
fail.

Isn’t the fail nice?

The fail is not on the agenda of Ulrich Neumerkel.

[Mostowski Collapse]

Also tests such as:
```
?- length([a|b], X).
ERROR: Type error: `list' expected, found `[a|b]' (a compound)
```
Are totally missing in the assesment page by Ulrich Neumerkel.
So what is the coverage?

[Mostowski Collapse]

Maybe a good start for Cooperative Multitasking would
be an engine API? Or a delimited continuation API?

Delimited continuation is sometimes favored to make
stare transformers. Here is a little benchmark from SWI-Prolog:

/* shift/1, warm run */
?- time(run_state(fib(28), 0, S)).
% 9,770,334 inferences, 2.437 CPU in 2.861 seconds (85% CPU, 4008342 Lips)
S = 514229 .

/* engine_yield/1, warm run */
?- time(run_state(fib(28), 0, S)).
% 3,599,612 inferences, 1.109 CPU in 1.177 seconds (94% CPU, 3244721 Lips)
S = 514229 .

Woa! engine_yield/1 was more than twice as fast as shift/1! Was using this
fibonacci with spurious choice points, as often found in pure DCG:

fib(0) :- inc(1).
fib(1) :- inc(1).
fib(N) :- N > 1, M is N-1, fib(M), J is M-1, fib(J).

Mostowski Collapse schrieb am Freitag, 12. August 2022 um 15:49:54 UTC+2:

[Mostowski Collapse]

Maybe an interesting experiment would be, whether SLG Tabling
can be implemented with engine_yield/1 instead of shift/1?

[Mostowski Collapse]

Quite an amazing test case:

/* SWI-Prolog 8.5.14 */
?- time(run_state(fib(25), 0, S)).
% 2,306,455 inferences, 0.453 CPU in 0.499 seconds
(91% CPU, 5090108 Lips)
S = 121393 .

/* Scryer Prolog 0.9.0 */
?- time(run_state(fib(25), 0, S)).
% CPU time: 12.532s
S = 121393

[Mostowski Collapse]

Its both with shift/1 !

[Mostowski Collapse]

Are the shift/1 solution and the engine_yield/1 solution
semantically equivalent at the moment. The answer is no!

/* shift/1 solution */
?- run_state((inc(1), (inc(2); inc(4))), 0, S).
S = 3 ;
S = 5.

/* engine_yield/1 solution */
?- run_state((inc(1), (inc(2); inc(4))), 0, S).
S = 3 ;
S = 7 ;
false.

Can we nevertheless possibly implement shift/reset
via engines? I wouldn't exclude this possibility, and

it might indeed give faster shift/reset.

[Mostowski Collapse]

Pitty the CW631 paper here:
https://www.cs.kuleuven.be/publicaties/rapporten/cw/CW631.pdf

doesn’t show Felleisen prompt/control implemented with difference lists:

/* Vanilla conjunction list solve/1 extended to control/3 */
control([], none, []).
control([prompt(A)|B], A, B).
control([control(L, P, Q)|B], X, Y) :- control(L, P, Q), control(B, X, Y).
control([A|B], X, Y) :- rule(A,C,B), control(C, X, Y).

/* Difference List variant of clause/2 */
rule(sum(A+B),[prompt(ask(A)),prompt(ask(B))|R],R).
rule(with_list(G,P,Q),[control(G,T,C),with_list2(T,C,P,Q)|R],R).
rule(with_list2(none,_,P,P),R,R).
rule(with_list2(ask(X),C,[X|P],Q),[with_list(C,P,Q)|R],R).

Works fine:

?- control([with_list([sum(S)],[1,2,3],L)],_,_).
S = 1+2,
L = [3] .

[Mostowski Collapse]

Its quite fast, since it does nothing with the continuation, just takes it as is,
which I suppose is the spirit of Felleisen. Here the good ole shift/reset:

/* SWI-Prolog 8.5.14 */
?- time(run_state(fib(25), 0, S)).
% 2,306,455 inferences, 0.453 CPU in 0.499 seconds (91% CPU, 5090108 Lips)
S = 121393 .

And here the prompt/control:

/* SWI-Prolog 8.5.14 */
?- time(control([run_state([fib(25)],0,S)],_,_)).
% 3,641,788 inferences, 0.891 CPU in 0.980 seconds (91% CPU, 4089025 Lips)
S = 121393 .

/* Jekejeke Prolog 1.5.4, only 512 MB allocated, JDK 16 */
?- time(control([run_state([fib(25)],0,S)],_,_)).
% Threads 1,656 ms, GC 404 ms, Up 2,086 ms (Current 08/23/22 04:19:56)
S = 121393

[Mostowski Collapse]

Mostowski Collapse schrieb am Donnerstag, 11. August 2022 um 17:27:36 UTC+2:
> Thats just your humanly ape brain acceptance curve:
> - Resistance
> - Denial
> - Exploration
> - Commitment

If you are a slow thinker, you pass these quadrants slowly,
maybe you never reach the end. Here is an example of a
slow thinker, Ulrich Neumerkel, concerning Unicode:

bla bla bla TPTP and bla bla bla WG21/N3146
https://www.tptp.org/TPTP/SyntaxBNF.html#lower_alpha
https://github.com/mthom/scryer-prolog/issues/1515#issuecomment-1225242783
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3146.html

LoL

[Mostowski Collapse]

If you would like to be TPTP compatible (versus v8.1.0.0),
you would need to ban this:

$ target/release/scryer-prolog -v
"v0.9.0-181-g8e9302ea"
$ target/release/scryer-prolog
?- X = hörgerät.
X = hörgerät.

So Scryer Prolog is now somewhere between TPTP
and a Prolog Unicode, neither fish nor fowl,
I cannot parse this, doesn't work in Scryer Prolog:

$ cat text.pl
text("«The Logos of Cybele is the idea that the Great Mother creates\n\
and kills everything. It is not eternity (Apollo) or the circle\n\
(Dionysus), but something that acts in her way with blind\n\
and absolute power. A form of progress: bottom-up growth.\n\
We are experiencing the final attack of Cybele, of the Great Risen\n\
Mother, with feminism, artificial intelligence, globalization,\n\
democracy, liberalism, and so on»").

I get this error:

$ target/release/scryer-prolog
?- ['text.pl'].
error(syntax_error(missing_quote),read_term/3:0).
false.

Works fine in SWI-Prolog:

?- text(X), write(X), nl, fail; true.
«The Logos of Cybele is the idea that the Great Mother creates
and kills everything. It is not eternity (Apollo) or the circle
(Dionysus), but something that acts in her way with blind
and absolute power. A form of progress: bottom-up growth.
We are experiencing the final attack of Cybele, of the Great Risen
Mother, with feminism, artificial intelligence, globalization,
democracy, liberalism, and so on»
true.

[Mostowski Collapse]

Holy Cow, Logtalk is the converse of Ciao Prolog.
Its kind of the inverted image of the library(lists)
of Ciao Prolog. What Logtalk has:

subtract([], _, []).
subtract([Head| Tail], List, Rest) :-
( memberchk(Head, List) ->
subtract(Tail, List, Rest)
; Rest = [Head| Tail2],
subtract(Tail, List, Tail2)
).

is not in Ciao Prolog. And vice versa, like
union/3, interaction/3, etc.. is missing from lists.
So if Ciao Prolog would load Logtalk it

would have everything?

[Mostowski Collapse]

Slow thinking can lead to constant pain. Like here:

And, indeed, in the case of an atom the escaping is not necessary:
?- write('foo"bar').
foo"bar true.
https://github.com/mthom/scryer-prolog/issues/1552#issuecomment-1207677642

What does "not necessary" mean. Thats nonsense.

Wouldn't it be a better and less confusing explanation that write/1
must behave as putatom/1, and everything else would be a bug,
since it has quoted(false)?

At least this is what the ISO core standard tells me:

- ISO/IEC 13211-1:1995(E), Page 100

So basically write/1 will behave, concerning atoms
as if (in DEC-10 Dialect):

[...]
write(X) :- atom(X), name(X,L), putlist(L).
[...]

putlist([X|Y]) :- put(X), putlist(Y).
putlist([]).

[Mostowski Collapse]

Also, I don't find a putatom/1 in DEC-10 Manual from year 1982 here:

Department of Artificial Intelligence. University of Edinburgh
DECsystem-10 PROLOG USER'S MANUAL - 10 November 1982
https://userweb.fct.unl.pt/~lmp/publications/online-papers/DECsystem-10%20PROLOG%20USER%27S%20MANUAL.pdf

Where do you have an evidence, for the year 1978?

[Mostowski Collapse]

Slow thinking can lead to constant pain! LoL

Logtalk is writing nonsense:
Union, intersection, ... are set operations. Subtract is not.
https://github.com/LogtalkDotOrg/logtalk3/issues/150#issuecomment-1226084264

Then correcting it to:
Union, intersection, ... are set operations. Subtract is not a set only operation.

Ha Ha, when you have no clue how Prolog works. Union and
Intersection are also "not a set only operation", what ever that
means. They are in the same category like Subtract.

Come on guys, its not so difficult! sets.lgt has also subtract:

subtract(Set, [], Set) :- !.
subtract([], _, []) :- !.
subtract([Head1| Tail1], [Head2| Tail2], Difference) :-
compare(Order, Head1, Head2),
subtract(Order, Head1, Tail1, Head2, Tail2, Difference).

subtract(=, _, Tail1, _, Tail2, Difference) :-
subtract(Tail1, Tail2, Difference).
subtract(<, Head1, Tail1, Head2, Tail2, [Head1| Difference]) :-
subtract(Tail1, [Head2| Tail2], Difference).
subtract(>, Head1, Tail1, _, Tail2, Difference) :-
subtract([Head1| Tail1], Tail2, Difference).

In SWI-Prolog its clearer since there is no name overloading:

https://www.swi-prolog.org/pldoc/doc_for?object=ord_subtract/3

P.S.: Mathematically we would write P \ Q for the set difference.

https://mathworld.wolfram.com/SetDifference.html

[Mostowski Collapse]

These are some ultra cute graphics!

Cooperative Multitasking Visualized - Tau Prolog, 2022
http://tau-prolog.org/files/prole2022/#/4/4

Noice!

[Mostowski Collapse]

Why not load small JavaScript snippets synchronously
in the browser? This one works for me. At least
some local testing works already, its not yet deployed:

let parent = "";
let cache = {};

/**
* Load a JavaScript text synchronously.
*
* @param url The url.
*/
function cludge(url) {
/* resolve and check */
if (parent !== "")
url = new URL(url, parent).href;
let map = cache[url];
if (map !== undefined)
return map;
map = {};
cache[url] = map;

/* load and execute */
let back = parent;
try {
parent = url;
let request = new XMLHttpRequest();
request.open('GET', url, false);
request.send();
let fun = new Function("exports", "require", request.responseText);
fun(map, cludge);
} finally {
parent = back;
}
return map;
}

Have Fun!

[Mostowski Collapse]

On second though, I will not use this piece
of code, rather should work more consequently
in keeping the overal code async and the examples

async as well, so that it wont happen, that the browser
blocks. From GUI viewpoint its not so trivial
to make it async compatible. There are few tricks

needed. If there is only one action button this
works to avoid that multiple Prolog interpreter
calls are spawned:

async function main_async() {
document.getElementById("launch").disabled = true;
try {
async bla bla
} finally {
document.getElementById("launch").disabled = false;
}
}

document.getElementById("launch")
.addEventListener("click", main_async);

But if there are multiple action buttons,
more work is needed.

Mostowski Collapse schrieb:

[Mostowski Collapse]

How would the world look like, if Paul Tarau would
not have written a paper about Prolog engines years
ago, but about Prolog promisses? Is there even such

a paper already? SWI-Prolog with its await/2 is quite
advanced, they have promise objects passing created
by (:=)/2 and then can be passed to await/2. In Dogelog

player promisses are still opaque, not visible in Prolog
sphere as it is actually the case in SWI-Prolog WASM.
But the idea of enginess still sticks to the mind!

[Mostowski Collapse]

Recent version of Dogelog uses import(), this is a JavaScript
function which returns a promise. Behind this promise
is loading and evaluating JavaScript code, and I could

show how to transpile WordNet to JavaScript, and then load
it via import(). Its quite fast! But can we make it parallel
without going into engines? Maybe use this syntax

for parallel import. A list wrapped into a singleton list. Or use
some flag to control it, since a default parallel behaviour
could be too dangerous?

/* Parallel Variant Syntax of Multi-Consult ? */
?- [['foo.p', 'bar.p', 'baz.p']].

Of course engines or what Tau Prolog did with tasks and
promises, could also lead to a parallel load. Maybe can get
away with a cheaper solution, not bring tasks to Dogelog

player, only juggle with import() ?

[Mostowski Collapse]

So what was rummaging in my brain. Why did I get repelled
by the idea of engines. I don’t know, or maybe I know?
Does SWI-Prolog have only this here, namely:

await(+Promise, -Result)
execution of await/2 completes when the Promise resolves
and Result is unified with the value passed to the
Promise.then() method.
https://www.swi-prolog.org/pldoc/doc_for?object=await/2

How about a new await_all/2, that takes a promise list?
What could be done with await_all/2? BTW: JavaScript
has more in stock than only await_all/2 which would correspond

to Promise.all(), there are like 1-2 other multi-waiters.

[Mostowski Collapse]

Dogelog Player is the avant garde of logic programming.
How long is the adoption time by other Prolog systems?
Depends, it can be 6 to 12 months. Example:

Auto-yield was explained here:

[Feb 3, 2022]
Non-Blocking Browser for Dogelog Player
https://medium.com/

It was adopted 6 months later by SWI-Prolog:

[Aug 6 2022]
ADDED: heartbeat mechanism: by setting the Prolog flag heartbeat,
the multifile predicate heartbeat/0 is called every N inferences.
https://www.swi-prolog.org/

And now it got adopted by Trealla Prolog:

[March 8 2023]
Yield tweaks for Wasm port #136
https://github.com/

[Mostowski Collapse]

I now have complete my take on Task Contexts, there is
an implementation, will soon write a medium article about it.
They are different from Engines. Not sure whether Task

Contexts will be adopted by some Prolog system. It requires
a new and different thinking. You need to think beyond engines.
Actually Task Contexts are independent of engines.

You can add Task Contexts to a Prolog system with engines or
to a Prolog system without engines. All you need is an event loop
and stackless (non-fibers) and stackfull coroutines (fibers).

No engines needed, unless you might want to implement the
event loop and the non-fibers and fibers from engines? Would
it perform? The problem is a calLlater/2 or create_task/1

as a side effect on the current engine loop is much more efficient
then yielding for such a command. Thats why these methods in
JavaScript are not some sort of promises that can be awaited:

setTimeout()
clearTimeout()

They have an immediate effect on the event loop.

[Mild Shock]

Now I have a first prototype running:

$ node dogelog.mjs
?- create_task((between(1,10,_), write(foo), nl, sleep(100), fail; true)).
true.
?- foo
foo
foo
foo
foo
foo
foo
foo
foo
foo

The task is running, because the console read is now async.
If it were not async and blocking, the task wouldn't continue running.

Mostowski Collapse schrieb am Mittwoch, 10. August 2022 um 09:28:41 UTC+2:
> Now I got an idea how to asyncify the ISO core standard I/O.
> The idea is very simple and I try to solve one problem: How
> can we make for example get_code/2 asyncified, without
>
> sacrificing performance? I came up with this sketch:
>
> get_code(S, C) :- get_code_buffered(S, C), !.
> get_code(S, C) :- async_read_buffer(S), get_code(S, C).
>
> There are some unresolved issues in the above code, like
> how return end of file, i.e. -1 or how to deal with surrogate pairs.
> But essentially the idea is that the stream S has a buffer
>
> somewhere and that the fast path is to read from this buffer,
> and we only need to yield when we replenish the buffer via
> some I/O. This could give a quite fast asyncified get_code/2.
>
> Is there any Prolog system that did already something like this?
> I always see that get_code/2 is a primitive and was also folllowing
> this approach in my systems so far. In the above I bootstrap it from two
>
> other primitives, a synchronous one and a asynchronous one,
> make it itself not anymore primitive. get_code_buffered/2 would
> fail if it has reached the end of the buffer.

[Mild Shock]

Whats also working is async file I/O with node fs:
Showing the promises (i.e. open_promise and file_read_promise)
that get fullfilled in the course of action:

?- open('foo.txt', read, S), get_atom(S,-1,A), close(S).
open_promise
file_read_promise
file_read_promise
S = 0rReference, A = 'Hello World!'.

This was not in the request for comments sketch that open
itself is also asynchronous. Astonishingly error handling
works also, as a side effect from interrupt signals I already

developed in the spirit of the JavaScript AbortController.
So I am just using the abort reason field to give the error back:

?- open('foo2.txt', read, S), get_atom(S,-1,A), close(S).
open_promise
Fehler: Datei 'foo2.txt' nicht gefunden.
user auf 5

This is a little bit ugly since it means I am using only 25% of
the JavaScript Promise features, i.e. resolve() the other 50%
and 25% are not using reject() and not using reolve with return

value. What I am using to return values is the stream handle
itself as a value holder. Maybe I can remove sometime in the future
this uglyness. Ideally a concept of arbitrary FFI async predicates

would be swell. Now its a little bit a hack. Next step asyncify
fetch() in the browser. So as to get rid of my current use of
synchronized XmlHttpRequest, which rightfully causes a warning

in the browser saying it might block.

[Mild Shock]

Not sure whether I will asynchify file_property/2 predicate
sa well. But I should somehow, since in the browser it is
based on XmlHttpRequest with method HEAD. So if

I don't get rid of that the browser will still issue a warning.

[Mild Shock]

Now that we have implemented Async I/O for Dogelog Player,
we braught some of the principles already also to formerly
Jekejeke Prolog. Like we could get rid of compression.

But the venture has multiple goals:

- Goal 1: Async I/O, decompressed makes it simpler.

- Goal 2: Novacore, have a smaller core.

So what about Goal 2? One idea behind Goal 2 is to have
a single interface for text and binary streams. Even maybe
allow changing the encoding midflight.

This can be helpful for example in MIME multipart/mixed writing
or reading. So we have to get rid of some Java-ism and ISO-ism:

- ISO-ism: Have separate get_byte/[1,2] and get_code/[1,2],
we have to get rid of that. We are planning to realize binary streams
by an encoding such as "8bit" or "latin1" not yet sure how this
works out. Also the encoding should be mutable.

- Java-ism: In Java starting with BufferedReader and on top
LineNumberReader line sparators such as '\n', '\r' and '\r\n'
area compressed into '\n'. We have also to get rid of that if want
to processs binary streams with the text API.

There is no encoding parameter yet in Dogelog Player, its all
hardcoded "utf8". And formerly Jekejeke Prolog has an
encoding parameter but its immutable. So we will work on this
as well.

[Mild Shock]

Getting rid of the Java-ism of compression was quite fun. We
tried to push it to its limits. So the tokenizer written 100%
in Prolog now preserves '\n', '\r' and '\r\n'. But then when we for

example generate HTML we have to replace the line terminator
by </div></div>. So how to do this without falling back to an
atom_split/2 with separator '\n' and do it correctly?

Its a litte bit tricky, like for example an input such as 'abc\r'
should have a line count of 2. So anything that views '\r' as
padding will go wrong. The following works fine:

/**
* sys_split_lines(L, I, O):
* The predicate succeeds in L with the lines
* of the input I and output O codes.
*/
% sys_split_lines(-List, +List, -List)
sys_split_lines([A|L]) -->
sys_split_line(X), {atom_codes(A,X)},
sys_split_more(L).

% sys_split_more(-List, +List, -List)
sys_split_more([A|L]) --> sys_convert_sep, !,
sys_split_line(X), {atom_codes(A,X)},
sys_split_more(L).
sys_split_more([]) --> [].

% sys_split_line(-List, +List, -List)
sys_split_line([X|L]) --> \+ sys_convert_sep, [X], !,
sys_split_line(L).
sys_split_line([]) --> [].

The above uses DCG (\+)/1 (% 7.14.11) banned (sic!) by Scryer Prolog.
Where the line separators are kind of plugable, currently defined as follows,
but can be an arbitrary set of arbitrary long code combinations:

% sys_convert_sep(+List, -List)
sys_convert_sep --> [0'\r, 0'\n].
sys_convert_sep --> [0'\n].
sys_convert_sep --> [0'\r].

BTW: I think SWI-Prolog already implements some of the ideas
like encoding switching, which we do not have a demonstrator
for yet. But its a little bit weak and stubborn concerning line

terminators refuses to support CRLF.

[Mild Shock]

Also Scryer Prolog and Trealla Prolog might have shoot
themselves into the foot by usually prefering chars over codes.
This somehow makes a greater divide between text and binary,

but for practical purposes such as MIME multipart/mixed
we have to work the other way around, closing the abyss between
text and binary. Just make it look the same. According to

this principle:

KISS, an acronym for "Keep it simple, stupid!"
https://en.wikipedia.org/wiki/KISS_principle

Just how these things such as MIME multipart/mixed possibly
evolved. People had mostlike switchable encoders and binary
streams, and not a wall between text and binary.

Not sure whether encoding such as EBCDIC is an argument
in favor of chars? But for example adding NEL to the line separators
shouldn't be a problem in itself. In case an EBCDIC to Unicode

encoder insists on produce this code point for line separators.

[Mild Shock]

Although the newline compression Java-ism is a pain
in the ass. Something else is quite nice to close the gap
between text and binary. Here is a comparison of two

string encodings, since we are planning to use simply
Prolog atoms and thus strings for binary payload. Both
Dogelog Player and formerly Jekejeke Prolog are already

good with huge amount of atoms based on native strings,
no artificial string Prolog type is needed. So what are the
options for atoms based on native strings:

- UTF-8 native strings: Doesn't work well with the envisioned
"latin1" payload, the range 0..255 is encoded in either 1 byte or
2 bytes. Would need to have clever strings that switch
encoding between "utf8" and "latin1".

- UTF-16 native strings: Does work well with the envisioned
"latin1" payload, the range 0..255 is encoded in one 16-bit word,
which is two bytes, but most programming language already
implement clever strings:

JEP 254: Compact Strings
We propose to change the internal representation of the String class
from a UTF-16 char array to a byte array plus an encoding-flag field.
The new String class will store characters encoded either as ISO-8859-1/
Latin-1 (one byte per character), or as UTF-16 (two bytes per character),
based upon the contents of the string. The encoding flag will indicate
which encoding is used.
https://openjdk.org/jeps/254

So the idea is not to introduce a special byte array type or
some such. The plan is to carry around byte arrays as ordinary
Prolog atoms, which are in effect mapped to the target

native strings, that are anyway already optimized for byte
payload, this will cost us nothing, just use the target
native strings for the targets Java, Python and JavaScript.

[Mild Shock]

Now asyncifying I/O of Dogelog Player for Python.
I guess we got our head around the equivalents
of our Java Surrogate async/await constructs

"Promise" and "Coroutine". The key utilities among
asyncio are asyncio.to_thread an asyncio.run_coroutine_threadsafe,
which seem toe especially made for the two use cases.

Namely what was our Java Surrogate "Promise" wrapper,
now looks like here, what a wonderful code gem:


async def console_promise(buf, stream):
try:
res = await asyncio.to_thread(blocking_readline, stream.data)
stream.buf = res
stream.pos = 0
except IOError as err:
register_signal(buf, map_stream_error(err))


def blocking_readline(data):
return data.readline()


And what was our Java Surrogate "Coroutine" wrapper,
now looks like here, what a wonderful code gem again:

def test_sys_http_server_on(args):
[...]
obj.func = lambda req, res: baby_come_back(
launch_async(clause, buf, [req, res]), loop)


def baby_come_back(coro, loop):
future = asyncio.run_coroutine_threadsafe(coro, loop)
return future.result()

LoL

[Mild Shock]

But its much more complicated than what we did
for JDK 21. Also starting the HTTP server in a separate
thread is extremly frightening:


def test_http_server_listen(args):
[...]
thread = threading.Thread(target=blocking_forever, args=(obj,))
thread.start()


def blocking_forever(obj):
obj.serve_forever()


Its extremly frightening since the Thread docu warns us:

Thread-based parallelism
In CPython, due to the Global Interpreter Lock,
only one thread can execute Python code at once
https://docs.python.org/3/library/threading.html

global interpreter lock
Also, the GIL is always released when doing I/O.
Past efforts to create a “free-threaded” interpreter
have not been successful
https://docs.python.org/3/glossary.html#term-global-interpreter-lock

But still, our use of asyncio.to_thread and
asyncio.run_coroutine_threadsafe capitalizes on
that the GIL is nevertheless released during I/O,

and we don't see much issue here.

Mild Shock schrieb:

[Mild Shock]

Conclusions so far:

Super fast Phyton makes you bald, loose your protein
filaments that grow from follicles found in your head dermis.
https://www.linkedin.com/in/jasonbrownlee/

Why? Because there is no super fast Python. By default
the Python byte code cannot use multiple cores!