What is the module of SWI-Prolog ?

[Kuniaki Mukai]

The module functionality of SWI-Prolog is one of
what we all have been using daily, but as for me without understanding precisely
what it is. I remember vaguely something like the
introductory description about SWI-Prolog module:

... SWI-Prolog module is based on "predicate space", but not
on "name space". ...

But, I did not see at all what the description meant at that time (ten years ago ?)
To be honest, the precise meaning of
the module is still out of my understanding, though I am able to
write complex codes, I believe, in SWI-Prolog, but which are based on
my private mental ad hoc model of the module concept.

I am a little bit family with "scope” of identifier in block
based programming languages like Pascal/Algol/Ada ...
though I am afraid they might be too old now to understand modern module.

I appreciate if clear explanation on difference between
"name space" and "predicate space” are given here by expert.

Thanks,

Kuniaki Mukai

[Jan Wielemaker]

The short explanation is that in a name based module system every atom
(in the Prolog sense) that appears in a module is tagged with the
module. On some systems this is done my simply prefixing the atom. So,
if in module 'a' you use the atom 'p', the real atom is 'a$p'. Now, if
you import an atom from another module, e.g., module 'b' imports 'p'
from 'a', writing 'p' in 'b' also produces 'a$p'. As you can see, this
provides molecularity. The nice thing is that eventually predicates are
still name/arity pairs living in a global space. No special declarations
are needed that tell the system that some name actually refers to a
predicate as the atom itself is tagged. That also brings the other
consequence: beside predicate names you also have to import atoms that
you share for whatever reason.

Opposed, in the predicate based module systems, atoms are left
untouched, but predicates are identified by the triple
module:name/arity. This means that if you need to resolve name/arity or
a goal term (p(....)) to a predicate, you need to know in what module to
look. Inside a module that is not a problem, but it becomes a problem if
one module passes a name/arity to another module that must do the
lookup. This is achieved using the meta_predicate declaration, which
tells the system to qualify the declared argument with the current
module *if not already qualified*. Thus, if we define

:- module(apply,
[ forall/2
]).
:- meta_predicate forall(0,0).

forall(Gen, Cond) :-
\+ (Gen, \+ Cond).

and we call forall/2 from some module q, it in fact calls

forall(q:Gen, q:Cond)

I think Quintus was the first system to implement that (correct me if
I'm wrong). From there it got into SICStus, Ciao, YAP and SWI-Prolog
(more?) Incorrect interpretation of the Quintus docs was one of the
reasons why the SWI-Prolog version was a little different. This
incompatibility was resolved in later versions. Over the years new use
cases were identified and solved in different ways, causing some more
diversion.

The motivation of Quintus was, if I recall correctly, that this module
system is easy to use except for meta-predicates that are exported (and
imported). People using this construct are supposed to be knowledgeable
in the first place and thus likely capable to deal with the complexity.

I think most of the confusion is the dual role of :, which may be used
to call p in m, as m:p(...) as well as qualify arguments as in
assert(m:p(...)). As the builtin assert/1 is available from all modules
this is the same as m:assert(p(...)). ECLiPSe solved this by
introducing Goal@Module, which allows running a predicate from one
module in the context of another module. SWI-Prolog has @/2, but it
is very rarely used.

The module system is defined and (I think) intended to avoid the need
for explicit qualification in almost all circumstances. Used that way it
works quite reasonable IMO. Many people seem to like explicit
qualification though and then it can get nasty. I never really
understood why people like that. Is it to be explicit or because they
copy this behaviour from another language?

Well, this is how I see it. I'm sure there are other sensible ways to
look at it that are completely different. Logtalk can be considered a
module system on steroids, compiling Logtalk specification to a plain
non-modular Prolog program. Whether or not you need steroids is an
entirely different discussion. Note that Quintus-like module system in
SWI-Prolog is build on top of more low-level primitives. These can also
be assembled in a different way to provide a different module system.

Cheers --- Jan

[Proof Easel]

But obviously a module system has somewhere a mapping:

   name <-> file name

Mapping the file name space to the module name space.
In SWI-Prolog, the module name space seems to be flat,
and the file name space is non flat, the usual file name segments.

You can easily query it:

Welcome to SWI-Prolog (threaded, 64 bits, version 7.7.1)

SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

?- current_module(lists).

true.

?- module_property(lists, file(F)).

F = '/Software/Logic/SWI-Prolog.app/Contents/swipl/library/lists.pl'.

[Proof Easel]

If you want to use a module system for querying a module
taxonomy, you only need a module property somewhere
that returns the semantic relationships between different

modules. In SWI-Prolog I guess you cannot query the
semantic relation with module_property/2, if I remember
well its a separate predicate import_module/2:

     True if Module inherits directly from Import.
     http://www.swi-prolog.org/pldoc/man?predicate=import_module/2

But I am not sure whether it really for example reflects
a reexport/1 directive of a module. Or has only special
uses for user and system.

Logtalk would do this work all again non-native. Modules
would have technical names such as object, prototypes, etc..
whatever and semantics relationships would be named

specializes, etc..  But I guess doing it natively is much faster,
and can result in a more seamless solution and much
more dynamic, even more steroids solution...

[Proof Easel]

The drawbacks of flat name spaces are well known:

Name Space

Flat Name Spaces
Name spaces can be flat or hierarchical. Flat name spaces do not
scale well because they can grow only so large before all available
names are used up. Once a name is used more than once in a name
space, the name space violates the unambiguously resolvable requirement.

Hierarchical Name Space
A hierarchical name space is divided into different areas, which can be
thought of as subname spaces. Each area is its own subname space
within the overall name space. Therefore, each object must have a
unique name only within its subname space in order to have an
unambiguously resolvable name within the name space hierarchy.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms682853%28v=vs.85%29.aspx

Am Montag, 11. Juni 2018 10:33:00 UTC+2 schrieb Proof Easel:

[Jan Wielemaker]

On 06/11/2018 10:52 AM, Proof Easel wrote:
> The drawbacks of flat name spaces are well known:

But the question was about name based vs. predicate based. That is
completely orthogonal to flat vs hierarchical. SWI-Prolog (and most
(all) others) module name space is flat. Import modules are again
a completely unrelated concept. They can define a hierarchy of modules,
but not of their name.

Cheers --- Jan

[Paulo Moura]

Hi,

This is a quite clear explanation of atom-based vs predicate-based module systems :-) A possible single sentence description of modules is that they are a (atom or predicate) *prefixing* mechanism.

A notable example of an atom-based module system is XSB. Ciao Prolog module system also exhibits some atom-based characteristics. Most Prolog module systems are, however, predicate-based.

> I think Quintus was the first system to implement that (correct me if
> I'm wrong).

Correct. Modules are invented, from the little historic information that I have been able to collect over the years, by Dave Bowen, back in 1984.

> From there it got into SICStus, Ciao, YAP and SWI-Prolog
> (more?) Incorrect interpretation of the Quintus docs was one of the
> reasons why the SWI-Prolog version was a little different. This
> incompatibility was resolved in later versions. Over the years new use
> cases were identified and solved in different ways, causing some more
> diversion.

WG17 tried to fix it by inventing a new module system (instead of trying to standardize the existing one), which was/is basically ignored (thankfully as it's a broken spec that e.g. allows two compliant systems to be incompatible!).

> The motivation of Quintus was, if I recall correctly, that this module
> system is easy to use except for meta-predicates that are exported (and
> imported). People using this construct are supposed to be knowledgeable
> in the first place and thus likely capable to deal with the complexity.

The main motivation was also that modules could double as an encapsulation mechanism with zero overhead. This zero overhead was (and mainly still is) accomplished by compile-time prefixing.

> I think most of the confusion is the dual role of :, which may be used
> to call p in m, as m:p(...) as well as qualify arguments as in
> assert(m:p(...)). As the builtin assert/1 is available from all modules
> this is the same as m:assert(p(...)). ECLiPSe solved this by introducing Goal@Module, which allows running a predicate from one
> module in the context of another module. SWI-Prolog has @/2, but it
> is very rarely used.

A nasty effect of the prefixing semantics is that qualified calls to module meta-predicates don't have the same semantics as implicitly qualified calls.

> The module system is defined and (I think) intended to avoid the need
> for explicit qualification in almost all circumstances.

Indeed I have seen other implementors/vendors claiming that explicit qualification should be banned!

> Used that way it
> works quite reasonable IMO. Many people seem to like explicit
> qualification though and then it can get nasty. I never really
> understood why people like that. Is it to be explicit or because they
> copy this behaviour from another language?

Carey questions, as other before him, clearly show that implicit qualification can also have nasty consequences. Using implicit or explicit qualification is (or should be) a programming style that should make no difference for program semantics. Unfortunately, that's not the case with module systems due to being a *prefixing* mechanism.

> Well, this is how I see it. I'm sure there are other sensible ways to
> look at it that are completely different. Logtalk can be considered a
> module system on steroids, compiling Logtalk specification to a plain
> non-modular Prolog program. Whether or not you need steroids is an
> entirely different discussion. Note that Quintus-like module system in
> SWI-Prolog is build on top of more low-level primitives. These can also
> be assembled in a different way to provide a different module system.

Steroids have usually a negative connotation... But such characterization is misleading considering that Logtalk provides *basic* features for an encapsulation mechanism that are not provided by modules. That includes (short list):

- separation between interface and implementation (protocols are first-class entities)
- same semantics for implicit and explicit qualified calls (resulting in clear and clean meta-predicate semantics)
- clear distinction between declaring a predicate and defining a predicate (clear closed world assumption semantics)
- simple scope rules for all flags (flag directives are always local to their scope)
- enforcement of encapsulation (which you may able to do without iff you have strong team discipline)

The point is, you don't need to be doing programming in the large to benefit. Logtalk may look like, as you wrote, a module system on steroids but only because module systems are in reality a prefixing mechanism stretching to double as a rather limited encapsulation mechanism.

Cheers,
Paulo

[Paulo Moura]

> On 11 Jun 2018, at 09:42, Proof Easel <janb...@easel.ch> wrote:
>
> ...

> Logtalk would do this work all again non-native. Modules
> would have technical names such as object, prototypes, etc..
> whatever and semantics relationships would be named
>
> specializes, etc..

Taking a page from the "Entities and roles" section on:

https://learnxinyminutes.com/docs/logtalk/

aren't you?

> But I guess doing it natively is much faster,
> and can result in a more seamless solution and much
> more dynamic, even more steroids solution...

If it was much faster, or indeed had any significant performance difference, that would mean that the Prolog system itself is broken as anything that trans-compiles to Prolog code (including Logtalk but also e.g. term-expansion mechanisms) is going to be compiled as, well, Prolog code. Is not like the Prolog compiler is looking into the original form of the source code and deciding "hum... the user is term-expanding this, let's do a shitty job then". In fact:

https://github.com/LogtalkDotOrg/logtalk3/wiki/Performance

A few days ago you were publicly citing this page on comp.lang.prolog.

Bye,
Paulo

[Kuniaki Mukai]

Thanks  Jan and Paulo,

Thanks for  clear explanation including historical notes. 

Now, I got the difference between predicate-based and name-based, which

is enough for using modules as a user of SWI-Prolog.

I don’t remember  well  where I was confused on the module of SWI-Prolog, but only 

vaguely which was like this:

… ,  m:p(X), ….

provided that p is transparent for the meta argument. I found that the variable X  is prefixed with the module m when 

passed to the module p/1,  which was  not what I expected at that time.  

Kuniaki Mukai

[Paulo Moura]

> On 11 Jun 2018, at 11:14, Kuniaki Mukai <kuniak...@gmail.com> wrote:
>
> Thanks Jan and Paulo,
>
> Thanks for clear explanation including historical notes.
> Now, I got the difference between predicate-based and name-based, which
> is enough for using modules as a user of SWI-Prolog.
>
> I don’t remember well where I was confused on the module of SWI-Prolog, but only
> vaguely which was like this:
>
> … , m:p(X), ….
>
> provided that p is transparent for the meta argument. I found that the variable X is prefixed with the module m when
> passed to the module p/1, which was not what I expected at that time.

This paper on meta-predicate semantics may be useful:

@incollection {pmoura12d,
author = "Paulo Moura",
title = "{Meta-predicate Semantics}",
booktitle = {Logic-Based Program Synthesis and Transformation},
series = {Lecture Notes in Computer Science},
editor = "Germán Vidal",
publisher = {Springer Berlin / Heidelberg},
isbn = {978-3-642-32210-5},
keyword = {Computer Science},
pages = {155-172},
volume = {7225},
url = {http://dx.doi.org/10.1007/978-3-642-32211-2_11},
doi = {10.1007/978-3-642-32211-2_11},
year = {2012}
}

Cheers,
Paulo

[Proof Easel]

Nope, Kuniaki Mukai wrote: "What is the module of SWI-Prolog ?"
He then hypothesized it has something to do with module lookup
versus predicate lookup.

In a flat namespace module lookup is trivial. You only have the
qualified call M:G. If the namespacce is hierarchical, you
can introduce the following directive:

     :- use_package(foo)

And then a call bar:G could mean foo/bar:G. If a module
foo/bar exists. module lookup and predicate lookup
can coexist, they are not mutually exclusive.

You can have both use_package/1 and use_module/1
as a directive in a Prolog system.

[Proof Easel]

Also meta-predicate driven module and/or predicate lookup and
and non-meta-predicate driven module and/or predicate lookup
is again orthogonal.

So there are like 4 dimensions:
- flat or hierarchical module name space
- module name lookup, for qualified calls
- predicate name lookup, for unqualified calls
  (and also for qualified calls in fact)
- meta-predicate driven or meta-predicate less

The problem is when you do module name lookup without
predicate name lookup, you get a poor qualified call M:G.
predicate name lookup is always involved.

What happens if you have a poor predicate name lookup
is seen in Logtalk. You need to wrap {}/1 to call  plain predicates.
This means the predicate name lookup is very poor.

This could be improved.

[Proof Easel]

I don't think the Prolog system is broken if it can do it
natively. You also don't say SWI-Prolog is broken,
because it implements between/3 natively.

Implementing a more richer module system natively,
up to providing an (:.)/2 operator natively, has a lot
of advantages, that are difficult or impossible

to realize in a transpiler. Thats why relinking WAMs
should be studied or somesuch. The problem gets
much more aggravated when you want an

interactive system. Where the module system has
included what has in AI become known as "truth
maintenance", so that a kind of "believe revision"

is possible at runtime. A transpiler cannot necessarily
do this. I don't say its impossible. You would need to
have a incremental transpiler, which has an additional

input. Here is the difference:

Non-Incremental Transpiler:

        Source --> Transpiler --> Results

Incremental Transpiler:

      Delta Source --+
                               +--> Transpiler --> Delta Result
       Old Result  ---+

Truth maintenance inside a native module system, during
consult and reconsult of modules, does all this for you directly
inside memory.

    A truth maintenance system, or TMS, is a knowledge representation   
    method for representing both beliefs and their dependencies and an
    algorithm called the "truth maintenance algorithm" that manipulates and
    maintains the dependencies. The name truth maintenance is due
    to the ability of these systems to restore consistency.
    https://en.wikipedia.org/wiki/Reason_maintenance

[Paulo Moura]

> On 11 Jun 2018, at 11:34, Proof Easel <janb...@easel.ch> wrote:
>
> What happens if you have a poor predicate name lookup
> is seen in Logtalk. You need to wrap {}/1 to call plain predicates.
> This means the predicate name lookup is very poor.

It means that the semantics of Logtalk code cannot be adversely affected by what happens to be loaded at the moment in the "user" module where plain predicates live unless that's precisely what you want. It's one of those things that contributes to writing robust, resilient code.

Here is a bold suggestion for you: instead of constantly misrepresenting Logtalk all over public places with lengthy posts full with misunderstandings, faulty assumptions, baseless conclusions, why don't you actually look into its documentation, sources, and use if for awhile? That's the best way to learn a system. Other people sharing their works publicly will also certainly appreciate that you do the same with their contributions.

-----------------------------------------------------------------
Paulo Moura
Logtalk developer

[Jan Wielemaker]

On 06/11/2018 11:37 AM, Paulo Moura wrote:
> Steroids have usually a negative connotation... But such

Steroids to me have no negative connotation :) I simply meant a system
that provides much more than the module systems as we know them. Whether
that is good or bad depends on what you need. If you need the stuff
Logtalk offers it is great. If you don't, it is not. It does come at a
price. It enlarges the footprint, it makes some of the development tools
work less transparently or not at all (in return you get some Logtalk
development tools), it makes loading code slower as it implements (I
think) a three-phase compiler (two phases compiling Logtalk, one loading
the result), whereas SWI-Prolog has a single-phase compiler. It requires
temporary files, which is typically no real problem but is on some
build and deployment scenarios. As we have seen, making Logtalk
compatible with SWISH is probably a difficult task.

> characterization is misleading considering that Logtalk provides*basic*
> features for an encapsulation mechanism that are not provided by
> modules. That includes (short list):
>
> - separation between interface and implementation (protocols are first-class
> entities)

That can indeed be quite useful. It can be done in plain (modularized)
Prolog, but it is a bit hacky and consistency (verifying the
implementation does indeed implement the interface) doesn't come
for free.

> - same semantics for implicit and explicit qualified calls (resulting in
> clear and clean meta-predicate semantics)

I think that also applies to ECLiPSe and, AFAIK, the ISO module
standard. I've played with the idea, but some testing indicated the
compatibility consequences are severe, while the added value is limited.
Without a legacy to worry about the ECLiPSe/Logtalk way is nicer.

> - clear distinction between declaring a predicate and defining a predicate
> (clear closed world assumption semantics)

Is basically multifile, no?

> - simple scope rules for all flags (flag directives are always local to
> their scope)

Flags (as in current_prolog_flag) are a nightmare. It is a single
mechanism that affects the compiler and runtime behavior. Flags that
affect the compiler should probably have lexical scoping. Some do in
SWI-Prolog but inconsistently either due to lazyness or compatibility
concerns. Flags that affect runtime behaviour are hard to handle and
having threads makes it even harder. If you have a golden solution I'm
glad to hear about it :) My general opinion is "don't touch them" with
as only exception to touch them at program startup to deal with
portability issues.

> - enforcement of encapsulation (which you may able to do without iff you
> have strong team discipline)

That is a choice. A good team should have coding guidelines. Not
breaking encapsulation should be one of these rules. IMO it is pretty
useful that these rules are not enforced by default during development
and to hack around reusing legacy code. It would not be very hard to
enforce them and given library(prolog_codewalk) it is just a few lines
to write a checker for cross-module calls to private predicates. This
can be fooled, although it is also quite easy to locate calls for which
the target cannot be deduced at compile time.

> The point is, you don't need to be doing programming in the large to
> benefit. Logtalk may look like, as you wrote, a module system on
> steroids but only because module systems are in reality a prefixing
> mechanism stretching to double as a rather limited encapsulation mechanism.

I don't think that is the main value of Logtalk vs module systems. It is
too easy to do that in a module system. The main value lies IMO in more
sophisticated mechanisms for code reuse such the notion of interfaces,
calling self/super methods, etc. All these things can be done in
modularized Prolog, but having it all available in a consistent, nicely
documented and well tested package is a clear win. I find myself
typically only wishing for tiny pieces of Logtalk for which I prefer
stretching Prolog a little over switching to Logtalk.

Cheers --- Jan

[Proof Easel]

For built-ins this is not ISO conform. At least the SICStus mini
interpreter suggest that, that an ISO module system, wouldn't
need a {}/1 braket for built-ins. You find the SICStus mini

interpreter here:

call_goal(asserta(X), _) :- !,
        asserta(X).
call_goal(asserta(X,R), _) :- !,
        asserta(X, R).
% and so on for all built-in predicates
call_goal(Goal, M) :-
        (M:Goal :- Body),
        icall(Body, M).
https://sicstus.sics.se/sicstus/docs/latest4/html/sicstus.html/ref_002dmod_002dsem.html#ref_002dmod_002dsem

Otherwise for user predicates you can anyway not do
the change, since you do not have truth maintenance.
So this is a rather hypothetical question for your system.

[Proof Easel]

I didn't misrepresent Logtalk. You only confirmed what I said.
I said this already yesterday on comp.lang.prolog:

https://groups.google.com/d/msg/comp.lang.prolog/eO8RrEfanmg/5vpCCtgHAwAJ

[Proof Easel]

The performance overhead is too large.
At some place its documented as:

Logtalk says, extra logical inferences:

    Static binding: 0
    Dynamic binding (object bound at compile time): 1
    Dynamic binding (object bound at runtime time): 2
    https://github.com/LogtalkDotOrg/logtalk3/wiki/Performance

With a native based system, you can bring it further down.
So Logtalk is not a practical system for real big taxonomies
and a lot of dynamic binding at runtime.

But if you have only object bound at compile time or
static binding, you don't need such a system.

[Proof Easel]

The reason is that the native (:)/2 can already work
in dynamic mode. So basically you should get for (::)/2
the same timing as for (:)/2 in a rich module system.

Especially in SWI-Prolog this could be archived,
since arguments, I guess, are pushed on a stack.
Right? So an operator (::)/2 is a piece of cake, could

be done much more efficient than for example in my system.

[Kuniaki Mukai]

Hi,

I have tested myself whether I can expect correctly results of five queries (1-5) below on using module prefixes.

For queries 2-5, results are exactly what I expect.

However output query 1 should be “hello” but in fact is “world” because of the transparency rule of module prefixes.
Hmm... there is still a gap in my understanding of the module of SWI-Prolog.

:- module(zdd,[]).
:- meta_predicate user:a(0).

user:a(X):- user:call(X).
user:b:- write(hello).

:- meta_predicate a(0).
a(X):- user:a(X).

b :- write(world).


?- module(zdd).
true.

(1) ?- a(b).
world
true.

(2) ?- user:a(b).
hello
true.

(3) ?- zdd:a(b).
world
true.

(4) ?- zdd:a(user:b).
hello
true.

(5) ?- a(zdd:b).
world
true.

Kuniaki Mukai

[Paulo Moura]

Hi Jan,

Let me start by saying that I'm fully aware (as other long time Prolog programmers are) that several of the module system issues we are discussing are inherited and fixing them would break havoc backwards compatibility.

> On 11 Jun 2018, at 13:05, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> On 06/11/2018 11:37 AM, Paulo Moura wrote:
>> Steroids have usually a negative connotation... But such
>
> Steroids to me have no negative connotation :) I simply meant a system
> that provides much more than the module systems as we know them. Whether
> that is good or bad depends on what you need. If you need the stuff
> Logtalk offers it is great. If you don't, it is not. It does come at a
> price. It enlarges the footprint,

Is the footprint (of loading Logtalk) that significant nowadays? Using the included script for embedding Logtalk we get:

$ ll
total 632
-rw-r--r-- 1 pmoura staff 322284 Jun 11 13:56 logtalk.qlf

This QLF files includes both the compiler and runtime.

> it makes some of the development tools
> work less transparently or not at all (in return you get some Logtalk
> development tools),

Some? :-) Logtalk provides a full set of developer tools. A good summary with usage examples is available at:

https://logtalk.pt/tools/

Which tool you find missing? But you can also use the SWI-Prolog graphical tracer and graphical profiler, thanks for the hooks that you have in SWI-Prolog.

> it makes loading code slower as it implements (I
> think) a three-phase compiler (two phases compiling Logtalk, one loading
> the result), whereas SWI-Prolog has a single-phase compiler.

14:04 $ ll large.*
-rw-r--r--@ 1 pmoura staff 1072336 Mar 3 2013 large.lgt
-rw-r--r--@ 1 pmoura staff 1072324 Nov 7 2013 large.pl

The Logtalk file simply encapsulates the contents of the large.pl file inside an object. The contents itself is one of the big Unicode tables.

?- time(true).
% 3 inferences, 0.000 CPU in 0.000 seconds (72% CPU, 300000 Lips)
true.

?- time(load_files('large.pl')).
% 827,212 inferences, 0.137 CPU in 0.144 seconds (95% CPU, 6045899 Lips)
true.

?- time(logtalk_load('large.lgt')).
% [ /Users/pmoura/Desktop/large.lgt loaded ]
% (0 warnings)
% 1,596,437 inferences, 0.360 CPU in 0.434 seconds (83% CPU, 4434658 Lips)
true.

So, roughly twice the number of inferences and 2.5x the compilation time. This may have some practical significance for large data files (this example can be considered a data file, albeit a small one). But giving Logtalk's make tool and how easy is to precompile Logtalk code into QLF files, that's only an issue the first time you compile a large file. Then again, why encapsulate large data in the first place?

> It requires
> temporary files, which is typically no real problem but is on some
> build and deployment scenarios.

Logtalk default scratch directory can be set to /tmp. Which operating-systems are people using that don't provide that or an equivalent? For deployment, the usual solution is to precompile and embed Logtalk and the Logtalk applications, not to be compiling application code in deployment scenarios (an observation which is also valid for Prolog itself).

> As we have seen, making Logtalk
> compatible with SWISH is probably a difficult task.

There's for quite some time a experimental Logtalk pack that works with SWISH. Douglas Miles as playing with it some time ago. Myself, have not looked into it at detail. Happy to revisit it.

>> characterization is misleading considering that Logtalk provides*basic*
>> features for an encapsulation mechanism that are not provided by
>> modules. That includes (short list):
>>
>> - separation between interface and implementation (protocols are first-class
>> entities)
>
> That can indeed be quite useful. It can be done in plain (modularized)
> Prolog, but it is a bit hacky and consistency (verifying the
> implementation does indeed implement the interface) doesn't come
> for free.

Is not that is just hacky with modules. The current and recommended practice with Prolog modules is that *exported* predicates should not clash. I.e. modules, as encapsulation units, in practice are there only to prevent clashes in *private* predicates. You likely remember the discussion over which module had the right to export a transpose/2 predicate some years ago? Or consider module libraries such as "avl" or "rbtrees" and other implementing the same data structure but with completely different interfaces. Contrast that with Logtalk where you have a dictionary protocol and several objects implementing it, which makes switching between alternative implementations trivial. Being able to define a protocol/interface and provide multiple implementations is one of the most basic features for an encapsulation mechanisms. Its absence in module systems twists practice and leads to non scalable, brittle solutions.

>> - same semantics for implicit and explicit qualified calls (resulting in
>> clear and clean meta-predicate semantics)
>
> I think that also applies to ECLiPSe and, AFAIK, the ISO module
> standard. I've played with the idea, but some testing indicated the
> compatibility consequences are severe, while the added value is limited.
> Without a legacy to worry about the ECLiPSe/Logtalk way is nicer.

Thanks. That's why I started this mail with a sentence about backwards compatibility.

>> - clear distinction between declaring a predicate and defining a predicate
>> (clear closed world assumption semantics)
>
> Is basically multifile, no?

Nothing to do with multifile (or dynamic) predicates. For a trivial example, consider the following file:

----- foo.pl -----
:- module(foo, [bar/1]).
------------------

?- [foo].
ERROR: Exported procedure foo:bar/1 is not defined
true.

The equivalent in Logtalk would be:

----- foo.lgt -----
:- object(foo).
:- public(bar/1).
:- end_object.
------------------

?- {foo}.
% [ /Users/pmoura/Desktop/foo1234.lgt loaded ]
% (0 warnings)
true.

?- foo::bar(_).
false.

I.e. Messages for declared but undefined predicates fail. Messages for unknown (not declared) predicates throw an error. Note that you can not have protocols/interfaces without being able to *declare* a predicate without necessarily *defining* it.

>> - simple scope rules for all flags (flag directives are always local to
>> their scope)
>
> Flags (as in current_prolog_flag) are a nightmare.

A mostly *inherited* nightmare :(

> It is a single
> mechanism that affects the compiler and runtime behavior. Flags that
> affect the compiler should probably have lexical scoping. Some do in
> SWI-Prolog but inconsistently either due to lazyness or compatibility
> concerns. Flags that affect runtime behaviour are hard to handle and
> having threads makes it even harder. If you have a golden solution I'm
> glad to hear about it :) My general opinion is "don't touch them" with
> as only exception to touch them at program startup to deal with
> portability issues.

The solution I use for Logtalk flags could certainly be used for Prolog flags by it would break backwards compatibility :(

>> - enforcement of encapsulation (which you may able to do without iff you
>> have strong team discipline)
>
> That is a choice. A good team should have coding guidelines. Not
> breaking encapsulation should be one of these rules. IMO it is pretty
> useful that these rules are not enforced by default during development
> and to hack around reusing legacy code. It would not be very hard to
> enforce them and given library(prolog_codewalk) it is just a few lines
> to write a checker for cross-module calls to private predicates. This
> can be fooled, although it is also quite easy to locate calls for which
> the target cannot be deduced at compile time.
>
>> The point is, you don't need to be doing programming in the large to
>> benefit. Logtalk may look like, as you wrote, a module system on
>> steroids but only because module systems are in reality a prefixing
>> mechanism stretching to double as a rather limited encapsulation mechanism.
>
> I don't think that is the main value of Logtalk vs module systems. It is
> too easy to do that in a module system.

Above you have several examples that contradict that. As I wrote, this was just a short list. I could easy include better term-expansion, message printing, question asking mechanisms. Or parametric entities. Or optionals and expecteds libraries (that take full advantage of parametric objects), ...

> The main value lies IMO in more
> sophisticated mechanisms for code reuse such the notion of interfaces,
> calling self/super methods, etc. All these things can be done in
> modularized Prolog, but having it all available in a consistent, nicely
> documented and well tested package is a clear win.

Thanks.

> I find myself
> typically only wishing for tiny pieces of Logtalk for which I prefer
> stretching Prolog a little over switching to Logtalk.

Logtalk was *designed* to extend Prolog, not to replace it. It's not a *switch to*, it's a *use with*. Logtalk does provide an alternative to Prolog modules but it happily co-exists with them. Is not a "either" choice. In fact, modules can send messages to objects with zero overhead (in SWI-Prolog and YAP) Objects can call module predicates also have zero overhead.

Cheers,
Paulo

[Paulo Moura]

> On 11 Jun 2018, at 13:08, Proof Easel <janb...@easel.ch> wrote:
>
> For built-ins this is not ISO conform. At least the SICStus mini
> interpreter suggest that, that an ISO module system, wouldn't
> need a {}/1 braket for built-ins. You find the SICStus mini

For builtin-predicates, you don't need to use {}/1 (or, in alternative, a :- uses(user, [...]). directive). Stop confusing your untested, baseless assumptions with what actually Logtalk does! It would take you a fraction of the time you spent writing length speculative posts to actually try the things that you are criticizing and cut down the noise!

> --
> You received this message because you are subscribed to the Google Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to swi-prolog+...@googlegroups.com.
> Visit this group at https://groups.google.com/group/swi-prolog.
> For more options, visit https://groups.google.com/d/optout.

[Paulo Moura]

[my last reply to you was I'm tired of your trolling]

> On 11 Jun 2018, at 13:21, Proof Easel <janb...@easel.ch> wrote:
>
> The performance overhead is too large.

An overhead of *two* extra logical inferences in the *worst case* "is too large"?

Are you aware that even a relatively slow Prolog system such as SWI-Prolog can do hundreds of thousands of logical inferences per second in consumer hardware? That fast Prolog systems can do millions of logical inferences per second in the same hardware?

Are you aware that applications usually do something useful when a predicate is called instead of just purely call other predicate across object/module boundaries?

> At some place its documented as:
>
> Logtalk says, extra logical inferences:
>
> Static binding: 0
> Dynamic binding (object bound at compile time): 1
> Dynamic binding (object bound at runtime time): 2
> https://github.com/LogtalkDotOrg/logtalk3/wiki/Performance
>
> With a native based system, you can bring it further down.

I guess you can always bring 2 down to 1 and 1 down 0. Progress!

> So Logtalk is not a practical system for real big taxonomies

You mean like this:

http://logtalk.org/diagrams/lgtstep.pdf

But by all means, please use something else for real big taxonomies. Please.

Bye,
Paulo

[Paulo Moura]

Hi,

If you do a trace for the a/1 call from inside the module you get:

zdd: [trace] ?- a(b).
^ Call: (8) zdd:a(b) ? creep
^ Call: (9) a(zdd:b) ? creep
Call: (10) zdd:b ? creep
Call: (11) write(world) ? creep
world
Exit: (11) write(world) ? creep
Exit: (10) zdd:b ? creep
^ Exit: (9) a(zdd:b) ? creep
^ Exit: (8) zdd:a(zdd:b) ? creep
true.

Note that the argument of the a/1 predicate call is qualified by the module: a(zdd:b). In as nutshell, this is how module prefixing works: it applies not only to predicates but also to the meta-arguments as per the meta_predicate directives. As you have, inside the zdd module:

:- meta_predicate a(0).
a(X):- user:a(X).

the clause is compiled as (equivalent to):

zdd:a(X) :- user:a(zdd:X).

I.e. X is called in the context of zdd, not in user. That explains why you get "world" printed instead of "hello".

Cheers,
Paulo

[Kuniaki Mukai]

Thanks Paulo,
I got it.


Now I wonder if the following three rules are all enough on using module prefixes for not confusing.

R1 m:n: A => n: A (cancellation right module prefix)
R2 m:p(X) => m:p(m:X) (propagate to meta argument)
R3 p(X) => m:p(X) (the context module prefix )

for example (1) ?- a(b).

a(b) => zdd:a(b) => zdd: a(zdd:b)

the rest works as you wrote.

I hope the three rules safely useful for the usual SWI-Prolog user like me.

Thanks for removing my last of confusions (I hope)

Kuniaki Mukai

[Proof Easel]

You call others trolling, when you don't have any arguments
anymore? Thats ad hominem. But the problem is rather
that Logtalk has suffienciently blurred the distinction

between module and class, respectively a module that
has methods. So that it thinks its hierarchy here in lgstep.pdf
is some class hierarchy that shows virtual methods?

Also your example:

14:04 $ ll large.*
-rw-r--r--@ 1 pmoura  staff  1072336 Mar  3  2013 large.lgt
-rw-r--r--@ 1 pmoura  staff  1072324 Nov  7  2013 large.pl

Is cheating. I tried once to find in Logtalk a single virtual
method. I didn't find one. Its all modules, not on steriod,
rather on valium. Even the date library doesn't use

virtual methods, you couldn't plug in a mayan calendar.
I call Logtalk rather a fraud. It doesn't perform for virtual
methods and has never had a use case

with a large taxonomy with a lot of virtual methods.

[Jan Wielemaker]

On 06/11/2018 04:28 PM, Paulo Moura wrote:
> An overhead of*two* extra logical inferences in the *worst case* "is

> too large"?
>
> Are you aware that even a relatively slow Prolog system such as
> SWI-Prolog can do hundreds of thousands of logical inferences per
> second in consumer hardware? That fast Prolog systems can do millions
> of logical inferences per second in the same hardware?

SWI-Prolog generally isn't that slow anymore :) Yes, there are some
extreme cases where other systems are possibly up to 10 times faster,
but there are also cases where the reverse is true. The overall effect
on large applications varies and in many cases the is margional.

> Are you aware that applications usually do something useful when a
> predicate is called instead of just purely call other predicate
> across object/module boundaries?

Yip. A little extra cost between object/module boundaries typically has
little impact on the overall performance. Again, of course there are
exceptions ...

Cheers --- Jan

[Proof Easel]

It could be that lgstep contains some virtual methods. But
something like an abstract module does not necessariely
mean a virtual method. You would need to show

use some call-site metric, that also proves polymorphic
calls information. And not simple code sharing through module
inheritance which can be already done in a standard ISO

module system through the following directive:

   :- module(foo, []).
   :- reexport(bar).

This allows already a semantic relationship between foo
and bar. And you would see it in a taxonomy diagram. But
this says nothing about some _;_ or _::_ call-site.

You are halucinating and confused. You think what needs
to be tested, dynamic calls, can be seen in a taxonomy
diagram. But actually in Prolog and any Logtalk based

language, where you don't have types, you cannot so
easily extract some information about dynamic calls. You
don't have so many types that tells you what M in

M:_ or T in T::_ is at runtime. Maybe a port statistics of
executing the code would give evidence that there are
polymorphic calls. But taxonomy diagram itself is useless.

You need to know what happens in the methods. And you
need to know what happens in the methods at runtime,
not only at compile time. You need to give statistics

about the last case, Nr.3, here:

        Static binding: 0
        Dynamic binding (object bound at compile time): 1
>>    Dynamic binding (object bound at runtime time): 2      <<
        https://github.com/LogtalkDotOrg/logtalk3/wiki/Performance

You don't see that last case, Nr.3, in a taxonomy diagram.

[Paulo Moura]

Hi Jan,

> On 11 Jun 2018, at 16:25, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> On 06/11/2018 04:28 PM, Paulo Moura wrote:
>> An overhead of*two* extra logical inferences in the *worst case* "is
>> too large"?
>>
>> Are you aware that even a relatively slow Prolog system such as
>> SWI-Prolog can do hundreds of thousands of logical inferences per
>> second in consumer hardware? That fast Prolog systems can do millions
>> of logical inferences per second in the same hardware?
>
> SWI-Prolog generally isn't that slow anymore :) Yes, there are some
> extreme cases where other systems are possibly up to 10 times faster,
> but there are also cases where the reverse is true. The overall effect
> on large applications varies and in many cases the is margional.

I would add that SWI-Prolog provides features other than raw performance that are often as important if not more critical when deploying applications. Not to mention its stability.

>> Are you aware that applications usually do something useful when a
>> predicate is called instead of just purely call other predicate
>> across object/module boundaries?
>
> Yip. A little extra cost between object/module boundaries typically has
> little impact on the overall performance. Again, of course there are
> exceptions ...

Exactly.

Cheers,
Paulo

[Paulo Moura]

Hi,

> On 11 Jun 2018, at 16:19, Kuniaki Mukai <kuniak...@gmail.com> wrote:
>
>
> Thanks Paulo,
> I got it.
>
>
> Now I wonder if the following three rules are all enough on using module prefixes for not confusing.
>
> R1 m:n: A => n: A (cancellation right module prefix)

As a side note, SWI-Prolog does a good job here. Some other Prolog systems have cases where the prefixes can accumulate.

> R2 m:p(X) => m:p(m:X) (propagate to meta argument)

Given meta_predicate/1 directives, yes.

> R3 p(X) => m:p(X) (the context module prefix )

Assuming that p/1 is not a meta-predicate, which you covered in R2.

The rules look good to me :-) The prefixing also apply to controls constructs. For example:

m:(a(X), b(X)) => m:a(X), m:b(X).

> for example (1) ?- a(b).
>
> a(b) => zdd:a(b) => zdd: a(zdd:b)
>
> the rest works as you wrote.
>
> I hope the three rules safely useful for the usual SWI-Prolog user like me.
>
> Thanks for removing my last of confusions (I hope)

You're most welcome.

Cheers,
Paulo

[Proof Easel]

OO-Prologs have a long tradition. Chris Moss once handed
me a copy of his book Prolog++. I still have it, I got
it in August 1994, when there was even no Logtalk yet.

   Chris Moss, Prolog++: The Power of Object-Oriented and Logic Programming
   https://en.wikipedia.org/wiki/Prolog%2B%2B

Of course the goal of any OO system is to make virtual
calls effortless. Otherwise you shouldn't even touch making
an OO system. This has to that without an effortless virtual

call, many OO typical programming patterns cannot be used.
For example visitor pattern etc.. Such patterns are often
solved Prolog-ish in different ways. But if you solve it

in another way, and if you don't use virtual calls, why
use Logtalk at all. Doesn't make any sense at all. You
can use modules and reexport or use_module to

share code. You also have in the ISO core standard
the following feature:

     ... have to look it up, either in core or module,
     a predicate can be in a state that it is half defined,
     i.e. it exists but has no clauses, and throws an
     error if invoked ...

Looking at the many modifiers that Java has I
noticed that Logtalk already implements public/1,
private/1 and protected/1.

I do not yet have protected/1. But I got the idea to
implement abstract/1. One could do what some of
your prototypes or who knows what. One could

for example make a pure abstract module as follows:

    :- module(arith, []).
    :- abstract add/3.
    :- abstract sub/3.
    :- abstract neg/2.
    ...

The effect would be that such a half backed predicate
is created, I have already an internal predicate for that,
that can create such predicates:
      

     sys_neutral_predicate(I):

     If no predicate has yet been defined for the predicate indicator I, defines a corresponding neutral predicate.

    http://www.jekejeke.ch/idatab/doclet/prod/en/docs/05_run/10_docu/02_reference/07_theories/01_kernel/03_body.html

What I do not yet have is an "abstract" attribute of a predicate.
But taling into account this "abstract" attribute, I could adapt
the current style check for modules,

that looks for unimplemented predicates. And maybe I could
also adapt the predicate lookup. This would satisfy some
of the Logtalk software engineering appeal.

     In the end I would have both performance and
     software engineering appeal.

Even this "abstract" attribute would come very handy for
my Java class auto loader. I could also assign the attribute
to auto loaded classes.

Maybe I should do the "abstract" attribute first, and the
"protected" modifier attribute later. Ha Ha. Can be used for
a lot more, already found in the Logtalk univers by

other names. But for example in Java interface classes,
are just classes where all the members are by default
"abstract" and "public". So such things could be also done.

I have actually some use cases for this scenario:
- It serves as a documentation tool, other than
  no predicate declared, its abstractly declared.
- It helps in making the code robust, other error
  than simply predicate undefined
- It helps in debugging the code, other error than
  simply predicate undefined.

Although its not really a contribution to speed. Its a contribution
to make the OO system more complete, and map some
OO patterns, OO practice to the OO system.

The next step would be to allow a modifier for the
module declaration, for example, as such:

:- abstract module(arith, []).

Then term objects and/or proxy objects from the module
arith would not anymore be allowed. Etc.. Etc..

But please note: This would again be based on the native
module system. So that the end-user doesn't have to learn
anything new, execpt the modifiers.

[Proof Easel]

BTW: The Prolog++ book is still an execellent book.
For example in section 3.6 it shows one way how
methods could be rewritten to Prolog with cur.

Its not the same approach that I now have with
(::)/2 which simply delegates to (:)/2, but it could
be for example interesting for Kuniaki Mukai.

In general the book discusses abstract datatypes
and dynamic binding. But Logtalk even doesn't
provide a single abstract datatype example

in its documentation. You can try yourself:

https://www.google.ch/search?q=Logtalk+abstract+datatype

Not a single hit. Whats wrong?

[Proof Easel]

But I take home the message from Logtalk. That there are more
critical elements than performance. And I guess Logtalk has
adopted this doctrine as well, and is a testbed for

other things than a performant OO-system. I hope the end-users
also honor this decision. SWI-Prolog already shows some bit rot,
last time I measured, I could not reach my usual 3100 ms,

the benchmark was only around 3500 ms. ECLiPSe Prolog
new release 7.0 same problem, some benchmarks show
degradation. Maybe I should do some Logtalk measurement,

especially late binding, and monitor against my system. Ha Ha.
Somehow I think there is still a little headroom to improve my
(::)/1 operator, got a few new ideas recently. But mostlikely I

already beat Logtalk in many scenario... Ha Ha. OO-system on
valium and not on stereoids. But if you say performance is
not critical, then I guess there are also no lessons learnt

for OO-system implementors from Logtalk?

Am Montag, 11. Juni 2018 17:56:34 UTC+2 schrieb Paulo Moura:

Hi Jan,
 

I would add that SWI-Prolog provides features other than raw performance

[Proof Easel]

Problem is, stability there is not much choice. If you provide
a multi-threaded Prolog system, and its not stable, you
should trash it.

Or if you provide pengines that are not stable, or GC that
us not stable, you should either trash it, or see it to it
that it gets fixed.

But Logtalk relies on existing Prolog systems, it can anyway
not influence stability. Could it be that Logtalk experienced that
some systems, possibly other than SWI-Prolog, are not

that stable, when it tried to provide adapters? What is
this stabiity issue, I don't understand and wonder? Please
tell us? Ha Ha

[Proof Easel]

Anyway, if you don't believe me that there was bit rot, I made picture:

SWI-Prolog performance (significant 200ms slower, 12 Jun 2018)?

    https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2616883

Compare with this timing on 11 Sep 2016:

     https://github.com/jburse/jekejeke-samples/blob/master/jekrun/benchmark/galery/ideapad_swi.gif

Cost of stability? New windows 10 version 1803 induced?

[Jan Wielemaker]

On 11/06/18 15:49, Paulo Moura wrote:
> Hi Jan,
>
> Let me start by saying that I'm fully aware (as other long time Prolog programmers are) that several of the module system issues we are discussing are inherited and fixing them would break havoc backwards compatibility.
>
>> On 11 Jun 2018, at 13:05, Jan Wielemaker <j...@swi-prolog.org> wrote:
>>
>> On 06/11/2018 11:37 AM, Paulo Moura wrote:
>>> Steroids have usually a negative connotation... But such
>>
>> Steroids to me have no negative connotation :) I simply meant a system
>> that provides much more than the module systems as we know them. Whether
>> that is good or bad depends on what you need. If you need the stuff
>> Logtalk offers it is great. If you don't, it is not. It does come at a
>> price. It enlarges the footprint,
>
> Is the footprint (of loading Logtalk) that significant nowadays? Using the included script for embedding Logtalk we get:
>
> $ ll
> total 632
> -rw-r--r-- 1 pmoura staff 322284 Jun 11 13:56 logtalk.qlf
>
> This QLF files includes both the compiler and runtime.

Thanks that current resources are zip :) I must admit that the memory
usage impact is small. Startup of a simple state goes from 12ms to about
20ms. That is significant, but relevant to a small minority of
applications such as using Prolog/Logtalk as traditional CGI script.

>> it makes some of the development tools
>> work less transparently or not at all (in return you get some Logtalk
>> development tools),
>
> Some? :-) Logtalk provides a full set of developer tools. A good summary with usage examples is available at:
>
> https://logtalk.pt/tools/
>
> Which tool you find missing? But you can also use the SWI-Prolog graphical tracer and graphical profiler, thanks for the hooks that you have in SWI-Prolog.

Yes, but the Edit menu in the graphical profiler doesn't work, nor
edit/1 at the prompt. I guess that is fixable using more hooks and
implementation thereof in Logtalk. Similar issues exist with the low
level tools such as jiti_list/1, vm_list/1, etc. to understand the
program at all its levels. I did some code analysis at Kyndi for Logtalk
code and these are the small things you stumble on. Of course I'm an
outlier as I know all the SWI-Prolog tools very well and not the Logtalk
replacements.

>> it makes loading code slower as it implements (I
>> think) a three-phase compiler (two phases compiling Logtalk, one loading
>> the result), whereas SWI-Prolog has a single-phase compiler.
>
> 14:04 $ ll large.*
> -rw-r--r--@ 1 pmoura staff 1072336 Mar 3 2013 large.lgt
> -rw-r--r--@ 1 pmoura staff 1072324 Nov 7 2013 large.pl
>
> The Logtalk file simply encapsulates the contents of the large.pl file inside an object. The contents itself is one of the big Unicode tables.
>
> ?- time(true).
> % 3 inferences, 0.000 CPU in 0.000 seconds (72% CPU, 300000 Lips)
> true.
>
> ?- time(load_files('large.pl')).
> % 827,212 inferences, 0.137 CPU in 0.144 seconds (95% CPU, 6045899 Lips)
> true.
>
> ?- time(logtalk_load('large.lgt')).
> % [ /Users/pmoura/Desktop/large.lgt loaded ]
> % (0 warnings)
> % 1,596,437 inferences, 0.360 CPU in 0.434 seconds (83% CPU, 4434658 Lips)
> true.
>
> So, roughly twice the number of inferences and 2.5x the compilation time. This may have some practical significance for large data files (this example can be considered a data file, albeit a small one). But giving Logtalk's make tool and how easy is to precompile Logtalk code into QLF files, that's only an issue the first time you compile a large file. Then again, why encapsulate large data in the first place?

The time difference is about what I expected. Why you use Logtalk for
data? I suspect many Logtalkers will do so. Note that compiler speed
does matter. At least I have a couple of users that frequently complain
about long compile times. There are people with really large programs
and not only for big data files ...

>> It requires
>> temporary files, which is typically no real problem but is on some
>> build and deployment scenarios.
>
> Logtalk default scratch directory can be set to /tmp. Which operating-systems are people using that don't provide that or an equivalent? For deployment, the usual solution is to precompile and embed Logtalk and the Logtalk applications, not to be compiling application code in deployment scenarios (an observation which is also valid for Prolog itself).

I recently understood that the nodes of some of the big tech companies
run on a completely read-only filesystem. Now as saved states seem to
work with recent Logtalk versions this is possible. Some people are a
bit scared about using saved states that indeed come with some problems
wrt. initialization and access to resources such as foreign libraries.

>> As we have seen, making Logtalk
>> compatible with SWISH is probably a difficult task.
>
> There's for quite some time a experimental Logtalk pack that works with SWISH. Douglas Miles as playing with it some time ago. Myself, have not looked into it at detail. Happy to revisit it.

Good to know. I thought the problem of compiling Logtalk objects to
(temporary) modules was not solved. Curious how Douglas managed to
hack around that. He's surely very creative :)

>>> characterization is misleading considering that Logtalk provides*basic*
>>> features for an encapsulation mechanism that are not provided by
>>> modules. That includes (short list):
>>>
>>> - separation between interface and implementation (protocols are first-class
>>> entities)
>>
>> That can indeed be quite useful. It can be done in plain (modularized)
>> Prolog, but it is a bit hacky and consistency (verifying the
>> implementation does indeed implement the interface) doesn't come
>> for free.
>
> Is not that is just hacky with modules. The current and recommended practice with Prolog modules is that *exported* predicates should not clash. I.e. modules, as encapsulation units, in practice are there only to prevent clashes in *private* predicates. You likely remember the discussion over which module had the right to export a transpose/2 predicate some years ago? Or consider module libraries such as "avl" or "rbtrees" and other implementing the same data structure but with completely different interfaces. Contrast that with Logtalk where you have a dictionary protocol and several objects implementing it, which makes switching between alternative implementations trivial. Being able to define a protocol/interface and provide multiple implementations is one of the most basic features for an encapsulation mechanisms. Its absence in module systems twists practice and leads to non scalable, brittle solutions.

It would probably be good practice to implement the same API for modules
that implement the same abstract data type using different technology.
That also works in Prolog. It seems pretty unlikely that someone wishes
to load rbtrees and assoc into the same module and in the case they wish
to do so, using :- use_module(library(assoc),[]) and assoc:empty_map(Map)
or something like that would work. All Logtalk offers for this is the
notion of a protocol. That is easily defined in Prolog. Just introduce
a syntax to define a protocol such as

:- protocol(map,
[ empty_map/1,
...
]).

And a term expansion that expands :- implements_protocol(map). into a
module header and you are done.

>>> - clear distinction between declaring a predicate and defining a predicate
>>> (clear closed world assumption semantics)
>>
>> Is basically multifile, no?
>
> Nothing to do with multifile (or dynamic) predicates. For a trivial example, consider the following file:
>
> ----- foo.pl -----
> :- module(foo, [bar/1]).
> ------------------
>
> ?- [foo].
> ERROR: Exported procedure foo:bar/1 is not defined
> true.
>
> The equivalent in Logtalk would be:
>
> ----- foo.lgt -----
> :- object(foo).
> :- public(bar/1).
> :- end_object.
> ------------------
>
> ?- {foo}.
> % [ /Users/pmoura/Desktop/foo1234.lgt loaded ]
> % (0 warnings)
> true.
>
> ?- foo::bar(_).
> false.
>
> I.e. Messages for declared but undefined predicates fail. Messages for unknown (not declared) predicates throw an error. Note that you can not have protocols/interfaces without being able to *declare* a predicate without necessarily *defining* it.

That is still what multifile/1 does. The only difference is that it
_allows_ to provide the implementation in multiple files. AFAIK the ISO
proposal also separates the interface from the implementation. This
suggests the idea lives, but apparently nobody bothered implementing
it in a module system while this is of course trivial.

>>> - simple scope rules for all flags (flag directives are always local to
>>> their scope)
>>
>> Flags (as in current_prolog_flag) are a nightmare.
>
> A mostly *inherited* nightmare :(
>
>> It is a single
>> mechanism that affects the compiler and runtime behavior. Flags that
>> affect the compiler should probably have lexical scoping. Some do in
>> SWI-Prolog but inconsistently either due to lazyness or compatibility
>> concerns. Flags that affect runtime behaviour are hard to handle and
>> having threads makes it even harder. If you have a golden solution I'm
>> glad to hear about it :) My general opinion is "don't touch them" with
>> as only exception to touch them at program startup to deal with
>> portability issues.
>
> The solution I use for Logtalk flags could certainly be used for Prolog flags by it would break backwards compatibility :(

Well, lexical scoping for flags that affect the compiler is easy. Scoped
switching between the different occurs_check values is a different matter.
Are you saying you can do that?

>>> - enforcement of encapsulation (which you may able to do without iff you
>>> have strong team discipline)
>>
>> That is a choice. A good team should have coding guidelines. Not
>> breaking encapsulation should be one of these rules. IMO it is pretty
>> useful that these rules are not enforced by default during development
>> and to hack around reusing legacy code. It would not be very hard to
>> enforce them and given library(prolog_codewalk) it is just a few lines
>> to write a checker for cross-module calls to private predicates. This
>> can be fooled, although it is also quite easy to locate calls for which
>> the target cannot be deduced at compile time.
>>
>>> The point is, you don't need to be doing programming in the large to
>>> benefit. Logtalk may look like, as you wrote, a module system on
>>> steroids but only because module systems are in reality a prefixing
>>> mechanism stretching to double as a rather limited encapsulation mechanism.
>>
>> I don't think that is the main value of Logtalk vs module systems. It is
>> too easy to do that in a module system.
>
> Above you have several examples that contradict that. As I wrote, this was just a short list. I could easy include better term-expansion, message printing, question asking mechanisms. Or parametric entities. Or optionals and expecteds libraries (that take full advantage of parametric objects), ...

I think I argued that a lot of this stuff is easily done in the context
of modules. You may of course ask why it hasn't been done. Well, quite a
bit has been done by some Prolog system, e.g., term-expansion in Ciao
and ECLiPSe, message printing in Quintus and SWI-Prolog, question asking
in SWI-Prolog. I'm not aware of parametric entities as a standard Prolog
facility. I've seen a talk at ICLP quite a few years proposing this for
modules. Surely there are applications doing this type of stuff as it is
quite natural in Prolog.

One thing that is hard is to consider a module as a set of methods and
perform clean inheritance on that.

>> The main value lies IMO in more
>> sophisticated mechanisms for code reuse such the notion of interfaces,
>> calling self/super methods, etc. All these things can be done in
>> modularized Prolog, but having it all available in a consistent, nicely
>> documented and well tested package is a clear win.
>
> Thanks.
>
>> I find myself
>> typically only wishing for tiny pieces of Logtalk for which I prefer
>> stretching Prolog a little over switching to Logtalk.
>
> Logtalk was *designed* to extend Prolog, not to replace it. It's not a *switch to*, it's a *use with*. Logtalk does provide an alternative to Prolog modules but it happily co-exists with them. Is not a "either" choice. In fact, modules can send messages to objects with zero overhead (in SWI-Prolog and YAP) Objects can call module predicates also have zero overhead.

Is that new? I always liked the idea that you could do

:- use_module(library(logtalk)).

:- object(...)
...

etc. I've always understood there was a Logtalk executable as swi-lgt.
Since years I have a little load library to load Logtalk dynamically
which is notably practical if I need to run SWI/Logtalk under gdb
debugger and thus need the raw executable rather than a shell script.

I know it is non-negotiable due to your portability requirements, but I
also would like to see most of Logtalk a shallow and possibly modular
wrapper around Prolog's modules. That would make the system more
transparent and invites using Logtalk for things it is good at rather
than an alternative module syntax. I still claim that modules solve most
of the problems needed to deploy Prolog for large programs. I do agree
it neglects some issues and poor standardization doesn't do much good
either. Still, module incompatibility is a minor problem for porting
Prolog in my experience. (Lack of) e.g., tabling, attributed variables,
mutable objects, threads, foreign interface, indexing, etc. are way
harder to deal with.

Cheers --- Jan

[Paulo Moura]

Hi Jan,

> On 12 Jun 2018, at 10:05, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> On 11/06/18 15:49, Paulo Moura wrote:
>> Hi Jan,
>>
>> Let me start by saying that I'm fully aware (as other long time Prolog programmers are) that several of the module system issues we are discussing are inherited and fixing them would break havoc backwards compatibility.
>>
>>> On 11 Jun 2018, at 13:05, Jan Wielemaker <j...@swi-prolog.org> wrote:
>>>
>>> On 06/11/2018 11:37 AM, Paulo Moura wrote:
>>>> Steroids have usually a negative connotation... But such
>>>
>>> Steroids to me have no negative connotation :) I simply meant a system
>>> that provides much more than the module systems as we know them. Whether
>>> that is good or bad depends on what you need. If you need the stuff
>>> Logtalk offers it is great. If you don't, it is not. It does come at a
>>> price. It enlarges the footprint,
>>
>> Is the footprint (of loading Logtalk) that significant nowadays? Using the included script for embedding Logtalk we get:
>>
>> $ ll
>> total 632
>> -rw-r--r-- 1 pmoura staff 322284 Jun 11 13:56 logtalk.qlf
>>
>> This QLF files includes both the compiler and runtime.
>
> Thanks that current resources are zip :)

The switch to zip is a great improvement indeed. Thanks!

> I must admit that the memory
> usage impact is small. Startup of a simple state goes from 12ms to about
> 20ms. That is significant, but relevant to a small minority of
> applications such as using Prolog/Logtalk as traditional CGI script.
>
>>> it makes some of the development tools
>>> work less transparently or not at all (in return you get some Logtalk
>>> development tools),
>>
>> Some? :-) Logtalk provides a full set of developer tools. A good summary with usage examples is available at:
>>
>> https://logtalk.pt/tools/
>>
>> Which tool you find missing? But you can also use the SWI-Prolog graphical tracer and graphical profiler, thanks for the hooks that you have in SWI-Prolog.
>
> Yes, but the Edit menu in the graphical profiler doesn't work, nor
> edit/1 at the prompt.

I will look into the graphical profiler edit menu issue. But for edit/1 at the top-level I get:

?- [aliases(loader)].
% [ /Users/pmoura/logtalk/examples/aliases/aliases.lgt loaded ]
% [ /Users/pmoura/logtalk/examples/aliases/loader.lgt loaded ]
% (0 warnings)
true.

?- edit(aliases).
Please select item to edit:

1 <file> aliases
2 <loaded file> 'logtalk/examples/aliases/aliases.lgt'
3 <file> aliases

Your choice?

Sounds good to me. Can we improve on it?

> I guess that is fixable using more hooks and
> implementation thereof in Logtalk. Similar issues exist with the low
> level tools such as jiti_list/1, vm_list/1, etc. to understand the
> program at all its levels. I did some code analysis at Kyndi for Logtalk
> code and these are the small things you stumble on.

They are indeed some small issues and I'm happy to work with you in fixing them.

> Of course I'm an
> outlier as I know all the SWI-Prolog tools very well and not the Logtalk
> replacements.
>
>>> it makes loading code slower as it implements (I
>>> think) a three-phase compiler (two phases compiling Logtalk, one loading
>>> the result), whereas SWI-Prolog has a single-phase compiler.
>>
>> 14:04 $ ll large.*
>> -rw-r--r--@ 1 pmoura staff 1072336 Mar 3 2013 large.lgt
>> -rw-r--r--@ 1 pmoura staff 1072324 Nov 7 2013 large.pl
>>
>> The Logtalk file simply encapsulates the contents of the large.pl file inside an object. The contents itself is one of the big Unicode tables.
>>
>> ?- time(true).
>> % 3 inferences, 0.000 CPU in 0.000 seconds (72% CPU, 300000 Lips)
>> true.
>>
>> ?- time(load_files('large.pl')).
>> % 827,212 inferences, 0.137 CPU in 0.144 seconds (95% CPU, 6045899 Lips)
>> true.
>>
>> ?- time(logtalk_load('large.lgt')).
>> % [ /Users/pmoura/Desktop/large.lgt loaded ]
>> % (0 warnings)
>> % 1,596,437 inferences, 0.360 CPU in 0.434 seconds (83% CPU, 4434658 Lips)
>> true.
>>
>> So, roughly twice the number of inferences and 2.5x the compilation time. This may have some practical significance for large data files (this example can be considered a data file, albeit a small one). But giving Logtalk's make tool and how easy is to precompile Logtalk code into QLF files, that's only an issue the first time you compile a large file. Then again, why encapsulate large data in the first place?
>
> The time difference is about what I expected. Why you use Logtalk for
> data?

I *don't*. This is an artificial benchmark constructed just to do an apples to apples (or oranges to oranges for you Linux users ;-) comparison of compiler overhead.

> I suspect many Logtalkers will do so.

Not that I'm aware. In fact, in most cases with big data files that I worked on for industrial clients (or where I gave feedback), the data files are usually plain Prolog files. When SWI-Prolog was used, those data files were precompiled into QLF files.

> Note that compiler speed
> does matter. At least I have a couple of users that frequently complain
> about long compile times. There are people with really large programs
> and not only for big data files ...

Isn't that more of any issue of proper modularization and use of make tools (which are available in both SWI-Prolog and Logtalk)? If everytime you make a change you need to recompiled everything, you have a *build* problem and there the compiler performance may indeed be a factor.

>>> It requires
>>> temporary files, which is typically no real problem but is on some
>>> build and deployment scenarios.
>>
>> Logtalk default scratch directory can be set to /tmp. Which operating-systems are people using that don't provide that or an equivalent? For deployment, the usual solution is to precompile and embed Logtalk and the Logtalk applications, not to be compiling application code in deployment scenarios (an observation which is also valid for Prolog itself).
>
> I recently understood that the nodes of some of the big tech companies
> run on a completely read-only filesystem.

That seems to be common indeed for security reasons.

> Now as saved states seem to
> work with recent Logtalk versions this is possible.

See the embedding script for SWI-Prolog distributed with Logtalk:

https://github.com/LogtalkDotOrg/logtalk3/tree/master/scripts/embedding

Try the saved state examples at:

https://github.com/LogtalkDotOrg/logtalk3/blob/master/scripts/embedding/SCRIPT.txt

> Some people are a
> bit scared about using saved states that indeed come with some problems
> wrt. initialization and access to resources such as foreign libraries.

Complex cases most likely still require some customization, fine tuning of the build infrastructure. But both Logtalk and SWI-Prolog got significantly better recently for deploying applications :-)

>>> As we have seen, making Logtalk
>>> compatible with SWISH is probably a difficult task.
>>
>> There's for quite some time a experimental Logtalk pack that works with SWISH. Douglas Miles as playing with it some time ago. Myself, have not looked into it at detail. Happy to revisit it.
>
> Good to know. I thought the problem of compiling Logtalk objects to
> (temporary) modules was not solved. Curious how Douglas managed to
> hack around that. He's surely very creative :)

I provided Douglas with a version of the pack that loads Logtalk into a module (other than "user"). No idea if he played with hacking it to use a *temporary* module, however.

>>>> characterization is misleading considering that Logtalk provides*basic*
>>>> features for an encapsulation mechanism that are not provided by
>>>> modules. That includes (short list):
>>>>
>>>> - separation between interface and implementation (protocols are first-class
>>>> entities)
>>>
>>> That can indeed be quite useful. It can be done in plain (modularized)
>>> Prolog, but it is a bit hacky and consistency (verifying the
>>> implementation does indeed implement the interface) doesn't come
>>> for free.
>>
>> Is not that is just hacky with modules. The current and recommended practice with Prolog modules is that *exported* predicates should not clash. I.e. modules, as encapsulation units, in practice are there only to prevent clashes in *private* predicates. You likely remember the discussion over which module had the right to export a transpose/2 predicate some years ago? Or consider module libraries such as "avl" or "rbtrees" and other implementing the same data structure but with completely different interfaces. Contrast that with Logtalk where you have a dictionary protocol and several objects implementing it, which makes switching between alternative implementations trivial. Being able to define a protocol/interface and provide multiple implementations is one of the most basic features for an encapsulation mechanisms. Its absence in module systems twists practice and leads to non scalable, brittle solutions.
>
> It would probably be good practice to implement the same API for modules
> that implement the same abstract data type using different technology.
> That also works in Prolog. It seems pretty unlikely that someone wishes
> to load rbtrees and assoc into the same module and in the case they wish
> to do so, using :- use_module(library(assoc),[]) and assoc:empty_map(Map)
> or something like that would work. All Logtalk offers for this is the
> notion of a protocol. That is easily defined in Prolog. Just introduce
> a syntax to define a protocol such as
>
> :- protocol(map,
> [ empty_map/1,
> ...
> ]).
>
> And a term expansion that expands :- implements_protocol(map). into a
> module header and you are done.

Not really done. Logtalk objects can implement any number of protocols. You can only have a single module header. Also, a protocol doesn't define just normal predicates but also e.g. meta-predicates. Thus, you need to pass more than just predicate indicators. Your simple example is simplistic and needs (much) more to be a practical solution.

multifile/1 does indeed more than you want. Is not just overkill. It's the wrong solution. The semantics are different. Certainly you are not advocating making any predicate declared in a protocol/interface hack for modules a multifile predicate?

> AFAIK the ISO
> proposal also separates the interface from the implementation. This
> suggests the idea lives, but apparently nobody bothered implementing
> it in a module system while this is of course trivial.

The ISO module standard do not support multiple implementations per interface. One must wonder why they even bother to publish it.

>>>> - simple scope rules for all flags (flag directives are always local to
>>>> their scope)
>>>
>>> Flags (as in current_prolog_flag) are a nightmare.
>>
>> A mostly *inherited* nightmare :(
>>
>>> It is a single
>>> mechanism that affects the compiler and runtime behavior. Flags that
>>> affect the compiler should probably have lexical scoping. Some do in
>>> SWI-Prolog but inconsistently either due to lazyness or compatibility
>>> concerns. Flags that affect runtime behaviour are hard to handle and
>>> having threads makes it even harder. If you have a golden solution I'm
>>> glad to hear about it :) My general opinion is "don't touch them" with
>>> as only exception to touch them at program startup to deal with
>>> portability issues.
>>
>> The solution I use for Logtalk flags could certainly be used for Prolog flags by it would break backwards compatibility :(
>
> Well, lexical scoping for flags that affect the compiler is easy. Scoped
> switching between the different occurs_check values is a different matter.
> Are you saying you can do that?

The occurs_check flag is a Prolog flag, not a Logtalk flag. Note that Logtalk acts as a pure add-on. It doesn't change Prolog builtin features (that would kill portability but would also cause confusion and havoc even when using a single backend Prolog system).

>>>> - enforcement of encapsulation (which you may able to do without iff you
>>>> have strong team discipline)
>>>
>>> That is a choice. A good team should have coding guidelines. Not
>>> breaking encapsulation should be one of these rules. IMO it is pretty
>>> useful that these rules are not enforced by default during development
>>> and to hack around reusing legacy code. It would not be very hard to
>>> enforce them and given library(prolog_codewalk) it is just a few lines
>>> to write a checker for cross-module calls to private predicates. This
>>> can be fooled, although it is also quite easy to locate calls for which
>>> the target cannot be deduced at compile time.
>>>
>>>> The point is, you don't need to be doing programming in the large to
>>>> benefit. Logtalk may look like, as you wrote, a module system on
>>>> steroids but only because module systems are in reality a prefixing
>>>> mechanism stretching to double as a rather limited encapsulation mechanism.
>>>
>>> I don't think that is the main value of Logtalk vs module systems. It is
>>> too easy to do that in a module system.
>>
>> Above you have several examples that contradict that. As I wrote, this was just a short list. I could easy include better term-expansion, message printing, question asking mechanisms. Or parametric entities. Or optionals and expecteds libraries (that take full advantage of parametric objects), ...
>
> I think I argued that a lot of this stuff is easily done in the context
> of modules.

For some simplistic scenarios, maybe. You will not get very far with them. There are also a good number of fine details with semantic implications that we are not even discussing here.

There have been people toying and arguing for years about how to expand module systems (reinventing parts of Logtalk in the process). Nothing of practical consequence have ever been done. One of the main reasons is that a predicate prefixing mechanism is the wrong basis to start with. There's also a big difference between talking about it and actually delivering it as a mature system that other can use.

It should be noted that the very same things that people find convenient when writing small programs are often the ones that get them in trouble when programs get more complex. Case in point, we see regularly in this mailing list how the consequences of implicit qualification in its illusory simplicity causes perplexity on program behavior and leads to length discussions of people trying to understand what's happening and how to fix it. Logtalk, despite being more featured, leads to easier to understand code as it's not based on a prefixing mechanism acting under the covers.

> You may of course ask why it hasn't been done.

Are you kidding? After all the work I did on Prolog standardization? :-D

> Well, quite a
> bit has been done by some Prolog system, e.g., term-expansion in Ciao
> and ECLiPSe, message printing in Quintus and SWI-Prolog, question asking
> in SWI-Prolog.

I wrote better, not new :-) Logtalk term-expansion mechanism for one is a significant improvement. We discuss it in the past in this mailing list.

> I'm not aware of parametric entities as a standard Prolog
> facility. I've seen a talk at ICLP quite a few years proposing this for
> modules. Surely there are applications doing this type of stuff as it is
> quite natural in Prolog.

Parametric entities are a wonderful versatile concept. For for several application scenarios, see e.g:

@inproceedings{pmoura09d,
author = {Paulo Moura},
title = "{Knowledge Representation Using Logtalk Parametric Objects}",
booktitle = {Proceedings of the 18th International Conference on Applications of Declarative Programming and Knowledge Management (INAP)},
editor = "Salvador Abreu and Dietmar Seipel",
publisher = "University of \'Evora, Portugal",
pages = {225-240},
month = nov,
year = 2009

}

> One thing that is hard is to consider a module as a set of methods and
> perform clean inheritance on that.

Indeed. Btw, Logtalk supports multiple inheritance, multiple instantiation, *and* a mechanism to override the default predicate lookup algorithm in case of inheritance conflict. And yes, multiple inheritance can be clean with a bit of care. But it can also blow your hands.

>>> The main value lies IMO in more
>>> sophisticated mechanisms for code reuse such the notion of interfaces,
>>> calling self/super methods, etc. All these things can be done in
>>> modularized Prolog, but having it all available in a consistent, nicely
>>> documented and well tested package is a clear win.
>>
>> Thanks.
>>
>>> I find myself
>>> typically only wishing for tiny pieces of Logtalk for which I prefer
>>> stretching Prolog a little over switching to Logtalk.
>>
>> Logtalk was *designed* to extend Prolog, not to replace it. It's not a *switch to*, it's a *use with*. Logtalk does provide an alternative to Prolog modules but it happily co-exists with them. Is not a "either" choice. In fact, modules can send messages to objects with zero overhead (in SWI-Prolog and YAP) Objects can call module predicates also have zero overhead.
>
> Is that new?

You mean the performance bits? That are indeed some performance improvements in the recent years. But Logtalk main design goals and choices are the same as when first released back in 1998.

> I always liked the idea that you could do
>
> :- use_module(library(logtalk)).
>
> :- object(...)
> ...
>
> etc. I've always understood there was a Logtalk executable as swi-lgt.
> Since years I have a little load library to load Logtalk dynamically
> which is notably practical if I need to run SWI/Logtalk under gdb
> debugger and thus need the raw executable rather than a shell script.

You can still do that. Look into the sources of the swilgt.sh script.

> I know it is non-negotiable due to your portability requirements, but I
> also would like to see most of Logtalk a shallow and possibly modular
> wrapper around Prolog's modules.

Not going to happen: a predicate-prefixing mechanism is simply the *wrong* foundation.

> That would make the system more
> transparent and invites using Logtalk for things it is good at rather
> than an alternative module syntax.

Is not an alternative syntax. Logtalk doesn't use a predicate-prefixing mechanism. That have fundamental consequences on semantics.

> I still claim that modules solve most
> of the problems needed to deploy Prolog for large programs.

It doesn't. Not the least due to current recommended practices.

I regularly talk with people that think that because they are writing big and complex applications they are doing programming in the large. To be blunt, anyone that can oversee all that is going on in an application is, be definition, programming in the small.

> I do agree
> it neglects some issues and poor standardization doesn't do much good
> either. Still, module incompatibility is a minor problem for porting
> Prolog in my experience. (Lack of) e.g., tabling, attributed variables,
> mutable objects, threads, foreign interface, indexing, etc. are way
> harder to deal with.

Writing portable code beats hands down porting code. But, as you say, there are too many things that have no portable solution. The challenge is often in trying, if possible, to modularize the non-portable bits. Not easy :(

Cheers,
Paulo

[Proof Easel]

I do not agree with the below statement. I think this is one way
of interpretation of the ISO module standard. But there are other
ways of interpretation of the ISO module standard.

As a fact the ISO module standard has:

    ISO/IEC 13211-2 First edition 2000-06-01
    6.2.4.4 Module interface directive reexport/1

So you can do an interface with multiple implementations
as follows (not using begin/end, but the modern module
directive that also exists in SWI-Prolog besides reexport):

    File interface:
    :- module(interface, []).

    File implementation1:
    :- module(implementation1, []).
    :- reexport(interface).

    File implementation2:
    :- module(implementation2, []).
    :- reexport(interface).

Pretty simple, isn't it? See also my thread:

Best kept secret of Prolog, Lesson 1: reexport/1 directive as IS-A link
https://groups.google.com/d/msg/comp.lang.prolog/4_2WRm2EXCQ/0a6Pjr68AgAJ

In as far I believe that you can do everything that Logtalk offers,
with plain  ISO modules. I don't think there is a single thing, that
can be done with Logtalk, that cannot be done with plain

ISO modules, using this best kept scecret...

[Proof Easel]

What you need to add to your native module system, to do
everything that Logtalk can do, and a little bit more, besides
the best kept secret #1 is the following

- preferably natively visibility modifiers for both predicates
   inside modules and plain predicates as well

- Use the other best kept secret #2 for late binding:

    Best kept secret of Prolog, Lesson 2: (:)/2 for late binding
    https://groups.google.com/d/msg/comp.lang.prolog/4_2WRm2EXCQ/O5hELd6-AgAJ
  
- preferably natively truth maintenance, so that you get
  a system on steriods, not a system on valium with
  clumsy full code recompile piplines.

- Keep improving!

[Jan Wielemaker]

Hi Paulo,

I think we simply disagree on what programmers want/expect/need. We knew
that all along ... Clearly there are programmers seeing it your way as
well as programmers seeing it my way and probably many more programmers
seeing it yet some other way.

Claiming that plain Prolog cannot be used for programming in the large
is IMHO proven to be wrong. I am aware of several programs developed by
teams and ranging in the 0.5-1M lines of code. Some people may claim
that is still small, but if we take the brevety of Prolog into account
these are probably equivalent to Java/C++/... programs of 5-10M lines.

One little answer:

> I will look into the graphical profiler edit menu issue. But for edit/1 at the top-level I get:
>
> ?- [aliases(loader)].
> % [ /Users/pmoura/logtalk/examples/aliases/aliases.lgt loaded ]
> % [ /Users/pmoura/logtalk/examples/aliases/loader.lgt loaded ]
> % (0 warnings)
> true.
>
> ?- edit(aliases).
> Please select item to edit:
>
> 1 <file> aliases
> 2 <loaded file> 'logtalk/examples/aliases/aliases.lgt'
> 3 <file> aliases
>
> Your choice?
>
> Sounds good to me. Can we improve on it?

A lot. The idea of edit/1 is that you can give it any term that
can be associated to a source location, so you can do

?- edit(member).
?- edit(lists:member/2).
?- edit(lists).

It is hookable at two levels: how to translated a term into its full
specification and get a source location (file, file+line or
file+line+linepos). So, edit(member) will produce lists:member/2, file
file and the line it is defined. The second hook specifies how to open
an editor at this location.

Together with <TAB> based completion that typically makes finding some
object from what is on the console and/or in your head very quick. For
Logtalk there is a lot to add: classes, methods, interfaces, etc.

This is also used by the editor, profiler and all other tools that
offer editing the source.

Cheers --- Jan

[Proof Easel]

You need of course get the reexport/1 right and powerful
enough. Here is an exercise. Can your Prolog system
do the following:

    File interface:
    :- module(interface, [op(500,yfx,++)]).

    File implementation1:
    :- module(implementation1, []).
    :- reexport(interface).

    File implementation2:
    :- module(implementation2, []).
    :- reexport(interface).

Does the plus syntax operator (++)/2 get visible in the
right places? Can your Prolog system do it. Can Logtalk do it?
I doubt that Logtalk can do it.

I have never seen syntax operator inheritance in connection
with Logtalk. But this is pretty standard for standard modules.
And it can be also lifted to OO modules by the

best kept secrets #1 and #2. But how should Logtalk do it?
It cannot tweak the existing Prolog readers, and it doesn't
implement its own Prolog text reader.

Thats impossible and out of reach for Logtalk. But for a grown
up OO modules, we want of course everything, the whole
cake, not only half of the cake.

Am Dienstag, 12. Juni 2018 13:22:39 UTC+2 schrieb Proof Easel:

- Keep improving!

[Proof Easel]

SWI-Prolog can do it partially. It cannot do it fully, since it uses meta_predicate
semantics for modules. The rexport/1 is respected during consult file, hence
during the read_term there:

   Welcome to SWI-Prolog (threaded, 64 bits, version 7.7.15)
   SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

   ?- assertz(user:prolog_file_type(p, prolog)).
   true.

   ?- ensure_loaded('C:\\Projects\\Jekejeke\\Prototyping\\experiment\\other\\prolog2\\fareast\\logtalk\\implementation1.p').
   ++(foo,bar)
   true.

   ?- ensure_loaded('C:\\Projects\\Jekejeke\\Prototyping\\experiment\\other\\prolog2\\fareast\\logtalk\\implementation2.p').
   foo++baz
   true.

But what doesn't work is the reverse, the write_term. This has
to do that the SWI-Prolog module system is meta_predicate driven.
This means calls and so on work. But term writing has no clue

at all about modules. The second write works, since ensure_loaded/1
polutes the user namespace. I made my own test with sys_auto_load/1,
I do not know currently whats the SWI-Prolog equivalent to sys_auto_load/1,

but I can make more demonstrate that the (++)/2 operator is properly in
its own namespace. This is not the real issue for SWI-Prolog, I guess
it has the (++)/2 operator also properly in its own namespace:

   Jekejeke Prolog 3, Runtime Library 1.3.0
   (c) 1985-2018, XLOG Technologies GmbH, Switzerland
   ?- sys_add_path('file:/C:/Projects/Jekejeke/Prototyping/experiment/other/prolog2/fareast/logtalk/').
   Yes

   ?- sys_auto_load('file:/C:/Projects/Jekejeke/Prototyping/experiment/other/prolog2/fareast/logtalk/implementation1.p').
   foo++bar
   Yes

   ?- sys_auto_load('file:/C:/Projects/Jekejeke/Prototyping/experiment/other/prolog2/fareast/logtalk/implementation2.p').
   foo++baz
   Yes

   ?- X = foo ++ bak.
   Error: Superfluous token.
   X = foo ++ bak.
           ^

I do not know what can be done to fix the writing in case of SWI-Prolog.
Some write option? My expectation: A write inside a module should do
the right thing automatically.

I uploaded some screenshots, in case you don't believe me. I put
the little Prolog text of interface, implementation1 and implementation2
also on the same gist:

https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2617329

[Paulo Moura]

Hi Jan,

> On 12 Jun 2018, at 12:22, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> Hi Paulo,
>
> I think we simply disagree on what programmers want/expect/need.

One (honest) question: how much time have you ever spend writing Logtalk code? E.g. running the examples, modifying them, writing your own, using the developer tools? Or is your experience with it mainly from the snippets of Logtalk code you see here and there?

Having programmed with plain Prolog and Prolog modules for +10 years before getting feed up and inventing Logtalk back in 1998 gives me a fair experience in both sides.

> We knew
> that all along ... Clearly there are programmers seeing it your way as
> well as programmers seeing it my way and probably many more programmers
> seeing it yet some other way.

Fair enough. Thanks for the constructive discussion.

> Claiming that plain Prolog cannot be used for programming in the large
> is IMHO proven to be wrong.

I never claimed *that*. Heck, people have done amazing stuff in assembler. But the constructs available in a language can make the process more painful or more pleasant and that have real implications, notably for maintenance (typically in these scenarios programmers come and go but applications need to keep working). I did and do argue that Logtalk is much better suited for programming in the large than Prolog modules, however. It also works nicely in programming in the small scenarios.

> I am aware of several programs developed by
> teams and ranging in the 0.5-1M lines of code. Some people may claim
> that is still small, but if we take the brevety of Prolog into account
> these are probably equivalent to Java/C++/... programs of 5-10M lines.

Agree with the Prolog vs Java comparison. But like you, I also experience first hand how painful it can be to understand, fix, port those little and not so little monsters :(

Thanks for the details. I assume that some of this functionality will only work, however, with the SWI-Prolog own editor? Is it possible to configure a different editor? Say Atom or VSC?

Cheers,
Paulo

[Proof Easel]

Logtalk was conceived already before 1998, by some additional participant
Ernesto Costa. At least this paper says so, and is dated September 1994:

Logtalk: Object-Oriented Programming in Prolog (1994)

by Paulo Moura, Ernesto Costa
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.48.2295

This is still after Prolog++. It seems that the design of Logtalk then
splitted a dynamic call into search and executed.

Is this stil the same pattern for dynamic call? Probably it is this
Ernesto Costa, the paper is also found in this automatic collection:

https://scholar.google.com/citations?user=myoBo8UAAAAJ

[Proof Easel]

Maybe a new model, a new tool chain, a new runtime approach, a
new eco system is needed. Like Java did.

Maybe this is needed for Prolog:

- Like Java did, one can do annotations in the code. The usual Prolog
   compiler might not see them, a tool further upstream in the tool chain,
   might use it for code manipulation or generate extra artefacts, like
   web server declarative security, etc.. etc...

- Like Java did, on the byte code level, there can be instrumentation
  agents. You can start your code via preMain() and all thinks can
  happen, like hooking into method calls.

- Like Java has, there are also post hoc tools, like javadoc, that can
  generate reports, etc.. Like a API documentation web site from
  Java code. The same as PL Doc already for SWI. Or something
  new for some code metrics.

This way somehow the Prolog interpreter is externally enhanced,
but for performance reasons the OO stuff would be directly done
by the Prolog interpreter.

But everything else would be done externally, and kind of optionally.
The Prolog interpreter would already have a sufficiently strong OO
to execute an application in normal mode.

For extra things, like compiling an application based on annotations,
or running an application with some hooks, or producting some
documentation, would be extra somewhere.

[Proof Easel]

For example you can do your Logtalk event hooks,
also with a debugger hooks of a native Prolog system.
I have such a protocol here:

Custom Debugger

http://www.jekejeke.ch/idatab/doclet/prod/en/docs/10_dev/10_docu/02_reference/07_theories/01_debug/02_custom.html

In the present case, by using Best kept secret #1 and #2
an OO call, would be simply a call to the builtin (::)/2, and
can be easily traced and analyzed.

You can do all kind of facy stuff, like drawing interaction
diagrams (not taxonomies, real interaction), from runtime
dumping huge protocal data, via this hook. etc.

Well I dump the protocol data into memory, and analyse
it later. But this all works for OO calls also, just watch for
(::)/2 invocations, remember that they are PythonStyleCalls,

and you have your runtime receiver object, and can do
all kind of analyzing. I am using this for coverage analysis...

[Jan Wielemaker]

On 12/06/18 13:56, Paulo Moura wrote:
> Hi Jan,
>
>> On 12 Jun 2018, at 12:22, Jan Wielemaker <j...@swi-prolog.org> wrote:
>>
>> Hi Paulo,
>>
>> I think we simply disagree on what programmers want/expect/need.
>
> One (honest) question: how much time have you ever spend writing
Logtalk code? E.g. running the examples, modifying them, writing your
own, using the developer tools? Or is your experience with it mainly
from the snippets of Logtalk code you see here and there?

I've never really developed Logtalk code. I considered it when in need
for proper inheritance with send self/super. That was long ago and doing
this in Logtalk seemed to imply moving the whole project to Logtalk
those days and that was, again those days, not trivial. Writing a simple
inheritance mechanism based on modules was easier.

Most of my experience with Logtalk comes from dealing with bug reports,
mostly by you and some poking around in code while at Kyndi. In both
cases I've been struggling to understand what exactly happens at the
Prolog level.

And, of course there are the cases where people want to achieve
something and you explain how to do it in Logtalk. I rarely, if ever,
saw a case that had no reasonable alternative in plain SWI-Prolog.

> Having programmed with plain Prolog and Prolog modules for +10 years before getting feed up and inventing Logtalk back in 1998 gives me a fair experience in both sides.

You surely have experienced with many more Prolog systems than I do.
I haven't found much evidence that the current module system is
inadequate in general though. Surely, there are things it cannot
(easily, cleanly) do.

>> We knew
>> that all along ... Clearly there are programmers seeing it your way as
>> well as programmers seeing it my way and probably many more programmers
>> seeing it yet some other way.
>
> Fair enough. Thanks for the constructive discussion.
>
>> Claiming that plain Prolog cannot be used for programming in the large
>> is IMHO proven to be wrong.
>
> I never claimed *that*. Heck, people have done amazing stuff in assembler. But the constructs available in a language can make the process more painful or more pleasant and that have real implications, notably for maintenance (typically in these scenarios programmers come and go but applications need to keep working). I did and do argue that Logtalk is much better suited for programming in the large than Prolog modules, however. It also works nicely in programming in the small scenarios.
>
>> I am aware of several programs developed by
>> teams and ranging in the 0.5-1M lines of code. Some people may claim
>> that is still small, but if we take the brevety of Prolog into account
>> these are probably equivalent to Java/C++/... programs of 5-10M lines.
>
> Agree with the Prolog vs Java comparison. But like you, I also experience first hand how painful it can be to understand, fix, port those little and not so little monsters :(

Given SWI-Prolog's tools, code is quite tractable. There can be a lot
that is hard to understand in a Prolog program. Following what calls
what or what is called by what is not one of them. Unless, of course,
people start constructing calls dynamically. I've seen code doing
name/2, append/3, name/2, =../2 and finally call/1. If code is called
that way it gets hard, but Logtalk won't solve this. What does is e.g., the
profiler which constructs a dynamic call graph (might be nice to make
that graphically available) or simply put a spy point on the predicate.

No. As said, the second hook goes from an identified source location to
calling the editor. That can do anything. The default is to call
PceEmacs when available. The next default is to look at the Prolog flag
`editor` and finally at the environment variable `EDITOR` and do
something clever with them. There is a multifile predicate that
specifies how a number of editors can be called to start at a specific
line (check the code, this is from my head). As you can hook before all
that default logic you can do anything Prolog can access. Please send
extensions for your popular editors.

Cheers --- Jan

>
> Cheers,
> Paulo
>

[Proof Easel]

Concering "super". It took me some time to figure out,
how to do it Python style. But its relatively simple.
You don't have the luxury of Java, that it determines

the class name for you, but lets take the same interface
multiple implementations scenario, and lets conceptually
use a new feature of Java, that interfaces can have methods.

Or lets assume our interface is just an abstract class.
Long talking short result, I will put the super in the interface
for this example. So how to do "super", with best kept secret

#1 and #2? I really had to figure this out, otherwise my
Java autoloader would never work. I need it in the overloading
branching. Well here is an example how it looks at runtime:

File implementation 1:

?- succ(1, X).
I am doing somethhing before super
super did its job, isn't that super
X = 2

File implementation 2:

?- succ(1, X).
super did its job, isn't that super
I am doing something after super
X = 2

And here is the code that uses "super":

File implementation 1:

succ(X, Y) :-
   write('I am doing somethhing before super'), nl,
   interface:succ(X, Y).

File implementation 2:

succ(X, Y) :-
   interface:succ(X, Y),
   write('I am doing something after super'), nl.

You can check the source code, super is a simply a qualified
call with (:)/2. Works also if you have pythonesk style OO calls.
You could maybe define a extra operator for it.

But I think conceptually it does what it should to. It delegates
a message processing to some method in some class,
by keeping the receiver, which might belong to some other class.

Here is the source code, with screenshots:
https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2617406

Give me some time. I will make some SWI-Prolog test.
I guess it will also work there...not sure... As I wrote,
**keep improving guys!** The key to all these secrets

has two incredients, #1 rexport/1 as IS-A relationship and
#2 usePythonesk OO. The receiver object in the first argument
is something I borrowed from Python. I posted about

that on comp.lang.prolog some years or months ago. The
Pythonesk approach works also for "super", not only for
"self". You can do all OO, you don't need anything else.

the Python idea itself was already documented on comp.lang.prolog
some years/months ago. Currently I am working on TMS
and hierarchical knowledgebases. But the core OO stuff is

already completely solved with native
modules for a couple of years/months.

[Proof Easel]

Works also in SWI-Prolog. Here is what SWI-Prolog does.
The only problem is a little fine tuning of the overriding:

  Welcome to SWI-Prolog (threaded, 64 bits, version 7.7.15)
  SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

  Please run ?- license. for legal details.

  For online help and background, visit http://www.swi-prolog.org
  For built-in help, use ?- help(Topic). or ?- apropos(Word).

  ?- assertz(user:prolog_file_type(p, prolog)).true.

  ?- ensure_loaded('C:\\Projects\\Jekejeke\\Prototyping\\experiment\\other\\prolog2\\fareast\\logtalk\\implementation1.p').
  ++(foo,bar)

  ERROR: c:/projects/jekejeke/prototyping/experiment/other/prolog2/fareast/logtalk/implementation1.p:33:12: Syntax error: Operator expected
  Warning: c:/projects/jekejeke/prototyping/experiment/other/prolog2/fareast/logtalk/implementation1.p:34:
        Local definition of implementation1:succ/2 overrides weak import from interface
  true.

  ?- succ(1, X).
  I am doing something before super

  super did its job, isn't that super

  X = 2.

The fine tuning of the overriding could be, the following:
- I am using the operator and directive override/1 to indicate
  overriding. Since this operator is not defined in SWI-Prolog
  I get an error in the above, but consult continues.

- Consult continues, and it seems that the SWI-Prolog
  policy is, that it accepts overriding, at least it accepts
  a new succ/2. This is exactly what is need to make
  the secret #2 work in a broader sense.

- But SWI-Prolog is a little bit verbose during consult. I don't
  know right now, what could direct SWI-Prolog to not be
  that verbose. Is there an equivalent of override/1 directive
  that would switch off the warning.

I uploaded the screenshot also to gist, to show what SWI-Prolog does:
https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2617418

[Proof Easel]

Disclaimer: Currently limitation of this super call, its like in
original Java, assumption is that a Java class has one super
class, so you just name this super class.

Doesn't work if the **appetite** is much greater than Java.
Like if a class has multiple super classes in the sense of Java,
which Java usally doesn't really support for ordinary classes,

that really all super classes provide some implementation. In
this case, you would have multiple reexpert/1 anyway, and you
could choose the qualified call, to prefer some of the reexports.

But you can still allow some search down the superclass
chain. So if you write this, at least in Jekejeke Prolog, and
possibly in many other Prolog systems:

   interface:succ(X, Y).

There is no requirement that succ/2 really exists in
interface. It could also exist in some super interface, and
would be found there. Thats basically the semantics of reexprt/1

from ISO module standard, how I interpret it. This little
thingy here, which can do a lot of magic. All the IS-A magic
that is needed for OO:

          ISO/IEC 13211-2 First edition 2000-06-01
         6.2.4.4 Module interface directive reexport/1

[Proof Easel]

The **appetite** could be greater, if you would like
to do some Scala mixin stuff (traits) in Prolog. You
can do that actually with the ISO core reexport/1

to some extend. But Java has also improved recently.
They added multiple inheritance for interfaces.
Their name is default methods, they use a keyword

from the Java language, which was spared, and not yet used:

Default Methods (JDK 8 feature):

https://docs.oracle.com/javase/tutorial/java/IandI/defaultmethods.html

You can do that with ISO reexport/1 to some extend.
If I have time, I could do an example. But if I remember
well this is also possible.

So the ISO reexport/1 is on par with JDK 8, viewed from
JDK 7, the ISO reexport/1 is even more powerful. This
JDK 8 feature was critizised since it introduces more

phaenomena in the direction of the

Fragile base class problem:

https://en.wikipedia.org/wiki/Fragile_base_class

If you have truth maintenance, you should have a problem
with that, that your system gets unstable technically. But
for the programmer, greater *appetite* means also

greater responsibility. You could also go on and forbid it...

[Proof Easel]

Corr.:

If you have truth maintenance, you should not have a problem

with that, that your system gets unstable technically.

[Paulo Moura]

Hi Jan,

> On 12 Jun 2018, at 13:57, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> ...

> And, of course there are the cases where people want to achieve
> something and you explain how to do it in Logtalk. I rarely, if ever,
> saw a case that had no reasonable alternative in plain SWI-Prolog.

There's an interesting issue here: there are a good number of abstract programming patterns, most of them born or made popular in the context of OOP languages, that have wide applicability and can be easily/naturally expressed in Logtalk but are often cumbersome with modules (e.g. several of them take full advantage of the idea of protocols/interfaces). Students are usually familiar with them thanks to OOP and Software Engineering classes. In fact, and sadly, there are far more likely to be familiar with these patterns than with logic and functional programming idioms. This is not to say, of course, that Prolog don't have its own, characteristics abstract programming patterns and idioms.

> I haven't found much evidence that the current module system is
> inadequate in general though. Surely, there are things it cannot
> (easily, cleanly) do.

Note that Logtalk objects *subsume* Prolog modules but the reverse is not true in general. As Prolog modules *are* objects, prototypes to be exact, any module-based solution can be easily translated into an object-based solution. So easy in fact that the Logtalk compiler does it for you (minus some proprietary stuff that choke it, which varies from Prolog system to Prolog system; blame lack of standardization).

:- module(foo, [bar/1, baz2]).
...

or

:- module(foo, []).
:- export([bar/1, baz2]).
...

is simply interpreted by the Logtalk compiler as:

:- object(foo).
:- public([bar/1, baz2]).
...
:- end_object.

It's equally readable, the performance is basically the same, and just one more directive, end_object/0, that a decent editor will type for you by expanding an object template.

But you can't go from an object solution to a module solution easily or without significant hacking when you're taking advantage of e.g. inheritance, self/super calls, protocols, parametric objects, ... Using a more general solution is worthy by itself and orthogonal to whole programming in the small/large perspective.

Cheers,
Paulo

-----------------------------------------------------------------
Paulo Moura
Logtalk developer

[Proof Easel]

Well the reverse is also true. Show an example that cannot be done?

[Proof Easel]

It would need to be an example with virtual methods of
course. And we would look at both performance and
benefits for the programmer.

The idea to do OO natively is an old idea of mine. I
already formulated it Dezember 2010, see here:

http://coding.derkeiler.com/Archive/Prolog/comp.lang.prolog/2010-12/msg00035.html

Only then it didn't occur to me that ISO module standard
offers everything. But now there is an implementation
around of a native OO-system.

And we could work for example on this claim of yours.
In 2010 you wrote: "That said, there is likely little to be gained
(in terms of runtime performance (*))".

But for example to have abstract datatypes you need good
virtual method invocation time. Just assume an abstract datatype
CharSequence which has different implementations,

and you use it with charAt(). You mentioned yourself the
interface/multiple implementation scenario. To have this practical
you need very good performance in the virtual calls.

I don't say I have already top speed. Actually my inline
caches are a little cludge, and some ways of using them,
you see a performance degradation, where the Logtalk

2 additional logical inferences (or are they more), has maybe
better performance, if underpinned by indexes. It depends on
the Prolog system. What are these additional logical inferences

exactly? Static predicates or dynamic predicates. In a
multithreaded environment, if they are dynamic predicates,
this would induce additional synchronization, even if a

concurrent datastructure is used. My inline caches use a
concurrent datastructure, which is ultra concurrent without
eventual synchronization, even not some atomic clear/test.

[Proof Easel]

I could replace the inline cache in the future by something
else. After all after a module has been consulted, all its
predicates are known. So it would be maybe also possible

to construct some virtual method call table, like some of C++
translators work. And make the virtual calls even more faster.
Such indirect adressing is the standard approach:

Virtual function

https://de.wikipedia.org/wiki/Virtuelle_Methode#L%C3%B6sung:_Indirekte_Adressierung

Doing it natively, much more natively, could give you just a
one machine instuction for the invocation. In Java you need
only a little bit more complex stuff (actually I don't know

how it looks exactly, might differ JRE to JRE) for interfaces,
but the usual classes which only have one super class, this
implementation is rather trivial. So that virtual methods

become, except for the indirection, neglible. Should all be
the same, static calls and dynamic calls. Not much difference.
For sure not a logical inference. I do not yet have such

an implementation, since I didn't implement a compiler.
But I was perplexed what the ISO module standard can do...

[Jan Wielemaker]

On 12/06/18 16:48, Paulo Moura wrote:
> Hi Jan,
>
>> On 12 Jun 2018, at 13:57, Jan Wielemaker <j...@swi-prolog.org> wrote:
>>
>> ...
>> And, of course there are the cases where people want to achieve
>> something and you explain how to do it in Logtalk. I rarely, if ever,
>> saw a case that had no reasonable alternative in plain SWI-Prolog.
>
> There's an interesting issue here: there are a good number of abstract programming patterns, most of them born or made popular in the context of OOP languages, that have wide applicability and can be easily/naturally expressed in Logtalk but are often cumbersome with modules (e.g. several of them take full advantage of the idea of protocols/interfaces). Students are usually familiar with them thanks to OOP and Software Engineering classes. In fact, and sadly, there are far more likely to be familiar with these patterns than with logic and functional programming idioms. This is not to say, of course, that Prolog don't have its own, characteristics abstract programming patterns and idioms.

That is a valid point.

>> I haven't found much evidence that the current module system is
>> inadequate in general though. Surely, there are things it cannot
>> (easily, cleanly) do.
>
> Note that Logtalk objects *subsume* Prolog modules but the reverse is not true in general. As Prolog modules *are* objects, prototypes to be exact, any module-based solution can be easily translated into an object-based solution. So easy in fact that the Logtalk compiler does it for you (minus some proprietary stuff that choke it, which varies from Prolog system to Prolog system; blame lack of standardization).
>
> :- module(foo, [bar/1, baz2]).
> ...
>
> or
>
> :- module(foo, []).
> :- export([bar/1, baz2]).
> ...
>
> is simply interpreted by the Logtalk compiler as:
>
> :- object(foo).
> :- public([bar/1, baz2]).
> ...
> :- end_object.
>
> It's equally readable, the performance is basically the same, and just one more directive, end_object/0, that a decent editor will type for you by expanding an object template.

It is impressive this actually works, although I guess it eventually
fails for some of the modules doing really nasty stuff, including using
reflection on the module system itself.

> But you can't go from an object solution to a module solution easily or without significant hacking when you're taking advantage of e.g. inheritance, self/super calls, protocols, parametric objects, ... Using a more general solution is worthy by itself and orthogonal to whole programming in the small/large perspective.

One thing I did forget as one of my experiences is fact that you catch
and re-throw many exceptions. That breaks one of the nice features of
SWI-Prolog to print a backtrace on uncaught exceptions as well as to
start the debugger when asked to (default when interactive). That
saves a lot of time during debugging.

We found similar issues with rewritten exceptions breaking the mapping
rules in the HTTP server that map exceptions to HTTP errors.

Cheers --- Jan

[Proof Easel]

Maybe the distinction should be made between semantic modelling,
and OO programming. For OO programming, for example if only
single inheritance is needed, its evident that the reexport/1 directive

builds a function table, and in the case of single inheritance, you
can easily turn this function table, into a virtual function table lookup
and use the module as class name for instances.

But in the case multiple inheritance, maybe one would anyway
use RDF, OWL, Description Logics, etc.. whatever, where its more
common to use logical inferences,

also in RDF, OWL, Description Logics, etc.. you might have
forward chaining, which would give you truth maintenance, if
you desire so. These systems again are more specialized

to also return meta information about their own objects. The
requirements for OO programming and semantic modelling
are slightly different in my opinion. What works for semantic

modelling doesn't necessary work for OO programing. Anyway,
I wont bother you anymore. For every problem the right tool.
In my opinion Logtalk is neither fish nor bird. It tries a balancing

act between semantic modelling and OO programming, but
in my opinion for sure harms OO programming. For OO
programming the most important is performance of

virtual methods and this kind. Java had in the past a couple
of patents which guarded their aproaches. From Python
we have PEPs with some technology. OO programming

languages is a total different turf which you need to approach
natively, otherwise you don't produce something useful...

[Proof Easel]

For example the goal of an OO-system to deliver
meta information is much more limited. You could provide
some reflection, which can be useful to discover services

etc.. But typically you don't bother providing bidirectional
predicates for meta information, that tell you everything
about the OO-system for all directions.

If you want an analysis of such data, you can for
example use in Java javadoc. Which does extract from
source code on its own a meta model of the program

and provide grouping in its output along diverse
axis etc.. Its just a separate tool, not directly married
at runtime with the OO system.

P.S.: Technically Java javadoc shares some of
the token scanner and parser of Java javac.
Basically they share tools.jar from the Java distribution.

But its really a separate tool. Thats probably one criteria
for the distinction between an OO-system and semantic
modelling. Logtalk might have the idea, that such a restriction

doesn't exist. But every restriction that is willfully lifted,
should not sacrifice performance. For an OO-system this
is really the holy grail. No performance, no OO-system.

I have a lot of example where this is extremly important,
for example computer algebra, deep learning, auto diff, etc...
No performance, no computer algebra, no deep learning,

no auto diff, etc... Without performance, with the usually
OO programming patterns how you program these things,
these things simple don't work anymore...

[Paulo Moura]

Hi,

> On 12 Jun 2018, at 17:26, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> One thing I did forget as one of my experiences is fact that you catch
> and re-throw many exceptions.

That is done *only* for top-level goals and the <</2 debugging control construct. The main reasons for doing it are:

1. Portability. Several Prolog systems don't provide standard compliant exceptions. That have improved recently for some (e.g. me and Theresa Swift fixed it for XSB a few years ago). Something that I will eventually revisit.

2. To provide exception context arguments using the same predicate names the user used in the source code instead of internal names that result from the compilation process. I understand that the backtrace mechanism can take care of this for SWI-Prolog. In fact, there's commented out code for that in the current Logtalk version that uses this mechanism.

> That breaks one of the nice features of
> SWI-Prolog to print a backtrace on uncaught exceptions as well as to
> start the debugger when asked to (default when interactive). That
> saves a lot of time during debugging.

I fully understand the value of backtraces and I have looked into it a couple of times. The requirement that the exception is not touched in any way for backtraces to work, however, conflicts with the points above. I'm yet to find an acceptable solution that would not be too cumbersome and would not break portability.

Cheers,
Paulo

[Jan Wielemaker]

I think you can do this using the SWI-Prolog exception hook. You can
put that inside the SWI-Prolog adapter. Then the only thing you need
is some way to specify the catch/rethrow code that you can change from
the adapter to do nothing. It would be one more step.

Cheers --- Jan

[Proof Easel]

Hi Jan W.,

You invented already OO via ISO modules. Maybe you never
tried it. But it seems to work, exactly as I suggest to use
reexport/1 for inheritance.

I am not taking my PythonStyleCall, but your DictStyleCall.
It is documented here. In your DictStyleCall the receiver
is always the but last argument, and you also assume a
last argument because you only think about functions:

     <module>:<name>(Arg1, ..., +Dict, -Value)
     http://www.swi-prolog.org/pldoc/man?section=ext-dict-user-functions

So lets make an inheritance example:

File subject.pl:

:- module(subject, [name/2]).
M.name() := N :-
   N is M.name.

File person.pl:

:- module(person, []).
:- reexport(subject).

And here is the result:

?- X = person{name:123}.name().

X = 123.

Works perfectly, or did I miss something. The only difference to my
OO-system, is that the PythonStyleCall can be used for both predicates
and functions, respectively the PythonStyleCall doesn't care what
mode the predicate has. You simply put the receiver in the first argument.

Then my OO-system didn't sacrifice ISO core standard compatibiity.
I didn't change consing, i.e. ('.')/2 is still the consing because I use
(::)/2 for message sending. Also in the PythonStyleCall you don't
need (:=)/2 to define a method, neither () in a call.

So my PythonStyleCall is less intrusive to the ISO core standard,
than your DictStyleCall. But its the same idea, as you have documented
in the link above, to use ordinary ISO module standard (:)/2 operator
for message dispatch.

Best Regards

[Proof Easel]

We do not have dict proper in my system, but we can 
simulate some of the dict functionality, and get 
the same result:

?- person(name,123)::name(X).
X = 123

I uploaded all the test sources, showing that inheritance
works, to Gist. Proof of concept for SWI-Prolog and
my system. There are also some screenshots:

    reexport/1 for inheritance in SWI-Prolog
    https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2620313

The operator (::)/2 can also do the dict functionality,
since for a term object which is a receiver, when extracting
the module from the given receiver term object, the arity

of the receiver term object is irrelevant. So I can also
pass arbitrary long dicts as a key value pairs coded
in the arguments of compound. But I do not have a

dict syntax. But basically the (::)/2 operator from my
system, can do the same as your (.)/2 operator. But my
(::)/2 operator does not involve any term expansion. Its

an ordinary builtin predicate. What I wonder is whether Logtalk
would support dicts in anyway, or whether it has already
such a term object data structure?

Disclaimer: I hope my SWI-Prolog and inheritance
as reexport/1 testing is correct, and the result is not
some artefact of the user context.

[Jan Wielemaker]

On 06/15/2018 02:10 AM, Proof Easel wrote:
> You invented already OO via ISO modules. Maybe you never
> tried it. But it seems to work, exactly as I suggest to use
> reexport/1 for inheritance.

I know that, although I must admit I never seriously used it. I am
pretty sure you can define a quite reasonable OO system on top of
modules with only a shallow layer that maps nice OO terminology to the
SWI-Prolog interface (including the low level interface that allows for
multiple modules in a single file and modules inheriting implementations
from other modules). You need something to maintain a notion of the
class context. I once implemented that by inspecting the environment
stack, but that is rather hacky and can become slow.

Cheers --- Jan

[Proof Easel]

Concerning what you wrote, you would need to define
what you mean by class context, and what you want to
do with it. In both PythonCallStyle and DictCallStyle,

you have the class of the receiver available from the
receiver, and you can do further message sending to
the receiver just by ordinary message sending.

In both PythonCallStyle and DictCallStyle the self is
just a formal parameter you can give any name you
want. In your DictCallStyle examples you use M.

Python typically uses "self", but its just an ordinary
formal parameter of a routine (or a function pointer).
Here is an example in your DictCallStyle setting calling self.

To illustrate my point I renamed your M to Self, its
based on your M here:

  M.multiply(F) := point{x:X, y:Y} :-
        X is M.x*F,
        Y is M.y*F.

     http://www.swi-prolog.org/pldoc/man?section=ext-dict-user-functions

To show some self calling, I blowed up the example
and accesed x and y by extra functions insted by the dict
accessor:
   

    Self.multiply(F) := point{x:X, y:Y} :-
          X1 = Self.x(), X is X1*F,
          Y1 = Self.y(), Y is Y1*F.

    Self.x() := X :-
           X is Self.x.

    Self.y() := Y :-
           Y is Self.y.

Here is the reult when you run it:

    Welcome to SWI-Prolog (threaded, 64 bits, version 7.7.15)
    SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

    ?- X = point{x:1, y:2}.multiply(2).
    X = point{x:2, y:4}.

Thats the same in Python. self is just a function formal
parameter (and of course also actual parameter), which 
could also give other names.

In my system I adopted the Python approach. Your
Dict approach has a different argument order and is
not directly compatible with the Python approach.

Here is the Python approach described:

      Python decided to do methods in a way that makes the instance to
      which the method belongs be passed automatically, but not received
      automatically: the first parameter of methods is the instance the
      method is called on.
     https://stackoverflow.com/a/2709832/502187

And here is that you can choose the name of self by yourself:

    That makes methods entirely the same as functions, and
    leaves the actual name to use up to you (although self is the
    convention, and people will generally frown at you when you use
    something else.) self is not special to the code, it's just
    another object.
     https://stackoverflow.com/a/2709832/502187

Thats a tremendous simplification. For automatic passing you
provide an operator, like in my system its the (::)/2 operator.
But automatic receiving you don't need anything.

The programmer just uses any name he likes for self. And
in PythonCallStyle its the first argument. In DictCallStyle its
effectively the last but one argument, but

you introduced (:=)/2. The (:=)/2 is not really necessary.
Especially if you would switch to PythonCallStyle since then
self is anyway the first argument, you get a nice reading.

If you would use PythonCallStyle you could also bind
foreign libraries more easily. The PythonCallStyle makes also
the FFI more easily. I use the PythonCallStyle to invoke

Java foreign functions that are virtual methods.

[Proof Easel]

For self sending you just use the receiver variable name,
that you can decide on your own. Works like a charm.

I have made a picture of the SWI-Prolog run, and also
uploaded the point.pl source on Gist:

https://gist.github.com/jburse/98796d20e58bbd6d5474dad5135b5bff#gistcomment-2620758

Works also in my system analogously:

?- point(x,1,y,2)::multiply(2,X).
X = point(x,2,y,4)

The self sending is inside the multiply method. You need
to look at the source code on Gist.

But its all implement Pythonesk.

[Proof Easel]

Admitedly everything is right before your nose, but gets
obsfuscated and restrained by the (:=)/2 operator. The
(:=)/2 operator assumes you want to define a function.

But so far all programming languages, modern ones,
Java, Python, etc.., do not restrict methods to functions.
A method can have return type:

a) void, like a subroutine
b) boolean, like a test
c) some value, like a function
d) value1, value2, what we like in Prolog returning multiple arguments,
     a predicate with a mode thta has multiple outputs.

With (:=)/2 OO gets awfully restricted. You assume you
would use methods only for c). If you throw (:=)/2 over
board (I don't say you should do it, maybe you have already

a user base using), but just mentally, to see what is right
before your nose, what was originally wrapped in (:=)/2
will straight away look like an ordinary predicate, and it is

an ordinary predicate. Just compare:

With (:=)/2 you got (and DictStyleCall last but one is the receiver):

Self.multiply(F) := point{x:X, y:Y} :-

    ...

Without (:=)/2 you can get (and PythonCallStyle first argument is the receiver):

multiply(Self, F, point(x, X, y, Y)) :-
    ...

With PythonCallStyle you easily get c), i.e. a function. But you
also get easily a) by using a predicate that always succeeds,
maybe with some side effect. With PythonCallStyle you can

further obviously also easily do b) by using a predicate that
succeeds or fails, you can use that as a test. And you can
do d), nobody prevents you from using more than one

argument as output. In as far with a), b), c) and d) you
cover all of Logtalk I guess. Logtalk cannot offer more,
because Prolog cannot do much more.

Logtalk has some further method annotations, co-inductive
and threaded or somesuch. If you want such stuff, it could
be the case that there are also Pythonesk solutions around,

that can be invented or adapted for Prolog. Not sure,
didn't try yet. Such things are also not available in Java. Whats
available in Java is marking a method synchronized. But

you already got with_mutex/2 in SWI-Prolog. So this is
not really an issue...

[Paulo Moura]

Hi Jan,

> On 15 Jun 2018, at 11:51, Jan Wielemaker <j...@swi-prolog.org> wrote:
>
> On 06/15/2018 02:10 AM, Proof Easel wrote:
>> You invented already OO via ISO modules. Maybe you never
>> tried it. But it seems to work, exactly as I suggest to use
>> reexport/1 for inheritance.
>
> I know that, although I must admit I never seriously used it. I am
> pretty sure you can define a quite reasonable OO system on top of
> modules with only a shallow layer that maps nice OO terminology to the
> SWI-Prolog interface (including the low level interface that allows for
> multiple modules in a single file and modules inheriting implementations
> from other modules).

It's always a good idea to try something more than Jan Burse's simplistic and misleading examples. Consider:

----- m1.pl -----
:- module(m1, [p/1]).
p(m1).
-----------------

----- m2.pl -----
:- module(m2, [q/1]).
:- reexport(m1).
q(m2).
-----------------

----- m3.pl -----
:- module(m3, [r/1]).
:- reexport(m2).
r(m3).
p(m3).
-----------------

?- [m1, m2, m3].
Warning: /Users/pmoura/Desktop/m3.pl:4:
Local definition of m3:p/1 overrides weak import from m1
ERROR: import/1: No permission to import m3:p/1 into user (already imported from m1)
true.

This can not be solved by having an empty export list in the module/2 directives given the reexport/1 directive semantics.

The Logtalk version:

----- o1.lgt -----
:- object(o1).

:- public(p/1).
p(o1).

:- end_object.
-----------------

----- o2.lgt -----
:- object(o2, extends(o1)).

:- public(q/1).
q(o2).

:- end_object.
-----------------

----- o3.lgt -----
:- object(o3, extends(o2)).

:- public(r/1).
r(o3).

p(o3).

:- end_object.

-----------------

?- {o1, o2, o3}.
% [ /Users/pmoura/Desktop/troll/o1.lgt loaded ]
% (0 warnings)
% [ /Users/pmoura/Desktop/troll/o2.lgt loaded ]
% (0 warnings)
% [ /Users/pmoura/Desktop/troll/o3.lgt loaded ]
% (0 warnings)
true.

?- o1::p(X).
X = o1.

?- o2::p(X).
X = o1.

?- o3::p(X).
X = o3.

> You need something to maintain a notion of the
> class context.

Which, btw, goes beyond just "self"...

> I once implemented that by inspecting the environment
> stack, but that is rather hacky and can become slow.

Cheers,
Paulo

[Jan Wielemaker]

Hi Paulo,

On 06/15/2018 04:04 PM, Paulo Moura wrote:
> It's always a good idea to try something more than Jan Burse's
> simplistic and misleading examples. Consider:

I don't think reexport/1 is the right building block. Module defaulting
as managed by add_import_module/3 delete_import_module/2 are more
promising. That is what I used in a small in-project implementation. I
used a goal that was not subject to last-call optimization to do what
you do using additional arguments. If these additional arguments were
needed I used the stack inspection predicates.

With that technology it is most likely possible to emulate part of
Logtalk in a lightweight manner where methods remain just predicates and
Logtalk objects are just modules. That has some advantages as all native
(development) tools keep working without any additional work.

That doesn't really give you Logtalk though. The whole design phylosophy
is rather different (as we discussed) and there are also the Logtalk
libraries, documentation, tutorials, etc.

I think it is wise end this comparison. Logtalk has its merits and as we
have seen also its limits. The same holds for plain (SWI-)Prolog. Let
people make up their mind and decide what is important for their
project.

Cheers --- Jan

[Paulo Moura]

> On 11 Jun 2018, at 16:21, Proof Easel <janb...@easel.ch> wrote:
>
> You call others trolling, when you don't have any arguments
> anymore? Thats ad hominem.

*I* have nothing to prove. Logtalk is open source, well documented, comes with ~130 examples, its design and implementation is described in my PhD thesis and several published papers, both reviewed by academic peers. Anyone can try if for free, anonymously, read the sources, run the examples, ...

*You*, on the other hand, have *years* of bashing Logtalk and wild speculation on how you can do Logtalk features with just modules features. What's the download URL for your solution again?

I'm also tired of proving you wrong. But let's do it again just for fun. Twice.

> On 11 Jun 2018, at 22:19, Proof Easel <janb...@easel.ch> wrote:
>
> In general the book discusses abstract datatypes
> and dynamic binding. But Logtalk even doesn't
> provide a single abstract datatype example
>
> in its documentation. You can try yourself:
>
> https://www.google.ch/search?q=Logtalk+abstract+datatype
>
> Not a single hit. Whats wrong?

Abstract data type is a theoretical concept that is commonly materialized in programming languages using interfaces/protocols. In Logtalk, protocols are first-class entities. The Logtalk user manual have a full section on protocols. The Logtalk library and the examples contains several protocols.

What's wrong with *you*?

> On 11 Jun 2018, at 16:21, Proof Easel <janb...@easel.ch> wrote:
>
> I tried once to find in Logtalk a single virtual
> method. I didn't find one.

Examples that make use of virtual methods in the Logtalk distribution include:

https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/carengines
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/classmethods
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/diamonds
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/instmethods
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/metaclasses
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/patching
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/people
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/points
https://github.com/LogtalkDotOrg/logtalk3/tree/master/examples/prototypes

Of course, there's also the "Inheritance" section in the user manual:

https://logtalk.org/manuals/userman/inheritance.html

> Its all modules, not on steriod,
> rather on valium.

I would say that readers with the patience to follow this thread have a pretty good idea who's skipping his medication.

> Even the date library doesn't use
>
> virtual methods, you couldn't plug in a mayan calendar.

The "date" object in the Logtalk library implements a "datep" protocol:

https://github.com/LogtalkDotOrg/logtalk3/blob/master/library/datep.lgt
https://github.com/LogtalkDotOrg/logtalk3/blob/master/library/date.lgt

> I call Logtalk rather a fraud.

Unlikely other public places where you rush into deleting your own words everytime you put your foot in your month, these ones from you will endure.

> It doesn't perform for virtual
> methods and has never had a use case

I guess all the third-party papers, contributions, and industrial sponsorship listed at:

https://logtalk.org/documentation.html

are from a parallel reality. Welcome to the multiverse!

Bye bye troll,
Paulo

[Proof Easel]

The below is not an example of virtual method call. It seems
you don't know what a virtual method call is. Are you sure
you understand whats a polymorphy virtual method.

Your example is only a super call:

File classic.pl

:- module(classic, [horsepower_hp/2, horsepower_rpm/2]).
_.horsepower_hp() := 94 :- true.
_.horsepower_rpm() := 4800 :- true.

File sport.pl:

:- module(sport, [horsepower_hp/2, horsepower_rpm/2]).
:- use_module(classic).

M.horsepower_hp() := HP :-
   classic:horsepower_hp(M, ClassicHP),
   HP is truncate(ClassicHP*1.23).

M.horsepower_rpm() := RPM :-
   classic:horsepower_rpm(M, ClassicRPM),
   RPM is truncate(ClassicRPM*0.762).

Here is an example run:

Welcome to SWI-Prolog (threaded, 64 bits, version 7.7.1)

SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

?- HP = sport{}.horsepower_rpm().

HP = 3657.

?- HP = sport{}.horsepower_hp().

HP = 115.

Why would one need Logtalk for that? Super calls already
work in plain SWI-Prolog Version 7 by and also together
with its DictCallStyle. The only problem of DictCallStyle is

that it cannot return multiple values. I already wrote about
that, and suggested to switch to PythonCallStyle and abandon
DictCallStyle or provide additionally PythonCallStyle.

[Proof Easel]

Anyway, you can call me troll a million times. Nobody cares.
It only shows that Paulo Moura is unprofessional, and not
interested in solving problems efficiently.

You already explained that performance is not on your
agenda. On the other hand performance is on my agenda.

The problem of multiple return values, which your Logtalk car
example can be a running example for, can be solved by switching
from (:=)/2 and DictCallStyle to PythonCallStyle. Here is the

same example with PythonCallStyle:

File classic.p:

:- module(classic, [horsepower_hp_rpm/3]).
horsepower_hp_rpm(_, 94, 4800).

File sport.p:

:- module(sport, [horsepower_hp_rpm/3]).
:- use_module(classic).

:- override horsepower_hp_rpm/3.
horsepower_hp_rpm(M, HP, RPM) :-
   classic:horsepower_hp_rpm(M, ClassicHP, ClassicRPM),
   HP is truncate(ClassicHP*1.23),
   RPM is truncate(ClassicRPM*0.762).

And this is the result:

  Jekejeke Prolog 2, Runtime Library 1.2.7
  (c) 1985-2018, XLOG Technologies GmbH, Switzerland

  ?- sport::horsepower_hp_rpm(HP, RPM).
  HP = 115.0,
  RPM = 3657.0

PythonCallStyle works also with more advanced stuff,
polymorphic virtual methods, self calls, etc.. For more
advanced stuff would need a better example

than your car example. And example with a real taxonomy,
and the real occurence of virtual method calls. The above is
only a super call where we mention the super class explicitly.

I have uploaded screenshots of the results, and also
the source code for both SWI-Prolog and my system:
https://gist.github.com/jburse/01cb2a3a63e5afcbaf6dee2fba06a44b#gistcomment-2620906

[Proof Easel]

Goal stack inspection doesn't give you self in a reliable way. An OO
call can happen any time. You can invoke from any call-site an OO call.
You need to be able to forget the goal stack, for last call optimization.
So if you try to get self from the Goal stack,

this will break last call optimization.

There is nothing more simpler or lightweight than the PythonCallStyle.
It is used by thousend of python programs, that use classes. If you
have deterministic recursive self calls, last call optimization possibly
amortizes the extra argument.

But I dont have some measurement right now.

For such measurement we would need a big example, of OO code.
Which is a big taxonomy, and preferably also a lot of virtual calls.
I checked Paulo Mouras first car example, and it was not a
virtual call, but something else.

I am not interested in checking more of Paulo Mouras examples,
especially since he called me a troll. Its of no use to take
Paulo Moura seriously. But fact is that in the PythonCallStyle
methods are just predicates with an extra argument.

To get more experience of this approach bigger examples would
be needed. Of course comparison with Logtalk could also be
done. But it would be more interesting to compare OO inside
Prolog with OO inside Prolog, than OO inside Prolog with

OO on top of Prolog. OO on top of Prolog will mostlike loose...

[Proof Easel]

Also if you get self from the goal stack, where will you store
it? Will you always retrieve it from stack? Assume you make
something intelligent, where self is salvaged during last call
optimization.

Then you still need to store it somewhere, after you have
slavaged it. Why not have it intially as a formal parameter, like
in PythonCallStyle? What comes to mind, if you would
try to optimize these things,

you could rewert to usual optimization in procedure
calls, where register files etc are used to pass paremeters
and somehow registers are reused. But this only of
interest if you would deal with assembler

But typically the extra parameter for self doesn't
hurt you so much in performance. See here:

?- time((between(1,500000,_), plain, fail; true)).
% Up 452 ms, GC 7 ms, Thread Cpu 454 ms (Current 06/09/18 15:12:50)
Yes

?- time((between(1,500000,_), obj::simple, fail; true)).
% Up 493 ms, GC 7 ms, Thread Cpu 468 ms (Current 06/09/18 15:12:52)
Yes

https://stackoverflow.com/a/50774703/502187

The performance overhead is from something else, which
is specific to the way I have implemented (::)/2. In SWI-Prolog
you could possibly implement (::)/2 a little better, without trying
to access the call stack in the method,

but simply in a very efficient way how you do the (::)/2 call
itself? You already have experience in SWI-Prolog from call/n
in these matters. You can use the same experience here, and
I am pretty sure you can beat the above figures.

P.S.: The self in the method clause is anyway a NOOP,
since it is underscore, this is already optimized away. The
example is an exemple where you really need to optimize
the (::)/2 call, no issue at all with the self in the method clause:

simple(_) :- fail.

Usually a Prolog interpeter optimizes away unification and
the underscore variable. At least in Jekejeke Prolog this gives
a NOOP for the head unification. So the problem is pimarily
(::)/2 and not the self variable.

In your dicts your (::)/2 is probably fast. Since in your dicts
you can access the module from the dict, by unpacting the
type of the dict. Which seems to be easy step. But still you
need to lookup the module.

[Proof Easel]

Strange question by Paulo Moura. The operator (::)/2
is part of the Jekejeke Prolog runtime library.
You can download it from here:

*Android Appstores:*
https://play.google.com/store/apps/details?id=jekpro.platform.headless

*Download:*
http://www.jekejeke.ch/idatab/doclet/prod/en/docs/05_run/05_down.jsp

The operator (::)/2 exists already for a couple of
releases. To be exact it was introduce as a native
built-in in the following release

Jekejeke Prolog 1.1.1 (Object-Orientation via ISO Moduls)
http://www.jekejeke.ch/idatab/doclet/intr/en/docs/5_ext/94_arc2016/099_releases/topic_999588.html

>> On 11 Jun 2018, at 16:21, Proof Easel <janb...@easel.ch> wrote:
>>
>> You call others trolling, when you don't have any arguments
>> anymore? Thats ad hominem.
>
> *I* have nothing to prove. Logtalk is open
> source, well documented, comes with ~130
> examples, its design and implementation is
> described in my PhD thesis and several
> published papers, both reviewed by academic
> peers. Anyone can try if for free, anonymously,
> read the sources, run the examples, ...
>
> *You*, on the other hand, have *years* of
> bashing Logtalk and wild speculation on how
> you can do Logtalk features with just modules
> features. What's the download URL for your
> solution again?

Am Freitag, 15. Juni 2018 16:49:02 UTC+2 schrieb Paulo Moura:

[Proof Easel]

I also don't understand why you want to shy away
from reexport/1. After all you need to lookup M:N
in both PythonCallStyle and DictCallStyle:

I would rather think about clause indexing:

- DictCallStyle: The passing of an OO call is translated
  from  X = Y.M(A1,..,An) to N:M(A1,..,An,Y,X) where N
  is the class of Y. For a very primitive Prolog system, as

  a last resort, which only has first argument indexing,
  it could still index A1. Thats probably the advantage of
  the DictCallStyle. It would work with very primitive

  Prolog systems, and could still take advantage
  of clause indexing.

- PythonCallStyle: The passing of an OO call is
  translated from Y::M(A1,..,An) to N:M(Y,A1,..,An) where
  N is the class of Y. This requires a more advanced

  Prolog system. Ideally the Prolog system decides on
  its own what kind of index it builds. And then since
  typically the self variable is in the first argument,

  it must have enough index building logic, to not unnessarely
  index this argument.

There are some Prolog systems that could probably already
deal with PythonCallStyle, since they have enough good
automatic indexing, what comes to mind are YAP, SWI, Jekejeke,

etc.. which all have just-in-time multi-argument indexing.
Multi-argument indexing could also index more than only
A1 in PythonCallStyle, depending on the sensitivity

further subsets or cascades of A2,..An could be index.
If the index is not just-in-time automatic, things get
annoying. One would need to manually direct indexes for

a Phytonesk like method predicate.

Am Freitag, 15. Juni 2018 16:24:36 UTC+2 schrieb Jan Wielemaker:

[Proof Easel]

The problem is for a virtual method call resolution,
you need more than "Module defaulting" as managed by
add_import_module/3 delete_import_module/2

I don't know what you mean by "Module defaulting" in
connection of a message dispatch. But the reexport/1
is exactly the right

semantics for message dispatch, take this statemet:

:- module(N, []).
:- reexport(K).

Lets say sig(N) is the export list of each module N.
Then tran_sig(N) is the visible predicates list (see
ISO core module standard) in each module. It has

the following algorithm

tran_sig(N) :
     Res := sig(N)
     for each reexport(K) of N
         Res := Res u tran_sig(K)
     return Res

So tran_sig is the transitive closure somehow
of what is visible. Thats exactly the semantics of
inheritance. Note when you have a call M:N,

you cannot default alone along the reexport chain,
by finding a M_0 alone. You need to find a pair M_0:N,
means M_0:N must be arealy a member of some

sig(N_0). So you cannot ignore the signature of each
involved module, otherwise you could not do overridng
of methods, and you OO-system would be a very weak

one. So any defaulting must know two things:
- The module M where to start the search
- The predicate N it needs to search the default for

reexport/1 does exactly this for you. An ISO core module
implementation, the (:)/2, can be implemented such that
it does the above search.

What helps more if (:)/2 can identifiy it self as pertaining
to some call site, then you can even do a little more
and support private, package locale, etc..

visibility rules.

[Proof Easel]

Over the last days some tests were conducted.
Paulo Moura insisted on testing a "with" solution.
The results were (both on SWI7):

   reexport/1 and (:)/2 on SWI "with" Solution:
      0.712 secs 
   Logtalk on SWI "with" Solution:
       0.812 secs

Complete test material was always tweeted. One gist
"with" solution, and other gist dict solution. With the
time to come more tests should be conducted with 


deeper and wider taxonomy, improved or novel
ISO module implementations. The latest report
can be found here:

Same "with" Solution SWI-Prolog 7 dispatcher beats Logtalk
https://plus.google.com/+JekejekeCh/posts/6YYKsVJ1ab9

[Proof Easel]

Reexport/1 and (:)/2 proof of concept: Who observed it first? I find for
example A. Pineda and M. Hermenegildo. O'ciao: An Object Oriented
Programming Model for (Ciao) Prolog. TR CLIP 5/99.0,
Facultad de Informática, UPM, July 1999.

[Proof Easel]

Last remark: In case somebody is critical
towards using (:)/2 with a first argument,
that is only determined at runtime.

Many Prolog systems will try to optimize
and have a good performance of their system
for this situation anyway.

Why?

Because under the hood of many attribute
variable implementations, here and then
also such a calls are used.

See also here when Ken Asked:
"Please tell me. Is there a DCG, CLP code
written within the ISO-Prolog standard?"
https://groups.google.com/d/msg/comp.lang.prolog/4_2WRm2EXCQ/VgU3XHHdAgAJ

My recommendation was and still is:
When you want CLP, try attribute
variables first. When you want attributed
variables, try modules first.

[Paulo Moura]

> On 21 Jun 2018, at 17:23, Proof Easel <janb...@easel.ch> wrote:
>
> Over the last days some tests were conducted.
> Paulo Moura insisted on testing a "with" solution.

I simply made available a sensible implementation (as your original Logtalk version used quite contrived code and had a code smell, which I mention below):

https://gist.github.com/pmoura/153a4dde11e12b2d820ab35acacd6e9d

> The results were (both on SWI7):
>
> reexport/1 and (:)/2 on SWI "with" Solution:
> 0.712 secs
> Logtalk on SWI "with" Solution:
> 0.812 secs

The percentage difference is what I also observed (in macOS) *and* expected (given the characteristics of the SWI-Prolog VM). I don't expect a meaningful difference in other Prolog VMs but I was only able to load Jan Burse Prolog version on YAP (there his Prolog version is 6x slower than the Logtalk version but, given the current beta status of YAP, it's not wise to draw conclusions from those numbers). The Prolog version doesn't load (i.e. we get compilation errors) in ECLiPSe 7.0 #41, SICStus Prolog 4.4.1 or XSB 3.8.0. Of course, the Logtalk version works as-is on all supported Prolog systems.

> Complete test material was always tweeted. One gist
>
> "with" solution, and other gist dict solution. With the time t
>
> o come more tests should be conducted with
>
>
>
> deeper and wider taxonomy, improved or novel
>
> ISO module implementations. The latest report
>
> can be found here:
>
>
>
> Same "with" Solution SWI-Prolog 7 dispatcher beats Logtalk

The solutions are indeed closer but not the same. In the Logtalk solution, which uses parametric objects, ancestor objects do not restrict the parameter positions in descendant objects. In the Prolog solution, ancestor modules dictates the order for the equivalent to the Logtalk parameters. Jab Burse previous Prolog version used SWI-Prolog proprietary dicts to be parameter order free. But, in my tests, that solution was significantly slower than the Logtalk version. The order issue can, however, be considered a minor code smell.

> https://plus.google.com/+JekejekeCh/posts/6YYKsVJ1ab9

Thanks for the interesting problem and the experiments with different solutions. Notably, allowing to compare a SWI-Prolog version based on dicts (which are proprietary) with a Logtalk version based on parameters (which is portable as parameters are *logical variables*).

Bye,
Paulo

> --
> You received this message because you are subscribed to the Google Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to swi-prolog+...@googlegroups.com.
> Visit this group at https://groups.google.com/group/swi-prolog.
> For more options, visit https://groups.google.com/d/optout.

[Proof Easel]

I have no clue how the SWI-Prolog VM works. And why its
that fast. Also sofar the taxonomy was very small, I think
more tests should be made, and maybe even problems
with the SWI-Prolog VM can be dedected.

That other Prolog systems cannot do it in the same time
as SWI-Prolog, or even cannot run it at all, I also didn't
expect that. I simply didn't know. Maybe I should also do
some testing, and look at some of the compilation errors.

Logtalk could provide a "ISO module" option, and for
those Prolog systems that have a good VM, it could do
a different compilation and directly compile to "ISO
modules". Maybe this would be also an

incentive to Prolog system vendors to advance their
module system. Possibly this would be only a Logtalk
subset that can work with this option for the time beeing.
Current module systems are most of the time not

advanced enough  to support all visibility modifiers.
Maybe in general the ISO module standard should
be a little bit be revived,

Begin usual Ulrich Neumerkel bash:
I dont think something from Vienna is to expect there,
they are in a very deep sleeping mode. You here
the snoring day and night. They only wake up when
some syntax error is made.
End usual Ulrich Neumerkel bash.

Begin usual Ulrich Neumerkel bash II:
Or they wake up, when some new error message
come to their mind, like length(L,L), etc.. only nonsens
nothing useful to build large scale Prolog system.
Sounds good, large scale Prolog systems, isn't it?
End usual Ulrich Neumerkel bash II.

[Proof Easel]

Paulo Moura wrote:
"In the Prolog solution, ancestor modules dictates the order for
the equivalent to the Logtalk parameters."

Not really. It depends how you access the parameters. When
you are at it to override some with/n methods object state changer,
you could also provide some xx/n methods and overriding
for object state accessors.
 
Thats probably what you think is smelly in my original first
dict solution. But there is nothing smelly. The granularity
is only much smaller what the with/n methods concerns,
there are a couple of set/x methods,

and I also realized accessor methods. The result of the
smaller granularity is of course that you need to updated
the state in more steps and that you get more intermediate
variables. But there is nothing smelly with it.

The only smelly thing I saw was your light yellow screenshot,
which looked a little bit like piss color.

[Proof Easel]

If you compare two solutions, that use the same setter and
getter granularity, also if the setter and getter don't do much,
you are comparing apples with apples. With was my first test
case, I only used =... for the setter, which turned out to

be a  speed brake for SWI-Prolog. But there is nothing wrong
with that. In Java and other programming languages, you often
code setters and getters, even if they don't do a lot. You do
this for various reasons:

- You can use the visibility modifiers of the setters and
   getters to regulate what can go out and in of an object.

 - This probably doesn't completely translate to parametric
   objects, which have their equivalent in Java and Scala
   as so called value objects. Usually value objects are
   immutable, so they don't have usually setters.

- But by using the name with/n to make a copy (in your case
  for a parametric object and not a map), you are in
  good company, this is found for example in Erlang:

   with(Ks, Map1) -> Map2
   http://erlang.org/doc/man/maps.html

- In usually OO-programming, some setters or getters
  might exists, so that subclasses override them. We
  might even see example code, where some setters
  and getters are declared abstract or from an interface.

- The later is no so much required for Prolog VM,
  since the dynamic dispatch doesn't need types. But
  we would need bigger test examples to test that.
  Theoretically I can call anything anytime with any
  method I desire. This is even more powerful than
  duck typing as found in the Go programming language.

What I am planning to do more on testing, is to test the
colorpoint problem against Java and Scala. Testing against
Scala is especially interesting, since they have pattern
matching, which is in competition with Prolog rules.

Writing a similar value object based code for Java and
Scala could give empirical results what is theoretical
possible, and found at the formost front of virtual methods.
One could of course also try C++ or Python variants

and compare. This would probably give a lower bound
on what a Prolog VM can practically reach in the future,
concering the ISO module required dispatch.

[Proof Easel]

Ok, I could already complete the Java comparison,
Java is kind of 50-times faster for the color point example:

  colorpoint(1,2,0)
  colorpoint(3,6,255)
  cpcount 1500003, time 16 (ms)

For more information see here:

ColorPoint in Java indicative of what future Prolog VMs could do
https://plus.google.com/+JekejekeCh/posts/G1Pa1EhtXJh

[Paulo Moura]

> On 21 Jun 2018, at 23:02, Proof Easel <janb...@easel.ch> wrote:
>
> Paulo Moura wrote:
> "In the Prolog solution, ancestor modules dictates the order for
> the equivalent to the Logtalk parameters."
>
> Not really. It depends how you access the parameters. When
> you are at it to override some with/n methods object state changer,
> you could also provide some xx/n methods and overriding
> for object state accessors.
>
> Thats probably what you think is smelly in my original first
> dict solution. But there is nothing smelly. The granularity
> is only much smaller what the with/n methods concerns,
> there are a couple of set/x methods,
>
> and I also realized accessor methods. The result of the
> smaller granularity is of course that you need to updated
> the state in more steps and that you get more intermediate
> variables. But there is nothing smelly with it.

Let's try it then by adding:

------ namedcolorpoint.pl ------------
:- module(namedcolorpoint, []).
:- reexport(colorpoint).
:- use_module(swiobj).

with_color(C, namedcolorpoint(N,C,X,Y), namedcolorpoint(N,_,X,Y)).

with_position(X, Y, namedcolorpoint(N,C,X,Y), namedcolorpoint(N,C,_,_)).
--------------------------------------

I.e. namedcolorpoint = name + color + point (position). If we just add this module we get, as expected:

?- namedcolorpoint(n,0,1,2)::multiply(3,X), X::invert(Y).
ERROR: Arithmetic: `n/0' is not a function
ERROR: In:
ERROR: [11] _3672 is n*3
ERROR: [10] point:multiply(3,_3712,namedcolorpoint(n,0,1,2)) at /Users/pmoura/Desktop/points_v3/point.pl:14
ERROR: [9] swiobj:namedcolorpoint(n,0,1,2)::multiply(3,_3772) at /Users/pmoura/Desktop/points_v3/swiobj.pl:5
ERROR: [8] '<meta-call>'(user:(...,...)) <foreign>
ERROR: [7] <user>

We may be tempted to simply override the exported multiply/3 predicate:

------ namedcolorpoint.pl ------------
:- module(namedcolorpoint, []).
:- reexport(colorpoint).
:- use_module(swiobj).

multiply(F, New, Self) :-
arg(3, Self, X1),
arg(4, Self, Y1),
X is X1 * F,
Y is Y1 * F,
Self::with_position(X, Y, New).

with_color(C, namedcolorpoint(N,C,X,Y), namedcolorpoint(N,_,X,Y)).

with_position(X, Y, namedcolorpoint(N,C,X,Y), namedcolorpoint(N,C,_,_)).
--------------------------------------

This, of course, also doesn't work:

?- [swiobj, point, colorpoint, namedcolorpoint].
Warning: /Users/pmoura/Desktop/points_v4/namedcolorpoint.pl:12:
Local definition of namedcolorpoint:multiply/3 overrides weak import from point
ERROR: import/1: No permission to import namedcolorpoint:multiply/3 into user (already imported from point)

Same problem I reported in June 15 in this mailing list: reexport/1 re-exports all exports predicates and we cannot export the same predicate twice to the same context. So, let's do as you suggest and add xx/n methods:

---------------- point.pl ------------
:- module(point, [multiply/3, x/2, y/2]).
:- use_module(swiobj).

multiply(F, New, Self) :-
Self::x(X1),
Self::y(Y1),
X is X1 * F,
Y is Y1 * F,
Self::with_position(X, Y, New).

x(X, Self) :-
arg(1, Self, X).
y(Y, Self) :-
arg(2, Self, Y).

with_position(X, Y, point(X,Y), _).
--------------------------------------

This works nice for point and colorpoint:

?- point(1,2,0)::multiply(3,X).
X = point(3, 6).

?- colorpoint(1,2,0)::multiply(3,X), X::invert(Y).
X = colorpoint(3, 6, 0),
Y = colorpoint(3, 6, 255).

But for namedcolorpoint we need to override the x/2 and y/2 predicates. But we can't as they're exported and if we try we get the same error as above:

?- make.
% Updating index for library /Users/pmoura/lib/swipl-7.7.16/library/
Warning: /Users/pmoura/Desktop/points_v5/namedcolorpoint.pl:12:
Local definition of namedcolorpoint:x/2 overrides weak import from point
Warning: /Users/pmoura/Desktop/points_v5/namedcolorpoint.pl:14:
Local definition of namedcolorpoint:y/2 overrides weak import from point
ERROR: import/1: No permission to import namedcolorpoint:y/2 into user (already imported from point)
ERROR: import/1: No permission to import namedcolorpoint:x/2 into user (already imported from point)

Where do we go from here? Back to using a dict as in your original version to, like in the Logtalk version, be free of the constraints of parameter position? But that solution is slower than the Logtalk version, nullifying your claim. Maybe we can add instead x/2 and y/2 as non-exported predicates... let's try it:

?- point(1,2,0)::multiply(3,X).
X = point(3, 6).

?- colorpoint(1,2,0)::multiply(3,X), X::invert(Y).
ERROR: Undefined procedure: colorpoint:x/2
ERROR: In:
ERROR: [12] colorpoint:x(_4804,colorpoint(1,2,0))
ERROR: [11] swiobj:colorpoint(1,2,0)::x(_4858) at /Users/pmoura/Desktop/points_v6/swiobj.pl:5
ERROR: [10] point:multiply(3,_4890,colorpoint(1,2,0)) at /Users/pmoura/Desktop/points_v6/point.pl:12
ERROR: [9] swiobj:colorpoint(1,2,0)::multiply(3,_4946) at /Users/pmoura/Desktop/points_v6/swiobj.pl:5
ERROR: [8] '<meta-call>'(user:(...,...)) <foreign>
ERROR: [7] <user>
Exception: (12) colorpoint:x(_4072, colorpoint(1, 2, 0)) ?

No, it also doesn't work. Clearly I failed to implement your suggestion of adding add accessor xx/n methods. Can you show us how so that we can learn?

> The only smelly thing I saw was your light yellow screenshot,
> which looked a little bit like piss color.

Classy.

> Am Donnerstag, 21. Juni 2018 23:55:13 UTC+2 schrieb Proof Easel:

> ...

> Begin usual Ulrich Neumerkel bash:
> I dont think something from Vienna is to expect there,
> they are in a very deep sleeping mode. You here
> the snoring day and night. They only wake up when
> some syntax error is made.
> End usual Ulrich Neumerkel bash.
>
> Begin usual Ulrich Neumerkel bash II:
> Or they wake up, when some new error message
> come to their mind, like length(L,L), etc.. only nonsens
> nothing useful to build large scale Prolog system.
> Sounds good, large scale Prolog systems, isn't it?
> End usual Ulrich Neumerkel bash II.

Another classy act.

> Am Donnerstag, 21. Juni 2018 20:15:35 UTC+2 schrieb Paulo Moura:
> > The results were (both on SWI7):
> >
> > reexport/1 and (:)/2 on SWI "with" Solution:
> > 0.712 secs
> > Logtalk on SWI "with" Solution:
> > 0.812 secs
>
> The percentage difference is what I also observed (in macOS) *and* expected (given
> the characteristics of the SWI-Prolog VM). I don't expect a meaningful difference in
> other Prolog VMs but I was only able to load Jan Burse Prolog version on YAP (there
> his Prolog version is 6x slower than the Logtalk version but, given the current beta
> status of YAP, it's not wise to draw conclusions from those numbers). The Prolog
> version doesn't load (i.e. we get compilation errors) in ECLiPSe 7.0 #41, SICStus
> Prolog 4.4.1 or XSB 3.8.0. Of course, the Logtalk version works as-is on
> all supported Prolog systems.
>
>

[Falco Nogatz]

Please stop attacking anyone. It makes our lovely community unnecessarily toxic.

Related reading recommendation, just of today: https://www.destroyallsoftware.com/blog/2018/a-case-study-in-not-being-a-jerk-in-open-source

Am Donnerstag, 21. Juni 2018 23:55:13 UTC+2 schrieb Proof Easel: