Runing SWISH with rserve_client on localhost (Linux Mint).

[nicos.ang...@gmail.com]

Dear all,

Does anyone have experience/advice for running Swish with Rserve on a local machine (Linux Mint in my case).

I have latest swipl-devel/ and swish/ and have installed rserve_client with pack_install(rserve_client).

The instructions in pack/rserve_client/README.md state:

"
 ## Using this package

This  package  is  primarily  intended  for    accessing   R  in  server
environments such as [SWISH](http://swish.swi-prolog.org).  We created a
[Docker container](https://github.com/JanWielemaker/rserve-sandbox) that
runs  Rserve  in  a  sandbox.  The  container  exposes  Rserve  using  a
Unix-domain socket at the following address:

    /home/rserve/socket

With SWISH and is interface  installed   in  adjacent directories, i.e.,
below the same parent, R may be linked to SWISH doing

    :- use_module(lib/r_swish).

Now, R is not safe. You should either run Rserve in a tight OS container
and load `library(r/r_sandbox)` or run SWISH   in  authenticated mode by
loading `lib/authenticate.pl.`

"

I found the above a bit difficult to digest

What I would like to do is just run the localhost:3050 via rul.pl  and have the Rserve example working
(http://localhost:3050/example/Rserve.swinb).
Currently I get syntax error  A <- 1:10 operator expected.
Sandboxing is not an issue for now.

Thanks in advance,

Nicos Angelopoulos
---
http://stoics.org.uk/~nicos


Btw, I spotted 2 typos:
With SWISH and is interface    -> With SWISH and ?this? interface
This is just a proof of context.   -> This is just a proof of concept.

[Fabrizio Riguzzi]

Dear Nicos,

installing rserve is not an easy task, a student of mine has created a swish-installer for that purpose

https://github.com/frnmst/swish-installer

see the manual there

He also prepared a bin version of reserve sandbox

https://gitlab.com/frnmst/rserve-sandbox-bin

Best

Fabrizio

--
You received this message because you are subscribed to the Google Groups "SWI-Prolog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swi-prolog+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/swi-prolog.
For more options, visit https://groups.google.com/d/optout.

[Jan Wielemaker]

On 05/11/2017 11:02 PM, nicos.ang...@gmail.com wrote:
>
> Dear all,
>
> Does anyone have experience/advice for running Swish with Rserve on a
> local machine (Linux Mint in my case).
>
> I have latest swipl-devel/ and swish/ and have installed rserve_client
> with pack_install(rserve_client).
>
> The instructions in pack/rserve_client/README.md state:

You started at the wrong end :( Since some months the configuration of
SWISH has been changed completely because the many things you may want
to configure required a more modular approach (although the old mostly
works, you are very close). After unpacking swish, go to the main dir
and do

% mkdir config-enabled
% cp config/available/r_serve.pl config-enabled

See comments in config-enabled/r_serve.pl for further steps. There is
lots of stuff in config-available that you can enable this way. The
README.md file there provides a brief overview. Details are in the
configuration files.

Note that swish uses something for which I eventually want a more
principled approach: connect packs to a single installed application
rather than to the entire Prolog installation. It does this by using
a local `pack` directory and git submodules to control the attached
packs.

Hope this helps

--- Jan

> --
> You received this message because you are subscribed to the Google
> Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to swi-prolog+...@googlegroups.com
> <mailto:swi-prolog+...@googlegroups.com>.

[nicos angelopoulos]

Dear Fabrizio,

Many thanks I will have a look soon.

I realised after posting that I could also possibly use Real as I am not that
concerned about sandboxing.

However being able to use rserve would be nice. Thanks to your student for setting those projects up.


Regards,

Nicos.

> > email to swi-prolog+...@googlegroups.com.

[nicos angelopoulos]

Dear Jan,

Many thanks for your reply.

> You started at the wrong end :( Since some months the configuration of
> SWISH has been changed completely because the many things you may want
> to configure required a more modular approach (although the old mostly
> works, you are very close). After unpacking swish, go to the main dir
> and do
>
> % mkdir config-enabled
> % cp config/available/r_serve.pl config-enabled
>

there is a tiny typo in the above, i think it should be:

cp config-available/r_serve.pl config-enabled

As an aside, there is a bit of an issue with user_profile.
On the latest swipl-devel/ and swish/ (on a Xubuntu 14.04), I see the following.

With or without config-enabled directory:

άμπελος;src/swish% swipl -f run.pl
ERROR: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
source_sink `library(user_profile)' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
Goal (directive) failed: swish_chat:use_module(library(user_profile))
% Started server at http://localhost:3050/
Welcome to SWI-Prolog (threaded, 64 bits, version 7.5.5-28-gf895284)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit http://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).


If i do:

% mkdir config-enabled
% cp config-available/user_profile.pl config-enabled/

% swipl -f run.pl
ERROR: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
source_sink `library(user_profile)' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
Goal (directive) failed: swish_chat:use_module(library(user_profile))
ERROR: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:39:
source_sink `library(user_profile)' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:39:
Goal (directive) failed: config_user_profile:use_module(library(user_profile))
ERROR: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:40:
source_sink `library(profile/backend/profile_prolog)' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:40:
Goal (directive) failed: config_user_profile:use_module(library(profile/backend/profile_prolog),[])
ERROR: /usr/local/users/na11/local/git/src/swish/lib/plugin/profile.pl:40:
source_sink `library(user_profile)' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/lib/plugin/profile.pl:40:
Goal (directive) failed: swish_plugin_user_profile:use_module(library(user_profile))
ERROR: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:44:
setting `backend' does not exist
Warning: /usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:44:
Goal (directive) failed: config_user_profile:set_setting(user_profile:backend,impl_profile_prolog)
ERROR: /usr/local/users/na11/local/git/src/swish/swish.pl:81:
/usr/local/users/na11/local/git/src/swish/config-enabled/user_profile.pl:45: Initialization goal raised
exception: '$run_init_goal'/1: Undefined procedure: config_user_profile:profile_open_db/1
% Started server at http://localhost:3050/
Welcome to SWI-Prolog (threaded, 64 bits, version 7.5.5-28-gf895284)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit http://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).



The standard swish server is working in both cases, I guess it is some "chat" function that is impaired ?.

> See comments in config-enabled/r_serve.pl for further steps.

The comments there are a bit terse and a tad confusing.
The fact that there are two alternative ways adds to the complexity.
I know that you are trying to keep 1 copy of all that config facts, but
splitting the commentary to method A and method B would be easier for the first time user.
In addition to sandbox or not dilemma there is also having root access or not on the machine.
I rarely install as root when compiling from source. On my work machine I don't have the option anyway.

> There is
> lots of stuff in config-available that you can enable this way. The
> README.md file there provides a brief overview. Details are in the
> configuration files.
>
> Note that swish uses something for which I eventually want a more
> principled approach: connect packs to a single installed application
> rather than to the entire Prolog installation.

This would be a nice feature also for static versions of required packages.
If the library expansion rules are module specific then loading should be easy enough.
You are already doing this, if you are picking local packs in Swish, right ?
For building, in addition to rules for the git case, it would be nice to have rules
for tgz files and specific/minimum versions of published packs (from the SWI pack server)

> It does this by using
> a local `pack` directory and git submodules to control the attached
> packs.
>
> Hope this helps
>
> --- Jan
>

Thanks it does. but not quite there yet; need to config Rserve and connect to it.

Many congrats for the excellent product SWISH has become.


Some other comments:

1. As far as I can see, Swish is not visible on SWI main page. Maybe at a minimum an additional link in the downloads
tab ?

2. Are there any long term plans for supporting other (interpreted) languages? Instead-of/in-addition-to Prolog that
is.

3. Do code comments from standard predicates automatically translate to Swish clickable queries?

4. Is there special markup to incorporate images generated by commented code ?
(Recently I rigged some crude marks with ever cruder code to include images on static html:
http://stoics.org.uk/~nicos/sware/b_real/doc/html/b_real.html#r_hist/2
In theory the doc server also supports images but the location is defined centrally i think)

5. Is there a way to have a visual history mechanism? (If user runs four 8-Queens queries and then closes the boards-
then
the history panel could navigate through the images with a simple fwd/bwd interface.)

6. can the config-enabled directory be placed somewhere outside the main installation ?
it is usually better not to mix the git sources with the site-specific stuff (similarly to Swi moving all site
specific things to $PREFIX/lib/swipl-Mj.Mn.Fx)

7. when a visual output is iconified (say the 8 queens chess board) ideally the "iconify" option in the iconified
window should read "expand" or some such and the graphical symbol be different ?

Regards,

Nicos

[nicos angelopoulos]

Dear Jan,

On Fri, 12 May 2017 12:55:11 +0200
nicos angelopoulos <nicos.ang...@gmail.com> wrote:


>
> As an aside, there is a bit of an issue with user_profile.
> On the latest swipl-devel/ and swish/ (on a Xubuntu 14.04), I see the following.
>
> With or without config-enabled directory:
>
> άμπελος;src/swish% swipl -f run.pl
> ERROR: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
> source_sink `library(user_profile)' does not exist
> Warning: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
> Goal (directive) failed: swish_chat:use_module(library(user_profile))

Please scrap that.

I believe this was due to web/bower_components missing (I made a clean install this morning, and forgot to get the .zip
file).

I now have installed Rserve via R and running it with vanilla call:

> library(Rserve)
> Rserve()
Starting Rserve:
/usr/local/users/na11/local/lib/R/bin/R CMD /usr/local/users/na11/local/lib/R/library/Rserve/libs//Rserve


R version 3.3.3 (2017-03-06) -- "Another Canoe"
....
Rserv started in daemon mode.


I changed rserve_client.pl to have configs:

% EDIT: COMMENT IF R IS NOT RUNNING IN AN OS SANDBOX.
% :- use_module(library(r/r_sandbox)).

% EDIT: Uncomment and edit these values if Rserve is accessible
% from another location. If a `socket` is configured and exists
% the socket is used. Otherwise it will try a TCP connection to
% the given host and port.
%
% IF R IS NOT RUNNING IN AN OS SANDBOX, COMMENT THE SANDBOX LINE
% ABOVE AND RUN SWISH WITH AUTHENTICATION SUPPORT.

%:- set_setting_default(rserve:socket, '/home/rserve/socket').
%:- set_setting_default(rserve:host, '127.0.0.1').
:- set_setting_default(rserve:host, 'localhost' ).
:- set_setting_default(rserve:port, 6311).


When executing R commands on SWISH I get:

No permission to call sandboxed `r_assign_(_1160,_1162,_1164)'
Reachable from:
r_call:r_assign(A,B,C)
r_call:r_bind(A)
r_call:'__aux_maplist/2_r_bind+0'(A)
r_call:r_execute(A,B,C)
<-(A,1:10)
swish_trace:swish_call( <-(A,1:10))
'$swish wrapper'(<-(A,1:10),B)

I suspect i need to run "SWISH WITH AUTHENTICATION SUPPORT" ?
Sandboxing is not important for me, I am testing at this stage. Can I by-pass it?
Also, I would also like to try running Real in Swish, what would be the simplest
way to tell SWISH to let it run <- commands ?



Regards,

Nicos

[nicos angelopoulos]

> % IF R IS NOT RUNNING IN AN OS SANDBOX, COMMENT THE SANDBOX LINE
> % ABOVE AND RUN SWISH WITH AUTHENTICATION SUPPORT.
>
> %:- set_setting_default(rserve:socket, '/home/rserve/socket').
> %:- set_setting_default(rserve:host, '127.0.0.1').
> :- set_setting_default(rserve:host, 'localhost' ).
> :- set_setting_default(rserve:port, 6311).
>
>
> When executing R commands on SWISH I get:
>
> No permission to call sandboxed `r_assign_(_1160,_1162,_1164)'
> Reachable from:
> r_call:r_assign(A,B,C)
> r_call:r_bind(A)
> r_call:'__aux_maplist/2_r_bind+0'(A)
> r_call:r_execute(A,B,C)
> <-(A,1:10)
> swish_trace:swish_call( <-(A,1:10))
> '$swish wrapper'(<-(A,1:10),B)
>
> I suspect i need to run "SWISH WITH AUTHENTICATION SUPPORT" ?

One way this can be done is by

1. adding

:- use_module('lib/plugin/http_authenticate').


after line
:- use_module(server).
in run.pl

2. on the first time run.pl is loaded
query with
?- swish_add_user.

See lib/plugin/http_authenticate.pl for details.


I have now a working local copy of (non-sandboxed, i assume) Swish with Rserve support.

Regards,

Nicos

[Jan Wielemaker]

Hi Nicos,

I was just about to start on your other mail. Thanks for the
observations. It is always good to get peoples first impressions
before they think the oddities are normal after all :)

On 05/12/2017 02:58 PM, nicos angelopoulos wrote:

>> With or without config-enabled directory:
>>
>> άμπελος;src/swish% swipl -f run.pl
>> ERROR: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
>> source_sink `library(user_profile)' does not exist
>> Warning: /usr/local/users/na11/local/git/src/swish/lib/chat.pl:62:
>> Goal (directive) failed: swish_chat:use_module(library(user_profile))
>
> Please scrap that.
>
> I believe this was due to web/bower_components missing (I made a clean install this morning, and forgot to get the .zip
> file).

I don't think that. Just did a fresh install on a different machine for
someone. Without the bower stuff you just get an almost empty page with
some scattered items. I did get the error above, which went away by running

git submodule update --init

Actually just initializing pack/profile does the job. Only, it should
work without.

> I now have installed Rserve via R and running it with vanilla call:
>
>> library(Rserve)
>> Rserve()
> Starting Rserve:
> /usr/local/users/na11/local/lib/R/bin/R CMD /usr/local/users/na11/local/lib/R/library/Rserve/libs//Rserve
>
>
> R version 3.3.3 (2017-03-06) -- "Another Canoe"
> ....
> Rserv started in daemon mode.

Good.

>
>
> I changed rserve_client.pl to have configs:
>
> % EDIT: COMMENT IF R IS NOT RUNNING IN AN OS SANDBOX.
> % :- use_module(library(r/r_sandbox)).
>
> % EDIT: Uncomment and edit these values if Rserve is accessible
> % from another location. If a `socket` is configured and exists
> % the socket is used. Otherwise it will try a TCP connection to
> % the given host and port.
> %
> % IF R IS NOT RUNNING IN AN OS SANDBOX, COMMENT THE SANDBOX LINE
> % ABOVE AND RUN SWISH WITH AUTHENTICATION SUPPORT.
>
> %:- set_setting_default(rserve:socket, '/home/rserve/socket').
> %:- set_setting_default(rserve:host, '127.0.0.1').
> :- set_setting_default(rserve:host, 'localhost' ).
> :- set_setting_default(rserve:port, 6311).

Ok.

> When executing R commands on SWISH I get:
>
> No permission to call sandboxed `r_assign_(_1160,_1162,_1164)'
> Reachable from:
> r_call:r_assign(A,B,C)
> r_call:r_bind(A)
> r_call:'__aux_maplist/2_r_bind+0'(A)
> r_call:r_execute(A,B,C)
> <-(A,1:10)
> swish_trace:swish_call( <-(A,1:10))
> '$swish wrapper'(<-(A,1:10),B)
>
> I suspect i need to run "SWISH WITH AUTHENTICATION SUPPORT" ?
> Sandboxing is not important for me, I am testing at this stage. Can I by-pass it?

Several ways. You were too obedient :) You followed this advice:

% EDIT: COMMENT IF R IS NOT RUNNING IN AN OS SANDBOX.

:- use_module(library(r/r_sandbox)).

You should not. If you leave this in, you can access Rserve. Note that
anyone with access to your machine can.

The alternative is to activate the auth_http_always.pl config and follow
the description to create a user. Next you need to login
before you get access to SWISH, but once you have access to swish,
you can do anything you can do in a normal Prolog shell.

Finally, you can load the file 'ide.pl' and run ?- swish. to start
SWISH. This also gives access without limitations. It is a little
experimental though.

> Also, I would also like to try running Real in Swish, what would be the simplest
> way to tell SWISH to let it run <- commands ?

Just use

:- use_module(swish:library(real)).

You probably need to disable the r_serve plugin as I doubt they will
cooperate. There are pros and cons between r_serve and real. In the
above case, threading may be an issue. Remember that each query in
SWISH is executed in a thread using a module created for executing
just that query. After the query finishes, both the thread and module
are reclaimed.

Cheers --- Jan

[Jan Wielemaker]

On 05/12/2017 03:30 PM, nicos angelopoulos wrote:
>
>
>> % IF R IS NOT RUNNING IN AN OS SANDBOX, COMMENT THE SANDBOX LINE
>> % ABOVE AND RUN SWISH WITH AUTHENTICATION SUPPORT.
>>
>> %:- set_setting_default(rserve:socket, '/home/rserve/socket').
>> %:- set_setting_default(rserve:host, '127.0.0.1').
>> :- set_setting_default(rserve:host, 'localhost' ).
>> :- set_setting_default(rserve:port, 6311).
>>
>>
>> When executing R commands on SWISH I get:
>>
>> No permission to call sandboxed `r_assign_(_1160,_1162,_1164)'
>> Reachable from:
>> r_call:r_assign(A,B,C)
>> r_call:r_bind(A)
>> r_call:'__aux_maplist/2_r_bind+0'(A)
>> r_call:r_execute(A,B,C)
>> <-(A,1:10)
>> swish_trace:swish_call( <-(A,1:10))
>> '$swish wrapper'(<-(A,1:10),B)
>>
>> I suspect i need to run "SWISH WITH AUTHENTICATION SUPPORT" ?
>
>
> One way this can be done is by
>
> 1. adding
>
> :- use_module('lib/plugin/http_authenticate').
>
>
> after line
> :- use_module(server).
> in run.pl

That was the way to do it. In the current version, simply copy/link
auth_http_always.pl from config-available to config-enabled.

> 2. on the first time run.pl is loaded
> query with
> ?- swish_add_user.
>
> See lib/plugin/http_authenticate.pl for details.
>
>
> I have now a working local copy of (non-sandboxed, i assume) Swish with Rserve support.

Yes, both SWISH and R should now allow you to do whatever you want.

I'm curious how you like it after getting used to it. Please record
the oddities you find. Probably not all can be solved, but some may
be trivial and adding a lot to the (first time) experience.

Cheers --- Jan

[nicos angelopoulos]

> > I believe this was due to web/bower_components missing (I made a clean install this morning, and forgot to get
> > the .zip file).
>
> I don't think that. Just did a fresh install on a different machine for
> someone. Without the bower stuff you just get an almost empty page with
> some scattered items.

yes. of course you are right. I got that both yesterday and today.

> I did get the error above, which went away by running
>
> git submodule update --init
>
> Actually just initializing pack/profile does the job. Only, it should
> work without.
>

yes. that was the reason. but there was a seminar in between
testing and reporting things I recalled wrongly

>
> Several ways. You were too obedient :) You followed this advice:
>
> % EDIT: COMMENT IF R IS NOT RUNNING IN AN OS SANDBOX.
> :- use_module(library(r/r_sandbox)).
>
> You should not. If you leave this in, you can access Rserve. Note that
> anyone with access to your machine can.
>

great, thanks. i will try this.

> The alternative is to activate the auth_http_always.pl config and follow
> the description to create a user. Next you need to login
> before you get access to SWISH, but once you have access to swish,
> you can do anything you can do in a normal Prolog shell.
>

once the architecture is stable it might be useful to have howtos
for different scenarios (happy to be of help if you need any)

> Finally, you can load the file 'ide.pl' and run ?- swish. to start
> SWISH. This also gives access without limitations. It is a little
> experimental though.
>
> > Also, I would also like to try running Real in Swish, what would be the simplest
> > way to tell SWISH to let it run <- commands ?
>
> Just use
>
> :- use_module(swish:library(real)).
>
> You probably need to disable the r_serve plugin as I doubt they will
> cooperate. There are pros and cons between r_serve and real. In the
> above case, threading may be an issue. Remember that each query in
> SWISH is executed in a thread using a module created for executing
> just that query. After the query finishes, both the thread and module
> are reclaimed.
>
> Cheers --- Jan
>

Thank you.

Currently I want to use it for browsing pipelines under construction or pipelines that have mutliple branches and
use Swish to pick the path more easily.

I will try to stick to Rserve as much as possible. It should be fine,
I normally only use <- in Real on purpose.


Thanks again,

Nicos

[Jan Wielemaker]

On 05/12/2017 03:57 PM, nicos angelopoulos wrote:

>> Actually just initializing pack/profile does the job. Only, it should
>> work without.
>>
> yes. that was the reason. but there was a seminar in between
> testing and reporting things I recalled wrongly

Pushed a fix such that it no longer requires this pack installed.

> once the architecture is stable it might be useful to have howtos
> for different scenarios (happy to be of help if you need any)

Configuration is indeed still a mess. I think the overall idea on how to
configure it is ok. The next step is probably to make it easy to install
some common scenarios. I just pushed a number of updates to the Makefile
to deal with common installation tasks. One option would be to add
targets for common use cases. Of course one could argue make isn't very
suitable for this given that it doesn't work on Windows. Possibly I
should use Chris Mungall's plmake pack?

Next question is what are common use cases? Virtually every install
I've been involved in had different requirements ... There are several
dimensions: single/multi-user, (not) sandboxed, social login vs secure
login and finally the data and libraries you want to make available.

> Thank you.
>
> Currently I want to use it for browsing pipelines under construction or pipelines that have mutliple branches and
> use Swish to pick the path more easily.

Nice use case. We came to the following way to do this type of things

- Create programs for bigger reusable code fragments
- Create a notebook for your scenarios
- Add a world-visible program at the end of the notebook
where you include the reusable code you need and add the
stuff that is common to the scenarios in your nodebook.
- Represent every scenario using
- (optionally) a Markdown cell explaining the scenario
- A program fragment with some facts that are used by
the general program to configure itself
- A query that calls a predicate in the generic code.

Note that the query will combine the code from the program cell just
above it with the `global' program cell. We used the above for creating
R charts. The included programs did the real computation. The global
program included these and provided the reusable code to generate the
R charts and the local program provided parameters to create the desired
chart.

> I will try to stick to Rserve as much as possible. It should be fine,
> I normally only use <- in Real on purpose.

Success --- Jan