call lisp from C to embed FriCAS

[Qian Yun]

One of the goals of FriCAS is:
make it easier for external programs to interface with FriCAS

Currently we have SageMath and jfricas to interface with FriCAS,
SageMath is using pexpect which is like terminal IO,
while jfricas is using hunchentoot which is IO via HTTP.

I think a C interface would be most performant, and convenient for
some use cases.

The disadvantage would be that for different Lisp, it needs different
code for the C interface.

(SageMath has a C interface for Maxima with ECL.)

Since SBCL has the best performance, I'd like to try to write
a C interface for it first.

This needs SBCL >= 2.1.11, and "libsbcl.so" built by
"make-shared-library.sh" in the sbcl repo.

I can call Lisp function from C now, but I have not called/exported
FriCAS functions yet. I'll keep you posted.

- Qian

[Dima Pasechnik]

On Tue, Sep 24, 2024 at 2:51 PM Qian Yun <oldk...@gmail.com> wrote:
>
> One of the goals of FriCAS is:
> make it easier for external programs to interface with FriCAS
>
> Currently we have SageMath and jfricas to interface with FriCAS,
> SageMath is using pexpect which is like terminal IO,
> while jfricas is using hunchentoot which is IO via HTTP.
>
> I think a C interface would be most performant, and convenient for
> some use cases.
>
> The disadvantage would be that for different Lisp, it needs different
> code for the C interface.
>
> (SageMath has a C interface for Maxima with ECL.)
>
> Since SBCL has the best performance, I'd like to try to write
> a C interface for it first.

Are you using sbcl-librarian, as I proposed in
https://groups.google.com/g/fricas-devel/c/ds7aqEd-lj4/m/pBKlFj47AgAJ
?

>
> This needs SBCL >= 2.1.11, and "libsbcl.so" built by
> "make-shared-library.sh" in the sbcl repo.

yes, sbcl-librarian uses libsbcl.so - but I have an impression that
you might be trying to write code which
is already there.

Just in case,
Dima

>
> I can call Lisp function from C now, but I have not called/exported
> FriCAS functions yet. I'll keep you posted.
>
> - Qian
>

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/fc876c08-879d-4651-acfa-3523e3958b0f%40gmail.com.

[Qian Yun]

On 9/24/24 10:18 PM, Dima Pasechnik wrote:
>
> Are you using sbcl-librarian, as I proposed in
> https://groups.google.com/g/fricas-devel/c/ds7aqEd-lj4/m/pBKlFj47AgAJ
> ?

I know about sbcl-librarian, but I'm using the raw SBCL API
right now.

>> This needs SBCL >= 2.1.11, and "libsbcl.so" built by
>> "make-shared-library.sh" in the sbcl repo.
>
> yes, sbcl-librarian uses libsbcl.so - but I have an impression that
> you might be trying to write code which
> is already there.
>
> Just in case,
> Dima
>

For next step I need to choose which fricas functions to expose,
and write C wrappers for them.

- Qian

[Dima Pasechnik]

On Tue, Sep 24, 2024 at 3:31 PM Qian Yun <oldk...@gmail.com> wrote:
>
>
>
> On 9/24/24 10:18 PM, Dima Pasechnik wrote:
> >
> > Are you using sbcl-librarian, as I proposed in
> > https://groups.google.com/g/fricas-devel/c/ds7aqEd-lj4/m/pBKlFj47AgAJ
> > ?
>
> I know about sbcl-librarian, but I'm using the raw SBCL API
> right now.

Isn't the main purpose of sbcl-librarian to automate doing C-wrappers?

>
> >> This needs SBCL >= 2.1.11, and "libsbcl.so" built by
> >> "make-shared-library.sh" in the sbcl repo.
> >
> > yes, sbcl-librarian uses libsbcl.so - but I have an impression that
> > you might be trying to write code which
> > is already there.
> >
> > Just in case,
> > Dima
> >
>
> For next step I need to choose which fricas functions to expose,
> and write C wrappers for them.
>
> - Qian
>

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/905ed36d-aa91-4f5e-a9f0-c926add1d201%40gmail.com.

[Waldek Hebisch]

On Tue, Sep 24, 2024 at 11:02:30PM +0100, Dima Pasechnik wrote:
> On Tue, Sep 24, 2024 at 3:31 PM Qian Yun <oldk...@gmail.com> wrote:
> >
> >
> >
> > On 9/24/24 10:18 PM, Dima Pasechnik wrote:
> > >
> > > Are you using sbcl-librarian, as I proposed in
> > > https://groups.google.com/g/fricas-devel/c/ds7aqEd-lj4/m/pBKlFj47AgAJ
> > > ?
> >
> > I know about sbcl-librarian, but I'm using the raw SBCL API
> > right now.
>
> Isn't the main purpose of sbcl-librarian to automate doing C-wrappers?

I have looked a bit sbcl-librarian. AFAICS sbcl-librarian has its
assumptions and its API. It is not clear if there is any gain compared
to sbcl API. And my impression was that sbcl-librarian API was poorly
documented, while sbcl documentation looked quite good.

--
Waldek Hebisch

[Qian Yun]

I was mistaken, I did not successfully call Lisp function
from C side.

I wonder if you can do this successfully:

1. put SBCL source code to $SRC
2. build SBCL and libsbcl.so:
cd $SRC
./make.sh
./make-shared-library.sh
3. dump a Lisp core:
==== test.lisp ====
(sb-alien:define-alien-callable fun1 sb-alien:int () 2)
(sb-ext:save-lisp-and-die "test.core" :callable-exports '(fun1))
==== end test.lisp ====

$SRC/run-sbcl.sh --load test.lisp

4. build a C executable:
==== test.c ====
#include <stdio.h>

extern int initialize_lisp(int argc, char *argv[], char *envp[]);

int main(int argc, char *argv[], char *envp[]){

int sargc=4;
char * sargv[] = {argv[0], "--core", "test.core", "--no-sysinit", };
printf("start init\n");
initialize_lisp(sargc, sargv, envp);

printf("finish init\n");
return 0;
}
==== end test.c ====

cc test.c $SRC/runtime/libsbcl.so -o app

5. execute the "./app" and you get error in
"foreign function: call_into_lisp_".

Looks like the error is because it can't initialize the foreign
function correctly.

- Qian

[Grégory Vanuxem]

Hello Qian,

I only did a quick test but no, not successfully, your test.core is unusable. Use the key ":executable t" when the core image is dumped to check this with ./test.core. I have no idea why the :callable-exports is causing this.

So even without your test.c file you see that the image is useless as it is. I guess you noticed it.

Greg

--
You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/6e3d477d-71e7-4980-b561-9409ec9f1ccd%40gmail.com.

[Qian Yun]

The purpose of this is to use Lisp image as library, so it is
intentionally to not use ":executable t".

Anyway, I have reported this issue to sbcl-bugs mail list, but
they have not replied yet.

- Qian

[Grégory Vanuxem]

Le lun. 30 sept. 2024 à 12:59, Qian Yun <oldk...@gmail.com> a écrit :

The purpose of this is to use Lisp image as library, so it is
intentionally to not use ":executable t".

This is what I understood. It was just to show that the image seems not usable, even on its own.

 

Anyway, I have reported this issue to sbcl-bugs mail list, but
they have not replied yet.

Seemed to be an issue for me also but I haven't the knowledge in this area (creating a Lisp image as a library). That would be interesting though.

Greg

 

- Qian

On 9/27/24 5:29 PM, Grégory Vanuxem wrote:
> Hello Qian,
>
> I only did a quick test but no, not successfully, your test.core
> is unusable. Use the key ":executable t" when the core image is dumped
> to check this with ./test.core. I have no idea why the :callable-exports
> is causing this.
>
> So even without your test.c file you see that the image is useless as it
> is. I guess you noticed it.
>
> Greg
>

--
You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/0878fa15-756f-4b7c-a703-b05c812ecb94%40gmail.com.

[Qian Yun]

On 9/24/24 10:31 PM, Qian Yun wrote:
>
>
> On 9/24/24 10:18 PM, Dima Pasechnik wrote:
>>
>> Are you using sbcl-librarian, as I proposed in
>> https://groups.google.com/g/fricas-devel/c/ds7aqEd-lj4/m/pBKlFj47AgAJ
>> ?
>
> I know about sbcl-librarian, but I'm using the raw SBCL API
> right now.
>

Hmm, good news here, I tried sbcl-librarian and I can make the
example work. (With small tweaks on Linux.)

It is strange that I can't make an minimal example with raw SBCL API
work. Maybe the secret sauce is this extra C declaration,
__attribute__ ((visibility ("default")))
but I'm still missing something.

Anyway, it seems that this call-lisp-from-c API is still very
experimental and not widely used. So perhaps not a good choice
to embed FriCAS into Sagemath.

- Qian

[Qian Yun]

OK, the key here is to compile the exported functions into
a shared library, example listed there won't work:
https://github.com/svetlyak40wt/sbcl-lib

1. put SBCL source code to $SRC
2. build SBCL and libsbcl.so:
cd $SRC
./make.sh
./make-shared-library.sh
3. dump a Lisp core:
==== test.lisp ====
(sb-alien:define-alien-callable fun1 sb-alien:int () 2)
(sb-ext:save-lisp-and-die "test.core" :callable-exports '(fun1))
==== end test.lisp ====

$SRC/run-sbcl.sh --load test.lisp

4. build a shared library:

==== lib.c ====
int(*fun1)();
==== end lib.c ====

cc lib.c -shared -o lib.so

5. build a C executable:

==== test.c ====
#include <stdio.h>

extern int initialize_lisp(int argc, char *argv[], char *envp[]);

int (*fun1)();

int main(int argc, char *argv[], char *envp[]){

int sargc=4;
char * sargv[] = {argv[0], "--core", "test.core", "--no-sysinit", };
printf("start init\n");
initialize_lisp(sargc, sargv, envp);

printf("finish init: %d\n", fun1());

return 0;
}
==== end test.c ====

cc test.c ./lib.so $SRC/runtime/libsbcl.so -o app

6. execute the "./app" and you get expected output.

- Qian