libcurl FFI interface

[Thomas McGuire]

I seem to be having a problem setting up a library interface for libcurl. For now it seems my problems stem from an unusable pointer frome curl_easy_init. 

Some C language externs: 

CURL_EXTERN CURL *curl_easy_init(void);

CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);

CURL_EXTERN CURLcode curl_easy_perform(CURL *curl);

CURL_EXTERN void curl_easy_cleanup(CURL *curl);

The CURL* is just a macro for void * and this ultimately gets played with deep in the library to become a pointer to a structure. You don’t really have control over that structure in C you have to make calls to curl_easy_setopt to make changes to the curl handle. 

I am only building out the parts of libcurl I need to do some simple http operations. Here is the pertinent J library FFI definitions: 

curl_global_init =: (LIBCURL,' curl_global_init n i') & cd

curl_global_cleanup =: (LIBCURL,' curl_global_cleanup n') & cd

NB. curl_easy_init=: [:{.(LIBCURL,' curl_easy_init * ') & cd

curl_easy_init=: (LIBCURL,' curl_easy_init x ') & cd

ccurl_easy_cleanup=: (LIBCURL,' curl_easy_cleanup n * ') & cd

curl_easy_setopt=: (LIBCURL,' curl_easy_setopt n x i x') & cd

curl_perform=: (LIBCURL,' curl_easy_perform i * ') & cd

curl_slist_append=: (LIBCURL,' curl_slist_append * * *') & cd

curl_slist_free_all=: (LIBCURL,' curl_slist_free_all n *') & cd

NB. print_slist =: (LIBJCURL,' print_slist n *') & cd

set_curl_string_opt =: (LIBJCURL,' set_curl_string_opt n x i *c') & cd

set_curl_long_opt =: (LIBJCURL,' set_curl_int_opt n * i i') & cd

set_curl_ptr_opt =: (LIBJCURL,' set_curl_ptr_opt n * i *') & cd

jcurlperform =: (LIBJCURL,' jcurlperform i x *c') & cd

free_response =: (LIBJCURL,' free_response n *') & cd

mystrdup =: (LIBJCURL,' mystrdup * *') & cd

This seems to center around the pointer I get back from curl_easy_init. I have defined the return value as a * and x as above but when I go to use that pointer in a C call the libcurl call curl_easy_perform segfaults and crashes J.

Here is the C code that is testing this stuff:

int jcurlperform(CURL* curl, char* comment) {

  CURLcode res;

  CURL *curl1;

  FILE *file_ptr; // Declare a FILE pointer

  file_ptr = fopen("jcurlget.log", "w");

    // Check if the file was opened successfully

    if (file_ptr == NULL) {

        perror("Error opening file"); // Print a system-specific error message

        return 100; // Indicate an error

    }

    // Write the string to the file

    fprintf(file_ptr, "initialize chunk\n");

  chunk = (struct memory){malloc(1), 0};

//   fprintf(file_ptr,"curl ptr from J: %p \n",(void *) curl);

   curl1 = curl_easy_init();

  if(curl1) {

    fprintf(file_ptr, "curl pointer seems valid (%p) check setup\n",(void *) curl1);

    curl_easy_setopt(curl1, CURLOPT_URL,comment );

    /* example.com is redirected, so we tell libcurl to follow redirection */

    curl_easy_setopt(curl1, CURLOPT_FOLLOWLOCATION, 1L);

    curl_easy_setopt(curl1, CURLOPT_WRITEFUNCTION, write_callback);

    curl_easy_setopt(curl1, CURLOPT_WRITEDATA, (void*)&chunk);

   fprintf(file_ptr,"curl write options added within C\n");

    /* Perform the request, res gets the return code */

    res = curl_easy_perform(curl1);

/* Check for errors */

    fprintf(file_ptr,"curl_easy_perform(curl1) completed\n");

    if(res != CURLE_OK) {

      fprintf(file_ptr, "curl_easy_perform() failed: %d, %s\n",res,

              curl_easy_strerror(res));

      curl_easy_cleanup(curl1);

      fclose(file_ptr);

      return res;

    }

    /* always cleanup */

    curl_easy_cleanup(curl1);

    fprintf(file_ptr,"almost done log response\n");

    fprintf(file_ptr,"%s\n",chunk.response);

    fclose(file_ptr);

    return res;

  }

  fprintf(file_ptr,"Problem setting up curl handle\n");

  fclose(file_ptr);

  return 200;

}

Here is the J routine that is trying to access this C code: 

chat =: 3 : 0

C =: >{. curl_easy_init''

if. C = 0 do.

NULL

return.

end.

smoutput 'curl initialized'

echo C

hlibjcurl =: (15!:20) LIBJCURL

mycbp =: hlibjcurl (15!:21) 'write_callback'

chunkp =: hlibjcurl (15!:21) 'chunk'

NB. headers =: mema 4

headers =: <0

NB. headers =: {. curl_slist_append (headers);'Content-Type: application/json'

NB. headers =: {. curl_slist_append (headers);'Authorization: Bearer sk-123'

NB. set_curl_string_opt C;CURLOPT_URL;'https://example.com'

NB. set_curl_string_opt C;CURLOPT_URL;url

NB. chkstr =: >{. chkptr <<url_base

NB. echo chkstr

NB. curl_easy_setopt C;CURLOPT_URL;url_base

NB. curl_setopt C;CURLOPT_HTTPHEADER;<headers

NB. set_curl_ptr_opt C;CURLOPT_HTTPHEADER;<headers

NB. curl_setopt C;CURLOPT_POSTEFIELDS;json

NB. curl_setopt C;CURLOPT_WRITEFUNCTION;<<mycbp

NB. curl_setopt C;CURLOPT_WRITEDATA;<<chunkp

NB. curl_setopt C;CURLOPT_VERBOSE;curlverbose

NB.

NB. res =: >{.curl_perform <C

NB.

NB. curl_easy_cleanup <C

NB. curl_slist_free_all <headers

NB. res_str =: 2000$' '

echo 'jcurlperform being called'

cmtstr =: 'https://example.com'

fullres =: jcurlperform C;cmtstr

res =: >{. fullres

echo 'return from jcurlperform'

respptr =. memr chunkp,0,1,JINT

respstr =. memr respptr,0,_1,JCHAR

NB. close the handle to the DLL

NB. free_response <respptr

(15!:22) hlibjcurl

if. res ~: CURLE_OK do.

echo 'curl error'

res

return.

end.

respstr

)

So the C =: >{. curl_easy_init’’ works and I get an integer result. But if I do nothing else but send it into the C program above it’s unusable for the curl_easy_perform and crashes J. In the console I get some meaningless domain error and an error message that libcurl tried to release an already defucnt pointer.

If I run all the libcurl calls in C. I can pass a string paramter from J into jcurlperform and that will be used as expected.

Any insight someone can provide would be helpful. I would like to be able to do as much libcurl in J as possible.

Tom McGuire

PS sorry for the extra long post

[bill lam]

you should provide your own write_function and chunk in J code.

IIRC there is a J lab on callback.

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Thomas McGuire]

Can’t even think about it if I can’t get a good CURL* back and forth from J to C. Until I can run curl_easy_perform in J on a CURL* handle that I get from curl_easy_init it doesn’t matter where I have the call back because nothing gets called.

C =: >{. curl_easy_init’'

set_curl_string_opt C;CURLOPT_URL;'https://example.com'

res =: >{.curl_perform <C

If I code up the chat verb to do just that much I get a error

|domain error in cd, executing dyad 15!:0
| res=:>{. curl_perform<C
Press ENTER to inspect
cder''
6 0

It doesn’t like the CURL* I have stored in the J noun ‘C’

So I’ve been splitting work between J and C trying to figure out why this doesn’t work. The cder’’ indicates the C parameter is the problem. But to change that I had to create a bigger C helper function, where essentially all the libcurl stuff gets done.

Tom McGuire

[bill lam]

I 'm not sure becasue I rarely use * . May be you try double box

curl_perform <<C

[Thomas McGuire]

That actually worked to run curl_perform and not the point stored at C crashed JQT when inside curl_perform

Here is a run from inside the jconsole:

   load '/users/tmcguire/j9.7-user/temp/jcurl.ijs'

   mchat =: 'https://example.com' conew 'JCurl'

   chat__mchat''

curl initialized

4487947264

jcurlperform being called

return from jcurlperform

JE has crashed, likely due to an internal bug.  Please report the code which caused the crash, as well as the following printout, to the J forum.

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

Could not generate stack trace: no debug info in Mach-O executable (-1)

-----------------------------------------------------------------------------

Abort trap: 6

logout

Thinking that this had been due to the fact that I did not have a write function and a data area to write to I added back in the following statements:

set_curl_ptr_opt (<C);CURLOPT_WRITEFUNCTION;<<mycbp

set_curl_ptr_opt (<C);CURLOPT_WRITEDATA;<<chunkp

This is a simple C helper function wrapping curl_easy_setopt calls with appropriate pointer arithmetic.

I also tried the direct libcurl call:

curl_easy_setopt C;CURLOPT_WRITEFUNCTION;mycbp

curl_easy_setopt C;CURLOPT_WRITEDATA;chunkp

with the same crash.

[Henry Rich]

Can you send me an example that crashes on Windows?

Henry Rich

[bill lam]

Did you ensure the mycbp and chunkp are valid?

[Thomas McGuire]

They are valid but I don’t know if moving them into the curl handle with curl_easy_setopt is moving the pointer in a valid way. When I do all the libcurl in C I have no trouble using ‘memr' to get the response from the example.com website using the chunkp address I get from the 15!:21 calls. Because chunkp is a pointer to a structure I have to dereference twice with ‘memr’ to get access to the string that holds the response: 

respptr =. memr chunkp,0,1,JINT

respstr =. memr respptr,0,_1,JCHAR

I was thinking that the CURL* address I get back from curl_easy_init may need to be dereferenced in a certain way. I had tried to use ‘mema’ as a way to allocate space for a pointer. But my attempts to fill that with the pointer returned by curl_easy_init. But I run into problems with those calls as well.

Tom McGuire

[bill lam]

How and where did you implement the callback function?

[Thomas McGuire]

Callback function still implemented in C. However I have had a minor break through. If I hide most of the libcurl functions in my own C function wrappers I get a usable CURL* pointer that can then drive an http get. I will provide some code snippets to show what is working:

NB. FFI definitions:

curl_global_init =: (LIBCURL,' curl_global_init n i') & cd
curl_global_cleanup =: (LIBCURL,' curl_global_cleanup n') & cd
NB. curl_easy_init=: [:{.(LIBCURL,' curl_easy_init * ') & cd
curl_easy_init=: (LIBCURL,' curl_easy_init x ') & cd
curl_easy_cleanup=: (LIBCURL,' curl_easy_cleanup n * ') & cd
curl_easy_setopt=: (LIBCURL,' curl_easy_setopt n x i x') & cd

curl_easy_perform=: (LIBCURL,' curl_easy_perform i * ') & cd

curl_slist_append=: (LIBCURL,' curl_slist_append * * *') & cd
curl_slist_free_all=: (LIBCURL,' curl_slist_free_all n *') & cd

NB. print_slist =: (LIBJCURL,' print_slist n *') & cd

get_curl_ptr =: (LIBJCURL,' get_curl_ptr * *') & cd
set_curl_string_opt =: (LIBJCURL,' set_curl_string_opt n * i *c') & cd
set_curl_long_opt =: (LIBJCURL,' set_curl_long_opt n * i i') & cd

set_curl_ptr_opt =: (LIBJCURL,' set_curl_ptr_opt n * i *') & cd

jcurlperform =: (LIBJCURL,' jcurlperform i *') & cd

// The C code that wraps the libcurl functions:

CURL *get_curl_ptr(CURL *jcurlptr) {
CURL *curl1 = curl_easy_init();
jcurlptr = curl1;
return curl1;
}

void set_curl_string_opt(void *curl, int opt_flag, char* param_str) {
CURL *curl1 = (CURL *) curl;
curl_easy_setopt(curl1, opt_flag, param_str);
}

void set_curl_long_opt(CURL *curl, int opt_flag, long lparam) {
curl_easy_setopt(curl, opt_flag, lparam);
}

void set_curl_ptr_opt(CURL *curl, int opt_flag, void* pparam) {
curl_easy_setopt(curl, opt_flag, pparam);
}

int jcurlperform(CURL* curl) {
CURLcode res;

FILE *file_ptr; // Declare a FILE pointer

file_ptr = fopen("jcurlget.log", "w");

// Check if the file was opened successfully
if (file_ptr == NULL) {
perror("Error opening file"); // Print a system-specific error message
return 100; // Indicate an error
}

chunk = (struct memory){malloc(1), 0};

if(curl) {

/* Perform the request, res gets the return code */

res = curl_easy_perform(curl);

/* Check for errors */

fprintf(file_ptr,"curl_easy_perform(curl) completed\n");

if(res != CURLE_OK) {
fprintf(file_ptr, "curl_easy_perform() failed: %d, %s\n",res,
curl_easy_strerror(res));

curl_easy_cleanup(curl);

fclose(file_ptr);
return res;
}
/* always cleanup */

curl_easy_cleanup(curl);

fprintf(file_ptr,"almost done log response\n");
fprintf(file_ptr,"%s\n",chunk.response);
fclose(file_ptr);

return res;
}
fprintf(file_ptr,"Problem setting up curl handle\n");
fclose(file_ptr);
return 200;
}

with that in place here is my chat function that makes the access (it’s called chat because this started as wrapper around an openai-c implementation I had found. But devolved into a libcurl interface.

chat =: 3 : 0
Cp =: <0
Cret =: get_curl_ptr <Cp
echo Cret
echo Cp
C =: {.Cret
echo C

if. C = 0 do.
NULL
return.
end.
smoutput 'curl initialized'

NB. get the C write callback and data ptr

hlibjcurl =: (15!:20) LIBJCURL
mycbp =: hlibjcurl (15!:21) 'write_callback'
chunkp =: hlibjcurl (15!:21) 'chunk'

'headers =: <0

NB. headers =: {. curl_slist_append (headers);'Content-Type: application/json'
NB. headers =: {. curl_slist_append (headers);'Authorization: Bearer sk-123'

'
echo 'set url'
set_curl_string_opt C;CURLOPT_URL;url

NB. curl_setopt C;CURLOPT_HTTPHEADER;<headers
NB. set_curl_ptr_opt C;CURLOPT_HTTPHEADER;<headers

NB. curl_setopt C;CURLOPT_POSTEFIELDS;json

NB. curl_easy_setopt C;CURLOPT_WRITEFUNCTION;mycbp
NB. curl_easy_setopt C;CURLOPT_WRITEDATA;chunkp
NB. echo 'set write callback and data loc'

set_curl_ptr_opt C;CURLOPT_WRITEFUNCTION;<<mycbp
set_curl_ptr_opt C;CURLOPT_WRITEDATA;<<chunkp
set_curl_long_opt C;CURLOPT_VERBOSE;1
echo 'jcurlperform being called'

NB.
NB. res =: >{.curl_easy_perform <C
NB.

fullres =: jcurlperform <C
echo fullres

res =: >{. fullres
echo 'return from jcurlperform'

if. res ~: CURLE_OK do.
echo 'curl error'

res
return.
end.
echo 'response OK get data'

respptr =. memr chunkp,0,1,JINT
respstr =. memr respptr,0,_1,JCHAR

echo respstr

echo 'clean up curl handle'
curl_easy_cleanup <C

NB. close the handle to the DLL
NB. free_response <respptr
(15!:22) hlibjcurl

respstr
)

This works for me on macOS. The response is easily converted to a J string via memr. I am using 15!:21 to get the addresses of the C variables and functions I need. I will make a repository on GITHUB of the full code so far for any one that is interested. I was hoping to get this as close to the curl_easy_* calls from libcurl as possible. But I suspect that libcurl’s reliance on untyped pointers is making it difficult for the J FFI to interface properly. My C interface provides a more standard mapping from J to C. Thanks to everyone for the help.

For Bill Lam: my next step is to see if I can use the J callback function interface and do the callback in J

For Henry Rich: I have been working on the mac but I have a Windows 11 box. I will try to whittle the code down to crash JQT. But the code does mess with pointers in C code. I’m sure it has created a memory leak that has crashed the system. Not sure if JQT should be expected to hold on when the underlying code is running amok.

Tom McGuire

[Henry Rich]

Yes, I was just offering to tell you where it was crashing. If you have it working there's no need. 

Bill has recently added a nifty name-trace feature that traces calls. If you can think of something we can add to that to help with what you're doing let us know. 

Henry Rich

[bill lam]

Oh, I didn't realize you wrapped libcurl into your own libjcurl.

If you intended to use libjcurl, why not hide all libcurl, wite_function etc and use J to call libjcurl only.

But if I were you, I would just call libcurl. 

[Thomas McGuire]

I wrapped it in C because the following code based directly on libcurl FFI doesn’t work:

curl_global_init =: (LIBCURL,' curl_global_init n i') & cd

curl_global_cleanup =: (LIBCURL,' curl_global_cleanup n') & cd

curl_easy_init=: (LIBCURL,' curl_easy_init x ') & cd

curl_easy_cleanup=: (LIBCURL,' curl_easy_cleanup n * ') & cd

curl_easy_setopt=: (LIBCURL,' curl_easy_setopt n x i x') & cd

curl_easy_perform=: (LIBCURL,' curl_easy_perform i * ') & cd

curl_slist_append=: (LIBCURL,' curl_slist_append * * *') & cd

curl_slist_free_all=: (LIBCURL,' curl_slist_free_all n *') & cd

NB. lots of constant definitions from libcurl defined here but removed for this email

runCurl =: 3 : 0

url =: y

curlverbose =: 1

curl_global_init <CURL_GLOBAL_DEFAULT

Cret =: curl_easy_init''

echo Cret

C =: {.Cret

echo C

if. C = 0 do.

NULL

return.

end.

NB. get the C write callback and data ptr

hlibjcurl =: (15!:20) LIBJCURL

mycbp =: hlibjcurl (15!:21) 'write_callback'

chunkp =: hlibjcurl (15!:21) 'chunk'

curl_easy_setopt C;CURLOPT_URL;url

curl_easy_setopt C;CURLOPT_VERBOSE;1

curl_easy_setopt C;CURLOPT_WRITEFUNCTION;<<mycbp

curl_easy_setopt C;CURLOPT_WRITEDATA;<<chunkp

res =: >{.curl_easy_perform <C

if. res ~: CURLE_OK do.

echo 'curl error'

res

return.

end.

respptr =. memr chunkp,0,1,JINT

respstr =. memr respptr,0,_1,JCHAR

echo respstr

curl_easy_cleanup <C

NB. close the handle to the DLL

(15!:22) hlibjcurl

respstr

curl_global_cleanup ''

)

Run of the above code under JQT 9.7

runCurl 'https://example.com'

┌──────────┐

│5154824192│

└──────────┘

┌──────────┐

│5154824192│

└──────────┘

|domain error in cd, executing dyad 15!:0

| curl_easy_setopt C;CURLOPT_URL;url

Press ENTER to inspect

[bill lam]

curl_easy_setopt=: (LIBCURL,' curl_easy_setopt n x i x') & cd

But url is a string, so this mismatch the x which expected an 8 byte integer.

You can define another protocol, eg

curl_easy_setopt2=: (LIBCURL,' curl_easy_setopt n x i *c') & cd

And use it for string arguments

curl_easy_setopt2 C;CURLOPT_URL;url

Hope this helps.

[Henry Rich]

[This is desk analysis, take with a grain of salt]

Cret =: curl_easy_init''

echo Cret

C =: {.Cret

echo C

(I think the echos show that C is a boxed integer)

...

curl_easy_setopt C;CURLOPT_URL;url

C is supposed to be a boxed integer (x), but it is actually a box.

cder'' will show what cd is complaining about.

Henry Rich

[Thomas McGuire]

Implementing Bill’s and Henry’s suggestions the code basically looks like this (missing curl constant definitions):

NULL =: <0

3 : 0''

select. UNAME

case. 'Win' do.

LIBCURL=: '"C:\Users\Thomas\Downloads\libcurl.dll" '

case. 'Darwin' do.

LIBCURL=: '/opt/homebrew/Cellar/curl/8.15.0/lib/libcurl.dylib'

LIBJCURL=: jpath '~/cppdev/jcurl/libjcurl.dylib'

case. 'Linux' do.

LIBCURL=: 'libcurl.dll '

case. do.

LIBCURL=: 'libcurl.dll '

end.

)

NB. some usage

NB. curl_global_init(CURL_GLOBAL_DEFAULT)

NB. all pointer parameters and return values use * or *c as the

NB. case may be. As suggested by Henry Rich

curl_global_init =: (LIBCURL,' curl_global_init n i') & cd

curl_global_cleanup =: (LIBCURL,' curl_global_cleanup n') & cd

curl_easy_init=: (LIBCURL,' curl_easy_init * ') & cd

curl_easy_cleanup=: (LIBCURL,' curl_easy_cleanup n * ') & cd

NB. multiple call signatures to the same curl_easy_setopt

NB. as suggested by Bill Lam

curl_easy_setopt_str=: (LIBCURL,' curl_easy_setopt n * i *c') & cd

curl_easy_setopt_ptr=: (LIBCURL,' curl_easy_setopt n * i *') & cd

curl_easy_setopt_int=: (LIBCURL,' curl_easy_setopt n * i i') & cd

curl_easy_perform=: (LIBCURL,' curl_easy_perform i * ') & cd

NB. not used yet but it's used for setting up header parameters

NB. to be used by curl_easy_perform in the http request

curl_slist_append=: (LIBCURL,' curl_slist_append * * *') & cd

curl_slist_free_all=: (LIBCURL,' curl_slist_free_all n *') & cd

runCurl =: 3 : 0

url =: y

curlverbose =: 1

curl_global_init <CURL_GLOBAL_DEFAULT

Cret =: curl_easy_init''

echo Cret

C =: {.Cret

echo C

if. C = 0 do.

NULL

return.

end.

NB. get the C write callback and data ptr

hlibjcurl =: (15!:20) LIBJCURL

mycbp =: hlibjcurl (15!:21) 'write_callback'

chunkp =: hlibjcurl (15!:21) 'chunk'

curl_easy_setopt_str C;CURLOPT_URL;url

NB. curl_easy_setopt_ptr C;CURLOPT_URL;<<mystr

curl_easy_setopt_int C;CURLOPT_VERBOSE;1

curl_easy_setopt_ptr C;CURLOPT_WRITEFUNCTION;<<mycbp

curl_easy_setopt_ptr C;CURLOPT_WRITEDATA;<<chunkp

fullres =: curl_easy_perform <C

res =: >{. fullres

if. res ~: CURLE_OK do.

echo 'curl error'

res

return.

end.

respptr =. memr chunkp,0,1,JINT

respstr =. memr respptr,0,_1,JCHAR

echo respstr

curl_easy_cleanup <C

NB. close the handle to the DLL

(15!:22) hlibjcurl

respstr

curl_global_cleanup ''

)

OUTPUT:

load '/users/tmcguire/j9.7-user/temp/curl.ijs'

runCurl 'https://example.com'

┌──────────┐

│5033224192│

└──────────┘

┌──────────┐

│5033224192│

└──────────┘

curl error

3

The error code corresponds to the following constant definition:

CURLE_URL_MALFORMAT=: 3

So after building up the curl handle with curl_easy_setopt calls the curl_easy_perform call doesn’t seem to be able to access anything. Yet if I wrap all these calls in a C-language wrapper I can get the curl_easy_perform to work properly in C when called from my wrapper.

This seems so straight forward and yet I can’t get it to work.

Tom McGuire

[bill lam]

I googled and found it's official page of examples 

https://curl.se/libcurl/c/example.html

I think trying the simple (writing to stdout) and then getinmemory (malloc memory) is sufficient.

Did you translate the simple.c to J ?

Also the examples there said the url is

"https://www.example.com/"

you missed the ending "/"

I will add a new libcurl addon for its api protocols and some simple examples.

To unsubscribe from this group and stop receiving emails from it, send an email to forum+un...@jsoftware.com.

[Thomas McGuire]

load '/users/tmcguire/j9.7-user/temp/curl.ijs'

runCurl 'https://www.example.com/'

┌──────────┐

│5339472896│

└──────────┘

┌──────────┐

│5339472896│

└──────────┘

curl error

3

So my understanding is that the default behavior for libcurl is to write to a file. So if I comment out the callback function and its write buffer I should be defaulting to this behavior. However, when I comment those lines and reload and rerun:

load '/users/tmcguire/j9.7-user/temp/curl.ijs'

runCurl 'https://www.example.com/'

┌──────────┐

│4773172736│

└──────────┘

┌──────────┐

│4773172736│

└──────────┘

curl error

3

Still getting the error. I suspect that curl_easy_perform returns immediately when it can’t locate a valid URL. We will be unable to even test default behavior until I can get a URL into the CURL* handle in libcurl that curl_perform_routine can access properly.

Thinking that curl_easy_setopt wanted a C string I even used the dll memory mangement routines in J to set up a string that way.

mema a location for the number of bytes in the URL

memw to write the J string into the memory as a null terminated string.

used both the curl_easy_setopt_str and curl_easy_setopt_ptr calls to see if curl_easy perform would be able to access the URL string. Unable to get that to work.

[Thomas McGuire]

I have placed my full code up on github if anyone wants to play along at home:

[bill lam]

I haved added api/curl addon. You can get it from pacman. There are 2 examples included under the test folder.

simple - write to stdout

getinmemory - use callback and write to a noun.

I tested them run ok on mac (arm64 and x86_64) and linux.

untested on windows, not sure the libcurl.dll is included in windows 10/11.

You must work on Apple arm64, the curl_easy_setopt is a variadic function and its ABI need special treatment.

The C compiler will handle it correctly. But using FFI, it is more complicated.

[Thomas McGuire]

Begin forwarded message:

From: bill lam <bbil...@gmail.com>

Subject: Re: [Jforum] libcurl FFI interface

Date: September 9, 2025 at 10:44:01 AM EDT

To: fo...@jsoftware.com

Reply-To: fo...@jsoftware.com

I haved added api/curl addon. You can get it from pacman. There are 2 examples included under the test folder.

simple - write to stdout

getinmemory - use callback and write to a noun.

I tested them run ok on mac (arm64 and x86_64) and linux.

untested on windows, not sure the libcurl.dll is included in windows 10/11.

You must work on Apple arm64, the curl_easy_setopt is a variadic function and its ABI need special treatment.

The C compiler will handle it correctly. But using FFI, it is more complicated.

THANKS SO MUCH FOR THE API.

Maybe one day I will figure out the FFI so I won’t have to bother you. 

Tom McGuire