Implementing support for HB_CURLOPT_WRITEFUNCTION

[Antonio Linares]

Dear friends,

These days I am testing using ollama from Harbour. It is working fine using hbcurl.lib.

Now the challenge is to implement HB_CURLOPT_WRITEFUNCTION in hbcurl.lib so every generated AI token is visible as its gets generated so the user does not have to wait until the end to see them. 

Viktor comments about it:

/* These are hidden on the Harbour level: */
/* HB_CURLOPT_WRITEFUNCTION */

With a few little changes I have been able to implement it, though I have found that CURLOPT_WRITEFUNCTION is already being used from hbcurl.lib core.c

from: HB_CURLOPT_DL_FILE_SETUP,  HB_CURLOPT_DL_FHANDLE_SETUP and HB_CURLOPT_DL_BUFF_SETUP. In my tested code I don't need those options, so I don't have any conflicts with them.

Basically I have added a new struct member PHB_ITEM pWriteFunctionCallback; to HB_CURL. And then from static void PHB_CURL_free( PHB_CURL hb_curl, HB_BOOL bFree ):

if( hb_curl->pProgressCallback )
{
   hb_itemRelease( hb_curl->pProgressCallback );
   hb_curl->pProgressCallback = NULL;
}

also from static HB_GARBAGE_FUNC( PHB_CURL_mark ):

if( hb_curl->pProgressCallback )
   hb_gcMark( hb_curl->pProgressCallback );

also:

size_t static hb_curl_writefunction_callback( void *buffer, size_t size, size_t nmemb, void *Cargo )
{
PHB_CURL hb_curl = ( PHB_CURL ) Cargo;

if( hb_curl->pWriteFunctionCallback && hb_vmRequestReenter() )
{
   hb_vmPushEvalSym();
   hb_vmPush( hb_curl->pWriteFunctionCallback );
   hb_vmPushString( buffer, size * nmemb );
   hb_vmSend( 1 );
   hb_vmRequestRestore();
   return size * nmemb;
}
return 0;
}

and

case HB_CURLOPT_WRITEFUNCTION:
{
   PHB_ITEM pWriteFunctionCallback = hb_param( 3, HB_IT_BLOCK | HB_IT_SYMBOL );

   if( hb_curl->pWriteFunctionCallback )
   {
      curl_easy_setopt( hb_curl->curl, CURLOPT_WRITEFUNCTION, NULL );

      hb_itemRelease( hb_curl->pWriteFunctionCallback );
      hb_curl->pWriteFunctionCallback = NULL;
   }

   if( pWriteFunctionCallback )
{
   hb_curl->pWriteFunctionCallback = hb_itemNew( pWriteFunctionCallback );
   /* unlock the item so GC will not mark them as used */
   hb_gcUnlock( hb_curl->pWriteFunctionCallback );

   curl_easy_setopt( hb_curl->curl, CURLOPT_WRITEFUNCTION,           hb_curl_writefunction_callback );
res = curl_easy_setopt( hb_curl->curl, CURLOPT_WRITEDATA, hb_curl );
}
}
break;

Basically thats all is needed and in my tests here is working fine. Now I can see ollama AI tokens as they get generated.

I do appreciate your feedback and comments before considering adding these changes to Harbour.

many thanks

[Aleksander Czajczynski]

Hello!

It's not common in Harbour business apps to use streaming transfers from HTTP or equivalent, usually it is audio, video streaming. Now we would like to see the AI storytelling realtime ;-)

Some typos in pasted code
if( hb_curl->pProgressCallback ) // should be pWriteFunctionCallback

{
   hb_itemRelease( hb_curl->pProgressCallback );
   hb_curl->pProgressCallback = NULL;
}

HB_IT_BLOCK | HB_IT_SYMBOL -> that's HB_IT_EVALITEM

What if someone would like to have both HB_CURLOPT_DL_* and "liveview"? If it's not implemented, then HB_CURLOPT_WRITEFUNCTION will be a mutually exclusive option to them. Otherwise looks like a useful addition to me, what about others?

Best regards, Aleksander

Antonio Linares wrote:

--
You received this message because you are subscribed to the Google Groups "Harbour Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-deve...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/harbour-devel/d7fdc2ec-4ef6-4a1d-8f37-d8205976b174n%40googlegroups.com.

[Gerald Drouillard]

I like it.  Anything to extend the curl functionality is greatly appreciated.

To view this discussion visit https://groups.google.com/d/msgid/harbour-devel/679FB663.3050802%40fki.pl.