Using AI models from HuggingFace from Harbour

[antonio....@gmail.com]

I have adapted llama.cpp from https://github.com/ggerganov/llama.cpp to be used from Harbour so you can run any HuggingFace GGUF AI model. Tested with Orca2, Phi-2 from Microsoft, etc.

These model are used locally so there is no need to pay OpenAI, etc.

If you are intesrested in this please contact me

regards,

[na richard roesnadi]

Yes, I saw the video.. 

https://www.facebook.com/groups/harbour.project/permalink/10163168124593776/

--
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: https://groups.google.com/group/harbour-users
---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/harbour-users/0f22f27b-bb5c-4f97-a311-5e2a65a468c5n%40googlegroups.com.

[antonio....@gmail.com]

We just published llama64.dll and the models urls and an example of use:

https://fivetechsupport.com/forums/viewtopic.php?p=266285&sid=0b14c7bd999dc25423f992b55bd97cec#p266285

best regards,

[antonio....@gmail.com]

Here you have the detailed instructions to build llama64.dll

https://fivetechsupport.com/forums/viewtopic.php?p=266316&sid=0169e77cddbd42d75192bd73c678c279#p266316

merry xmas 

[Mario H. Sabado]

Many thanks for sharing this Antonio and Merry Christmas!

To view this discussion on the web visit https://groups.google.com/d/msgid/harbour-users/686f9b3b-e0b4-4546-adb5-a3e43cf1dc44n%40googlegroups.com.

[cod...@outlook.com]

Can  llama64.dll  be used with HMG. 

Anybody have similar example ?

Regards,

Simo.

[antonio....@gmail.com]

HMG example:

https://fivetechsupport.com/forums/viewtopic.php?p=266361&sid=8e0a0897e7823dc99e0f13498643350d#p266361

best regards

[cod...@outlook.com]

Thank you Antonio.

Also many thanks to Jimmy for his effort to adapt code to HMG.

Best Regards,

Simo.

[cod...@outlook.com]

Information about AI models using HMG.

 Antonio example from https://fivetechsupport.com/forums/viewtopic.php?f=3&t=44028&sid=8d3e3a13527e4a85c423dc2f68d14031

works on HMG.

 First I tried it with  llama64.dll  I have downloaded on FW forum. But it did not worked. Then I built  llama64.dll on my own,  using detailed instructions Antonio presented in some post above.

 After that with new llama64.dll example worked.

 I am not good with HMG, just used it to try Antonio example. So I made it to work, but my GUI design is very bad.  Someone who use HMG can make more beter GUI. So here I will only write main part of example, one where we must call llama64.dll:

 

CallDll32 ( "Llama" , "llama64.dll" ,"phi-2_Q4_K_M.gguf"  , ;

   ALLTRIM(Form_1.cPrompt.value), ;

   CallBackAI({ | cStr | Form_1.cAnswer.SetFocus, Form_1.cAnswer.value:=Form_1.cAnswer.value+ cStr }) )

 

where Form_1 is name from DEFINE WINDOW, cPrompt is name from first TEXTBOX (in this we type question), cAnswer is name from second TEXTBOX (in this we display answer).

 In my HMG adaptation I have one WINDOW, two TEXTBOX and two BUTTON, same as is in Antonio example, but using HMG commands. I did not know how to make multiline text i cAnswer TEXTBOX.

 Some qualified HMG user can make proper GUI, adapting Antonio example. Key command is CallDll32.

 

Best regards,

Simo.

[cod...@outlook.com]

I have these TEXTBOX and BUTTON:

@ 30, 10 TEXTBOX cPrompt ;

    HEIGHT 25 ;

    WIDTH 500 ;

    VALUE "List 10 possible uses of AI from my Windows apps."

 

 @  70, 50 BUTTON buttonStart  ;

        CAPTION 'Start'   ;

        WIDTH 100                 ;

        HEIGHT 50                 ;

        ONCLICK StartAI()

 

 

@ 150, 50 TEXTBOX cAnswer ;

        WIDTH 500                 ;

        HEIGHT 400

 

And next functions:

 

FUNCTION StartAI()

 

 CallDll32 ( "Llama" , "llama64.dll" ,"phi-2_Q4_K_M.gguf"  , ;

   ALLTRIM(Form_1.cPrompt.value), ;

   CallBackAI({ | cStr | Form_1.cAnswer.SetFocus, Form_1.cAnswer.value:=Form_1.cAnswer.value+ cStr }) )

RETURN NIL

* Next is copy from Antonio example:

#pragma BEGINDUMP

#include <Windows.h>

#include <hbapi.h>

#include <hbapiitm.h>

 static PHB_ITEM pBlock;

static void callbackAI( char * szMsg )

{

   PHB_ITEM pStr = hb_itemPutC( NULL, szMsg );

      hb_evalBlock1( pBlock, pStr );

         hb_itemRelease( pStr );

         }

 HB_FUNC( CALLBACKAI )

{

   pBlock = hb_gcGripGet( hb_param( 1, HB_IT_BLOCK ) );

      hb_retnll( ( HB_LONGLONG ) callbackAI );

      }

 #pragma ENDDUMP

  

I have changed original function name Callback to CallbackAI, to be more clear for me.

Regards, Simo.

[Auge & Ohr]

hi,

your CODE work using CallDll32 ( i have try HMG:Calldll )

but it does not react like Fivewin Sample as i does not type "live"

it will show "Answer" when finish and all if in "one Text"

did you got same ?

Jim

[cod...@outlook.com]

Sorry I was absent.

All is done 64 bit.  HMG is 3.4.4 version, I could not download version 3.5.

Here is all my code, it is very bad, but it somehow works:

………………..

*

#include "hmg.ch"

*

Function Main()

LOCAL cAnswer

LOCAL cPrompt

  DEFINE WINDOW Form_1 ;

    AT 0,0 ;

    WIDTH 600 HEIGHT 800 ;

    TITLE 'Help Demo' ;

    ICON 'HelpDemo.ico' ;

    MAIN ;

    FONT 'MS Sans Serif' SIZE 10

 @ 30, 10 TEXTBOX cPrompt ;

    HEIGHT 25 ;

    WIDTH 500 ;

    VALUE "List 10 possible uses of AI from my Windows apps."

 

 @  70, 50 BUTTON buttonStart  ;

        CAPTION 'Start'   ;

        WIDTH 100                 ;

        HEIGHT 50                 ;

        ONCLICK StartAI()

 

  @ 150, 50 TEXTBOX cAnswer ;

        WIDTH 500                 ;

        HEIGHT 400

 

 @ 600, 50 BUTTON buttonClear  ;

        PARENT Win_1             ;

        CAPTION 'Clear'   ;

        WIDTH 100                 ;

        HEIGHT 50                 ;

        ONCLICK PressedClear()

 

  END WINDOW

 

  CENTER WINDOW Form_1

  ACTIVATE WINDOW Form_1

*

RETURN NIL

*

*--------------------------------------------------------Main-----------

*

FUNCTION StartAI()

*

MsgInfo( 'Question='+ALLTRIM(Form_1.cPrompt.value) )

 

 CallDll32 ( "Llama" , "llama64.dll" ,"phi-2_Q4_K_M.gguf"  , ;

   ALLTRIM(Form_1.cPrompt.value), ;

   CallBackAI({ | cStr | Form_1.cAnswer.SetFocus, Form_1.cAnswer.value:=Form_1.cAnswer.value+CHR(10)+cStr }) )

 

MsgInfo( 'Response='+ALLTRIM(Form_1.cAnswer.value) )

**oBtn:Enable()   // in Antonio example, did not know how to implement

**oBtn:SetFocus()  // in Antonio example, did not know how to implement

*

RETURN NIL

*

*-----------------------------------------------------StartAI--------------

*

FUNCTION PressedClear()  //  not realized …

   *Form_1.width := 500

   *Form_1.height := 400

   MsgInfo('Clicked Button CLEAR')

RETURN NIL

*

*-----------------------------------------------------PressedClear---------

*

#pragma BEGINDUMP

 

#include <Windows.h>

#include <hbapi.h>

#include <hbapiitm.h>

 

static PHB_ITEM pBlock;

 

static void callbackAI( char * szMsg )

{

   PHB_ITEM pStr = hb_itemPutC( NULL, szMsg );

      hb_evalBlock1( pBlock, pStr );

         hb_itemRelease( pStr );

         }

 

HB_FUNC( CALLBACKAI )

{

   pBlock = hb_gcGripGet( hb_param( 1, HB_IT_BLOCK ) );

      hb_retnll( ( HB_LONGLONG ) callbackAI );

      }

 

#pragma ENDDUMP
……………….

 

To mention again, all worked when I made new llama64.dll, built according  to detailed procedure by Antonio.

 

It works this way:

After click on Start BUTTON  I must wait about 30 seconds, then in cAnswer starts to display answer text, something like typing all in one line. I could not make it multiline as Antonio did. I have low knowledge in HMG. I can not conclude is it alive or not. Answer is not displayed all in a second, it is displayed as I did typing it, on speed as I typed it word by word. You can see progress of typing. It takes about one minute to display all answer.

 

Regards, Simo.

[Auge & Ohr]

hi,

found out why Answer look bad , it is CHR(10) and not CRLF

   CallBackAI({ | cStr | Form_1.cAnswer.SetFocus , ;

            IF(cStr=CHR(10),cStr:=CRLF,cStr) ,;     // add this line

            Form_1.cAnswer.value:=Form_1.cAnswer.value+cStr }) )

Jim

[antonio....@gmail.com]

There are many GGUF to test from HuggingFace

This one works also very well:

mistral-7b-instruct-v0.1.Q4_K_M.gguf

merry christmas

[antonio....@gmail.com]

The reason why llama64.dll has to be created on everyone's PC is because it takes full advantage of your CPU features, 

so if it does not work for you, then you have to build it yourself on your PC

[cod...@outlook.com]

Many thanks Jim !

Now it is clear how to use AI models with HMG.

Merry Christmas.

Best Regards, 

Simo.

[cod...@outlook.com]

Also many thanks to Antonio for sharing his work !

Regards,

Simo.

[antonio....@gmail.com]

next is to learn how to create our own GGUF files, as we already have the engine to use them :-)

so we can have private "chatGPTs" with our own data and from our Harbour apps

[cod...@outlook.com]

HMG and AI model.

 When I replaced EDITBOX with LABEL control, we can see progress of writing answer in label area, same way as in Antonio example.

 This is how I define LABEL:

  @ 60, 20 LABEL   cAnswer ;

        WIDTH 590  ;

        HEIGHT 420 ;

        VALUE 'AI mode response:'

After click on Start button, I wait about 35 seconds, then function starts to write answer, row by row, word by word, in a way as Antonio example do. Response display takes about 1 minute.

EDITBOX takes same time, but with no progress, whole answer displays after 90 seconds.

Needed time is for sure consequence of my machine (4 GB RAM) and internet I have.

It appears that HMG do not have proper control to serve this type application.  For now LABEL is best displaying area, but lacks horizontal and vertical scrollbar. Truth is that no LABEL control have nor need to have scrollbars.

Also, when answer displaying starts, if I do not click on some other windows, all works well. But if I click on some other appliaction (move focus from HMG app), after I return focus on HMG app display progress is stopped until whole answer is formed and then is displayed (as with EDITBOX). I think that some qualified HMG user can fix this.

 

Regards,

Simo.