Symbol Tables

[Eric Lendvai]

I am a little confused between all the following terms:

1. Table of Symbols
2. Local Symbol Table
3. Dynamic Symbol Table

If I am correct, 3. will store all private, public and class properties/methods names.

None will store name of Static or Local variables, since the compiler resolves them, meaning not at runtime.

Could someone clearly state the names of the Symbol Tables and what they are used for?

Also any links to documentation would be great.

I am currently working on the "Variable Types and Scope" section of the Developers Guide.

By the way the latest sections I wrote are: 

• "Assignments"
• "Conditional Statements"
• "Loop"
• "Conditional Loops"
• "NULL,NIL and EMPTY"

Any reviews, comments or corrections would be great, thanks, Eric

[Eric Lendvai]

Forgot to mention the link to the guide:  https://harbour.wiki/index.asp?page=PublicArticles&mode=show&id=220304024106&sig=2327201296

[Antonio Linares]

Dear Eric,

Using a simple test.prg:

function Main()

   ? "Hello world"

return nil

if you compile it: harbour test.prg /n

you get this test.c

/*
 * Harbour 3.2.0dev (r2004201301)
 * Microsoft Visual C 19.21.27702 (64-bit)
 * Generated C source from "test.prg"
 */

#include "hbvmpub.h"
#include "hbinit.h"


HB_FUNC( MAIN );
HB_FUNC_EXTERN( QOUT );


HB_INIT_SYMBOLS_BEGIN( hb_vm_SymbolInit_TEST )
{ "MAIN", {HB_FS_PUBLIC | HB_FS_FIRST | HB_FS_LOCAL}, {HB_FUNCNAME( MAIN )}, NULL },
{ "QOUT", {HB_FS_PUBLIC}, {HB_FUNCNAME( QOUT )}, NULL }
HB_INIT_SYMBOLS_EX_END( hb_vm_SymbolInit_TEST, "test.prg", 0x0, 0x0003 )

#if defined( HB_PRAGMA_STARTUP )
   #pragma startup hb_vm_SymbolInit_TEST
#elif defined( HB_DATASEG_STARTUP )
   #define HB_DATASEG_BODY    HB_DATASEG_FUNC( hb_vm_SymbolInit_TEST )
   #include "hbiniseg.h"
#endif

HB_FUNC( MAIN )
{
    static const HB_BYTE pcode[] =
    {
        36,3,0,176,1,0,106,12,72,101,108,108,111,32,
        119,111,114,108,100,0,20,1,36,5,0,100,110,7
    };

    hb_vmExecute( pcode, symbols );
}

From the above output, this is what we name the "local symbol table":

HB_INIT_SYMBOLS_BEGIN( hb_vm_SymbolInit_TEST )
{ "MAIN", {HB_FS_PUBLIC | HB_FS_FIRST | HB_FS_LOCAL}, {HB_FUNCNAME( MAIN )}, NULL },
{ "QOUT", {HB_FS_PUBLIC}, {HB_FUNCNAME( QOUT )}, NULL }
HB_INIT_SYMBOLS_EX_END( hb_vm_SymbolInit_TEST, "test.prg", 0x0, 0x0003 )

It is a C structure that contains the symbols used in test.prg. Each PRG that we compile, generates a "local symbol table"

This was learned and imported from clipper.exe way of working.

All the "local symbol tables" need to be referenced and accessible from a "global symbol table": One table that can access all of them. Again

this way of working was learned and imported from Clipper. 

When Harbour boots, all the local symbol tables must be stepped through to build the "global symbol table". Again this is done

in a similar way as Clipper did it, but using modern C compilers segments management techniques. This code in each C/OBJ does

the "magic":

#if defined( HB_PRAGMA_STARTUP )
   #pragma startup hb_vm_SymbolInit_TEST
#elif defined( HB_DATASEG_STARTUP )
   #define HB_DATASEG_BODY    HB_DATASEG_FUNC( hb_vm_SymbolInit_TEST )
   #include "hbiniseg.h"
#endif

Once that the global symbol table is built, besides being used by Harbour, we can "do" several things with it. We will

review it on a next msg.

best regards

[Antonio Linares]

The main technical reason for this local symbols table is execution speed. 

In the times when Clipper was born, computers were very slow and execution speed was a must. It was the key between a good fast app or a slow, useless one. Nantucket guys did a wonderful job with this design. Not sure if the idea was theirs or they learnt it from another language (smalltalk, objective-C, etc). The fact is that the local symbol table is extremely fast as we don't have to search for the functions (and other symbols) to use.

They are placed by the linker in the right position so when the pcode, executed by the virtual machine, is going to use them, no search at all is required. 

This is one of the reasons for the excellent execution speed of Clipper, and still used by all of us today, when we use our loved Harbour.

best regards,

[Eric Lendvai]

Hello Antonio,

Thank you very much for your explanations. 

So there is one "Dynamic Symbol Table" for the entire EXE. Are all the local symbol tables copied into the Dynamic one?

I see that Pete has a page with all the functions to query the Dynamic Symbol Table: https://github.com/Petewg/harbour-core/wiki/Symbols

So if I create at runtime a new class, it's name, methods and properties get added to the Dynamic Symbol Table?

I think that it stays in there even If try to recreate the class later, and thus the past property names are left in there.

Also you mentioned searches, so if each PRG as a local symbol table, internally it is searched first, then only then the global one is searched?

Thanks, Eric

[alexwi]

"So if I create at runtime a new class, it's name, methods and properties get added to the Dynamic Symbol Table?

I think that it stays in there even If try to recreate the class later, and thus the past property names are left in there."

I know nothing about the internals of harbour (and am only learning about those from clipper now), but my guess is that this falls in the realm of harbour's garbage collector. If you create an object and discard it (i.e. it falls out of focus or is explicitly destroyed) it's up to harbour to keep it somewhere in memory or dispose of it immediately.

One way to test either theory would be to have a program create and explicitly destroy an object a few thousand times (probably a few tens of thousands, to be able to notice the phenomenon unequivocally) and check the executable's memory usage. If it keeps going up, then it follows that a new symbol table is created(or entries added to an existing one) at every instantiation. Just my guess.

Alex

[Eric Lendvai]

Hello Antonio,

Looking at all the functions Pete listed at https://github.com/Petewg/harbour-core/wiki/Symbols

Do you have any practical use for them? Any example for using them. (Just so I could use them in the developers guide).

We already have  hb_IsFunction() and __mvExist() for private/public variable?

Also since we now have local and static variables, would it not be good practice to reduce the use of public and private variable?

Aren't private variables declared in main behaving as public variables?

Also I think xHarbour as GLOBAL "static" variables. Do we have something like that in Harbour?
Thanks, Eric

[Antonio Linares]

Dear Eric,

Here you have a public repo that allows you to build a Harbour app, building Harbour itself, from Microsoft Visual Studio IDE so you can low level debug it and see it running step by step, you can set breakpoints, inspect variables, structures, etc. You can see how it boots, how it initializes everything, etc. its a great way to see how Harbour works internally:

https://github.com/FiveTechSoft/harbour_app_msvc64_sources

I do recommend it to enhance our Harbour low level architecture understanding.

best regards,

[Antonio Linares]

[Antonio Linares]

Regarding your symbols question, this is a portion of code of one of the mod_harbour examples named symbols.prg:

https://github.com/FiveTechSoft/mod_harbour/blob/master/samples/symbols.prg

  for n = __DynsCount() to 1 step -1
     if lAll .or. lCode
        if __DynsIsFun( n )
           ?? '<p style="color:springgreen;">' + AllTrim( Str( __DynsCount() - n + 1 ) ) + ". " + __DynsGetName( n ) + "</p>" + CRLF
        endif
     endif
     if lAll .or. lNonCode
        if ! __DynsIsFun( n )
           ?? '<p style="color:yellow;">' + AllTrim( Str( __DynsCount() - n + 1 ) ) + ". " + __DynsGetName( n ) + "</p>" + CRLF
        endif
     endif
  next

best regards,

[Eric Lendvai]

Real interesting, Thanks !!