set datatypes support in Harbour

[Eric Lendvai]

To my knowledge Harbour does not have have the datatype "set". We have Hashes, which is fantastic, but sometimes a simpler datatype like set() would be very practical. Python and JavaScript have it.

So I asked the following questions to ChatGPT:

1. "How to add to the Harbour computer language the set() data type, the same kind that is available in Python and JavaScript for example?"

2. "Could you provide the C code that would need to be added to the compiler to provide native support to the set() data structure?"

The C code provided by the second question seems pretty good already. Could anyone implement the "set" datatype or help me work on this?

If you want to know more about the "set" datatype, ask the following from ChatGPT:
"Explain the set datatype available in Python and JavaScript."

In summary, a "set" is an unordered collection of unique elements.

We should also add support to "set" in the FOR EACH command.

Thanks, Eric

[Aleksander Czajczynski]

btw. you can already use Hash like this (pardon unordered vs ordered naming)
hb_hKeepOrder( hVar, .T. ) , then adding elements, then FOR EACH will process the data in the same order as you've added it.
Hash is then used for checking for uniqueness of the key, but not ordered. You don't have to set values.

LOCAL hVar := { => }
hb_HKeepOrder( hVar, .T. )
hVar[ "z" ] := NIL
hVar[ "x" ] := NIL
hVar[ "c" ] := NIL
? hb_ValToExp( hb_HKeys( hVar ) )

Best regards, Aleksander

[Bacco]

Also, a simple array can be used on FOR EACH...

Riight now i can't see any use of set that isn't covered

by Harbour in some similar or better form, can you

please provide any example not covered by current

implementation?

Regards,

Bacco

--
You received this message because you are subscribed to the Google Groups "Harbour Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-deve...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/harbour-devel/67A35BCF.5090300%40fki.pl.

[Eric Lendvai]

Yes it is possible to use arrays or hashes and make them work as sets. 
But I was thinking for people coming to Harbour from other languages, it would be nice to have this natively.

Adding features to the language will also confirm the language is not stagnant or dead.

Other features that would be nice to add to Harbour:

1. The "yield" command from Python, which would make it easier to make generators. 

2. Named parameters with default values for functions.

3. Native SQL support on in-memory tables.

Use cases:

• Store unique values:
• The primary purpose of a Set is to ensure that all values are unique. This is useful for scenarios like:

  • Removing duplicates from an array: Convert an array to a Set, then convert it back to an array to eliminate duplicates.
  • Tracking unique visitors to a website: Store each visitor's ID in a Set to easily count the number of unique visitors.
  • Implementing a tag system: Store tags in a Set to prevent users from adding the same tag multiple times.
• Perform fast membership checks:

  Sets use a hash table internally, which makes checking if an element exists in a Set much faster than checking if it exists in an array, especially for large datasets. This is beneficial for:

  • Caching: Quickly check if a value is already cached.
  • Filtering: Efficiently filter a collection of items based on whether they exist in a Set.
• Perform mathematical set operations:

  JavaScript Sets have built-in methods for performing set operations like:

  • Union: Combine two Sets to create a new Set containing all the unique elements from both.
  • Intersection: Find the common elements between two Sets.
  • Difference: Find the elements that are in one Set but not the other.

In JavaScript:

 1. Removing Duplicates from an Array

const array = [1, 2, 2, 3, 4, 4, 5];
const uniqueArray = [...new Set(array)];
console.log(uniqueArray);  // [1, 2, 3, 4, 5]

2.Checking for Uniqueness

const set = new Set();
set.add(1);
set.add(2);
set.add(2); // Duplicate won't be added
console.log(set);  // Set { 1, 2 }

3. Fast Lookups

const set = new Set([1, 2, 3, 4, 5]);
console.log(set.has(3));  // true
console.log(set.has(6));  // false

4. Performing Set Operations (Union, Intersection, Difference)

const set1 = new Set([1, 2, 3, 4]);
const set2 = new Set([3, 4, 5, 6]);

// Union
const union = new Set([...set1, ...set2]);

// Intersection
const intersection = new Set([...set1].filter(x => set2.has(x)));

// Difference
const difference = new Set([...set1].filter(x => !set2.has(x)));

console.log(union);       // Set { 1, 2, 3, 4, 5, 6 }
console.log(intersection); // Set { 3, 4 }
console.log(difference);  // Set { 1, 2 }

 5. Tracking State or History

const visitedPages = new Set();
visitedPages.add('home');
visitedPages.add('about');
visitedPages.add('home');  // Will not be added again

console.log(visitedPages);  // Set { 'home', 'about' }

 6. Efficient Filtering and Set Operations

const array1 = [1, 2, 3, 4, 5];
const array2 = [4, 5, 6, 7, 8];
const uniqueValues = new Set([...array1, ...array2]);
console.log([...uniqueValues]);  // [1, 2, 3, 4, 5, 6, 7, 8]

Summary

In general, Sets are perfect for use cases where:

• You need to store unique values.
• You want fast membership checking or lookups.
• You're performing set-based operations like unions, intersections, and differences.
• You want to remove duplicates from arrays or other data structures.

[Aleksander Czajczynski]

Eric Lendvai wrote:
> Yes it is possible to use arrays or hashes and make them work as sets.
> But I was thinking for people coming to Harbour from other languages,
> it would be nice to have this natively.
>
> Adding features to the language will also confirm the language is not
> stagnant or dead.
>
> Other features that would be nice to add to Harbour:
> 1. The "yield" command from Python, which would make it easier to make
> generators.
> 2. Named parameters with default values for functions.
> 3. Native SQL support on in-memory tables.

I plan to make some work this year on supporting SQL queries on top of
RDD (any opened workarea - not only in-memory). But first something else
has priority!

Who would work on ideas such as these? Because we have many more ideas,
that is not backed by manpower.

You can make a some sort Unique( <aArray>/<hHash> ) that will return an
array that has only unique elements/keys, if the source is hash use
HB_HKeys(). JavaScipt seems to have a very long spec to such simple
functionality, really doesn't look like it's necessary to make a new
ValType() out of it. Also Set() name collides with DBASE/Clipper SET's

Best regards, Aleksander

[Eric Lendvai]

Hello Aleksander,
It would be amazing to have native SQL support in Harbour!
VFP has it, but its a dead language. It was amazing to be able to run several remote SQL from any backends like PostgreSQL for example, then run more local SQL on those results.

Maybe this will also help xBase++ developers move to Harbour, since it also has some local SQL support.
Being able to run thousands of SQL queries locally makes it so much easier, especially for ETL jobs.
The local SQL could also access local variables, and even call local functions or methods from in-scope objects.

I will gladly write some documentation for it, article in harbour.wiki, and of course help you test it.

Best, Eric

[Aleksander Czajczynski]

Hello Eric!

Eric Lendvai wrote:
> It would be amazing to have native SQL support in Harbour!
> VFP has it, but its a dead language. It was amazing to be able to run
> several remote SQL from any backends like PostgreSQL for example, then
> run more local SQL on those results.
> Maybe this will also help xBase++ developers move to Harbour, since it
> also has some local SQL support.
> Being able to run thousands of SQL queries locally makes it so much
> easier, especially for ETL jobs.
> The local SQL could also access local variables, and even call local
> functions or methods from in-scope objects.
> I will gladly write some documentation for it, article in
> harbour.wiki, and of course help you test it.

Thank you very much for that, I'll reach you as soon as I have something
usable and worth your time.

Best regards, Aleksander

[Antonio Linares]

Dear Aleksander,

> I plan to make some work this year on supporting SQL queries on top of
> RDD (any opened workarea - not only in-memory)

That would be really great! I will happily help you on that!

best regards

[Diego Pego]

I've created a yield prototype using macros and detached locals :) I'd love to have it too.

[Diego Pego]

I'd love to have Native SQL support on tables, and I would be happy to help.

[Aleksander Czajczynski]

Hello, Diego!

Such a prototype is welcome, if you want to share. Basically it's a form
of coroutine/green thread, where called function pauses it's execution
to return a single value to caller. Harbour has task threads on
platforms without threading, maybe something can be reused, to
temporarily switch VM stacks.

Best regards, Aleksander

[Diego Pego]

#include "hbclass.ch"

/*
   Create a class that behaves like a simple coroutine/generator.
   Harbour doesn't have coroutines, so we emulate one using code blocks
   (blocks here act like closures that keep internal state).
*/
CREATE CLASS MyCoroutine
   VAR  coroutineBlock
   VAR  cachedNext
   VAR  finished
   METHOD New( bBlock )
   METHOD HasNext()
   METHOD Next()
   METHOD Reset( bBlock )   // rebinds a new block; blocks keep their own state
ENDCLASS

METHOD New( bBlock ) CLASS MyCoroutine
   ::coroutineBlock := bBlock
   ::cachedNext     := NIL
   ::finished       := .F.
RETURN Self

METHOD HasNext() CLASS MyCoroutine
   IF ::finished
      RETURN .F.
   ENDIF

   IF ValType( ::cachedNext ) == "U"
      ::cachedNext := Eval( ::coroutineBlock )
      IF ValType( ::cachedNext ) == "U"
         ::finished := .T.
         RETURN .F.
      ENDIF
   ENDIF

RETURN .T.

METHOD Next() CLASS MyCoroutine
   LOCAL uValue

   IF ! ::HasNext()
      RETURN NIL
   ENDIF

   uValue        := ::cachedNext
   ::cachedNext  := NIL
RETURN uValue

METHOD Reset( bBlock ) CLASS MyCoroutine
   // Rebind a new block (or the same factory again) to restart.
   ::coroutineBlock := bBlock
   ::cachedNext     := NIL
   ::finished       := .F.
RETURN Self


PROCEDURE Main()
   LOCAL bCounter := Counter( 10 )
   LOCAL oGen     := MyCoroutine():New( bCounter )
   LOCAL nValue

   DO WHILE oGen:HasNext()
      nValue := oGen:Next()
      ? nValue
   ENDDO

   // Example: reset with a new block to iterate again up to 5
   oGen:Reset( Counter( 5 ) )
   DO WHILE oGen:HasNext()
      ? oGen:Next()
   ENDDO
RETURN


/*  Counter factory:
    Returns a block that yields 1..nMax and then returns NIL (no more values).
*/
FUNCTION Counter( nMax )
   LOCAL nCount := 0
RETURN {|| IF( nCount < nMax, ( nCount += 1 ), NIL ) }

[Алексей Запольский (alex;)]

Hello, all.

I was thinking about implementing SQL Engine in Horbour. And the first place to start, I think, is to create a definition of indexes with fields separated by commas, as in the ORDER BY expression. Maybe an extension .cd2? What do you think?
WBR, alex;

вторник, 12 августа 2025 г. в 06:54:58 UTC+3, Diego Pego: