Schemas in Firebird - legacy API

[Adriano dos Santos Fernandes]

Hi!

We have functions like this, that was not even created in the new API:

ISC_STATUS API_ROUTINE isc_array_lookup_bounds(
ISC_STATUS* userStatus,
FB_API_HANDLE* dbHandle,
FB_API_HANDLE* traHandle,
const SCHAR* relationName,
const SCHAR* fieldName,
ISC_ARRAY_DESC* desc)

I think this (client) function, when dealing with database engine that
supports schemas, should look for relationName using the a search path
like for others names resolutions.

But should also we create a new function (isc_array_lookup_bounds2) also
expecting an explicit relationSchema?

Relying on user encoding a schema inside relationName is not going to
work, as currently it's possible to create object names that have dots
and double quotes and would make things ambiguous.

Comments?


Adriano

[Dimitry Sibiryakov]

Adriano dos Santos Fernandes wrote 20.05.2024 13:16:
> But should also we create a new function (isc_array_lookup_bounds2) also
> expecting an explicit relationSchema?

I see no reasons to create new functions for ISC API. It is
backward-compatibility API and nobody will use them because they are "too new".

--
WBR, SD.

[Pavel Cisar]

Hi,

Dne 20. 05. 24 v 13:19 'Dimitry Sibiryakov' via firebird-devel napsal(a):

>
>   I see no reasons to create new functions for ISC API. It is
> backward-compatibility API and nobody will use them because they are
> "too new".

Well, this particular function is important for implementation of ARRAY
support in any driver. Have no idea why it was not added to OO API, but
it is as it is. So you either add it to OO API with few others
(get_slice/put_slice) and then you can solve this problem at interface
level, or add the new ISC version in old API so Firebird driver and few
other could continue to support ARRAY type with Schemas.

best regards
Pavel Cisar
IBPhoenix

[Dimitry Sibiryakov]

Pavel Cisar wrote 20.05.2024 13:49:
>
> Well, this particular function is important for implementation of ARRAY support
> in any driver. Have no idea why it was not added to OO API, but it is as it is.

The answer is simple: ARRAYs were considered desupported when this API was
designed.

> So you either add it to OO API with few others (get_slice/put_slice) and then
> you can solve this problem at interface level, or add the new ISC version in old
> API so Firebird driver and few other could continue to support ARRAY type with
> Schemas.

Array is basically a binary BLOB. OO API allows to get its content. The
problem only to parse it.

--
WBR, SD.

[James Starkey]

Perhaps you are unaware that different machine architectures have different representations of binary numbers?

With similarity vector searching, arrays are more important than in the past.

Jim Starkey

--
You received this message because you are subscribed to the Google Groups "firebird-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-deve...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebird-devel/eaa86c89-3df0-47b9-a54d-8c4839186239%40ibphoenix.com.

[Pavel Cisar]

Dne 20. 05. 24 v 13:54 'Dimitry Sibiryakov' via firebird-devel napsal(a):

> Pavel Cisar wrote 20.05.2024 13:49:
>>
>> Well, this particular function is important for implementation of
>> ARRAY support in any driver. Have no idea why it was not added to OO
>> API, but it is as it is.
>
>   The answer is simple: ARRAYs were considered desupported when this
> API was designed.

Wonderful. I know that almost nobody uses them, but as long as
EMPLOYEE.FDB uses them, we have to support it. I'm no fan of ARRAYs and
would gladly get rid of them as they are pain in the ass, but there was
not even proper deprecation initiated! Hence leaving them out of IUtils
was (and still is) a great mistake that should have been fixed long time
ago (in fact, it could be fixed at any moment as extending IUtils is
simple and cause no problems even in minor versions), because anyone who
want to write a driver using OO API AND support array type must use both
APIs, hence full transition to new API is not possible.

>> So you either add it to OO API with few others (get_slice/put_slice)
>> and then you can solve this problem at interface level, or add the new
>> ISC version in old API so Firebird driver and few other could continue
>> to support ARRAY type with Schemas.
>
>   Array is basically a binary BLOB. OO API allows to get its content.
> The problem only to parse it.
>

Well, one could theoretically parse it, but first we need to know the
layout etc. That's what isc_array_lookup_bounds does, returning the
arraydesc structure with all the necessary information. However
isc_array_put/get_slice are good helpers to move data in/out, otherwise
everyone would have to implement their own version of these functions
anyway.

[Dimitry Sibiryakov]

Pavel Cisar wrote 20.05.2024 14:23:
> Well, one could theoretically parse it, but first we need to know the layout
> etc. That's what isc_array_lookup_bounds does, returning the arraydesc structure
> with all the necessary information.

This function does nothing more but simple query to system tables to fill the
structure:

select f.rdb$field_name,
f.rdb$field_type,
f.rdb$field_scale,
f.rdb$field_length,
f.rdb$dimensions
from rdb$relation_fields rf
join rdb$fields f
on f.rdb$field_name = rf.rdb$field_source
where rf.rdb$relation_name = ? and
rf.rdb$field_name = ?

And it does it _from client_.

--
WBR, SD.

[Alex Peshkoff]

May be we should just implement it in OO API? Certainly taking into an
account schemas.

[Dimitry Sibiryakov]

Alex Peshkoff wrote 20.05.2024 14:51:
> May be we should just implement it in OO API? Certainly taking into an account
> schemas.

I don't think that this function has any value by itself. Interface IArray
handled the same way as IBlob might be.
I see the problem that it takes names as parameters. This is the only
function that does it. No such functions for BLOBs, tables, SPs, etc.
And what the point in names if you already have "array ID" (ISC_QUAD) at hand
to get the content?

--
WBR, SD.

[Alex Peshkoff]

Can you suggest IArray design?

[Dimitry Sibiryakov]

Alex Peshkoff wrote 20.05.2024 15:25:
> Can you suggest IArray design?

No: I have no practical needs in arrays to create something useful and
interfaces created from theoretical needs used to be non-useable.

--
WBR, SD.

[Jim Starkey]

The impetus for the Interface array feature set was a requirement from the Boeing Aircraft noise group.  They collected large number of sound samples from microphone arrays and needed to store and crunch these.  They had been using an ad hoc database with array support and didn't like having to use to two different database systems.

In retrospect, I'm inclined to think that it was seriously over designed, but it had to work in an environments of tiny main memory (by today's standards) and networks that topped out at 10mbs.  Then, a complete transfer of a large array was problem; now, not much.

When I revisited the issues for Amorphous, I decided that single dimension arrays were good enough (am I completely sure? No.).

In today's AI obsessed world, arrays, hierarchical navigable small world indexes, and a nearest operator are sine quo non if you want to play with the state of the art.

The Amorphous array info call and array descriptors are:

  virtual int                getArray(Array* descriptor) = 0;        // returns length of array in bytes    
 

struct Array
    {
    int            size;                // number of elements
    int            type;                // data type of element
    union
        {
        const double*        doubles;
        const float*         floats;
        const int*           ints;
        const unsigned char* bytes;
        const void*           data;            // used for generic references
        } array;
    };

But, like I said, I may extend/redefine this for a vector of array bounds.  And, incidentally, for AI stuff, 16 bit floats are very important; similarity vectors care a great deal more about magnitude than precision.

--
Jim Starkey, AmorphousDB, LLC

[Mark Rotteveel]

That depends. Driver implementations that rely on the legacy API might
very well not want to switch to the OO API, but still be able to use
some of the newer features in this way.

Mark
--
Mark Rotteveel